Skip to main content
The specification describes the AIScaleTarget Custom Resource that defines how Thoras scales workloads. The following is a sample AIScaleTarget definition:
ast.yaml
apiVersion: thoras.ai/v1
kind: AIScaleTarget
metadata:
  name: "{{ YOUR_AST_NAME }}"
  namespace: "{{ YOUR_NAMESPACE }}"
spec:
  scaleTargetRef:
    kind: Deployment
    name: "{{ YOUR_AST_NAME }}"
  model:
    forecast_blocks: 15m
    forecast_buffer_percentage: 0%
    forecast_cron: "*/15 * * * *"
  horizontal:
    mode: recommendation
    scaling_behavior:
      scale_up:
        type: percent
        percent: 50
      scale_down:
        type: percent
        percent: 50
  vertical:
    containers:
      - name: "{{ CONTAINER_NAME }}"
        cpu:
          lowerbound: 20m
          upperbound: 1
        memory:
          lowerbound: 50Mi
          upperbound: 2G
    mode: recommendation
    scaling_behavior:
      scale_up:
        type: percent
        percent: 50
      scale_down:
        type: percent
        percent: 50
    in_place_resizing:
      enabled: true
      allow_restart_on_memory_limit_decrease: false

​
metadata

ast.yaml
metadata:
 name: {{ YOUR_AST_NAME }}
  namespace: {{ YOUR_NAMESPACE }}
It’s recommended that metadata.name matches the name of the workload being scaled.

​
scaleTargetRef

ast.yaml
scaleTargetRef:
  kind: Deployment
  name: {{YOUR_AST_NAME}}
  namespace: {{YOUR_NAMESPACE}}
  apiVersion: apps/v1
scaleTargetRef is the reference to a specific workload that Thoras will scale up or down. The workload must reside in the same namespace as the AIScaleTarget. apiVersion specifies the API group and version that defines the resource, while kind identifies the type of resource (e.g., Deployment, StatefulSet, Rollout, etc). Note: scaleTargetRef and selector are mutually exclusive. Use scaleTargetRef to target a single workload by name, or use selector to target multiple workloads by label.

​
selector

ast.yaml
selector:
  matchLabels:
    app: my-service
    environment: production
selector allows you to target multiple workloads by pod labels instead of specifying a single workload by name. This is useful when you want to apply the same scaling policy to multiple workloads that share common labels. Important constraints:
  • selector and scaleTargetRef are mutually exclusive—you must use one or the other, not both.
  • When using selector, horizontal scaling is not supported. You must leave spec.horizontal empty or omit it entirely.
  • Pods must be managed by a Deployment, StatefulSet, or Argo Rollout. Standalone pods or other controller types are not supported.
How it works: When a selector is defined, Thoras identifies all pod controllers (Deployments, StatefulSets, Rollouts) in the same namespace whose pods match the label selector. In autonomous mode:
  • Thoras restarts each matching pod controller one by one when a resource request suggestion comes in from the forecaster.
  • If in_place_resizing is enabled, pods are resized in place without restarts (when possible).

​
model

ast.yaml
model:
  forecast_blocks: 15m
  forecast_buffer_percentage: 0%
  forecast_cron: "*/15 * * * *"

​
model.forecast_blocks

Describes how far into the future Thoras’ scaler should prepare you for. For example, if forecast_blocks is 15m Thoras will forecast the maximum load of this workload over the next 15 minutes and then scale to that maximum. Important: forecast_blocks should be at least equal to your forecast_cron interval. For example, if your forecast_cron is set to run every 15m, your forecast_blocks should be at least 15m. This ensures forecasts always cover the next scaling window and prevents gaps in scaling coverage. You will need to specify either the “m” (for minutes) or “h” (for hours) unit for the value of model.forecast_blocks.

​
model.forecast_buffer_percentage

Defines an additional buffer applied on top of the forecasted resource usage to reduce the risk of under-provisioning. For example, if the forecasted CPU usage is 500m and forecast_buffer_percentage is set to 10%, the final recommended value will be 550m. Setting this to 0% means no buffer will be added.

​
model.forecast_cron

The forecast_cron setting determines how often forecasts are generated and, consequently, how often Thoras may trigger scaling events. Shorter intervals (e.g., every 5 minutes) can lead to more accurate forecasts but may also result in more frequent scaling. forecast_cron controls how often forecasts are made, while model.forecast_blocks defines the time window each forecast covers.

​
horizontal

ast.yaml
horizontal:
  mode: recommendation
  scaling_behavior:
    scale_up:
      type: percent
      percent: 50
    scale_down:
      type: percent
      percent: 50
Thoras horizontal scaling adjusts the number of replicas based on forecasted or observed workload demands. In autonomous mode:
  • Horizontally-scaled workloads must have CPU and/or memory requests defined in the pod spec if you plan to scale on averageUtilization.
  • The target deployment of your AIScaleTarget must have a single existing Horizontal Pod Autoscaler (HPA).
In recommendation mode :
  • System does not require an existing HPA. Thoras will assume an 80% utilization target for CPU and memory when generating scaling suggestions.
Thoras does not replace your HPA—it works alongside it by feeding forecasted metrics into your existing HPA configuration. You can customize behavior by:
  • Opting in additional metrics using spec.horizontal.additional_metrics
  • Opting out of default metrics using spec.horizontal.exclude_metrics
This setup gives you full control over how scaling decisions are made while benefiting from Thoras’ predictive intelligence. To enable horizontal scaling recommendations, set spec.horizontal.mode to recommendation. It is recommended to start in recommendation mode to allow the model time to learn workload patterns and validate scaling suggestions. Once the recommendations align with your performance expectations, you can switch to autonomous mode for automated scaling. Feel free to reach out to the Thoras Engineering Team to discuss model performance before switching into autonomous mode. Visit Predictive Horizontal Pod Autoscaling with Thoras Guide for more info.

​
vertical

ast.yaml
vertical:
  containers:
    - name: {{CONTAINER_NAME}}
      cpu:
        lowerbound: 20m # Mandatory
        upperbound: 1 # Optional
      memory:
        lowerbound: 50Mi # Mandatory
        upperbound: 2G # Optional
  mode: recommendation
  scaling_behavior:
    scale_up:
      type: percent
      percent: 50
    scale_down:
      type: percent
      percent: 50
  in_place_resizing:
    enabled: true
    allow_restart_on_memory_limit_decrease: false
The Thoras vertical scaler adjusts container-level CPU and memory resource requests based on forecasted utilization. To define a vertical scaling policy, you’ll want to set the following two fields:
  • vertical.mode: recommendation - we recommend running Thoras in recommendation mode for at least a day for the model to train before enabling autonomous.
  • vertical.containers[0].memory.lowerbound is always required and vertical.containers[0].memory.upperbound is optional if you had a preference for a memory floor and ceiling for your target workload.
Feel free to reach out to the Thoras Engineering Team to discuss model performance before switching into autonomous mode. Visit Predictive Vertical Pod Rightsizing Guide for more info.

​
scaling_behavior

The scaling_behavior field controls the rate and pattern of scaling actions. It helps smooth out rapid scaling and prevents Thoras from scaling too aggressively or too conservatively.
  • scale_up defines how quickly Thoras is allowed to increase the number of replicas.
  • scale_down defines how quickly it can decrease the number of replicas.
scaling_behavior:
  scale_up:
    type: percent
    percent: 50
  scale_down:
    type: percent
    percent: 50
scaling_behavior gives you a dial to control how sensitive Thoras is to triggers a scaling event. This setting helps rate-limit how much Thoras can scale up or down, making your scaling behavior more predictable and safe, based on your needs.

​
in_place_resizing

vertical:
  in_place_resizing:
    enabled: true
    allow_restart_on_memory_limit_decrease: false
The in_place_resizing field enables Kubernetes in-place pod resizing, which allows Thoras to adjust container resource requests and limits without recreating pods. This feature reduces disruption during vertical scaling operations. Prerequisites:
  • Kubernetes 1.33+
Configuration:
  • enabled (boolean): Enable in-place pod resizing. When true, Thoras will attempt to resize pods without eviction. If in-place resizing fails (e.g., due to QoS class changes), pods will be evicted and rolled out gracefully.
  • allow_restart_on_memory_limit_decrease (boolean): Allow pod restarts when memory limits decrease. Note: If you set memory limits in vertical.containers[*].memory.limit, you must set this to true to avoid validation errors.
How it works:
  1. When enabled, Thoras attempts to resize pods in place using the Kubernetes resize API
  2. If the pod’s QoS class would change or the resize operation is not supported, Thoras falls back to evicting the pod
  3. Evicted pods are rolled out gradually rather than all at once, minimizing service disruption
Benefits:
  • Reduced disruption: Pods are not recreated unless necessary
  • Faster scaling: Resource adjustments take effect immediately
  • Better node utilization: Enables more efficient bin-packing and cost savings, especially when combined with node autoscaling tools like Karpenter
  • Graceful fallback: Automatic eviction when in-place resizing is not possible

​
additional_configuration

ast.yaml
horizontal:
  mode: recommendation
  additional_metrics:
    - external:
        metric:
          name: {{EXTERNAL_METRIC_NAME}}
          selector:
            matchLabels:
              app: {{APP_NAME}}
        target:
          averageValue: "1"
          type: AverageValue
      type: External
  exclude_metrics:
    - name: cpu
    - type: Resource
  scaling_behavior:
    scale_up:
      type: percent
      percent: 50
    scale_down:
      type: percent
      percent: 50
When you enable horizontal scaling in Thoras, it defaults to using the same metrics as your HPA (Horizontal Pod Autoscaler) to guide predictive scaling. However, there may be cases where you want Thoras to:
  • Exclude certain metrics (e.g., CPU) from its predictions, even if they are used by the HPA.
  • Add new metrics for prediction that are not part of the HPA.

​
Example 1: Exclude CPU from Predictive Scaling

This configuration tells Thoras not to use CPU for predictive scaling, even if it’s included in the HPA:
ast.yaml
spec:
  horizontal:
    exclude_metrics:
      - name: cpu
        type: Resource

​
Example 2: Add Custom Metric for Predictive Scaling

In this example, custom metric is configured to predictively scale on CPU by explicitly including it in additional_metrics. Thoras will predictively scale on CPU even though it is not defined in the HPA.
ast.yaml
spec:
  horizontal:
    additional_metrics:
      - external:
          metric:
            name: {{EXTERNAL_METRIC_NAME}}
            selector:
              matchLabels:
                app: {{APP_NAME}}
          target:
            averageValue: "1"
            type: AverageValue
        type: External