Predictive Vertical Pod Rightsizing

​

Prerequisites

• Thoras is installed and running in your Kubernetes cluster.
• The Kubernetes Metrics Server is installed in your cluster.
• Your workloads are instrumented with resource requests and have some usage history.

​

Sample Configuration

apiVersion: thoras.ai/v1
kind: AIScaleTarget
metadata:
  name: {{YOUR_AST_NAME}}
  namespace: {{YOUR_NAMESPACE}}
spec:
  scaleTargetRef:
    kind: Deployment
    name: {{YOUR_DEPLOYMENT_NAME}}
  model:
    forecast_blocks: 15m0s
    forecast_buffer_percentage: 0%
    forecast_cron: "*/15 * * * *"
    mode: balanced
  vertical:
    mode: recommendation
    containers:
      - name: {{YOUR_CONTAINER_NAME}}
        cpu:
          lowerbound: 20m
          upperbound: 1
        memory:
          lowerbound: 50Mi
          limit:
            ratio: 1.5
    update_policy:
      update_mode: in_place_or_recreate

​

Vertical Breakdown

• spec.vertical.mode set to recommendation ensures Thoras suggests optimal requests and limits but does not apply them.
• spec.vertical.containers[].name should be set to the name of the container in the pods for the workload you are targeting with this AIScaleTarget.
• spec.vertical.containers[].RESOURCE.lowerbound is the lowest Thoras is allowed to set the request for that resource and is a required field.
• spec.vertical.containers[].RESOURCE.upperbound is the highest Thoras is allowed to set the request for that resource and is an optional field.
• spec.vertical.containers[].RESOURCE.limit allows thoras to modify the container’s limit along with the request for this resource and is optional. If unset, Thoras will not modify limits for this resource.
• spec.vertical.containers[].RESOURCE.limit.ratio the ratio used to keep the limit inline with the suggested request for this resource. E.g. for a ratio 2.5 Thoras will make the limit 2.5G if the suggestion for the request was for 1G.

​

How Limits Work With Predictive Right-Sizing

​

Limit Enforcement Rules

• AIScaleTarget upperbound — If you define an upperbound in your AIScaleTarget, Thoras will never right-size a pod above that value.
• Pod spec limit — If a limit exists on the pod spec, Thoras will never right-size a pod above that limit.

​

Scaling Limits Proportionally

​

Update Policy

• update_mode - Determines how pod updates are applied:
  • recreate - Pods are recreated when resource changes are applied
  • in_place - Changes applied without pod recreation (requires Kubernetes 1.33+)
  • in_place_or_recreate - Attempts in-place updates with fallback to recreation (recommended)
  • initial - Recommendations apply only during pod creation
• recreate_resources (optional; defaults to memory only) - Array specifying which resources (memory, cpu) trigger pod recreation when using recreate or fallback modes.

update_policy:
  update_mode: in_place_or_recreate

update_policy:
  update_mode: in_place
  recreate_resources: ["memory"]

update_policy:
  update_mode: in_place_or_recreate
  recreate_resources: ["memory", "cpu"]

Note: Enabling in-place resizing does not guarantee that pods will never restart during resizing operations. Pods may still be evicted and restarted if the resize would change the pod’s QoS class or if the cluster does not support the specific resize operation.

1. When update_mode is in_place or in_place_or_recreate, Thoras attempts to resize pods in place using the Kubernetes resize API
2. When update_mode is in_place_or_recreate and the pods QoS class would change or resize is not supported, Thoras falls back to recreating the pod
3. Evicted pods are rolled out gradually to minimize service disruption

• Reduced disruption: Pods continue running during resource adjustments
• Faster scaling: Changes take effect immediately without pod recreation
• Better node utilization: Enables more efficient bin-packing and cost savings, especially when combined with node autoscaling tools like Karpenter
• Graceful fallback: Automatic eviction when in-place resizing isn’t possible
• Fine-grained control: Specify which resources trigger pod recreation

• Kubernetes 1.33+

​

Additional Details

• Thoras recommendations are based on historical usage data collected from the Kubernetes Metrics Server.
• Recommendations are updated on the schedule defined by spec.model.forecast_cron.
• If resource usage patterns change significantly, Thoras will adjust its suggestions accordingly.
• You can monitor the impact of rightsizing by reviewing resource utilization and cost metrics in the Thoras Dashboard.
• For workloads with multiple containers, you can specify vertical settings for each container individually under spec.vertical.containers.
• spec.vertical.containers[].RESOURCE can be either cpu or memory. If either is unset for a container, Thoras will not scale that resource for that container.

​

Viewing Recommendations

​

Next Steps

• Change spec.vertical.mode to autonomous if you want Thoras to automatically apply recommendations.
• Add optional scaling_behavior if you want to fine-tune the thresholds Thoras uses to determine if a recommendation is worth restarting the entire workload.

  ...
  vertical:
    mode: autonomous
    # Only restart workload if a new resource recommendation is at least 50% greater or lower than the current pod resource allocation
    scaling_behavior:
      scale_up:
        type: percent
        percent: 50
      scale_down:
        type: percent
        percent: 50
    ...

​

Autonomous Mode

Note: Thoras mutates the resource requests and limits of running pods directly. It does not modify the Deployment or StatefulSet object itself.

Note: Only one scaling direction (horizontal or vertical) can be in autonomous mode at a time. See Configuring AIScaleTargets for additional details.