Predictive Vertical Pod Rightsizing

​

Prerequisites

• Thoras is installed and running in your Kubernetes cluster.
• The Kubernetes Metrics Server is installed in your cluster.
• Your workloads are instrumented with resource requests and have some usage history.

​

Sample Configuration

apiVersion: thoras.ai/v1
kind: AIScaleTarget
metadata:
  name: {{YOUR_AST_NAME}}
  namespace: {{YOUR_NAMESPACE}}
spec:
  scaleTargetRef:
    kind: Deployment
    name: {{YOUR_DEPLOYMENT_NAME}}
  model:
    forecast_blocks: 4h
    forecast_interval: 1h
    forecast_buffer_percentage: 0%
    mode: balanced
  vertical:
    mode: recommendation
    containers:
      - name: {{YOUR_CONTAINER_NAME}}
        cpu:
          lowerbound: 20m
          upperbound: 1
        memory:
          lowerbound: 50Mi
          limit:
            ratio: 1.5
    update_policy:
      update_mode: in_place_or_recreate

​

Vertical Breakdown

• spec.vertical.mode set to recommendation ensures Thoras suggests optimal requests and limits but does not apply them.
• spec.vertical.containers[].name should be set to the name of the container in the pods for the workload you are targeting with this AIScaleTarget. This field supports wildcard patterns (e.g., app-*, *-sidecar) to match multiple containers. See Container Name Wildcard Matching for details.
• spec.vertical.containers[].RESOURCE.lowerbound is the lowest Thoras is allowed to set the request for that resource and is a required field.
• spec.vertical.containers[].RESOURCE.upperbound is the highest Thoras is allowed to set the request for that resource and is an optional field.
• In autonomous mode, if you change lowerbound or upperbound and live pod requests fall outside the new range, Thoras applies the updated bounds immediately according to update_policy (in-place resize and/or recreate). With update_mode: initial, bounds still apply only when pods are next created.
• spec.vertical.containers[].RESOURCE.limit allows thoras to modify the container’s limit along with the request for this resource and is optional. If unset, Thoras will not modify limits for this resource.
• spec.vertical.containers[].RESOURCE.limit.ratio the ratio used to keep the limit inline with the suggested request for this resource. E.g. for a ratio 2.5 Thoras will make the limit 2.5G if the suggestion for the request was for 1G.

​

How Limits Work With Predictive Right-Sizing

​

Limit Enforcement Rules

• AIScaleTarget upperbound: If you define an upperbound in your AIScaleTarget, Thoras will never set requests to above that value during steady-state operation.
• Pod spec limit: If a limit exists on the pod spec, Thoras will never right-size a pod above that limit during steady-state operation.

​

Scaling Limits Proportionally

​

Update Policy

• update_mode - Determines how pod updates are applied:
  • recreate - Pods are recreated when resource changes are applied
  • in_place - Changes applied without pod recreation (requires Kubernetes 1.33+)
  • in_place_or_recreate - Attempts in-place updates with fallback to recreation (recommended)
  • initial - Recommendations apply only during pod creation
• recreate_resources (optional; defaults to memory only) - Array specifying which resources (memory, cpu) trigger pod recreation when using recreate or fallback modes.

update_policy:
  update_mode: in_place_or_recreate

update_policy:
  update_mode: in_place
  recreate_resources: ["memory"]

update_policy:
  update_mode: in_place_or_recreate
  recreate_resources: ["memory", "cpu"]

Note: Enabling in-place resizing does not guarantee that pods will never restart during resizing operations. Pods may still be evicted and restarted if the resize would change the pod’s QoS class or if the cluster does not support the specific resize operation.

1. When update_mode is in_place or in_place_or_recreate, Thoras attempts to resize pods in place using the Kubernetes resize API. Resizes are rolled out in paced batches across the workload rather than all at once, governed by disruption. If the Kubernetes API rate-limits these requests, Thoras backs off and resumes the remaining pods on a later run
2. When update_mode is in_place_or_recreate and the pods QoS class would change or resize is not supported, Thoras falls back to recreating the pod
3. Evicted pods are also rolled out gradually to minimize service disruption

• Reduced disruption: Pods continue running during resource adjustments
• Faster scaling: Changes take effect immediately without pod recreation
• Better node utilization: Enables more efficient bin-packing and cost savings, especially when combined with node autoscaling tools like Karpenter
• Graceful fallback: Automatic eviction when in-place resizing isn’t possible
• Fine-grained control: Specify which resources trigger pod recreation

• Kubernetes 1.33+

​

Additional Details

• Thoras recommendations are based on historical usage data collected from the Kubernetes Metrics Server.
• Recommendations are updated on the interval defined by spec.model.forecast_interval.
• If resource usage patterns change significantly, Thoras will adjust its suggestions accordingly.
• You can monitor the impact of rightsizing by reviewing resource utilization and cost metrics in the Thoras Dashboard.
• For workloads with multiple containers, you can specify vertical settings for each container individually under spec.vertical.containers.
• spec.vertical.containers[].RESOURCE can be either cpu or memory. If either is unset for a container, Thoras will not scale that resource for that container.

​

Viewing Recommendations

​

Next Steps

• Change spec.vertical.mode to autonomous if you want Thoras to automatically apply recommendations.
• Add optional scaling_behavior if you want to fine-tune the thresholds Thoras uses to determine if a recommendation is worth restarting the entire workload.

  ...
  vertical:
    mode: autonomous
    # Only restart workload if a new resource recommendation is at least 50% greater or lower than the current pod resource allocation
    scaling_behavior:
      scale_up:
        type: percent
        percent: 50
      scale_down:
        type: percent
        percent: 50
    ...

​

Autonomous Mode

Note: Thoras mutates the resource requests and limits of running pods directly. It does not modify the Deployment or StatefulSet object itself.

Note: Only one scaling direction (horizontal or vertical) can be in autonomous mode at a time. See Configuring AIScaleTargets for additional details.

​

Identifying Thoras-Managed Pods