Thoras Documentation

Overview

ClusterAIScaleTemplate watches for Deployments, StatefulSets, and Argo Rollouts that match a label selector, and automatically creates corresponding AIScaleTargets with a predefined configuration. When workloads are added, updated, or removed, Thoras automatically manages the lifecycle of the associated AIScaleTargets.

Use Cases

Apply AI-powered scaling to all workloads with a specific label (e.g., thoras.ai/enabled: "true")
Standardize scaling policies across multiple microservices
Automatically enable AI scaling for new workloads as they’re deployed
Manage scaling configuration for entire application tiers or environments

Basic Example

apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: production-workloads
spec:
  # Match workloads with specific labels
  selector:
    matchLabels:
      env: production
      thoras.ai/enabled: "true"

  # Template to apply to matching workloads
  template:
    model:
      forecast_interval: 1h
      forecast_blocks: 4h
      forecast_buffer_percentage: 10%
      mode: balanced

    vertical:
      mode: auto
      update_policy:
        update_mode: in_place_or_recreate
      containers:
        - name: "*"
          cpu:
            lowerbound: "10m"
          memory:
            lowerbound: 0

This template will automatically create AIScaleTargets for all Deployments, StatefulSets, and Argo Rollouts in any namespace that have the labels env: production and thoras.ai/enabled: "true".

Specification

ClusterAIScaleTemplateSpec

Field	Type	Required	Description
`selector`	`metav1.LabelSelector`	Yes	Matches workloads to apply this template to. Supports `matchLabels` and `matchExpressions`.
`template`	`AIScaleTemplateSpec`	Yes	Defines the AIScaleTarget configuration to apply.
`policy`	`TemplatePolicy`	No	Controls template application behavior.

AIScaleTemplateSpec

The template spec contains the same fields as AIScaleTarget spec, minus the scaleTargetRef (which is automatically populated based on the matched workload):

Field	Type	Required	Description
`model`	`ModelSpec`	Yes	Model configuration (mode, forecast settings, buffer percentage).
`vertical`	`VerticalSpec`	No	Vertical scaling configuration.
`horizontal`	`HorizontalSpec`	No	Horizontal scaling configuration.

TemplatePolicy

Field	Type	Required	Default	Description
`conflictResolution`	`string`	No	`skip`	How to handle conflicts with existing AIScaleTargets. Options: `skip`, `override`.
`includedNamespaces`	`[]string`	No	-	Allowlist of namespaces to target. Mutually exclusive with `excludedNamespaces`.
`excludedNamespaces`	`[]string`	No	-	Denylist of namespaces to skip. Mutually exclusive with `includedNamespaces`.

Behavior

Automatic AIScaleTarget Creation

When a workload (Deployment, StatefulSet, or Argo Rollout) matches the template’s selector:

Thoras creates an AIScaleTarget with the name <workload-name>-ast
The AIScaleTarget is labeled with thoras.ai/managed-by-template: <template-name>
The AIScaleTarget has an owner reference to the workload (so it’s deleted when the workload is deleted)
The scaleTargetRef is automatically populated with the workload’s details

Lifecycle Management

Workload Created: AIScaleTarget is automatically created
Workload Updated: If labels change and no longer match, the AIScaleTarget is deleted
Workload Opted Out: If the workload is annotated with thoras.ai/skip-template: "true", the template stops managing it. Any existing AIScaleTarget is retained, not deleted; see Opting a Workload Out
Workload Deleted: AIScaleTarget is automatically deleted via owner reference
Template Updated: All managed AIScaleTargets are updated to match the new template
Template Deleted: All managed AIScaleTargets are automatically deleted

Conflict Resolution

When reconciling a controller (e.g. Deployment, Rollout, etc.), Thoras checks whether an AIScaleTarget already exists that targets the controller via scaleTargetRef and applies the following conflictResolution policies:

skip (default): The existing AIScaleTarget is left unchanged. Changes made to the template will not be reflected in the conflicting AIScaleTarget.
override: The template takes ownership of the conflicting AIScaleTarget. The template’s configuration is applied to the AIScaleTarget.

Note: Thoras checks the thoras.ai/managed-by-template label to determine ownership. Only AIScaleTargets created by that specific template will be updated or deleted by it.

Opting a Workload Out

A workload that matches a template’s selector can still opt out of template management by setting an annotation on the workload itself (Deployment, StatefulSet, or Argo Rollout) — not on its pods:

metadata:
  annotations:
    thoras.ai/skip-template: "true"

The behavior depends on whether an AIScaleTarget already exists for the workload:

No AIScaleTarget exists yet: none is created while the annotation is present.
An AIScaleTarget already exists: it is left in place. The template stops updating it, but its history, model state, and suggestions are retained.

A retained AIScaleTarget is not paused — it keeps scaling using its last template-applied configuration. Opting out of a template stops the template from managing the AIScaleTarget; it does not stop autoscaling. See Stopping Autonomous Scaling if your goal is to halt scaling actions.

Removing the annotation (or setting it to "false") resumes template management on the next reconcile, updating any retained AIScaleTarget in place. The annotation value is parsed as a boolean: "true", "True", "TRUE", and "1" opt the workload out. "false", "0", an absent annotation, and empty or unparseable values are all treated as not opted out.

Namespace Filtering

Use policy.includedNamespaces or policy.excludedNamespaces to control which namespaces the template targets. The two fields are mutually exclusive, set one or the other, not both.

includedNamespaces (allowlist): only the listed namespaces are in scope. All other namespaces are ignored.
excludedNamespaces (denylist): all namespaces are in scope except those listed.

No namespaces are excluded by default. If you want to protect system namespaces, add them explicitly:

policy:
  excludedNamespaces:
    - kube-node-lease
    - kube-public
    - kube-system

Advanced Examples

Using matchExpressions

apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: microservices-tier
spec:
  selector:
    matchExpressions:
      - key: tier
        operator: In
        values:
          - frontend
          - backend
      - key: thoras.ai/enabled
        operator: Exists

  policy:
    excludedNamespaces:
      - kube-system
      - default
      - monitoring

  template:
    model:
      forecast_blocks: 4h
      forecast_buffer_percentage: 0%
      forecast_interval: 1h
      mode: balanced
    horizontal:
      mode: autonomous

Restricting to Specific Namespaces

Use includedNamespaces to target only the namespaces you want:

apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: production-only
spec:
  selector:
    matchLabels:
      thoras.ai/enabled: "true"
  policy:
    includedNamespaces:
      - production
      - staging
  template:
    model:
      forecast_blocks: 4h
      forecast_buffer_percentage: 0%
      forecast_interval: 1h
      mode: balanced
    vertical:
      mode: auto
      containers:
        - name: "*"
          cpu:
            lowerbound: "10m"
          memory:
            lowerbound: 0

Multiple Templates

You can create multiple templates with different selectors for different workload categories:

---
apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: critical-services
spec:
  selector:
    matchLabels:
      priority: critical
  template:
    model:
      forecast_blocks: 4h
      forecast_buffer_percentage: 20%
      forecast_interval: 1h
      mode: balanced
    vertical:
      mode: auto
      containers:
        - name: "*"
          cpu:
            lowerbound: "500m"
          memory:
            lowerbound: "1Gi"

---
apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: standard-services
spec:
  selector:
    matchExpressions:
      - key: priority
        operator: NotIn
        values:
          - critical
      - key: thoras.ai/enabled
        operator: Exists
  template:
    model:
      forecast_blocks: 4h
      forecast_buffer_percentage: 0%
      forecast_interval: 1h
      mode: balanced
    vertical:
      mode: auto
      containers:
        - name: "*"
          cpu:
            lowerbound: "0"
          memory:
            lowerbound: "0"

Status and Monitoring

ClusterAIScaleTemplate includes a status subresource with conditions to track reconciliation state:

status:
  conditions:
    - type: Ready
      status: "True"
      reason: ReconciliationSucceeded
      message: "Successfully reconciled 15 namespaces"
      lastTransitionTime: "2024-01-15T10:30:00Z"
      observedGeneration: 2

Condition Types

Type	Status	Reason	Description
Ready	True	ReconciliationSucceeded	Template successfully reconciled all namespaces
Ready	False	ReconciliationFailed	Failed to reconcile one or more namespaces
Ready	False	NamespaceListFailed	Failed to list namespaces

Check template status:

kubectl get clusteraiscaletemplate production-workloads -o yaml
# or using short name
kubectl get clusterast production-workloads -o yaml

View all managed AIScaleTargets:

kubectl get aiscaletarget -A -l thoras.ai/managed-by-template=production-workloads

Best Practices

Use Specific Selectors: Use precise label selectors to avoid unintended matches. Consider using a dedicated label like thoras.ai/enabled: "true" on workloads you want to scale.
Test in Non-Production First: Create a template in a development environment first to verify the selector matches the intended workloads.
Monitor Template Status: Regularly check the template’s status conditions to ensure successful reconciliation.
Namespace Filtering: No namespaces are excluded by default. Use policy.excludedNamespaces to block system namespaces (e.g., kube-system, kube-public, kube-node-lease) and any namespaces where automated scaling could cause issues. Use policy.includedNamespaces when you want to restrict the template to a known set of namespaces.
Template Naming: Use descriptive names that indicate the purpose or scope (e.g., production-frontend, staging-all-services).
Label Workloads Explicitly: Rather than relying on existing labels, add a specific label to workloads you want managed by templates (e.g., thoras.ai/template: "production-workloads").
Avoid Overlapping Templates: Ensure template selectors don’t overlap in ways that would cause multiple templates to match the same workloads.

Troubleshooting

Template Not Creating AIScaleTargets

Check the template status:

kubectl get clusterast <template-name> -o yaml

Verify workloads match the selector:

kubectl get deployments -A --show-labels

Check namespace filtering configuration:

kubectl get clusterast <template-name> -o jsonpath='{.spec.policy.includedNamespaces}'
kubectl get clusterast <template-name> -o jsonpath='{.spec.policy.excludedNamespaces}'

View operator logs:

kubectl logs -n thoras -l app=thoras-operator

Workload Matches the Selector But Has No AIScaleTarget

If a workload matches the template’s selector and namespace filtering, but no AIScaleTarget is created (or an existing one stops being updated), check for the opt-out annotation:

kubectl get deployment <name> -n <namespace> -o jsonpath='{.metadata.annotations.thoras\.ai/skip-template}'

If this returns true, the workload has opted out of template management. See Opting a Workload Out. Remove the annotation to resume management.

AIScaleTarget Not Updating When Template Changes

Ensure the AIScaleTarget has the thoras.ai/managed-by-template label:

kubectl get ast <ast-name> -n <namespace> -o jsonpath='{.metadata.labels}'

If the label is missing or points to a different template, the AIScaleTarget was not created by this template and won’t be updated by it.

Deleting Template But AIScaleTargets Remain

Check if the finalizer is stuck:

kubectl get clusterast <template-name> -o jsonpath='{.metadata.finalizers}'

View operator logs for errors during cleanup:

kubectl logs -n thoras -l app=thoras-operator | grep "cleaning up managed AIScaleTargets"

Migration from Manual AIScaleTargets

To migrate existing manually created AIScaleTargets to template management:

Create a ClusterAIScaleTemplate with the desired configuration
Add matching labels to your workloads
The template will skip existing AIScaleTargets (due to default skip conflict resolution)
Manually delete existing AIScaleTargets - they will be automatically recreated by the template
Verify new AIScaleTargets have the thoras.ai/managed-by-template label

Alternatively, manually add the label to existing AIScaleTargets:

kubectl label ast <ast-name> -n <namespace> thoras.ai/managed-by-template=<template-name>

Warning: Only do this if you’re certain the template configuration matches your existing AIScaleTarget configuration, as the template will overwrite the spec on the next reconciliation.

AIScaleTarget Definition - Complete reference for AIScaleTarget specifications
Configuring AIScaleTargets - Step-by-step guide for creating AIScaleTargets
Forecasting - Understanding how Thoras forecasts resource needs
Predictive Horizontal Pod Autoscaling - Guide to horizontal scaling configuration
Predictive Vertical Pod Rightsizing - Guide to vertical scaling configuration
Enabling Autonomous Mode - Prerequisites for autonomous scaling
Pod Annotations - The thoras.ai/managed family of annotations the webhook writes to mutated pods, distinct from the thoras.ai/managed-by-template label on this page
Stopping Autonomous Scaling - Options for halting Thoras autoscaling, from cluster-wide to per-workload

​Overview

​Use Cases

​Basic Example

​Specification

​ClusterAIScaleTemplateSpec

​AIScaleTemplateSpec

​TemplatePolicy

​Behavior

​Automatic AIScaleTarget Creation

​Lifecycle Management

​Conflict Resolution

​Opting a Workload Out

​Namespace Filtering

​Advanced Examples

​Using matchExpressions

​Restricting to Specific Namespaces

​Multiple Templates

​Status and Monitoring

​Condition Types

​Best Practices

​Troubleshooting

​Template Not Creating AIScaleTargets

​Workload Matches the Selector But Has No AIScaleTarget

​AIScaleTarget Not Updating When Template Changes

​Deleting Template But AIScaleTargets Remain

​Migration from Manual AIScaleTargets

​Related Documentation

Overview

Use Cases

Basic Example

Specification

ClusterAIScaleTemplateSpec

AIScaleTemplateSpec

TemplatePolicy

Behavior

Automatic AIScaleTarget Creation

Lifecycle Management

Conflict Resolution

Opting a Workload Out

Namespace Filtering

Advanced Examples

Using matchExpressions

Restricting to Specific Namespaces

Multiple Templates

Status and Monitoring

Condition Types

Best Practices

Troubleshooting

Template Not Creating AIScaleTargets

Workload Matches the Selector But Has No AIScaleTarget

AIScaleTarget Not Updating When Template Changes

Deleting Template But AIScaleTargets Remain

Migration from Manual AIScaleTargets

Related Documentation