Skip to main content

​
Overview

ClusterAIScaleTemplate watches for Deployments, StatefulSets, and Argo Rollouts that match a label selector, and automatically creates corresponding AIScaleTargets with a predefined configuration. When workloads are added, updated, or removed, Thoras automatically manages the lifecycle of the associated AIScaleTargets.

​
Use Cases

  • Apply AI-powered scaling to all workloads with a specific label (e.g., thoras.ai/enabled: "true")
  • Standardize scaling policies across multiple microservices
  • Automatically enable AI scaling for new workloads as they’re deployed
  • Manage scaling configuration for entire application tiers or environments

​
Basic Example

apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: production-workloads
spec:
  # Match workloads with specific labels
  selector:
    matchLabels:
      env: production
      thoras.ai/enabled: "true"

  # Template to apply to matching workloads
  template:
    model:
      forecast_interval: 15m
      forecast_blocks: 15m
      forecast_buffer_percentage: "10%"
      mode: balanced

    vertical:
      mode: auto
      update_policy:
        update_mode: in_place_or_recreate
      containers:
        - name: "*"
          cpu:
            lowerbound: "10m"
          memory:
            lowerbound: 0
This template will automatically create AIScaleTargets for all Deployments, StatefulSets, and Argo Rollouts in any namespace that have the labels env: production and thoras.ai/enabled: "true".

​
Specification

​
ClusterAIScaleTemplateSpec

FieldTypeRequiredDescription
selectormetav1.LabelSelectorYesMatches workloads to apply this template to. Supports matchLabels and matchExpressions.
templateAIScaleTemplateSpecYesDefines the AIScaleTarget configuration to apply.
policyTemplatePolicyNoControls template application behavior.

​
AIScaleTemplateSpec

The template spec contains the same fields as AIScaleTarget spec, minus the scaleTargetRef (which is automatically populated based on the matched workload):
FieldTypeRequiredDescription
modelModelSpecYesModel configuration (mode, forecast settings, buffer percentage).
verticalVerticalSpecNoVertical scaling configuration.
horizontalHorizontalSpecNoHorizontal scaling configuration.

​
TemplatePolicy

FieldTypeRequiredDefaultDescription
conflictResolutionstringNoskipHow to handle conflicts with existing AIScaleTargets. Options: skip, override.
includedNamespaces[]stringNo—Allowlist of namespaces to target. Mutually exclusive with excludedNamespaces.
excludedNamespaces[]stringNo—Denylist of namespaces to skip. Mutually exclusive with includedNamespaces.

​
Behavior

​
Automatic AIScaleTarget Creation

When a workload (Deployment, StatefulSet, or Argo Rollout) matches the template’s selector:
  1. Thoras creates an AIScaleTarget with the name <workload-name>-ast
  2. The AIScaleTarget is labeled with thoras.ai/managed-by-template: <template-name>
  3. The AIScaleTarget has an owner reference to the workload (so it’s deleted when the workload is deleted)
  4. The scaleTargetRef is automatically populated with the workload’s details

​
Lifecycle Management

  • Workload Created: AIScaleTarget is automatically created
  • Workload Updated: If labels change and no longer match, the AIScaleTarget is deleted
  • Workload Deleted: AIScaleTarget is automatically deleted via owner reference
  • Template Updated: All managed AIScaleTargets are updated to match the new template
  • Template Deleted: All managed AIScaleTargets are automatically deleted

​
Conflict Resolution

If an AIScaleTarget already exists with the same name:
  • skip (default): The template will not modify the existing AIScaleTarget. This prevents overriding manually created or differently managed AIScaleTargets.
  • override: Currently not implemented. The template will skip AIScaleTargets not managed by itself.
Note: Thoras checks the thoras.ai/managed-by-template label to determine ownership. Only AIScaleTargets created by that specific template will be updated or deleted by it.

​
Namespace Filtering

Use policy.includedNamespaces or policy.excludedNamespaces to control which namespaces the template targets. The two fields are mutually exclusive — set one or the other, not both.
  • includedNamespaces (allowlist): only the listed namespaces are in scope. All other namespaces are ignored.
  • excludedNamespaces (denylist): all namespaces are in scope except those listed.
No namespaces are excluded by default. If you want to protect system namespaces, add them explicitly: 
policy:
  excludedNamespaces:
    - kube-node-lease
    - kube-public
    - kube-system

​
Advanced Examples

​
Using matchExpressions

apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: microservices-tier
spec:
  selector:
    matchExpressions:
      - key: tier
        operator: In
        values:
          - frontend
          - backend
      - key: thoras.ai/enabled
        operator: Exists

  policy:
    excludedNamespaces:
      - kube-system
      - default
      - monitoring

  template:
    model:
      forecast_blocks: 15m
      forecast_buffer_percentage: 0%
      forecast_interval: 15m
      mode: balanced
    horizontal:
      mode: autonomous

​
Restricting to Specific Namespaces

Use includedNamespaces to target only the namespaces you want: 
apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: production-only
spec:
  selector:
    matchLabels:
      thoras.ai/enabled: "true"
  policy:
    includedNamespaces:
      - production
      - staging
  template:
    model:
      forecast_blocks: 15m
      forecast_buffer_percentage: 0%
      forecast_interval: 15m
      mode: balanced
    vertical:
      mode: auto
      containers:
        - name: "*"
          cpu:
            lowerbound: "10m"
          memory:
            lowerbound: 0

​
Multiple Templates

You can create multiple templates with different selectors for different workload categories: 
---
apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: critical-services
spec:
  selector:
    matchLabels:
      priority: critical
  template:
    model:
      forecast_blocks: 4h
      forecast_buffer_percentage: 20%
      forecast_cron: "0 0,6,12,18 * * *"
      mode: balanced
    vertical:
      mode: auto
      containers:
        - name: "*"
          cpu:
            lowerbound: "500m"
          memory:
            lowerbound: "1Gi"

---
apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: standard-services
spec:
  selector:
    matchExpressions:
      - key: priority
        operator: NotIn
        values:
          - critical
      - key: thoras.ai/enabled
        operator: Exists
  template:
    model:
      forecast_blocks: 15m
      forecast_buffer_percentage: 0%
      forecast_interval: 15m
      mode: balanced
    vertical:
      mode: auto
      containers:
        - name: "*"
          cpu:
            lowerbound: "0"
          memory:
            lowerbound: "0"

​
Status and Monitoring

ClusterAIScaleTemplate includes a status subresource with conditions to track reconciliation state: 
status:
  conditions:
    - type: Ready
      status: "True"
      reason: ReconciliationSucceeded
      message: "Successfully reconciled 15 namespaces"
      lastTransitionTime: "2024-01-15T10:30:00Z"
      observedGeneration: 2

​
Condition Types

TypeStatusReasonDescription
ReadyTrueReconciliationSucceededTemplate successfully reconciled all namespaces
ReadyFalseReconciliationFailedFailed to reconcile one or more namespaces
ReadyFalseNamespaceListFailedFailed to list namespaces
Check template status: 
kubectl get clusteraiscaletemplate production-workloads -o yaml
# or using short name
kubectl get clusterast production-workloads -o yaml
View all managed AIScaleTargets: 
kubectl get aiscaletarget -A -l thoras.ai/managed-by-template=production-workloads

​
Best Practices

  1. Use Specific Selectors: Use precise label selectors to avoid unintended matches. Consider using a dedicated label like thoras.ai/enabled: "true" on workloads you want to scale.
  2. Test in Non-Production First: Create a template in a development environment first to verify the selector matches the intended workloads.
  3. Monitor Template Status: Regularly check the template’s status conditions to ensure successful reconciliation.
  4. Namespace Filtering: No namespaces are excluded by default. Use policy.excludedNamespaces to block system namespaces (e.g., kube-system, kube-public, kube-node-lease) and any namespaces where automated scaling could cause issues. Use policy.includedNamespaces when you want to restrict the template to a known set of namespaces.
  5. Template Naming: Use descriptive names that indicate the purpose or scope (e.g., production-frontend, staging-all-services).
  6. Label Workloads Explicitly: Rather than relying on existing labels, add a specific label to workloads you want managed by templates (e.g., thoras.ai/template: "production-workloads").
  7. Avoid Overlapping Templates: Ensure template selectors don’t overlap in ways that would cause multiple templates to match the same workloads.

​
Troubleshooting

​
Template Not Creating AIScaleTargets

  1. Check the template status: 
    kubectl get clusterast <template-name> -o yaml
  2. Verify workloads match the selector: 
    kubectl get deployments -A --show-labels
  3. Check namespace filtering configuration: 
    kubectl get clusterast <template-name> -o jsonpath='{.spec.policy.includedNamespaces}'
kubectl get clusterast <template-name> -o jsonpath='{.spec.policy.excludedNamespaces}'
  4. View operator logs: 
    kubectl logs -n thoras -l app=thoras-operator

​
AIScaleTarget Not Updating When Template Changes

Ensure the AIScaleTarget has the thoras.ai/managed-by-template label: 
kubectl get ast <ast-name> -n <namespace> -o jsonpath='{.metadata.labels}'
If the label is missing or points to a different template, the AIScaleTarget was not created by this template and won’t be updated by it.

​
Deleting Template But AIScaleTargets Remain

Check if the finalizer is stuck: 
kubectl get clusterast <template-name> -o jsonpath='{.metadata.finalizers}'
View operator logs for errors during cleanup: 
kubectl logs -n thoras -l app=thoras-operator | grep "cleaning up managed AIScaleTargets"

​
Migration from Manual AIScaleTargets

To migrate existing manually created AIScaleTargets to template management:
  1. Create a ClusterAIScaleTemplate with the desired configuration
  2. Add matching labels to your workloads
  3. The template will skip existing AIScaleTargets (due to default skip conflict resolution)
  4. Manually delete existing AIScaleTargets - they will be automatically recreated by the template
  5. Verify new AIScaleTargets have the thoras.ai/managed-by-template label
Alternatively, manually add the label to existing AIScaleTargets: 
kubectl label ast <ast-name> -n <namespace> thoras.ai/managed-by-template=<template-name>
Warning: Only do this if you’re certain the template configuration matches your existing AIScaleTarget configuration, as the template will overwrite the spec on the next reconciliation.