ClusterAIScaleTemplate

​

Overview

​

Use Cases

• Apply AI-powered scaling to all workloads with a specific label (e.g., thoras.ai/enabled: "true")
• Standardize scaling policies across multiple microservices
• Automatically enable AI scaling for new workloads as they’re deployed
• Manage scaling configuration for entire application tiers or environments

​

Basic Example

apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: production-workloads
spec:
  # Match workloads with specific labels
  selector:
    matchLabels:
      env: production
      thoras.ai/enabled: "true"

  # Template to apply to matching workloads
  template:
    model:
      forecast_cron: "*/15 * * * *"
      forecast_blocks: "15m"
      forecast_buffer_percentage: "10%"

    vertical:
      mode: auto
      in_place_resizing:
        enabled: true
      containers:
        - name: "*"
          cpu:
            lowerbound: "10m"
          memory:
            lowerbound: 0

​

Specification

​

ClusterAIScaleTemplateSpec

​

AIScaleTemplateSpec

​

TemplatePolicy

​

Behavior

​

Automatic AIScaleTarget Creation

1. Thoras creates an AIScaleTarget with the name <workload-name>-ast
2. The AIScaleTarget is labeled with thoras.ai/managed-by-template: <template-name>
3. The AIScaleTarget has an owner reference to the workload (so it’s deleted when the workload is deleted)
4. The scaleTargetRef is automatically populated with the workload’s details

​

Lifecycle Management

• Workload Created: AIScaleTarget is automatically created
• Workload Updated: If labels change and no longer match, the AIScaleTarget is deleted
• Workload Deleted: AIScaleTarget is automatically deleted via owner reference
• Template Updated: All managed AIScaleTargets are updated to match the new template
• Template Deleted: All managed AIScaleTargets are automatically deleted

​

Conflict Resolution

• skip (default): The template will not modify the existing AIScaleTarget. This prevents overriding manually created or differently managed AIScaleTargets.
• override: Currently not implemented. The template will skip AIScaleTargets not managed by itself.

​

Namespace Exclusions

• kube-node-lease
• kube-public
• kube-system

​

Advanced Examples

​

Using matchExpressions

apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: microservices-tier
spec:
  selector:
    matchExpressions:
      - key: tier
        operator: In
        values:
          - frontend
          - backend
      - key: thoras.ai/enabled
        operator: Exists

  policy:
    excludedNamespaces:
      - kube-system
      - default
      - monitoring

  template:
    model:
      forecast_blocks: 15m
      forecast_buffer_percentage: 0%
      forecast_cron: "*/15 * * * *"
    horizontal:
      mode: autonomous

​

Multiple Templates

---
apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: critical-services
spec:
  selector:
    matchLabels:
      priority: critical
  template:
    model:
      forecast_blocks: 4h
      forecast_buffer_percentage: 20%
      forecast_cron: "0 0,6,12,18 * * *"
    vertical:
      mode: auto
      containers:
        - name: "*"
          cpu:
            lowerbound: "500m"
          memory:
            lowerbound: "1Gi"

---
apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: standard-services
spec:
  selector:
    matchExpressions:
      - key: priority
        operator: NotIn
        values:
          - critical
      - key: thoras.ai/enabled
        operator: Exists
  template:
    model:
      forecast_blocks: 15m
      forecast_buffer_percentage: 0%
      forecast_cron: "*/15 * * * *"
    vertical:
      mode: auto
      containers:
        - name: "*"
          cpu:
            lowerbound: "0"
          memory:
            lowerbound: "0"

​

Status and Monitoring

status:
  conditions:
    - type: Ready
      status: "True"
      reason: ReconciliationSucceeded
      message: "Successfully reconciled 15 namespaces"
      lastTransitionTime: "2024-01-15T10:30:00Z"
      observedGeneration: 2

​

Condition Types

kubectl get clusteraiscaletemplate production-workloads -o yaml
# or using short name
kubectl get cast production-workloads -o yaml

kubectl get aiscaletarget -A -l thoras.ai/managed-by-template=production-workloads

​

Best Practices

1. Use Specific Selectors: Use precise label selectors to avoid unintended matches. Consider using a dedicated label like thoras.ai/enabled: "true" on workloads you want to scale.
2. Test in Non-Production First: Create a template in a development environment first to verify the selector matches the intended workloads.
3. Monitor Template Status: Regularly check the template’s status conditions to ensure successful reconciliation.
4. Namespace Exclusions: Always exclude system namespaces and any namespaces where automated scaling could cause issues.
5. Template Naming: Use descriptive names that indicate the purpose or scope (e.g., production-frontend, staging-all-services).
6. Label Workloads Explicitly: Rather than relying on existing labels, add a specific label to workloads you want managed by templates (e.g., thoras.ai/template: "production-workloads").
7. Avoid Overlapping Templates: Ensure template selectors don’t overlap in ways that would cause multiple templates to match the same workloads.

​

Troubleshooting

​

Template Not Creating AIScaleTargets

1. Check the template status:
   Copy

   kubectl get cast <template-name> -o yaml
   
2. Verify workloads match the selector:
   Copy

   kubectl get deployments -A --show-labels
   
3. Check if namespace is excluded:
   Copy

   kubectl get cast <template-name> -o jsonpath='{.spec.policy.excludedNamespaces}'
   
4. View operator logs:
   Copy

   kubectl logs -n thoras -l app=thoras-operator
   

​

AIScaleTarget Not Updating When Template Changes

kubectl get ast <ast-name> -n <namespace> -o jsonpath='{.metadata.labels}'

​

Deleting Template But AIScaleTargets Remain

kubectl get cast <template-name> -o jsonpath='{.metadata.finalizers}'

kubectl logs -n thoras -l app=thoras-operator | grep "cleaning up managed AIScaleTargets"

​

Migration from Manual AIScaleTargets

1. Create a ClusterAIScaleTemplate with the desired configuration
2. Add matching labels to your workloads
3. The template will skip existing AIScaleTargets (due to default skip conflict resolution)
4. Manually delete existing AIScaleTargets - they will be automatically recreated by the template
5. Verify new AIScaleTargets have the thoras.ai/managed-by-template label

kubectl label ast <ast-name> -n <namespace> thoras.ai/managed-by-template=<template-name>

​

Related Documentation

• AIScaleTarget Definition - Complete reference for AIScaleTarget specifications
• Configuring AIScaleTargets - Step-by-step guide for creating AIScaleTargets
• Forecasting - Understanding how Thoras forecasts resource needs
• Predictive Horizontal Pod Autoscaling - Guide to horizontal scaling configuration
• Predictive Vertical Pod Rightsizing - Guide to vertical scaling configuration
• Enabling Autonomous Mode - Prerequisites for autonomous scaling