Skip to main content
Thoras can optimize ephemeral workloads that share the same deployment manifest across dynamic namespaces. When you designate a group of related workloads, Thoras begins producing recommendations within the first hour, with no historical training period required. This is useful for environments where identical microservices are deployed into short-lived namespaces (e.g. per-pull-request previews, per-branch test environments, or CI namespaces). These workloads typically run at their deploy-time resource requests for their entire lifetime because they are torn down before the standard forecasting pipeline has enough data. Grouped workloads get recommendations much sooner.

Prerequisites

The grouped workloads feature requires a minimum chart version of 5.14.0 and must be enabled in your Helm values:
values.yaml

Configuration

Create a ClusterAIScaleTemplate that targets your ephemeral namespaces. Because these namespaces are created and destroyed dynamically, use excludedNamespaces to list your persistent namespaces and let everything else match:
ephemeral-workloads.yaml
Apply it:
Thoras creates an AIScaleTarget for every Deployment, StatefulSet, and Argo Rollout in the matching namespaces. When a new namespace is created, Thoras picks it up on the next reconciliation cycle (typically within 30 seconds).

Key settings

A string that enables grouped forecasting for the workloads created by this template. Each AIScaleTarget still receives its own independent recommendation based on its own metrics. A single group name for all your ephemeral workloads is recommended.
How long Thoras must observe a workload before applying recommendations in autonomous mode. The default is 24h, which is longer than most ephemeral workloads live. Set this to 60m so recommendations become actionable within the first hour.See model.min_lookback_to_scale for details.
Adds a fixed percentage on top of the forecasted recommendation. Because ephemeral workloads have a shorter observation window, a 10% buffer provides additional headroom while the forecast has less data to work with.See model.forecast_buffer_percentage for details.
The vertical section is required. Without it, Thoras will not generate forecasts for the grouped workloads. Include at least a vertical spec with mode, as shown in the example above.

Namespace strategy

excludedNamespaces works best when your ephemeral namespaces are dynamic and unpredictable. List your persistent namespaces (the ones that should not use this template), and every other namespace is in scope automatically. If your ephemeral namespaces are known ahead of time, you can use includedNamespaces instead. See Namespace Filtering for details.

Existing templates

Each AIScaleTarget is owned by one template at a time. If you already have a ClusterAIScaleTemplate whose scope includes your ephemeral namespaces, you need to make the scopes disjoint before the new template can manage those workloads. First, update the existing template’s scope so it no longer covers ephemeral namespaces (e.g. switch it to includedNamespaces listing only your persistent namespaces). Then apply the new ephemeral template. For workloads in existing ephemeral namespaces, you have two options:
  • Immediate: Delete the existing AIScaleTargets so the new template can recreate them with the grouped configuration. The operator processes finalizers through its normal reconcile loop, so this completes in minutes.
    Use -n <namespace> instead of -A if the existing template also manages persistent workloads you want to keep.
  • Gradual: Leave them in place. Existing ephemeral namespaces continue with their current configuration until they are torn down. New namespaces are claimed by the ephemeral template automatically.

What to expect

After applying the template: When an ephemeral namespace is torn down and later recreated with the same workloads, Thoras starts fresh. New AIScaleTargets are created automatically and the recommendation cycle begins again from the first minute.

Validating

After applying the template, verify everything is working: 1. Check that AIScaleTargets were created:
2. Verify the configuration propagated correctly:
You should see group, min_lookback_to_scale, and forecast_buffer_percentage matching your template. 3. Check for a recommendation (after 15 minutes):
4. Verify hasEnoughDataToScale (after 1 hour):
This should return true once the workload has been running for at least 1 hour (with min_lookback_to_scale: 60m).

Transitioning to autonomous mode

Start with mode: observation to review recommendations before enabling autonomous scaling. Once you are confident the recommendations are appropriate:
  1. Update the template to mode: auto
  2. All managed AIScaleTargets will be updated on the next reconciliation
  3. Thoras will begin applying recommendations automatically for workloads with hasEnoughDataToScale: true
See Enabling Autonomous Mode for prerequisites and considerations.