> ## Documentation Index
> Fetch the complete documentation index at: https://docs.thoras.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Ephemeral Workloads

> Configure Thoras to optimize short-lived workloads that share the same deployment manifest across dynamic namespaces, such as per-pull-request preview environments.

Thoras can optimize ephemeral workloads that share the same deployment manifest
across dynamic namespaces. When you designate a group of related workloads,
Thoras begins producing recommendations within the first hour, with no
historical training period required.

This is useful for environments where identical microservices are deployed into
short-lived namespaces (e.g. per-pull-request previews, per-branch test
environments, or CI namespaces). These workloads typically run at their
deploy-time resource requests for their entire lifetime because they are torn
down before the standard forecasting pipeline has enough data. Grouped workloads
get recommendations much sooner.

## Prerequisites

The grouped workloads feature requires a minimum chart version of **5.14.0** and
must be enabled in your Helm values:

```yaml values.yaml theme={null}
featureFlags:
  enableModelGroups: true
```

## Configuration

Create a [ClusterAIScaleTemplate](https://docs.thoras.ai/reference/cluster-aiscale-template.md) that
targets your ephemeral namespaces. Because these namespaces are created and
destroyed dynamically, use `excludedNamespaces` to list your persistent
namespaces and let everything else match:

```yaml ephemeral-workloads.yaml theme={null}
apiVersion: thoras.ai/v1
kind: ClusterAIScaleTemplate
metadata:
  name: ephemeral-workloads
spec:
  policy:
    excludedNamespaces:
      - production
      - staging
      - monitoring
      - kube-system
      - kube-public
      - kube-node-lease
      # Add all persistent namespaces here
    conflictResolution: override
  selector: {}
  template:
    model:
      group: "ephemeral" # All ephemeral workloads can share one group name
      min_lookback_to_scale: "60m"
      forecast_buffer_percentage: "10%"
    vertical:
      containers:
        - name: "*"
          cpu:
            lowerbound: "10m"
          memory:
            lowerbound: "32Mi"
      mode: observation
```

Apply it:

```sh theme={null}
kubectl apply -f ephemeral-workloads.yaml
```

Thoras creates an AIScaleTarget for every Deployment, StatefulSet, and Argo
Rollout in the matching namespaces. When a new namespace is created, Thoras
picks it up on the next reconciliation cycle (typically within 30 seconds).

### Key settings

> [accordion] model.group
>
> A string that enables grouped forecasting for the workloads created by
> this template. Each AIScaleTarget still receives its own independent
> recommendation based on its own metrics. A single group name for all
> your ephemeral workloads is recommended.

> [accordion] min\_lookback\_to\_scale
>
> How long Thoras must observe a workload before applying recommendations
> in autonomous mode. The default is `24h`, which is longer than most
> ephemeral workloads live. Set this to `60m` so recommendations become
> actionable within the first hour.
>
> See [`model.min_lookback_to_scale`](https://docs.thoras.ai/reference/ast-definition#modelmin_lookback_to_scale)
> for details.

> [accordion] forecast\_buffer\_percentage
>
> Adds a fixed percentage on top of the forecasted recommendation.
> Because ephemeral workloads have a shorter observation window, a `10%`
> buffer provides additional headroom while the forecast has less data to
> work with.
>
> See [`model.forecast_buffer_percentage`](https://docs.thoras.ai/reference/ast-definition#model-forecast-buffer-percentage)
> for details.

> [!WARNING]
>
> The `vertical` section is required. Without it, Thoras will not generate
> forecasts for the grouped workloads. Include at least a vertical spec with
> `mode`, as shown in the example above.

### Namespace strategy

`excludedNamespaces` works best when your ephemeral namespaces are dynamic and
unpredictable. List your persistent namespaces (the ones that should not use
this template), and every other namespace is in scope automatically.

If your ephemeral namespaces are known ahead of time, you can use
`includedNamespaces` instead. See
[Namespace Filtering](https://docs.thoras.ai/reference/cluster-aiscale-template.md#namespace-filtering)
for details.

### Existing templates

Each AIScaleTarget is owned by one template at a time. If you already have a
ClusterAIScaleTemplate whose scope includes your ephemeral namespaces, you need
to make the scopes disjoint before the new template can manage those workloads.

First, update the existing template's scope so it no longer covers ephemeral
namespaces (e.g. switch it to `includedNamespaces` listing only your persistent
namespaces). Then apply the new ephemeral template.

For workloads in **existing** ephemeral namespaces, you have two options:

* **Immediate**: Delete the existing AIScaleTargets so the new template can
  recreate them with the grouped configuration. The operator processes
  finalizers through its normal reconcile loop, so this completes in minutes.

  ```sh theme={null}
  # Delete across all namespaces
  kubectl delete ast -A -l thoras.ai/managed-by-template=<existing-template>

  # Or target a specific namespace (replace -A with -n)
  kubectl delete ast -n <namespace> -l thoras.ai/managed-by-template=<existing-template>
  ```

  Use `-n <namespace>` instead of `-A` if the existing template also manages
  persistent workloads you want to keep.

* **Gradual**: Leave them in place. Existing ephemeral namespaces continue with
  their current configuration until they are torn down. New namespaces are
  claimed by the ephemeral template automatically.

## What to expect

After applying the template:

| Time | What happens |
| - | - |
| **0-1 min** | AIScaleTargets created, metric collection begins |
| **First forecast cycle** (up to 15 min) | Initial recommendation produced |
| **1 hour** | Recommendations become more precise as more data is available |
| **1 hour** (with `min_lookback_to_scale: 60m`) | `hasEnoughDataToScale` becomes true, enabling autonomous scaling |

When an ephemeral namespace is torn down and later recreated with the same
workloads, Thoras starts fresh. New AIScaleTargets are created automatically and
the recommendation cycle begins again from the first minute.

## Validating

After applying the template, verify everything is working:

**1. Check that AIScaleTargets were created:**

```sh theme={null}
kubectl get ast -A -l thoras.ai/managed-by-template=ephemeral-workloads
```

**2. Verify the configuration propagated correctly:**

```sh theme={null}
kubectl get ast <ast-name> -n <namespace> -o jsonpath='{.spec.model}'
```

You should see `group`, `min_lookback_to_scale`, and
`forecast_buffer_percentage` matching your template.

**3. Check for a recommendation (after 15 minutes):**

```sh theme={null}
kubectl get ast <ast-name> -n <namespace> -o jsonpath='{.status.verticalSuggestion}'
```

**4. Verify `hasEnoughDataToScale` (after 1 hour):**

```sh theme={null}
kubectl get ast <ast-name> -n <namespace> \
  -o jsonpath='{.status.verticalSuggestion.hasEnoughDataToScale}'
```

This should return `true` once the workload has been running for at least 1 hour
(with `min_lookback_to_scale: 60m`).

## Transitioning to autonomous mode

Start with `mode: observation` to review recommendations before enabling
autonomous scaling. Once you are confident the recommendations are appropriate:

1. Update the template to `mode: auto`
2. All managed AIScaleTargets will be updated on the next reconciliation
3. Thoras will begin applying recommendations automatically for workloads with
   `hasEnoughDataToScale: true`

See [Enabling Autonomous Mode](https://docs.thoras.ai/guides/enabling-autonomous-mode.md) for
prerequisites and considerations.

## Related

* [ClusterAIScaleTemplate](https://docs.thoras.ai/reference/cluster-aiscale-template.md): full reference
  for template configuration
* [Choosing a Forecast Optimization Strategy](https://docs.thoras.ai/guides/forecast-optimization-strategies.md):
  mode selection and tuning
* [Enabling Autonomous Mode](https://docs.thoras.ai/guides/enabling-autonomous-mode.md): prerequisites
  for autonomous scaling
* [Configuring AIScaleTargets](https://docs.thoras.ai/guides/configuring-asts.md): manual AST
  configuration


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.