Rollout Strategy

Rolling updates represent a critical operational strategy for online services aiming to achieve zero downtime. In the context of LLM inference services, the implementation of rolling updates is important to reduce the risk of service unavailability.

ModelServing supports rolling updates at either the ServingGroup or Role level. Select exactly one granularity with spec.rolloutStrategy.type:

• ServingGroupRollingUpdate uses spec.rolloutStrategy.rollingUpdateConfiguration. Its maxUnavailable limits unavailable ServingGroups, and its partition protects existing ServingGroups from updates.
• RoleRollingUpdate uses the inline maxUnavailable and partition fields on each entry in spec.template.roles. The settings are applied independently to each Role in every ServingGroup. The ModelServing-level rollingUpdateConfiguration must not be set and does not participate in the Role availability budget.

Configure rolling update settings only at the selected granularity. maxUnavailable accepts an absolute number or a percentage and defaults to 1. A percentage is calculated from spec.replicas for ServingGroupRollingUpdate and from the corresponding Role's replicas for RoleRollingUpdate.

partition protects the first N existing replicas in ascending ordinal order. The remaining replicas are eligible for rolling update. With a normal contiguous ordinal set, this is equivalent to protecting ordinals in [0, partition). Defining protection by list position also gives deterministic behavior for legacy or temporarily non-contiguous sets.

ServingGroup rolling update

When a protected replica is missing and must be recreated, the ordinal itself selects the template: a missing ordinal below partition uses CurrentRevision and its historical template, while other missing ordinals use UpdateRevision and the current template. Before creating a ServingGroup that references a new revision, the controller must successfully persist its ControllerRevision; otherwise reconciliation stops and retries without creating a partial ServingGroup.

Here's a ModelServing configured with rollout strategy:

spec:
  rolloutStrategy:
    type: ServingGroupRollingUpdate
    rollingUpdateConfiguration:
      maxUnavailable: 1
      partition: 0

In the following we'll show how rolling update processes for a ModelServing with four replicas. Three Replica status are simulated here:

• ✅ Replica has been updated
• ❎ Replica hasn't been updated
• ⏳ Replica is in rolling update

	R-0	R-1	R-2	R-3	Note
Stage1	✅	✅	✅	✅	Before rolling update
Stage2	❎	❎	❎	⏳	Rolling update started; R-3 is selected first in this example
Stage3	❎	❎	⏳	✅	R-3 is updated. The next replica (R-2) is now being updated
Stage4	❎	⏳	✅	✅	R-2 is updated. The next replica (R-1) is now being updated
Stage5	⏳	✅	✅	✅	R-1 is updated. The last replica (R-0) is now being updated
Stage6	✅	✅	✅	✅	Update completed. All replicas are on the new version

During a rolling upgrade, the controller selects an eligible outdated replica while respecting partition and availability constraints, then deletes and rebuilds it. Unhealthy outdated replicas are prioritized; ordinal order is used within the applicable candidate ordering. The controller does not proceed beyond the availability budget until replacement capacity is ready.

Role rolling update

Use RoleRollingUpdate when only the changed Roles should be recreated instead of rebuilding an entire ServingGroup. Configure the availability budget and partition directly on each Role:

spec:
  rolloutStrategy:
    type: RoleRollingUpdate
  template:
    roles:
      - name: prefill
        replicas: 4
        maxUnavailable: 1
        partition: 0
        # entryTemplate and other Role fields are omitted
      - name: decode
        replicas: 2
        maxUnavailable: 1
        partition: 0
        # entryTemplate and other Role fields are omitted

Kthena evaluates Role updates across all ServingGroups. Because each ServingGroup applies the per-Role availability budget independently, RoleRollingUpdate is recommended for a ModelServing with a single ServingGroup.

If recoveryPolicy is ServingGroupRecreate, deleting an outdated Role triggers recreation of its entire ServingGroup, which removes the resource-saving benefit of RoleRollingUpdate. Use RoleRecreate when only the outdated Role should be rebuilt.