When the AI provider is unavailable: design a fallback the business can use

A customer can still submit a support request when an AI summary is unavailable. They may not be able to complete the same request if the application makes that summary a mandatory step before saving the ticket.

That difference is an architectural choice. An AI provider outage does not have to become a full application outage.

A useful fallback plan classifies the work. Optional enrichment can be skipped, background tasks can queue and time-sensitive interactions can follow a deterministic or manual path. A second provider is appropriate only when it has been tested for the same task and is permitted to receive the same data.

Identify the real service dependency

List where the model sits in each user journey. Is it needed to accept input, validate a record, prepare a draft or perform an external action?

The point of failure matters more than the presence of an AI feature. A model used after a ticket is durably saved creates a different continuity risk from a model that must respond before the ticket can exist.

Separate the core transaction from optional analysis wherever the business permits it. Save the customer's request first, then enrich it asynchronously. Preserve a clear status so staff know which enrichment is missing.

Avoid pretending enrichment completed

If an AI field is unavailable, leave it explicitly pending or unavailable. Do not fill it with an old answer or a generic guess merely to keep the interface looking complete.

The user should be able to distinguish the original record from the generated interpretation. That distinction becomes especially valuable during degraded operation.

Define deadlines at the workflow level

A provider call needs a timeout, but the business task also needs a deadline. Several individually reasonable retries can exceed the time a user is willing to wait.

Give the workflow a total time budget and allocate it across retrieval, model processing and any fallback. When the remaining budget is insufficient, choose the defined degraded path.

For a background report, the deadline may be the review meeting rather than a few seconds. For a live agent-assist panel, a late answer can be less useful than a clear unavailable state.

Do not use the same timeout policy for every feature simply because they share one SDK.

Classify failure before retrying

A transient connectivity problem differs from an invalid request, revoked credential or policy refusal. Repeating a permanently invalid request adds cost and load without improving the result.

Use the provider's documented error semantics and the application's context. Bound retries, space them appropriately and avoid several layers of the stack independently retrying the same request.

A request that may have initiated a business action needs additional care. A timeout does not prove that the downstream action failed. Reconcile its outcome before deciding whether to repeat it.

Choose a fallback for each task class

The right alternative depends on the outcome, not on a preference to keep using AI.

A deterministic fallback can be a normal product feature. It is not necessarily a failure of the AI strategy. It keeps the business process available while a dependency is impaired.

Use a circuit breaker to stop unproductive pressure

When a dependency is failing repeatedly, continuing to send every request can increase queueing and consume the application's own resources. A circuit-breaker policy temporarily limits calls and lets the system use its degraded path.

Define how the circuit opens, how recovery is tested and which requests are allowed through during recovery. Keep the state visible to operators.

Do not let the breaker become a permanent unexplained switch. Record why it opened and provide a controlled way to test recovery. A provider status page can help investigation, but it does not establish that your account, region or particular endpoint is healthy.

Preserve capacity for essential work

Not every task deserves equal priority during an outage. A background enrichment job should not exhaust workers needed to save customer requests or process a manual queue.

Separate queues or concurrency limits can protect important paths. Choose the mechanism according to the application, but make the priority policy explicit.

A second provider must preserve the contract

Before enabling cross-provider failover, confirm that the alternative can meet the output schema, evidence requirements, tool behaviour and acceptable quality for the task.

Also confirm that the alternative is allowed to receive the data. A fallback must not bypass a processing-region restriction or send confidential records to an unapproved service.

Store the actual provider and configuration used for the completed job. Reviewers and support staff need to know when the normal route changed.

Do not call an untested route a backup

A provider configured with an old prompt and an expired credential is only a possibility. Exercise the fallback with current representative inputs and operational conditions.

Verify that its output is still usable by the application. Differences in tool calling, refusal behaviour and event handling can matter even when both providers accept text input.

Keep business state outside the provider session

The application should know which task was requested, which steps completed and which actions remain pending. A provider session identifier can be recorded as supporting detail, but it should not be the only durable record.

This makes fallback and recovery possible without asking a new model to infer the workflow's history. Pass the original task and permitted evidence to the alternative route, together with confirmed completed operations.

Do not present a failed model's draft as authoritative source material for the fallback. Preserve the difference between retrieved facts and generated interpretation.

Recover queued work without creating a second incident

When service returns, releasing every queued job at once can overwhelm the provider, downstream systems or human reviewers. Drain the queue at a controlled rate.

Check whether old jobs are still relevant. A customer case may have been resolved manually while the summary was pending. A report may no longer need generation after its review deadline passed.

Use expiry and cancellation rules. A stale job should not send a delayed message or overwrite a record simply because the model service is healthy again.

Reconcile manual work

Operators need a way to mark a task completed through the manual path. The recovery worker should respect that state and avoid duplicating the result.

Keep the evidence of manual completion. It helps explain why an automated job was skipped and allows reporting to distinguish degraded operation from lost work.

Communicate the effect in business terms

An operator needs to know whether customers can still submit requests, whether draft preparation is delayed and whether any external actions are uncertain. “Model API errors increased” is useful technical information but an incomplete operational update.

Provide the affected feature, start time, current fallback and next review point. Avoid claiming a recovery time that the team cannot substantiate.

For users, keep the message specific and actionable. Offer the available manual path or explain that the request is saved and processing will continue. Reassure through accurate status, not by hiding the failure.

Run an outage exercise

In a test environment, make the model endpoint unavailable and follow a representative user journey. Verify that the core record is preserved and that the interface does not claim success for an unfinished action.

Then introduce a partial failure: retrieval works, generation times out and the retry completes late. Check cancellation and duplicate handling.

Finally, restore the dependency with a backlog present. Observe queue drainage, stale-job handling and review capacity. The exercise should end with evidence that normal operation resumed without replaying work already completed manually.

A hypothetical service portal

A portal uses AI to draft responses after new support cases are saved. During an outage, the portal continues accepting cases and assigns them to the normal team.

The AI drafting queue pauses. Staff can write responses through the established interface. When the provider recovers, the worker skips closed cases and cases with an approved manual response.

The portal records which drafts were delayed and which cases used the manual path. This lets the service owner assess the actual effect of the outage without treating every unavailable draft as a failed customer interaction.

Rehearse an outage with ambiguous work in flight

A provider outage is easiest to handle when nothing has started. Real incidents often occur after the application sends a request but before it receives a conclusive result. The task may be unfinished, completed remotely or waiting on a tool whose result has not reached the application.

Create an exercise around this uncertainty. Start a representative task, interrupt the connection and inspect what the operator can determine from the task record. The application should preserve the action identifier, last confirmed stage and any committed business effects. A generic failure message does not provide enough information to resume safely.

Separate generation from execution during the exercise. Regenerating a draft may be acceptable. Repeating a payment request, customer notification or account update requires a check against the downstream system. The fallback path must inherit those controls rather than treating an outage as permission to bypass them.

Ask the operator to recover the task using the documented procedure. If recovery requires a developer to inspect a production database and guess which step ran, the workflow needs a clearer state model before it receives more autonomy.

Decide what customers should see

An internal retry can be invisible when it is brief and harmless. A longer delay needs an honest status message and a supported next step. Tell the user whether the request is queued, awaiting review or unavailable. Avoid showing a successful completion state before the business outcome is confirmed.

For customer-facing systems, prevent repeated submissions from creating multiple tasks. A user who refreshes a page during an outage should be able to see the existing request. If a new submission is genuinely required, explain why and connect it to the earlier attempt for reconciliation.

Fallback quality also affects communication. A secondary model may support a narrower task or require more human review. Make the reduced service behavior deliberate. A useful draft delivered for review is preferable to an apparently final answer that fails the application's normal acceptance rules.

Maintain a recovery inventory

Document which dependencies have an alternate route and which do not. Include identity, retrieval, storage, messaging and approval services as well as the model provider. Switching models does not restore a workflow whose document store or credentials are unavailable.

For each dependency, name the owner, the failure signal and the recovery action. Include a way to return from fallback to normal operation without releasing duplicate queued work. Recovery is part of the design, not an automatic consequence of the provider becoming reachable again.

Review the inventory when new tools or data sources are added. A workflow that was safe in read-only mode can acquire a different risk profile after it starts writing to a CRM. Its fallback test should change at the same time.

A practical resilience review starts with the business commitments: which tasks may wait, which require a person and which must stop when evidence is incomplete. KYCONNECTS can help map those commitments to application states and recovery paths. A second provider becomes valuable when it serves a defined continuity plan, rather than existing as an unused configuration option.

Questions about AI continuity

Keep the business process larger than the model

A continuity plan should describe the service users receive while AI is unavailable, not just how quickly another endpoint can be called.