Retry, defer, and escalation patterns

Retry is a technical recovery. Defer is a business choice. Escalation is an ownership transfer. Calling all three “retry” creates duplicate work and noisy alerts.

Choose the path by error class

Idempotency first

Before a retryable action, derive a stable action key such as run_id:step_id. Store it with the external request or the created task. On retry, check whether the action already succeeded. A timeout means the response is unknown; it does not mean the action did not happen.

Bounded retry policy

step: create-task
retry:
  max_attempts: 3
  backoff: exponential
  retryable: [timeout, rate_limit, 502]
  stop_on: [validation_error, permission_denied]
after_max_attempts:
  state: failed
  owner: revops
  notify: true

Use the platform's current retry behavior as an implementation detail. Keep this policy in the workflow specification so a change of tool does not change the business promise.

Escalation must transfer ownership

An alert without an owner is an announcement. On escalation, set the next owner, due time, and allowed action. Preserve the original owner and attempt history. Review failed and deferred work separately from successful work.

Keep retry state explicit

Store the attempt number, last error class, last attempted time, next attempt time, and idempotency key. A deferred item should also carry the condition that can release it, such as a restored credential, completed enrichment, or manual approval. Otherwise “waiting” becomes a permanent hiding place.

Use backoff and jitter for transient service failures, but set a maximum age as well as a maximum count. A task that succeeds three days later may no longer be useful. Before each retry, re-check safety conditions that may have changed, including consent, ownership, lifecycle, and whether another run already completed the outcome.

Review recovery quality

Report first-attempt success, recovered after retry, deferred, permanently failed, and manually resolved as separate outcomes. Group failures by reason and workflow version. Repeated transient errors may indicate capacity trouble; repeated permanent errors usually indicate a broken contract or bad source data. Fix the recurring class instead of expanding the retry count.

Related: Workflow states and job history, Reconciliation between CRM and external tools, and GTM workflow runbook template.