AI Agent Reliability: Approvals, Retries & Duplicate Prevention

You approve the reply. The agent calls the email tool. Then the request times out.

The screen shows an error. The recipient may already have the message.

Now your system has to decide whether to try again, wait, or ask someone to investigate. A second attempt could fix a temporary failure, or it could send the same email twice.

An agent's reliability becomes visible when the next action is uncertain.

To decide safely, you need to know what was proposed, what the reviewer approved, what the tool attempted, and what actually happened. Each of those answers needs evidence that outlives the model's next response.

At Imversion, we approach the problem through explicit action contracts: typed descriptions of what the system intends to do, which code can validate and a reviewer can inspect. Around those contracts, our orchestration code keeps approval checks, pending state, retry policies, and tests for duplicate completion events.

This post follows one email reply through five questions:

1. What exactly is being proposed? (sections 1 and 2)
2. Who can authorize it? (section 3)
3. What survives an interruption? (section 4)
4. What makes another attempt safe? (sections 5 and 6)
5. What proves completion? (section 7)

1. Give the action a boundary before giving the agent more freedom

Start with an ordinary request: “Reply to this message and confirm that we will review the details.”

It sounds like one task, but the system has several decisions to make. Which message? Which mailbox? What will the reply say? Who can approve it? What evidence will establish that it was sent?

Making those decisions explicit gives the workflow its structure:

1. Retrieve a source record through a permitted read tool.
2. Produce a proposed action with structured arguments.
3. Validate the proposal and resolve its source identity.
4. Persist a proposal that a reviewer can inspect.
5. Accept approval through a trusted application boundary.
6. Execute the permitted action and inspect the tool result.
7. Record whether the action completed, failed, or still needs reconciliation, meaning a check of what actually happened before anything else is attempted.

The model can help interpret a request and prepare a reply. The application and its backend must enforce which records the caller may access, which actions require approval, and what counts as completion.

Our tool configuration reflects that split by selecting chat tools and approval tools separately. The read tools include search_source_items and get_source_item, while actions go through propose_action and approve_action, so the separation is visible in the integration contract.

Tool selection is only one layer of control, though. The code that executes the action still has to authenticate the caller and check permissions itself, because a model-produced field saying that an action is approved carries no authority.

2. A plausible reply still needs a valid target

The agent has written the reply. Before anyone approves it, inspect what the system intends to do.

A paragraph of text leaves too much implicit. A structured proposal names the action, its source, its input, and its identity, which means a reviewer can inspect it and application code can reject it.

Our email contract uses Pydantic models for that proposal. It includes a provider, action type, input, preview, source identifiers, and an idempotency key: a unique identifier for this one intended action, so that a repeat of the same request can be recognized as a repeat. Actions aimed at an existing message also require a source-item identifier.

The following synthetic JSON illustrates the shape of a proposal:

{
  "provider": "gmail",
  "type": "email.reply",
  "target_source_item_id": "synthetic-message-001",
  "data_source_id": "synthetic-mailbox-001",
  "input": {
    "body": "Thanks for the update. We will review the details."
  },
  "preview": {
    "body": "Thanks for the update. We will review the details."
  },
  "idempotency_key": "synthetic-reply-intent-001"
}

The typed boundary gives the application something concrete to reject. In our implementation, email.reply, email.mark_read, and email.apply_labels proposals cannot omit their source target.

Now suppose the proposal names a message that has been deleted.

The JSON can be perfectly valid while the action has no valid source, because structural validation only establishes that the required fields are present. A lookup must establish that the record exists, and the backend must establish that the caller may use it.

If the lookup fails, preserve the unresolved identifier and route the task to review, so the reviewer can see what is missing. Substituting a plausible message would turn missing evidence into an unintended action.

Our contract test test_targeted_canonical_actions_require_source_item_id covers the first case: a proposal without a source identifier is rejected. The deleted-message case is a different condition and needs its own fixture and assertion.

3. Approval belongs to the action the reviewer saw

Assume the source exists and the reviewer accepts the draft. Before execution, another step changes the reply's target.

The conversation still contains “approved,” but the reviewer never approved this new action.

Approval needs a stable object to refer to.

Our chat approval handler provides one by storing a hash of the pending action. When an approval arrives, the handler checks that pending state exists, compares the requested action ID, parses the stored action, and recomputes its hash to compare with the stored one. If the approval payload supplies its own action or hash, that is checked against the pending proposal too.

This excerpt shows two of those checks:

pending_hash = str(pending_action.get("action_hash") or "")
if hash_action(action) != pending_hash:
    return None, "Pending action integrity check failed"

requested_hash = request.approved_payload.get("action_hash")
if requested_hash is not None and str(requested_hash) != pending_hash:
    return None, "approved_payload.action_hash does not match pending action"

The practical rule is that an edited proposal needs approval of the revised action. Changing recipients, targets, or content can change the consequences even when the surrounding conversation sounds the same.

It is worth being precise about what the hash does. It detects disagreement between two representations of an action; it is not a digital signature and it proves nothing about who the reviewer is. Authentication, authorization, and protection of the stored proposal remain separate responsibilities. In our tests, the mock action backend covers one of them by rejecting any approval call that arrives without trusted approval context.

4. Waiting is part of the workflow

The reviewer closes the tab. An hour passes. The service restarts.

When the reviewer returns, the system needs to recover the same proposal with the same approval requirements. Silence has not granted permission, and a restart has not cancelled the need for review. That makes pending approval a state-management problem as well as a user-interface problem.

In our code, the conversation context includes a pending_action, and the Redis-backed conversation store keeps it for 24 hours by default. We treat that as short-lived operational state, which has a consequence: a reviewer who returns after the window may find the proposal gone. Whatever the product promises about how long work remains recoverable has to agree with the storage policy.

A longer-lived approval process needs a durable action store that records the proposal, its version, the reviewer's identity, the decision, timestamps, and the final outcome. Expiration should be explicit there too, so that an expired proposal has a defined review or renewal path.

While approval is pending, no action that requires it should occur. Recovery should restore the proposal and its status, and it should never infer approval from the fact that work was interrupted.

5. Decide what a retry means before the first timeout

Return to the timeout at the beginning.

If it happened during source retrieval, the system can probably repeat the read. If it happened after the email provider accepted the reply, the system needs to discover the outcome before attempting another send. Both failures can look like the same exception in application code, but their consequences are very different.

Our retry logic lives outside the email flow, in the coordinator that runs multi-step agent graphs. When a step fails, the coordinator compares the step's retry count with a configured maximum; if there is budget left, it requeues the step and increments the count. The failure policy can also consider alternate agents and partial success. The corresponding test, test_failed_step_retries_until_retry_limit, configures one retry, records a failed task, and asserts that a replacement is queued with retry_count set to one.

That mechanism bounds how many times the system tries. It cannot say whether another attempt is safe, and for the reply that is the question that matters: did the first attempt already create an effect? Answering it requires a stable action identity and a way to ask the provider what happened.

Switching to another agent does not resolve that uncertainty either, since every execution path has to respect the same action record and the same permission checks.

When you write the policy down, a few definitions prevent confusion later:

• Count the initial attempt separately from retries. “Two retries” normally permits three total attempts.
• State which errors qualify for a retry.
• Set a per-attempt timeout and a total elapsed-time budget.

6. Follow the duplicate all the way to the effect

Suppose the reviewer clicks twice. Or a queue delivers the same completion event twice. Or the worker restarts after the provider sends the reply.

All three can produce duplicate work, but they enter the system at different points, so each needs its own control.

Duplicate proposals

Our mock backend indexes proposals by tenant, user, and idempotency key:

scope_key = (tenant_id, user_id, idempotency_key)
action = MOCK_ACTIONS.get(scope_key)

Repeated proposals within that scope reuse the stored action, and a test exercises repeated proposals and approval calls across independently created server sessions. Because the mock keeps its dictionary in process memory, that test covers reuse within one process only; recovery after a restart and coordination between replicas need persistent storage and tests of their own.

Duplicate orchestration events

The graph tests reconcile the same completion twice and assert that downstream steps are queued once. This handles duplicate completion notifications at the orchestration layer, but it says nothing about whether a downstream tool's external side effect happens once. That is the third boundary.

Duplicate external effects

A durable implementation needs an atomic way to claim an action, preserve its outcome, and recognize a replay. Where a provider supports idempotency keys, the same key should accompany every retry of the same intent.

A database record alone cannot make an external API call atomic. If the provider accepts a write and the connection drops before the application records success, the workflow needs either provider-side deduplication or reconciliation. Without one of those, an explicit “uncertain” state is more accurate than a blind retry or a success message.

For the same reason, reusing a key with a different payload should have a defined conflict behavior, because a repeated key should never silently authorize a different action.

Figure 1. Proposal deduplication, event handling, and external effects require separate controls. The same-key example assumes the same scope and payload; a lost response leaves the effect uncertain.

7. Let the result determine what you tell the user

The tool finally responds. Its status is executing.

The agent now has enough information to say that work is in progress, but it has no evidence that the reply was sent. The distinction is small, and its consequence is visible: whether the user leaves believing the task is finished.

Our chat service requires an approve_action result to report completed before treating it as completion. One unit test supplies this response:

{
  "action_id": "action-1",
  "status": "executing"
}

The test then asserts:

assert response.response_type == "error"
assert "status: executing" in response.payload["message"]

The contract is that an in-progress action never becomes a successful confirmation. Our current handler represents that condition as an error; a system with asynchronous actions could instead expose a dedicated pending state and a status lookup.

Either way, the user-facing message should follow the recorded state. An honest pending status gives the user a reason to wait and the system a reason to keep tracking the action.

Turn the design into something you can challenge

By now, the reply has exposed the questions a happy-path demo leaves unanswered. What if the source is missing? What if the payload changes? What if approval never arrives? What if the first write succeeds and its response disappears?

Write down the acceptable outcomes before injecting those faults. Otherwise it becomes too easy to explain away whatever the implementation happens to do.

A local demonstration needs no model provider and no real email delivery. Synthetic source records, fixed extraction outputs, a persistent action store, and a mock outbox as the only place an effect can land are enough to exercise every control boundary above.

The following matrix specifies the expected behavior. These are acceptance criteria, not reported run results.

The three-attempt budget is an illustrative choice for this fixture, not a platform-wide default. The duplicate fixture should include a restart and concurrent submissions. A separate ambiguous-write fixture should simulate an effect that completes before its response is lost, because that failure is different from a source-read timeout.

For each run, save the code revision, dependency versions, run date, fixture inputs, expected outcome, actual outcome, and relevant state transitions. Keep the expected behavior fixed while investigating failures.

Figure 2. An illustrative state model. Validated means structurally validated here; source resolution can still require review. Restoring pending state does not grant approval, and an uncertain outcome is not automatically retried.

Review one action from beginning to end

You can apply this approach to an email reply, a CRM update, or a document approval. Choose one action with an effect outside your system and trace it through the same five questions:

1. What exactly is being proposed? Record the action type, target, payload, and identity. Validate its structure and resolve its source.
2. Who can authorize it? Bind the decision to the proposal the reviewer inspected. Define what happens when it changes or expires.
3. What survives an interruption? Preserve pending work and action outcomes for a documented period. Test recovery after a process restart.
4. What makes another attempt safe? Set attempt and time budgets. Define replay behavior, conflicting payloads, and reconciliation for uncertain writes.
5. What proves completion? Inspect the tool result and action record. Test duplicate submissions and concurrent execution, then report only outcomes you can reproduce.

The reply at the beginning may have been sent. It may still be waiting. A well-defined workflow gives engineers a way to find out, gives the reviewer a recoverable task, and gives the user an accurate status.

That is the standard we want the surrounding software to make testable: every action with a real-world effect has a known target, a permission check, and an outcome supported by evidence.

For teams applying this approach to business processes, our AI automation work provides the broader context. Start with one workflow whose decisions and failures you can inspect. Then use what you learn to decide where more autonomy is justified.