Hard20 minDistributed Systems
UpdatedAug 6, 2026
Edit

RabbitMQ: Duplicate Payment Jobs

Question Variations

  • "What happens when a consumer crashes after charging but before `ack`?"
  • "Where would you store RabbitMQ deduplication state?"
  • "Why is automatic acknowledgement unsafe for payment work?"

Why This Is Asked

A worker charges a payment provider, then crashes before acknowledging its RabbitMQ message. This tests whether a candidate can prevent a redelivery from charging the customer twice while retaining at-least-once delivery.

Key Concepts

  • Manual acknowledgement: Acknowledge only after the durable business effect is complete.
  • Idempotency key: Use a stable payment or message ID to recognize a repeated charge attempt.
  • Atomic state: Persist the processed message and payment outcome together when possible.
  • Failure policy: Retry transient failures, but dead-letter invalid or exhausted messages for investigation.

Question Variations

  • “What happens when a consumer crashes after charging but before ack?”
  • “Where would you store RabbitMQ deduplication state?”
  • “Why is automatic acknowledgement unsafe for payment work?”

Answers by Technology

+ Add Variant
RabbitMQImprove this answer ✏️

Expected Answer

Use manual acknowledgements. The worker calls the payment provider with an idempotency key derived from the business payment or RabbitMQ message ID, persists the successful outcome and processed-message record, then acknowledges. If it crashes after charging but before ack, RabbitMQ redelivers; the provider or local deduplication record returns the prior result instead of creating a second charge. If the call times out, query the provider by idempotency key before retrying rather than assuming it failed. Automatic acknowledgement is inappropriate because it transfers responsibility before the business effect is durable.

Why It Matters

At-least-once delivery protects against lost jobs, but only idempotent business handling protects customers from duplicate charges.

Example Code

const key = `payment:${job.orderId}`;
const payment = await provider.charge({ key, cents: job.cents });
await db.processedMessage.upsert({ where: { id: job.messageId }, create: { id: job.messageId, paymentId: payment.id }, update: {} });
channel.ack(message);

Common Mistakes

  • Acknowledging before recording success: A crash loses the work record and makes recovery ambiguous.
  • Generating a new provider key per retry: The provider cannot recognize the retry as the same charge.

Follow-up Questions

  • Where can deduplication live? (Answer: In the payment provider, a database unique constraint, or both.)
  • What if the provider times out? (Answer: Look up the stable idempotency key before attempting another charge.)

Related Questions

References