Distributed Transactions - Design Patterns for Maintaining Consistency

Challenges of Distributed Transactions

In microservices, a single business operation often spans multiple services. Since each service has its own database, traditional ACID transactions cannot be used.

flowchart LR
    subgraph Order["E-commerce site order"]
        O1["1. Order Service<br/>Create order"] --> O2["2. Inventory Service<br/>Reduce stock"]
        O2 --> O3["3. Payment Service<br/>Process payment"]
        O3 --> O4["4. Shipping Service<br/>Arrange delivery"]
    end
    O3 -.->|"What if something fails?"| Rollback["Roll back everything?"]

Two-Phase Commit (2PC)

This protocol commits all participants simultaneously only after everyone confirms they are ready.

sequenceDiagram
    participant C as Coordinator
    participant A as Participant A
    participant B as Participant B
    C->>A: 1. Prepare
    C->>B: 1. Prepare
    A-->>C: 2. Ready
    B-->>C: 2. Ready
    C->>A: 3. Commit
    C->>B: 3. Commit
    A-->>C: 4. Done
    B-->>C: 4. Done

Problems with 2PC

2PC is not recommended in modern microservices

Saga Pattern

Divides a long-running transaction into a series of local transactions. On failure, compensating transactions are used to roll back.

flowchart LR
    subgraph Normal["Normal flow"]
        T1["T1<br/>Create Order"] --> T2["T2<br/>Reserve Stock"] --> T3["T3<br/>Payment"] --> T4["T4<br/>Arrange Shipping"]
    end

flowchart LR
    subgraph Failure["On failure (T3 fails)"]
        F1["T1"] --> F2["T2"] --> F3["T3 ❌"]
        F3 --> C2["C2<br/>Release Stock"]
        C2 --> C1["C1<br/>Cancel Order"]
    end

Orchestration Approach

A central orchestrator controls the Saga.

class OrderSaga {
  async execute(orderData) {
    const sagaId = uuid();
    let currentStep = 0;

    try {
      // Step 1: Create order
      const order = await orderService.create(orderData);
      currentStep = 1;

      // Step 2: Reserve inventory
      await inventoryService.reserve(order.items);
      currentStep = 2;

      // Step 3: Process payment
      await paymentService.process(order.total);
      currentStep = 3;

      // Step 4: Arrange shipping
      await shippingService.schedule(order);
      currentStep = 4;

      return { success: true, orderId: order.id };

    } catch (error) {
      // Execute compensating transactions
      await this.compensate(currentStep, order);
      throw error;
    }
  }

  async compensate(step, order) {
    if (step >= 3) await paymentService.refund(order.id);
    if (step >= 2) await inventoryService.release(order.items);
    if (step >= 1) await orderService.cancel(order.id);
  }
}

Choreography Approach

Each service publishes and subscribes to events to coordinate.

flowchart TB
    subgraph NormalFlow["Normal Flow"]
        OS["Order Service"] -->|"OrderCreated"| IS["Inventory Service"]
        IS -->|"InventoryReserved"| PS["Payment Service"]
        PS -->|"PaymentProcessed"| SS["Shipping Service"]
        SS -->|"ShippingScheduled"| Done["✓"]
    end

    subgraph FailureFlow["On Failure"]
        PS2["Payment Service"] -->|"PaymentFailed"| IS2["Inventory Service"]
        IS2 -->|"InventoryReleased (compensation)"| OS2["Order Service"]
        OS2 -->|"OrderCancelled (compensation)"| Rollback["↩"]
    end

Comparison

Compensating Transactions

Instead of rolling back on failure, execute the reverse operation.

// Original operation
async function reserveInventory(items) {
  for (const item of items) {
    await db.inventory.update({
      where: { productId: item.productId },
      data: { quantity: { decrement: item.quantity } }
    });
  }
}

// Compensating transaction
async function releaseInventory(items) {
  for (const item of items) {
    await db.inventory.update({
      where: { productId: item.productId },
      data: { quantity: { increment: item.quantity } }
    });
  }
}

Considerations for Compensation

Outbox Pattern

Ensures both database update and event publishing succeed together.

// 1. Execute both within a transaction
async function createOrder(orderData) {
  await db.transaction(async (tx) => {
    // Save business data
    const order = await tx.orders.create(orderData);

    // Save event to outbox table
    await tx.outbox.create({
      eventType: 'OrderCreated',
      payload: JSON.stringify(order),
      status: 'pending'
    });
  });
}

// 2. Poll outbox in a separate process
async function publishOutboxEvents() {
  const events = await db.outbox.findMany({
    where: { status: 'pending' }
  });

  for (const event of events) {
    await messageQueue.publish(event.eventType, event.payload);
    await db.outbox.update({
      where: { id: event.id },
      data: { status: 'published' }
    });
  }
}

Eventual Consistency

Accepts that consistency will be achieved eventually rather than immediately.

Handling Eventual Consistency

// Example UI handling
async function placeOrder(orderData) {
  const result = await api.createOrder(orderData);

  // Display as "processing" instead of immediately confirmed
  return {
    orderId: result.orderId,
    status: 'processing',
    message: 'Your order has been received. We will notify you by email once confirmed.'
  };
}

Summary

Distributed transactions are a challenging problem in microservices architecture. 2PC is not suitable for modern distributed systems, and the Saga pattern with compensating transactions is recommended. It is important to accept eventual consistency and choose the consistency model that fits your business requirements.