Handling Webhooks

Building a reliable webhook handler requires careful attention to response timing, idempotency, error handling, and asynchronous processing. This guide covers best practices to ensure you never miss an event and process each one exactly once.

Quick Response Pattern

The most critical rule for webhook handling: respond immediately, process asynchronously. Lettr waits up to 30 seconds for your endpoint to respond. If processing takes longer, the webhook is considered failed and will be retried.

import express from 'express';

const app = express();

app.post('/webhooks/lettr', express.json(), (req, res) => {
  // Acknowledge receipt immediately
  res.sendStatus(200);

  // Process asynchronously (after response is sent)
  setImmediate(() => {
    const events = req.body;
    processEvents(events).catch(err => {
      console.error('Failed to process events:', err);
    });
  });
});

Never perform long-running operations before responding. Database writes, external API calls, and complex processing should happen after you’ve acknowledged the webhook.

Idempotency

Webhooks may be delivered more than once due to retries, network issues, or edge cases. Your handler must be idempotent—processing the same event twice should have the same effect as processing it once.

Using Event IDs

Every webhook event has a unique id. Store processed event IDs to detect and skip duplicates:

import { Redis } from 'ioredis';

const redis = new Redis(process.env.REDIS_URL);

async function processEventIdempotently(event) {
  const eventKey = `webhook:processed:${event.id}`;

  // Try to set the key with NX (only if not exists) and EX (expiry)
  const isNew = await redis.set(eventKey, '1', 'NX', 'EX', 86400); // 24 hour expiry

  if (!isNew) {
    console.log(`Event ${event.id} already processed, skipping`);
    return;
  }

  // Process the event
  await processEvent(event);
}

Database-Based Idempotency

For applications without Redis, use your database:

async function processEventIdempotently(event) {
  // Use a transaction with a unique constraint
  try {
    await db.transaction(async (trx) => {
      // This will fail if event already exists
      await trx('processed_webhooks').insert({
        event_id: event.id,
        event_type: event.type,
        processed_at: new Date()
      });

      // Process the event within the transaction
      await processEvent(event, trx);
    });
  } catch (err) {
    if (err.code === '23505') { // PostgreSQL unique violation
      console.log(`Event ${event.id} already processed`);
      return;
    }
    throw err;
  }
}

Idempotency with Business Logic

Sometimes you need idempotency at the business level, not just event level:

async function handleEmailBounced(data) {
  const { to, bounceType, emailId } = data;

  if (bounceType !== 'hard') {
    return; // Only suppress hard bounces
  }

  // Idempotent operation: adding to suppression list
  // If already suppressed, this is a no-op
  await db.suppressions.upsert({
    email: to,
    reason: 'hard_bounce',
    source_email_id: emailId,
    suppressed_at: new Date()
  }, {
    conflictFields: ['email'],
    updateFields: [] // Don't update if exists
  });
}

Queue-Based Processing

For high-volume applications or complex processing, use a message queue to decouple webhook receipt from processing:

import { Queue } from 'bullmq';

const webhookQueue = new Queue('webhooks', {
  connection: { host: 'localhost', port: 6379 }
});

// Webhook endpoint - just enqueue and respond
app.post('/webhooks/lettr', express.json(), async (req, res) => {
  try {
    const events = req.body;

    // Add each event to queue
    for (const event of events) {
      await webhookQueue.add('webhook-event', event, {
        attempts: 3,
        backoff: {
          type: 'exponential',
          delay: 1000
        }
      });
    }

    res.sendStatus(200);
  } catch (err) {
    console.error('Webhook error:', err.message);
    res.sendStatus(400);
  }
});

// Worker - processes events from the queue
import { Worker } from 'bullmq';

const worker = new Worker('webhooks', async (job) => {
  const event = job.data;
  const eventType = event.msys ? Object.keys(event.msys)[0] : null;
  const data = eventType ? event.msys[eventType] : null;

  if (!data) return;

  switch (data.type) {
    case 'delivery':
      await handleDelivery(data);
      break;
    case 'bounce':
      await handleBounce(data);
      break;
    case 'relay_tempfail':
    case 'relay_permfail':
      await handleRelayEvent(data);
      break;
    // ... handle other events
  }
}, {
  connection: { host: 'localhost', port: 6379 },
  concurrency: 10
});

Error Handling

Proper error handling ensures you don’t lose events and can debug issues effectively.

Categorizing Errors

async function processEvent(event) {
  try {
    await handleEvent(event);
  } catch (err) {
    // Determine if error is retryable
    if (isRetryableError(err)) {
      // Log and let the webhook retry mechanism handle it
      console.error(`Retryable error for event ${event.id}:`, err.message);
      throw err; // Will trigger webhook retry
    }

    // Non-retryable errors - log and store for manual review
    console.error(`Non-retryable error for event ${event.id}:`, err);
    await storeFailedEvent(event, err);
  }
}

function isRetryableError(err) {
  // Network errors, timeouts, temporary database issues
  const retryableCodes = ['ECONNRESET', 'ETIMEDOUT', 'ENOTFOUND', '503', '429'];
  return retryableCodes.some(code =>
    err.code === code || err.message?.includes(code)
  );
}

async function storeFailedEvent(event, error) {
  await db.failedWebhooks.insert({
    event_id: event.id,
    event_type: event.type,
    payload: JSON.stringify(event),
    error_message: error.message,
    error_stack: error.stack,
    failed_at: new Date()
  });

  // Optionally alert your team
  await slack.send({
    channel: '#webhook-failures',
    text: `Webhook failed: ${event.type} (${event.id})\nError: ${error.message}`
  });
}

Graceful Degradation

When external dependencies fail, don’t block the entire webhook:

async function handleEmailDelivered(data) {
  const { emailId, to, metadata } = data;

  // Critical operation - must succeed
  await db.emails.update(emailId, {
    status: 'delivered',
    delivered_at: new Date()
  });

  // Non-critical operations - fail gracefully
  try {
    await analytics.track('email_delivered', { emailId, to });
  } catch (err) {
    console.warn('Analytics tracking failed:', err.message);
    // Don't throw - the critical operation succeeded
  }

  try {
    if (metadata?.notifyOnDelivery) {
      await notifications.send(metadata.userId, 'Your email was delivered');
    }
  } catch (err) {
    console.warn('Notification failed:', err.message);
    // Don't throw - still a success
  }
}

Relay Events and Inbound Email

Relay events (relay.relay_injection, relay.relay_delivery, relay.relay_rejection, relay.relay_tempfail, relay.relay_permfail) are status events about inbound email on your inbound domains. They tell you that a message was received, forwarded, rejected, or failed to reach your endpoint. They are not the inbound message itself and do not include its body or attachments. The inbound message is posted to the Forwarding URL you set on the inbound domain, as a separate request with its own payload. Handle message content (routing by recipient, threading replies, extracting attachments from the raw MIME) in that endpoint, not in your webhook handler. See Email Parsing for the payload format and Email Routing and Reply Tracking for handler patterns. In your webhook handler, treat relay events like any other status event, for example to alert when deliveries to your forwarding URL start failing:

async function handleRelayEvent(data) {
  if (data.type === 'relay_tempfail' || data.type === 'relay_permfail') {
    // Your forwarding URL did not accept an inbound message
    await alerts.notify(`Inbound delivery failed: ${data.type}`);
  }
}

Monitoring and Observability

Track webhook processing health with metrics and logging:

import { Counter, Histogram } from 'prom-client';

const webhooksReceived = new Counter({
  name: 'webhooks_received_total',
  help: 'Total webhooks received',
  labelNames: ['event_type', 'status']
});

const webhookProcessingTime = new Histogram({
  name: 'webhook_processing_seconds',
  help: 'Webhook processing time',
  labelNames: ['event_type']
});

async function processEvent(event) {
  const timer = webhookProcessingTime.startTimer({ event_type: event.type });

  try {
    await handleEvent(event);
    webhooksReceived.inc({ event_type: event.type, status: 'success' });
  } catch (err) {
    webhooksReceived.inc({ event_type: event.type, status: 'error' });
    throw err;
  } finally {
    timer();
  }
}

Framework-Specific Examples

Next.js App Router

// app/api/webhooks/lettr/route.ts
import { NextRequest, NextResponse } from 'next/server';

export async function POST(request: NextRequest) {
  const events = await request.json();

  // Queue for async processing (e.g., using Vercel's waitUntil)
  // @ts-ignore - waitUntil is available in edge runtime
  const ctx = (globalThis as any).__nextHandlerContext;
  ctx?.waitUntil?.(processEvents(events));

  return NextResponse.json({ received: true });
}

async function processEvents(events: any[]) {
  for (const event of events) {
    const eventType = event.msys ? Object.keys(event.msys)[0] : null;
    const data = eventType ? event.msys[eventType] : null;
    if (data) {
      await handleEvent(data);
    }
  }
}

Express with TypeScript

import express, { Request, Response } from 'express';

const app = express();

app.post(
  '/webhooks/lettr',
  express.json(),
  async (req: Request, res: Response) => {
    res.sendStatus(200);

    // Process async
    setImmediate(() => {
      const events = req.body;
      processEvents(events).catch(console.error);
    });
  }
);

Authorization

Secure your endpoints with authentication

Retries

Understand retry behavior and failure handling

Event Types

Complete reference of all webhook events

Testing

Test webhooks in development and production

Documentation

Quickstart

Learn

Integrations

Resources

Handling Webhooks

Quick Response Pattern

Idempotency

Using Event IDs

Database-Based Idempotency

Idempotency with Business Logic

Queue-Based Processing

Error Handling

Categorizing Errors

Graceful Degradation

Relay Events and Inbound Email

Monitoring and Observability

Framework-Specific Examples

Next.js App Router

Express with TypeScript

Authorization

Retries

Event Types

Testing

​Quick Response Pattern

​Idempotency

​Using Event IDs

​Database-Based Idempotency

​Idempotency with Business Logic

​Queue-Based Processing

​Error Handling

​Categorizing Errors

​Graceful Degradation

​Relay Events and Inbound Email

​Monitoring and Observability

​Framework-Specific Examples

​Next.js App Router

​Express with TypeScript

​Related Topics

Authorization

Retries

Event Types

Testing

Quick Response Pattern

Idempotency

Using Event IDs

Database-Based Idempotency

Idempotency with Business Logic

Queue-Based Processing

Error Handling

Categorizing Errors

Graceful Degradation

Relay Events and Inbound Email

Monitoring and Observability

Framework-Specific Examples

Next.js App Router

Express with TypeScript

Related Topics