Tutorial: Response Caching

Caching is one of the most impactful things you can add to an API gateway. This tutorial shows you how to add response caching with Stoma - from simple in-memory caching to production-grade Cloudflare caching.

What We’ll Cover

1. Basic in-memory caching (works everywhere)
2. Cloudflare Cache API caching (production-ready)
3. Cache invalidation strategies
4. Advanced: Vary headers and cache keys

Step 1: Basic In-Memory Caching

The simplest way to add caching uses InMemoryCacheStore:

// Basic response caching with an in-memory store.
// The cache policy stores upstream responses and serves them
// directly on subsequent requests, reducing upstream load.
// Demo API: https://stoma.opensource.homegrower.club/demo-api

import { createGateway, cache, cors, InMemoryCacheStore } from "@homegrower-club/stoma";

// Create a cache store — InMemoryCacheStore works on all runtimes
const cacheStore = new InMemoryCacheStore({
  maxEntries: 100, // Max number of cached responses (LRU eviction)
});

const gateway = createGateway({
  name: "cached-api",
  basePath: "/api",
  policies: [cors()],
  routes: [
    {
      path: "/products/*",
      pipeline: {
        policies: [
          cache({
            store: cacheStore,
            ttlSeconds: 120, // Cache for 2 minutes
            methods: ["GET"], // Only cache GET requests
            cacheStatusHeader: "x-cache", // Response header shows HIT/MISS
          }),
        ],
        upstream: {
          type: "url",
          target: "https://stoma.opensource.homegrower.club/demo-api",
        },
      },
    },
  ],
});

export default gateway;

Open in Editor

Testing It

# First request - MISS (upstream is called)
curl -v https://your-gateway.com/api/products/123
# x-cache: MISS

# Second request - HIT (cache serves the response)
curl -v https://your-gateway.com/api/products/123
# x-cache: HIT

Tip

In-memory caching works great for development and testing. For production on Cloudflare Workers, use the Cache API (shown below) for durability across requests.

Step 2: Cloudflare Cache API

For production deployments, use Cloudflare’s built-in cache:

import { createGateway, cache, cors } from "@homegrower-club/stoma";
import { cloudflareAdapter } from "@homegrower-club/stoma/adapters";

const adapter = cloudflareAdapter({
  cache: caches.default,  // Cloudflare's Cache API
});

const gateway = createGateway({
  name: "cached-api",
  basePath: "/api",
  policies: [cors()],
  routes: [
    {
      path: "/products/*",
      pipeline: {
        policies: [
          cache({
            store: adapter.cacheStore,
            ttlSeconds: 300,        // Cache for 5 minutes
            methods: ["GET"],
            varyHeaders: ["accept-language", "accept-encoding"],
            respectCacheControl: true,
          }),
        ],
        upstream: {
          type: "url",
          target: "https://api.example.com",
        },
      },
    },
  ],
});

Key Options Explained

Option	Purpose
ttlSeconds	How long to cache the response
methods	Which HTTP methods to cache (GET by default)
varyHeaders	Different cached versions based on these headers
respectCacheControl	Honor upstream’s Cache-Control headers
cacheKeyFn	Custom function to build cache keys

Step 3: Custom Cache Keys

Sometimes you need more control over what’s used as the cache key:

cache({
  store: adapter.cacheStore,
  cacheKeyFn: (c) => {
    // Include path, user region, and accept language in cache key
    const path = c.req.path;
    const region = c.req.header("cf-ipcountry");
    const lang = c.req.header("accept-language");
    return `${path}:${region}:${lang}`;
  },
})

Step 4: Cache Invalidation

Stoma’s cache policy respects upstream Cache-Control headers, but sometimes you need manual invalidation:

import { createGateway, cache, cors } from "@homegrower-club/stoma";
import { cloudflareAdapter } from "@homegrower-club/stoma/adapters";

const adapter = cloudflareAdapter({ cache: caches.default });

// Create a route to manually invalidate cache
const gateway = createGateway({
  name: "api",
  basePath: "/api",
  routes: [
    {
      path: "/products/*",
      pipeline: {
        policies: [
          cache({ store: adapter.cacheStore, ttlSeconds: 300 }),
        ],
        upstream: { type: "url", target: "https://api.example.com" },
      },
    },
    // Invalidation endpoint
    {
      path: "/admin/invalidate-cache",
      methods: ["POST"],
      pipeline: {
        policies: [
          // You'd add admin auth here in production
        ],
        upstream: {
          type: "handler",
          handler: async (c) => {
            const url = new URL(c.req.url);
            const pathToInvalidate = url.searchParams.get("path");

            if (!pathToInvalidate) {
              return c.json({ error: "Missing path parameter" }, 400);
            }

            // Build the cache key the same way the cache policy does
            const cacheKey = `https://api.example.com${pathToInvalidate}`;

            // Delete from cache
            await adapter.cacheStore.delete(cacheKey);

            return c.json({ success: true, invalidated: cacheKey });
          },
        },
      },
    },
  ],
});

Alternative: Time-Based Invalidation with Tags

If your upstream supports cache tags (Cloudflare Workers KV + custom header), you can implement tag-based invalidation:

cache({
  store: adapter.cacheStore,
  ttlSeconds: 300,
  // Cache policy adds x-cache-tags header on responses
  // You can read it and use it for invalidation
})

Step 5: Handling Edge Cases

Don’t Cache Errors

cache({
  store: adapter.cacheStore,
  ttlSeconds: 300,
  // Don't cache 5xx errors
  bypassDirectives: ["no-store", "no-cache", "private"],
})

Only Cache Successful Responses

The cache policy only caches responses with status 200-299 by default. Error responses (400, 401, 500, etc.) always bypass the cache.

Streaming Responses

Streaming responses (Transfer-Encoding: chunked) cannot be cached. The cache policy automatically bypasses streaming responses.

Complete Example: E-commerce API

Here’s a real-world configuration for a product catalog:

// Advanced caching example: e-commerce product catalog with tiered TTLs.
// Demonstrates different cache durations for listings vs. details,
// rate limiting, and language-aware cache keys via varyHeaders.
// Demo API: https://stoma.opensource.homegrower.club/demo-api

import { createGateway, cache, rateLimit, cors, InMemoryCacheStore } from "@homegrower-club/stoma";

const cacheStore = new InMemoryCacheStore({ maxEntries: 500 });

const gateway = createGateway({
  name: "catalog-api",
  basePath: "/api",
  policies: [
    cors({ origins: ["https://shop.example.com"] }),
  ],
  routes: [
    // Product listings — short TTL, rate limited
    {
      path: "/products",
      methods: ["GET"],
      pipeline: {
        policies: [
          rateLimit({ max: 100, windowSeconds: 60 }),
          cache({
            store: cacheStore,
            ttlSeconds: 60, // Short TTL for listings
            varyHeaders: ["accept-language"],
          }),
        ],
        upstream: { type: "url", target: "https://stoma.opensource.homegrower.club/demo-api" },
      },
    },
    // Product details — cached longer
    {
      path: "/products/:id",
      methods: ["GET"],
      pipeline: {
        policies: [
          rateLimit({ max: 200, windowSeconds: 60 }),
          cache({
            store: cacheStore,
            ttlSeconds: 3600, // 1 hour for product details
            varyHeaders: ["accept-language"],
          }),
        ],
        upstream: { type: "url", target: "https://stoma.opensource.homegrower.club/demo-api" },
      },
    },
    // Admin mutations — never cached
    {
      path: "/products/*",
      methods: ["POST", "PUT", "PATCH", "DELETE"],
      pipeline: {
        policies: [
          rateLimit({ max: 20, windowSeconds: 60 }),
        ],
        upstream: { type: "url", target: "https://stoma.opensource.homegrower.club/demo-api" },
      },
    },
  ],
});

export default gateway;

Open in Editor

What’s Next?

• Traffic Policies - all caching options
• Public Read API Recipe - complete example
• Real-World Example - production gateway with caching