Fix the Page, Not the API

The homepage of a news site I run is busy. Featured stories, a hero carousel, the main feed, top stories, market prices, commodity prices, football teams, a daily brief, trending topics from three different sources, and a category spotlight. Thirteen sections.

Behind them were 19 API calls. Each section fetched its own data, so the page assembled itself in front of the reader one piece at a time, and the whole thing was only as fast as the slowest call.

The obvious fix

The textbook answer to “too many requests” is GraphQL. One endpoint, one query that asks for exactly what the page needs, one response.

So I built it. In January the API got a full GraphQL layer, with queries for the homepage, the feed, articles, trending topics and markets, running on a staging server. It worked.

It also meant redesigning the API around a new schema, and it did not remove the slow part. A GraphQL server answering a homepage query still has to go and fetch all of that data when the reader asks for it. The work moved from the browser to the server. The reader still waited for it.

The branch is still there, parked “for later”. GraphQL is not bad. I just did not need to redesign my API to solve this problem.

What the problem actually was

The number of requests was not really the problem. The problem was doing the work at the moment the reader asked for it.

A news homepage is the same for everyone who loads it in the same few minutes. There is no reason to build it fresh for every visitor. Build it once, ahead of time, and hand every reader the finished result.

So instead of changing the API, I put something in front of it.

Precompute the page at the edge

Each multi-call page now has an aggregator: a Cloudflare Worker with two jobs.

On a schedule, it builds the page. Every few minutes (five for the homepage, three for other pages), a cron trigger calls every endpoint the page needs, combines the results into one JSON object, and stores it in Workers KV, Cloudflare’s key-value store at the edge.

On request, it serves the page. The frontend makes one call, GET /homepage/complete, and gets everything back from the edge. The API is not involved.

The build step fetches sections in batches, with a short pause between them, so a refresh never hits the API with 19 requests at once. And each section can fail on its own:

const [featured, feed, topStories] = await Promise.allSettled([
  fetchFeatured(env),
  fetchFeed(env),
  fetchTopStories(env),
]);

const page = {
  featured: featured.status === 'fulfilled' ? featured.value : null,
  feed: feed.status === 'fulfilled' ? feed.value : null,
  top_stories: topStories.status === 'fulfilled' ? topStories.value : null,
  cached_at: new Date().toISOString(),
};

Promise.allSettled is the important choice. With Promise.all, one slow trending-topics endpoint would take down the whole homepage. With allSettled, it becomes one empty section.

The last good copy

Every time the aggregator builds a valid page, it writes it twice: once as the current page with a five-minute lifetime, and once as a “last good” copy that lives for an hour.

Serving a request then goes through a short list:

// 1. Fresh copy in the cache? Serve it.
const cached = await env.CACHE.get(KEY);
if (cached) return respond(cached, 'HIT');

// 2. Cache empty and the API is healthy? Rebuild, store both copies, serve.
if (await originIsHealthy(env)) {
  const page = await buildPage(env);
  if (hasMainContent(page)) {
    await store(env, page); // current + last good
    return respond(page, 'MISS');
  }
}

// 3. API down or returning nothing? Serve the last good copy.
const lastGood = await env.CACHE.get(KEY + ':last_good');
if (lastGood) return respond(lastGood, 'LAST_GOOD');

The response header says which path served it, which makes debugging much easier than guessing.

I built that third step as insurance. I did not expect to find out how well it worked without noticing.

The outage I did not see

One day I got an email from my hosting provider: the API was down.

I opened the site. It was fine. The homepage loaded, the markets section had prices, the feed had stories. Readers were being served the last good copy from the edge, and nothing on the page said anything was wrong. If it had not been for the email, I would not have known.

That is the strange thing about this design. When it works, it hides problems from you as well as from your readers. That is why the platform’s health checks look at what the backend actually produces, not at what the site looks like from outside. I wrote about that in A Green Checkmark Is Not Evidence.

The bot flood I did not see either

The second test was not planned at all.

On 16 September, analytics flagged an anomaly. It expected about 74 active users that day. It recorded 108,343.

Google Analytics showing a spike in active users on 16 September: expected 74, actual 108,343

It was not a viral story. It was bots, mostly from Singapore, sending hundreds of thousands of requests. My hosting bill showed it too. Edge requests for the month came to 22.82 million against a 10 million allowance, and the 16th alone cost close to $60, against a few dollars on a normal day.

Daily hosting cost for September: a few dollars most days, and close to $60 on 16 September, with 22.82M edge requests against a 10M allowance

The site stayed fast the whole time. The bots were requesting pages, and the data behind those pages came from the precomputed copies at the edge, so the API behind them barely noticed. The flood cost money at the edge, but it never became an outage.

I only found out because I happened to look at analytics. I turned on bot protection the same day. A design built to make pages fast had also, without my asking, made the site hard to knock over.

What it costs

This is not free, and the trade-offs are worth stating plainly.

Freshness has a floor. The homepage is never more than five minutes old, but it can be up to five minutes old. For most news that is fine. For breaking news, there is an endpoint that rebuilds the page immediately.

The refresh runs whether anyone is reading or not. The aggregator calls the API on a schedule, even at 4 a.m. with no visitors. In exchange, the load on the API is fixed and predictable: the same number of requests every few minutes, no matter how many readers or bots show up.

Personal data cannot be precomputed. A reader’s saved articles or settings are different for every reader, so they stay out of the aggregator and out of every shared cache.

What is still not right

Two things I would change.

A partial failure can erase good data. If three of the thirteen sections fail during a refresh, the page is stored with those sections empty, and the empty version replaces both the current copy and the last good one. The fix is to keep the last good value for each section and only replace a section when a fresh fetch of it succeeds. That fix is written up and next on the list.

A cold cache makes one reader wait. When the cache is empty, the Worker checks the API’s health before rebuilding the page. That is the right call, but whoever triggers it waits a few seconds for the check and the rebuild. With a refresh every few minutes this rarely happens, but it can.

Where it is now

The pattern started on the homepage. Now every page that needs more than one API call has an aggregator: category pages, markets, the football league, currency pages, trending. Each page makes one request, served from the edge, built a few minutes ago.

I still have the GraphQL branch. If I ever need clients to ask flexible questions of the data, I may use it. But for a page that looks the same for everyone, the fastest request is the one the API never has to answer.