How I Used Redis to Speed Up Dinogoal’s Backend

Dinogoal’s backend has always been simple, a Node.js server and a MongoDB database. Every request did the same thing: go to MongoDB, read, respond. It worked fine, until I looked at what the server was actually spending its time on.

Most of it was answering the same question over and over again.

The Problem

The match list is the heart of Dinogoal. To keep live scores fresh, every client polls /getMatches every 10 seconds while there is a match in play.

Each one of those requests read seven days of matches from MongoDB and serialized them. Ten users watching a match meant ten full reads of the same collection, all returning exactly the same matches.

The data only changes when the server syncs with the football data provider. Everything in between was wasted work.

Before Redis: Fix the Queries

A cache hides a slow query, it doesn’t fix it. So Redis was the last step, not the first one.

Before touching it, I:

• Added 12 missing MongoDB indexes
• Removed the N+1 queries (a league with 100 members went from 101 queries to 2)
• Enabled gzip compression (the anonymous match list went from 11,251 to 1,390 bytes)

Only then did caching make sense.

Rule Number One: Redis Is Optional

I didn’t want a cache failure to become a server outage. Adding a dependency that can take the whole backend down would be worse than not having it at all.

So every Redis command goes through a small wrapper:

async function safe(operation, fallback = null) {
  if (!isAvailable()) return fallback;

  try {
    return await operation(client);
  } catch (error) {
    console.error("[redis] command failed:", error.message);
    return fallback;
  }
}

If Redis is not configured, or it goes down, the backend keeps working:

• The cache is skipped and the query goes straight to MongoDB
• Rate limiting falls back to in-memory counters
• The client keeps retrying and reconnects on its own when Redis comes back

I tested it the hard way: with the server running, I stopped the Redis container. Endpoints kept returning 200, login kept working and the brute-force protection kept blocking.

Redis itself runs as one more container next to the backend, capped at 256 MB with the allkeys-lru policy. Everything stored in it can be rebuilt, so if it fills up it can drop the least used keys without consequences. It publishes no ports: it is only reachable from the internal Docker network.

The Cache Helper

All the caching is built on a single function. This is a simplified version of it:

async function cached(key, ttlSeconds, producer) {
  if (!isAvailable()) return producer();

  const hit = await safe((client) => client.get(key));
  if (hit) return JSON.parse(hit);

  const value = await producer();
  await safe((client) =>
    client.set(key, JSON.stringify(value), { EX: ttlSeconds }),
  );

  return value;
}

Using it is one line:

function cachedMatches() {
  return cached("matches:all", 10, getAllMatches);
}

Only data that is the same for every user goes in:

Caching a Personal Endpoint

But the expensive part, reading and serializing a week of matches, is identical for everyone. The only personal part is the predictions, and that is a small, indexed query.

So I split it in two. The match list comes from the shared Redis entry, and the user’s predictions are merged on top.

With ten users polling, the server went from 10 reads of the matches collection to 1.

Invalidating Instead of Waiting

A 10 second TTL on its own would mean a goal could take 10 extra seconds to show up.

Instead, the job that syncs the matches deletes the key as soon as it writes something:

if (synced) await invalidate("matches:all");

A new score is visible as soon as the server knows about it. The TTL is just the safety net.

Sponsors and the card catalogue go one step further and have no TTL at all. They only change when someone edits them in the admin panel, and the panel invalidates the key when that happens.

The Cache Stampede

Invalidation exposed another problem.

The moment a key is deleted, every request in flight misses the cache at the same time and runs the same query at the same time. It is exactly the worst moment: the match list is invalidated when the sync job finishes, with as many polls in flight as users watching.

The fix is a map of regenerations in progress. The first request that misses regenerates the value, and the rest wait for its promise instead of doing the same work again:

const inflight = new Map();

const existing = inflight.get(key);
if (existing) return existing;

const regenerate = producer().finally(() => inflight.delete(key));
inflight.set(key, regenerate);

return regenerate;

10 simultaneous requests right after an invalidation used to trigger 10 full reads. Now they trigger 1.

More Than a Cache

Once Redis was there, it solved three other problems:

• Distributed locks. Periodic jobs take a lock with SET NX before running, so two instances can never hand out the league prizes twice. If Redis is down, the prize job simply doesn’t run: skipping it is safer than paying twice.
• Rate limiting. Failed login attempts are counted per account and per IP, and those counters are shared between instances.
• Socket.IO adapter. Card battle events reach the player no matter which instance their socket is connected to.

What I Didn’t Cache

The user document is the most frequent read in the whole backend, it is fetched in 61 different places. It was the most tempting thing to cache.

I didn’t.

Doing it properly means every write to a user has to go through a single place that invalidates the cache. Today those writes are spread across around 40 places, and coin balances are involved. Serving a stale balance is much worse than making one extra query.

The rule I follow is simple: the cache is for painting screens, never for deciding whether someone can afford something. Those checks stay atomic in the database.

Lessons Learned

Redis didn’t make Dinogoal fast by itself. It stopped the server from answering the same question over and over again.

If you are adding a cache to your backend:

• Fix your queries first
• Make the cache optional, so losing it never takes you down
• Cache the shared part of a response and compute the personal part on top
• Invalidate when the data changes and keep the TTL as a safety net
• Don’t cache what you can’t invalidate reliably