Articles

The N+1 Query Problem and How to Fix It

The N+1 query problem turns one database request into hundreds by issuing a separate query per row. Here's how to spot it and fix it.

The Lycoris Team The Lycoris Team · · 5 min read
A library card catalog with rows of small wooden drawers

The N+1 query problem happens when code fetches a list of N records with one query, then loops over that list and issues one additional query per record — turning what should be a single round trip to the database into N+1 of them. It’s one of the most common performance bugs in web applications, and one of the easiest to introduce without noticing, because the code that causes it often looks perfectly reasonable.

What it looks like in practice

Say you’re rendering a blog’s homepage: a list of posts, each with its author’s name. A natural first pass looks like this:

posts = db.query("SELECT * FROM posts LIMIT 20")
for post in posts:
    author = db.query("SELECT * FROM authors WHERE id = ?", post.author_id)
    render(post, author)

That’s 1 query to fetch the posts, plus 20 more — one per post — to fetch each author. Twenty rows isn’t a disaster, but the same pattern applied to a paginated API response, a nested comment thread, or a dashboard with hundreds of rows can turn a page load that should take milliseconds into one that takes seconds, because each query pays the full cost of a network round trip to the database.

Why it’s easy to miss

The bug is subtle because each individual line of code is correct — fetch the posts, then fetch each author. It’s the aggregate behavior that’s the problem, and that aggregate behavior often only shows up under realistic data volumes. A local dev database with five seeded rows won’t reveal it; a production table with tens of thousands of rows will. ORMs make this worse by design: lazy-loaded associations (post.author.name) look like a simple property access in code, hiding the fact that accessing it triggers a fresh query if the data wasn’t already loaded.

How to detect it

The most reliable way to catch N+1 queries is to look at what’s actually hitting the database, not just what the application code appears to do:

  • Query logging. Turn on your database driver’s or ORM’s query log in development and watch for a burst of near-identical queries differing only by an ID.
  • APM and tracing tools. Application performance monitoring tools that trace a request end-to-end will show a request that spawns dozens of child spans against the same table — a strong N+1 signal.
  • Query counters in tests. Some test frameworks let you assert a maximum query count for a given code path, which turns N+1 regressions into a failing test instead of a silent slowdown.

Combining this with database indexing matters too: an unindexed per-row query is doubly expensive, since each of the N queries does a slow scan on top of the network overhead.

Fixing it: eager loading and batching

The standard fix is to replace “fetch, then fetch again per row” with a single query that gets everything up front.

Eager loading with a join. Fetch posts and authors in one query using a JOIN, so the database does the correlation instead of the application:

SELECT posts.*, authors.name
FROM posts
JOIN authors ON authors.id = posts.author_id
LIMIT 20

Most ORMs expose this as an explicit “include” or “with” option on a query, precisely so you opt into eager loading instead of relying on lazy loading by default.

Batching with a second query. When a join would duplicate too much data (for example, loading each post’s list of tags), it’s often cleaner to run exactly two queries: one for the posts, and one WHERE id IN (...) query for all the related rows at once, then stitch them together in application code. This is the same idea behind the DataLoader pattern popularized in the GraphQL ecosystem, which batches and caches lookups within a single request so that resolving nested fields never degrades into N+1 behavior — see REST vs GraphQL for how that ecosystem shaped this pattern.

Selecting only what you need. Whether joining or batching, pull only the columns the view actually renders. A wide SELECT * across a join multiplies the amount of redundant data returned, which matters more as row counts grow.

Lazy loading vs eager loading

Lazy loadingEager loading
Query countOne per accessed relationOne (or a small fixed number) total
Best forRelations rarely accessedRelations rendered for every row
RiskN+1 queries under loadOver-fetching if the relation isn’t used
Typical fixConvert to eager load or batchAlready batched

Neither approach is universally right — a detail page that occasionally needs one related record is a fine candidate for lazy loading. The problem is specifically lazy loading inside a loop over many rows.

Caching as a complementary fix, not a replacement

A layer like Redis can absorb repeated identical lookups (see Redis vs Memcached), and a materialized view can precompute a joined result set so the read path never touches multiple tables at request time. Both help, but they’re best applied after fixing the query pattern itself — caching a query that shouldn’t exist in that form just hides the underlying inefficiency and adds cache-invalidation complexity on top of it. If your API sits behind connection pooling, an N+1 pattern also burns through pooled connections far faster than a batched equivalent, since each request holds a connection open across dozens of round trips instead of one.

The takeaway

The N+1 query problem is what happens when fetching a list and fetching each list item’s related data become two separate, unbatched steps — one query becomes N+1. Detect it by watching actual query logs or traces under realistic data volumes, not by reading the code in isolation. Fix it with a join or a single batched follow-up query instead of a per-row lookup, and treat caching as a supplement to that fix rather than a substitute for it.

The Lycoris Team The Lycoris Team · · 6 min read

How to Read a Postgres EXPLAIN ANALYZE Query Plan

A step-by-step guide to running EXPLAIN ANALYZE in PostgreSQL and reading the query plan it returns — node types, costs, and where the real time went.

#Databases #SQL #Performance
The Lycoris Team The Lycoris Team · · 5 min read

Postgres Index Types: B-Tree vs GIN vs GiST

Postgres offers several index types beyond the default B-tree. When GIN and GiST outperform it for arrays, JSONB, full-text search, and ranges.

#Databases #SQL #Performance
The Lycoris Team The Lycoris Team · · 5 min read

What Is a Covering Index?

A covering index holds every column a query needs, letting the database answer from the index alone without a lookup back to the table.

#Databases #SQL #Performance