MongoDB count queries can slow down your API

Sometimes front-end team or other API clients ask to show them total number of objects in API replies. But if you use MongoDB, try not to do this. It slows down your API.

I say that if you have response format like that one placed below, you are in trouble.

{
  "total": XX,
  "limit": ZZ,
  "offset": YY,
  "objects": [...]
}

If you have a big collection and quite heavy query, count will take prohibitively long time. Find may work fast, because of the limit. But count will scan entire result set.

Some times if you add some additional conditions, that should slow down query (e.g. regexp search), but it shrinks the result set, API may respond faster.

What is the solution?

First of all, if you can change API format, do it. Response with prev next links (like Facebook API does) can work faster.

{
  "pre": "link-to-previous-x-objects",
  "next": "link-to-next-x-objects",
  "limit": x,
  "objects": [...]
}

Second option is to cache total. Then your response will look like search engines do. They can show you that they have more results than they actually have. If it's OK in your case, do like this.

In any case, it matters only for huge collections, heavy objects and complex queries. On the other cases you can do what you want without any problem.