How we moved search from Typesense to Postgres for a tenth of the cost

Every time you type into the search bar in our app and results come back across hundreds of millions of messages in the blink of an eye, something quietly expensive is happening behind the scenes. This is the story of how we made that same search cost us a tenth of what it used to, survived two failed attempts to get there, and let a swarm of AI agents finish the job overnight while I slept.

The whole story in one picture: 64 GB of RAM down to 4, and the monthly bill with it.

The ticking timebomb

Search data only grows, and every time it crosses the line the machine (and the bill) has to double.

For the last five years, search in our app has been powered by a tool called Typesense. It is blazing fast for one reason: it keeps its entire search index in RAM.

RAM is the fastest memory a computer has. It is also one of the most expensive, and there is a hard ceiling on how much of it you can put in a single machine. That combination is what kept me up at night. Our Typesense server already holds 45 GB of search data inside its 64 GB of RAM and costs about $370 a month. The moment we crossed 60 GB, our only option was to double the machine to roughly $740 a month, with a wall not far beyond that. And search data only ever grows.

To buy time, one of our engineers, Kishore, had been manually deleting the search data of churned users every few months. It is not a clean job. It needs downtime, so he did it late on a Saturday night when traffic was low. It worked, but it was a patch on a leak. We were sitting on a ticking timebomb.

A year ago, we gave up

The obvious escape was Postgres, the database that already runs almost everything in our app. Postgres keeps its data on disk instead of RAM, and disk is almost free by comparison. A hundred gigabytes of disk costs about $8 a month and can be grown more or less forever.

But cheap comes with a catch. Disk-based search is nowhere near as fast as searching in RAM, and Typesense had spent years perfecting an algorithm that actually understands what a person means when they type a random phrase into a search bar. Matching that on Postgres was never going to be a weekend job.

Alban, another engineer on the team, took the first swing at it a year ago and spent nearly a month. Here is the thing I want to be honest about: the problem was never that it was unsolvable. It was that solving it meant endless trial and error, and 95% of that time went into grunt work. Export the data out of Typesense. Write a loader. Spin up a server. Wait hours for millions of rows to load. Build an index. Run a few queries. Realize the schema is wrong. Drop it and start the whole loop over again. Every lap took forever, and the payoff was never guaranteed. We could not justify sinking that much of Alban's time into a maybe. So we dropped it.

The Meilisearch detour

Then, a few weeks ago, Kishore took another crack at it, this time with a different tool called Meilisearch. Like Postgres, Meilisearch stores its data on disk instead of RAM. And this time we had AI on our side, so a lot of the menial setup that had drained Alban could be automated away. On paper it was close to perfect.

Then we pushed it to production, and reality showed up.

The first crack was size. Meilisearch took our roughly 40 GB of search data and bloated it into an index of over 400 GB, more than ten times bigger. Suddenly the cheap-disk story needed 500 GB to 1 TB just to hold the thing.

But storage was the least of it. The real test was the one that actually matters: could Meilisearch keep indexing brand new messages while also serving live searches, on a normal-sized server? Because that is exactly what our app does every second of every day.

It could not. And here is the irony that stung the most: we moved to Meilisearch to escape a RAM problem, and it died of a RAM problem. The moment we resumed indexing on a normal box and pointed real traffic at it, indexing and search started clawing at each other for memory. A single new message could take 15 to 20 minutes to become searchable. Messages that were "accepted" were not actually searchable yet. Live searches came back empty, half-complete, or timed out. Its health check cheerfully reported "available" while it was effectively broken. And deleting churned users, the exact chore we were trying to make painless, triggered a full index rebuild that ran the box out of memory.

So Kishore made the honest call. He rolled search back to Typesense, shut Meilisearch down, and we were right back where we started. Two serious attempts in, and the timebomb was still ticking.

Note: Meilisearch is a genuinely good tool. Our failure was specific to our workload, a very large corpus with continuous high-rate indexing and periodic bulk deletes on a memory-constrained box. Your mileage will differ.

The tool was there all along

Then it hit me. The answer had been sitting under our noses the entire time. Postgres.

The thing that had beaten Alban was never Postgres itself. It was the 95% of menial grunt work you had to grind through before you could even reach the 5% where the real problem lived. And that grind is precisely the part that has stopped being a human's job. Every bit of that repetitive setup can now be handed to Claude.

Claude did not make me smarter about search. It collapsed the distance between me and the work, so for the first time I could put all of my attention on the 5% that actually mattered. So I took a crack at it.

Native Postgres search, and its one fatal flaw

Postgres has full-text search built right in: you build a GIN index over a tsvector column and rank matches with ts_rank. That was the obvious first move, and for a while it looked like I had won. The results were as good as Typesense and it was easy on memory.

In plainer terms, picture a diligent office assistant sorting a mountain of letters, which is basically what those three pieces do. The tsvector is the assistant skimming each letter and noting only its meaningful words, dropping filler like "the" and "and" and treating "running" and "ran" as one. The GIN index is the master lookup sheet from those notes: name any word and it points straight to every letter with it, without re-reading the pile. And ts_rank is how well each letter matches, so the strongest rise to the top. All three come free with Postgres, which is why this felt like such an easy win at first.

Then I searched a common word inside one of our biggest accounts, and it took over 90 seconds to come back.

Here is why. Postgres' built-in search cannot stop early. GIN will happily find every matching row, but ts_rank has no notion of top-k termination. To return the best ten results, it has to fetch and score every matching row, sort them all, and only then apply the LIMIT. Picture a librarian who, asked for the three best books on a topic, insists on reading every book in the building cover to cover before naming even one. On a common word inside a huge account, that meant scoring millions of messages on a single keystroke. For a search bar people expect to feel instant, ninety seconds is not a slow answer. It is a broken one, and it is architectural, not a setting you can tune away.

-- The built-in approach: a GIN index over a tsvector, ranked by ts_rank.
CREATE INDEX messages_fts ON messages
  USING gin (to_tsvector('english', body));

SELECT id
FROM   messages
WHERE  to_tsvector('english', body) @@ plainto_tsquery('english', 'refund')
ORDER  BY ts_rank(to_tsvector('english', body),
                  plainto_tsquery('english', 'refund')) DESC
LIMIT  10;
-- On a common word, this scores every matching row before it can honour the LIMIT.

pg_search: a real search engine inside Postgres

The built-in ranker scores every match before it can sort; BM25's WAND keeps a shortlist and skips the rest.

So I switched to an extension called pg_search. It embeds Tantivy, a Rust search engine in the Lucene lineage, directly inside Postgres as a first-class index. You build it with CREATE INDEX ... USING bm25, query it through a new @@@ operator, and it ranks results with the BM25 algorithm.

That Lucene lineage is worth pausing on. Lucene is the open-source search library that Elasticsearch and Apache Solr are both built on, and between them those two power search and log analytics at a huge slice of the internet: Wikipedia's search, GitHub code search, and the logging stacks at companies like Netflix and Uber all sit on top of it. Tantivy is a from-scratch Rust re-implementation of the same core ideas, so switching to pg_search meant running the same battle-tested machinery those engines use, only right inside our own Postgres instead of in a separate cluster.

-- The BM25 index, built once over the fields we search.
CREATE INDEX messages_bm25 ON messages
  USING bm25 (id, account_id, name, email, subject, body, created_at)
  WITH (key_field = 'id');

Crucially, unlike ts_rank, BM25 inside Tantivy uses a technique called WAND to stop the moment it has found the best matches instead of grinding through everything. That single capability killed the deal-breaker. The same broad query that took ninety seconds now finishes in a couple of seconds at the very worst, and usually in single-digit milliseconds. The timeouts went to zero.

So how does it stop early without missing the best results? Every search term carries a known maximum possible score, so the engine keeps a running shortlist of the best K matches (2,000, in my case) plus a threshold equal to the weakest score on that shortlist. Any document whose best possible score cannot beat that threshold is skipped without ever being fully scored, and as stronger matches arrive the threshold rises and more of the tail falls away. On a broad query that discards the overwhelming majority of matches, the early exit that ts_rank could never make.

If that is hard to picture, here is the same idea with a bag of oranges. Say you want the three sweetest oranges out of a huge crate, but the only way to truly know an orange's sweetness is to cut it open and taste it, which is slow. So you keep a small tray of the sweetest three you have tasted so far, and the least sweet one on that tray becomes your bar. For every new orange you just give it a sniff, and the smell tells you the sweetest it could possibly be. If even that best-case smell is fainter than the worst orange already on your tray, you toss it without ever cutting it open. As sweeter oranges make the tray, the bar rises and you throw away more and more on smell alone. Tasting is fully scoring a result, the sniff is that cheap best-possible score, and the tray is the shortlist. That is WAND.

Letting agents solve it while I slept

A fast engine was only half the battle. The other half was taste: making Postgres rank results the way a human actually expects, the way Typesense had quietly perfected over years. That is hundreds of tiny judgment calls, the exact kind of endless trial and error that had killed both previous attempts.

So I did not grind through it by hand. I split the work along the line that actually mattered: the high-level, abstract calls stayed with me, and the low-level, soul-crushing grind went to the agents. My half was the thinking. Deciding what a good result even means, sketching the scoring rules, and reading the previous night's output to figure out where it was still wrong. Their half was the part that had beaten every human who tried before: taking those rules and running them against tens of thousands of our real historical searches, tuning a hundred little knobs, measuring, discarding, and trying again, thousands of times over.

And this was not one magic night. It was about a week. Each day I did the thinking and set the direction, then pointed the agents at a throwaway server with a clear finish line (keep scoring yourself against real searches, do not stop until the results clear a quality bar) and went to bed. Each night they ground through attempt after attempt on their own while I slept. Each morning I woke up to a fresh batch of results, saw where they still diverged from Typesense, adjusted the rules, and set them loose again.

By the end of that week they had landed on a formula that matched Typesense on quality and returned results in milliseconds. Weeks of trial and error that would have crushed a person, done in a handful of nights I spent unconscious.

Note: the agents did not just guess. For every one of tens of thousands of our real past searches, they ran the query through both the new Postgres search and the old Typesense, compared the two result sets side by side, and nudged the scoring rules until they lined up. That side-by-side loop is what turned "close" into "nobody notices the difference."

How the search actually works

A regular index starts from a record; an inverted index starts from a word, which is exactly what a search needs.

Before I get to the interesting part, the scoring, it is worth understanding how the engine finds anything at all.

The secret is something called an inverted index, and you already use one every time you open a textbook. To find every mention of a word in a 900-page book, you do not read all 900 pages. You flip to the index at the back, find the word, and it tells you the exact pages it appears on.

Why is it called inverted? Think about the natural direction of a book: page to words, where each page carries the list of words printed on it. The index at the back turns that around, word to pages, where each word carries the list of pages it appears on. A regular database index is built for that first direction, start from a record and look at its fields. An inverted index flips the relationship on its head and stores the reverse, "given this word, which records contain it," which happens to be the exact question a search is asking. That flip is the whole trick.

An inverted index is exactly that, built for our messages: for every word that has ever appeared, it keeps a ready-made list of which messages contain it. So the engine never reads through hundreds of millions of messages. It flips to the right index entries and pulls the lists.

Here is what happens, start to finish, the moment you hit enter:

1. Break the query into words. What you typed is split into search terms and turned into a checklist of what a result must contain and what is nice to have.
2. Look each word up in the index. For every word, the engine grabs the pre-built list of messages that contain it. The table itself is never read.
3. Combine the lists. Those lists are merged to find the messages that satisfy the whole query.
4. Score each match. Every surviving message gets a relevance number, based on how often your word appears, how rare the word is overall, and how long the message is.
5. Keep the best, skip the rest. The engine holds a running shortlist of the best matches and uses it to skip scoring anything that clearly cannot make the cut. This is the stop-early trick that killed the ninety-second problem.
6. Hand the winners back. The best matches go back to the database, best-first, ready to show you.

Where the default scoring fell short

Steps one through three, and step five, were perfect out of the box. Step four was the problem.

BM25 is superb at judging pure text relevance. But "most textually relevant" is not always "what the user actually wanted," and two examples made that painfully clear.

First, it has no sense of time. Say you search a customer's first name to pull up the chat you had with them yesterday. BM25 has no idea what "yesterday" means. It only knows who mentions that name and how strongly, so it will happily rank a years-old thread above your recent one. And because our app shows results newest-first from whatever set the search hands back, if that recent conversation never makes it into the set, you simply never see it. Recent, relevant messages were getting buried alive.

Second, it can be gamed by repetition. Imagine we once sent a batch of outreach messages that repeated a name in the subject, the body, and the contact field. Out of the box, BM25 adds a bit of score for every field that matches, so those junk messages, matching in four places, outscored a single real conversation that matched strongly in just one place, the customer's actual name. The noise drowned out the signal.

Both problems came down to the same realization: the engine was brilliant at finding matches quickly, but its definition of a "good" result was not the same as mine.

Teaching the engine my taste

The engine finds and ranks at full speed; my rules just tell it what a good result looks like.

So I did not fight the engine, and I did not replace it. I kept every bit of its speed and taught it my own set of scoring rules on top.

• Where the match is. A match in the customer's name counts far more than the same word buried in a long message, so a name match scores highest, then email, then subject, and the message body counts the least. Typing a person's name pulls up their conversation instead of every message that ever mentioned that word.
• Only your best match counts. I judge a conversation on its single strongest match, not on how many places it matches. A clean hit on the name is what it rides on, and matching a few other fields only adds a small nudge. This is the fix for the outreach-spam problem above.
• Exact phrases stick together. When your words appear together and in order, I give a big extra boost. Someone searching "order refund" should see that exact phrase ahead of conversations that mention "order" in one place and "refund" somewhere far away.
• Numbers get a strong, even boost. If you search a number, like a phone or order number, I boost it hard and equally across every field. With a number you almost always want the one exact record, not a fuzzy ranking.
• How recent it is. Finally, a freshness boost. I sort conversations into age buckets (today, this week, this month, this year), and the more recent one is, the bigger its bump. This is the fix for the no-sense-of-time problem above.

If this all sounds a lot like how Typesense already worked, that is exactly the point. These rules are, more or less, the same instincts Typesense had spent years refining, which is why the switch felt invisible to everyone using our app.

Mechanically, all of this is one Postgres function. I assemble a single scoring query, hand it to one @@@ index scan, and finish with ORDER BY score DESC LIMIT. The "best field wins" rule is a disjunction_max (take the strongest matching field rather than summing them all), and the freshness boost is a set of constant-score date bands folded into that same scan. So recency is decided while the index is still skipping, never in a slow second pass over the results.

-- Simplified: one @@@ scan matches, scores with my rules, and stops early.
SELECT id
FROM   messages
WHERE  id @@@ paradedb.boolean(
  -- best field wins: take the strongest matching field, not the sum
  must => ARRAY[
    paradedb.disjunction_max(tie_breaker => 0.2, disjuncts => ARRAY[
      paradedb.boost(factor => 100, query => paradedb.term('name',    'refund')),
      paradedb.boost(factor => 70,  query => paradedb.term('subject', 'refund')),
      paradedb.boost(factor => 20,  query => paradedb.term('body',    'refund'))
    ])
  ],
  -- freshness: newer rows fall into more date bands, so they score higher
  should => ARRAY[
    paradedb.const_score(2000, paradedb.range('created_at', int8range(now_1d,  NULL))),
    paradedb.const_score(500,  paradedb.range('created_at', int8range(now_30d, NULL)))
  ])
ORDER BY paradedb.score(id) DESC
LIMIT 2000;

And here is the elegant part: all of this custom scoring rides inside the engine's single, efficient pass, the same one that skips the millions of hopeless matches. I did not trade away any speed to get my own sense of taste. I taught a world-class engine my preferences and let it do all the heavy lifting.

The numbers

Same corpus, same queries, a fraction of the hardware.

So did it actually work? I measured the new search against tens of thousands of our real, historical searches.

• Does it return the right thing? For 98.8% of real searches, the new search returned a genuinely correct, relevant result. The rare miss was almost always a vague one-word query where there is not really a single right answer.
• Does it match what people were used to? For the typical search, the results came back identical to Typesense's, position for position. The gaps that remain are mostly on broad single-word searches, where even Typesense's own ordering was more or less a coin toss. Nobody's day-to-day searches suddenly started looking wrong.
• Is it fast? Half of all searches come back in about an eighth of a second, and nine out of ten in under half a second, on a small, cheap server. In hard numbers, measured through the real application path on that 4 GB box: a median of 125 ms, a 90th percentile of 450 ms, and a 99th percentile of 1.3 seconds, against a 2.5-second database timeout. Searches slow enough to time out: about four in every ten thousand. Phone-number lookups, the single most common kind, never time out at all. This is the same engine that, in its raw form, took ninety seconds.

And here is the part that still amazes me. Look at the size of the machine each approach needed to do the exact same job.

I did not match the old search by throwing a similar machine at it. I did it on a box with less memory than either previous attempt, because Postgres serves from cheap disk instead of hoarding everything in pricey RAM. And it does that without the storage blowup either: the exact same corpus that Meilisearch ballooned into a 420 GB index sits in about 45 GB on Postgres, barely larger than the raw data itself, with the BM25 search index accounting for only about 15 GB of that. Same job, less than a tenth of the cost, and room to grow more or less forever.

The timebomb is defused.

Closing notes

• pg_search (ParadeDB) is the Postgres extension that embeds the Tantivy search engine and the BM25 index I built on.
• The original WAND paper (Broder et al., 2003) describes the "stop early" top-k algorithm that lets the engine skip most matches.
• Typesense and Meilisearch are the two search engines that came before, both excellent, both a poor fit for this particular workload.
• Postgres full-text search is the built-in option I tried first, and why "cannot stop early" was its dealbreaker at our scale.

Fun fact

Our new production search server runs on 4 GB of RAM, which is less than the phone in your pocket, and it serves search across hundreds of millions of messages. The whole ranking formula that two engineers and a year of false starts could not land fits in a single Postgres function, and the agents wrote and rewrote it dozens of times in one night. I reviewed the winner over coffee.

Have a nice day and happy coding!