A vector database gets sold as a new tier of infrastructure you have to go stand up. Strip off the marketing and you have a feature: nearest-neighbor search over a column of floats, which the database most teams already run will do with a single extension. Three vector databases raised money this week. I lost count of the launch posts. Each comes with a benchmark where it wins, a logo wall, and a tutorial that has you running new infrastructure before you have written a single eval. The pitch never changes: embeddings are the future, and you need a real database built for them.
You probably don’t. Not yet.
Most teams reaching for a vector database this year are adding semantic search or a RAG feature to a product that already runs on Postgres. They have a few hundred thousand chunks, maybe a couple million. One team, a few on-call engineers, a finite appetite for new systems to babysit. For that team, the honest answer is to put the vectors in the Postgres you already operate, with the pgvector extension, and not think about a dedicated store until the numbers force you to.
I have been on both sides of this line. On a marketplace I led, the search workloads eventually justified specialized infrastructure, and we built it. But the first version of nearly every embedding feature I have shipped started in Postgres, and a surprising number never had a good reason to leave.
What pgvector actually is
The extension adds a vector column type to Postgres and a couple of index types for nearest-neighbor search, and that is the whole of it. Your embeddings live in a normal table, next to the columns you already have: the document id, the tenant id, the created_at, the soft-delete flag. You query them with SQL.
That last part is the whole argument, so let me make it concrete.
create extension if not exists vector;
create table doc_chunks (
id bigserial primary key,
document_id bigint not null references documents(id),
tenant_id bigint not null,
chunk_text text not null,
embedding vector(1536), -- ada-002 dims; change if you swap models
created_at timestamptz not null default now()
);
-- IVFFlat, not exact search. ANN is a recall/speed trade, not a bug.
-- Build this AFTER you have a meaningful amount of data, not on an empty table.
create index on doc_chunks
using ivfflat (embedding vector_cosine_ops)
with (lists = 100);The query that everyone writes a dedicated database to do:
-- `<=>` is cosine distance. Smaller is closer.
-- Note the tenant filter sits right next to the vector search,
-- in the same WHERE, hitting the same table. That's the point.
select id, document_id, chunk_text,
embedding <=> $1 as distance
from doc_chunks
where tenant_id = $2
order by embedding <=> $1
limit 8;There is no second system in that query. No copy of the data kept in sync. No “now go look up the row in Postgres” round trip after the vector search hands you back a list of ids. The tenant filter, the join back to documents, the created_at you sort by when two chunks tie, all of it is right there in the database that already holds your source of truth.
The launch posts benchmark the wrong thing. In production RAG the hard part is almost never raw nearest-neighbor speed but filtered search: “the closest chunks that this user is allowed to see, from documents not archived, in this workspace.” A dedicated vector store treats your metadata as a second-class citizen bolted onto the side of the index. Postgres treats it as the main event, because that is what Postgres is.
The tradeoff, stated honestly
A dedicated vector database will give you better recall at a given latency once your index gets large, and it will hold that under heavy concurrent query load better than Postgres will. That advantage is real: the HNSW and IVF implementations in the purpose-built stores are tuned harder than what ships in an extension, and they were designed for this one job.
The cost is a whole system: a new datastore to provision, secure, back up, monitor, version, and keep in sync with your primary database. Every embedding update now has two write paths and a chance to drift. Every “why is this document showing up for the wrong user” bug now spans two systems instead of one. That sync layer is unglamorous, it is where the incidents come from, and nobody benchmarks it.
The question is not “which is faster.” At small scale they are both fast enough that your network and your LLM call dominate the latency budget anyway. The real question is whether you are at the scale where the recall-under-load gap is large enough to be worth operating a second system to close.
For most teams, in 2023, the answer is no. A few million vectors with an IVFFlat index, behind a sane filter, returns in single-digit to low-double-digit milliseconds on hardware you are already paying for. Your embedding API call takes longer than the search does. Optimizing the search at that point is optimizing the wrong thing.
Where pgvector starts to hurt
I am not selling pgvector as the answer forever. It has edges, and you should know where they are before you commit, because nothing is worse than discovering them in an incident.
Index builds are not free and not online in the way you want. IVFFlat needs to see data to pick its cluster centroids, so you build the index after loading, and rebuilding on a large table locks you up for a while. The lists parameter is a real tuning knob: too few and you scan too much, too many and recall drops because each list holds too little, and nothing chooses it for you. (I have watched a team set lists = 1000 on a table of 50,000 rows and wonder why recall fell off a cliff. Each list had fifty vectors in it.)
Recall is a dial you set by hand. With IVFFlat you tune probes at query time, trading latency for how many lists you actually search. Set it once during a calm afternoon and it will degrade as your data grows and your distribution shifts, so you have to keep watching it.
And Postgres is one box, mostly. You can read-replica the search load, but the vectors and the index live with everything else competing for the same buffer cache and the same CPU. A heavy embedding workload sharing a database with your transactional traffic is a noisy-neighbor problem waiting to happen. At some volume, isolation alone justifies moving the vectors out, separate from any recall argument.
When the dedicated DB genuinely earns it
There is a real line, and crossing it is not a failure. Reach for a purpose-built vector database when:
- You are past tens of millions of vectors and growing, and the IVFFlat recall-versus-latency curve has stopped being good enough no matter how you tune it.
- Filtered ANN is your hot path and you need the filter applied during the search, not as a pre-scan or a post-filter that throws away most of your candidates. The good dedicated stores do this well, and at high cardinality it matters.
- You need an index type or a quantization scheme that the extension does not have, because memory at your vector count has become the binding constraint.
- The search workload is large and bursty enough that isolating it from your primary database is worth a system on its own.
None of those are about hype; each is a workload that has outgrown the convenient answer. When you hit them, the dedicated database is the right call, and the engineers who built those products built them for exactly this. Be fair to them: they are solving a genuinely hard problem at the top end. Most teams are just not at the top end yet, and adopting the top-end solution on day one buys you the operational cost without the workload that justifies it.
How to actually decide
Start in Postgres. Add the extension, write the schema above, ship the feature. Keep a few hundred real queries with answers you trust, and measure two things on every change: recall against those queries, and p95 latency under something like your real concurrency. Run those measurements on your data, with your filters, not on a vendor’s benchmark.
Let the numbers move you from there. When recall at acceptable latency stops being achievable in Postgres after honest tuning, or when the search load starts hurting your transactional traffic, that is your signal to migrate. By then you will know what your workload looks like, what filters you actually run, and what recall you actually need, which means you will pick the right dedicated store instead of the loudest one.
The gold rush sells the destination before anyone has measured the trip. Measured honestly, most of these features never leave Postgres. For the teams running a few million vectors behind a real filter, the boring extension on the database they already operate does the job, and one fewer system to run beats a benchmark they were never going to hit.