Billion-scale similarity search with GPUs · HackerPapers discussions

Billion-scale similarity search with GPUs — HN submission

Thu, 04 Jun 2026 13:54:28 GMT

Inside FAISS: Billion-Scale Similarity Search

Author here. I wrote this as a visual companion to the 2017 FAISS paper (https://arxiv.org/abs/1702.08734), focused on the parts I found hardest to grok from text alone. The article covers a subset of what FAISS does, with the paper as the source of truth. NSG, FastScan, IMI are not covered here, they'll get their own articles. I'd be especially interested in feedback on: - the IVFPQ / IVFADC explanation, particularly the LUT reuse argument - whether the GPU part captures enough of the actual complexity Happy to answer questions.

74 points; 6 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Billion-scale similarity search with GPUs — mentioned by jhj

Fri, 12 Jan 2024 23:19:49 GMT

The implementations that I am aware of, including the one in Faiss which I wrote (described in detail in https://arxiv.org/abs/1702.08734), do not index the vector based on its PQ encoding (e.g., in IVFPQ). The IVF cell chosen in which to put the vector is based on its pre-quantized (full precision floating point) representation. It would lose too much precision to perform all comparisons in the compressed space. Also, distance comparisons are usually of the "ADC" (asymmetric distance comparison) form: the query vector is in full floating point format, and vectors, if quantized/compressed in…

Paper and discussion history. A mention is not an endorsement.

Billion-scale similarity search with GPUs — HN submission

Wed, 24 May 2023 03:33:10 GMT

Billion-scale similarity search with GPUs (2017)

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Billion-scale similarity search with GPUs — mentioned by PaulHoule

Sat, 11 Mar 2023 12:44:51 GMT

Depends on what you want to optimize for. See this paper https://arxiv.org/abs/1702.08734 And this library that it describes https://github.com/facebookresearch/faiss Which is an optimal use of your time as you can install it in a minute with anaconda if you use Python.

Paper and discussion history. A mention is not an endorsement.

Billion-scale similarity search with GPUs — HN submission

Tue, 23 Feb 2021 17:09:49 GMT

Billion-scale similarity search with GPUs (FB AI research, 2017)

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Billion-scale similarity search with GPUs — HN submission

Mon, 06 Mar 2017 02:07:52 GMT

Billion-scale similarity search with GPUs

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Billion-scale similarity search with GPUs — HN submission

Sat, 04 Mar 2017 13:02:06 GMT

Billion-scale Similarity Search with GPUs

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.