Skip to content

Benchmarks

How SEMQ's operators compare with the vector formats a retrieval pipeline already uses: how much search quality survives, how many bytes each vector takes, and how fast it encodes. Every number comes from a frozen result file with its inputs, commit and hardware.

10.7×smaller than FP32 · SEMQ quant at 3 bits
95–100%of FP32 search quality kept · six runs
798kvectors encoded per second · 768 dimensions

Search quality kept

For each format, the line spans the lowest and the highest nDCG@10, relative to FP32, across three embedding models and two datasets (SciFact and FiQA). Formats are ordered by storage, fewest bits per dimension first. SEMQ is in blue.

40% 60% 80% 100% Binary · 1 bit/dim: 38.4% to 98.7% of FP32 nDCG@10 across 6 runs Binary · 1 bit/dim 38–99% SEMQ phase · 2 bits/dim: 75.5% to 99.6% of FP32 nDCG@10 across 6 runs SEMQ phase · 2 bits/dim 75–100% SEMQ quant · 2 bits/dim: 81.8% to 99.4% of FP32 nDCG@10 across 6 runs SEMQ quant · 2 bits/dim 82–99% Faiss PQ · 2.0–2.1 bits/dim: 94.9% to 100.5% of FP32 nDCG@10 across 6 runs Faiss PQ · 2.0–2.1 bits/dim 95–101% TurboQuant · 2.3–8.4 bits/dim: 81.9% to 99.9% of FP32 nDCG@10 across 6 runs TurboQuant · 2.3–8.4 bits/dim 82–100% SEMQ quant · 3 bits/dim: 94.7% to 100.5% of FP32 nDCG@10 across 6 runs SEMQ quant · 3 bits/dim 95–100% SEMQ quant · 4 bits/dim: 98.8% to 100.0% of FP32 nDCG@10 across 6 runs SEMQ quant · 4 bits/dim 99–100% RaBitQ · 4.6–10.5 bits/dim: 99.5% to 101.0% of FP32 nDCG@10 across 6 runs RaBitQ · 4.6–10.5 bits/dim 100–101% SEMQ orbit · 8 bits/dim: 97.4% to 100.2% of FP32 nDCG@10 across 6 runs SEMQ orbit · 8 bits/dim 97–100% INT8 · 8 bits/dim: 99.8% to 100.4% of FP32 nDCG@10 across 6 runs INT8 · 8 bits/dim 100–100% nDCG@10 kept relative to FP32 · the line spans the lowest and highest of six runs
Table view
Format Bits per dimension Lowest Highest
Binary, sentence-transformers 1 38.4% 98.7%
SEMQ phase · 16 sectors 2 75.5% 99.6%
SEMQ quant · 2 bits 2 81.8% 99.4%
Faiss PQ · 2 bits 2.0–2.1 94.9% 100.5%
TurboQuant · 2 bits 2.3–8.4 81.9% 99.9%
SEMQ quant · 3 bits 3 94.7% 100.5%
SEMQ quant · 4 bits 4 98.8% 100.0%
RaBitQ · 4 bits 4.6–10.5 99.5% 101.0%
SEMQ orbit · scale 50 8 97.4% 100.2%
INT8, sentence-transformers 8 99.8% 100.4%

A value above 100% means the ranking scored slightly better against the available relevance judgments; it is not a quality improvement in general. Queries search decoded rows by cosine, with no rescoring.

Explore one run

Pick a model and a dataset to see every measured format at its storage cost.

Loading measured results…

Storage per vector

Bytes per 768-dimension vector, codes only. A .semq file adds an 8-byte id per row and a fixed header; Faiss PQ also stores a codebook once.

0 1,000 2,000 3,000 4,000 Binary: 96 B bytes per vector Binary 96 B SEMQ quant · 2 bits: 192 B bytes per vector SEMQ quant · 2 bits 192 B SEMQ phase · 16 sectors: 192 B bytes per vector SEMQ phase · 16 sectors 192 B Faiss PQ: 192 B bytes per vector Faiss PQ 192 B SEMQ quant · 3 bits: 288 B bytes per vector SEMQ quant · 3 bits 288 B SEMQ quant · 4 bits: 384 B bytes per vector SEMQ quant · 4 bits 384 B SEMQ orbit · scale 50: 768 B bytes per vector SEMQ orbit · scale 50 768 B INT8: 772 B bytes per vector INT8 772 B FP16: 1,536 B bytes per vector FP16 1,536 B FP32: 3,072 B bytes per vector FP32 3,072 B bytes per vector
Table view
Format Bytes per vector Smaller than FP32 GB for 100M vectors
Binary 96 32.0× 9.6
SEMQ quant · 2 bits 192 16.0× 19.2
SEMQ phase · 16 sectors 192 16.0× 19.2
Faiss PQ 192 16.0× 19.2
SEMQ quant · 3 bits 288 10.7× 28.8
SEMQ quant · 4 bits 384 8.0× 38.4
SEMQ orbit · scale 50 768 4.0× 76.8
INT8 772 4.0× 77.2
FP16 1,536 2.0× 153.6
FP32 3,072 1.0× 307.2

Encode and decode speed

Vectors per second for 768-dimension rows, measured from Python on one arm64 CPU with the neon backend, median of five runs. Encode includes the unit-norm check and the state's digests.

0 2.0M 4.0M 6.0M SEMQ quant · 2 bits SEMQ quant · 2 bits, encode: 875,934 vectors per second 876k SEMQ quant · 2 bits, decode: 1,165,874 vectors per second 1.2M SEMQ quant · 3 bits SEMQ quant · 3 bits, encode: 798,074 vectors per second 798k SEMQ quant · 3 bits, decode: 902,340 vectors per second 902k SEMQ quant · 4 bits SEMQ quant · 4 bits, encode: 795,248 vectors per second 795k SEMQ quant · 4 bits, decode: 712,510 vectors per second 713k SEMQ phase · 16 sectors SEMQ phase · 16 sectors, encode: 691,664 vectors per second 692k SEMQ phase · 16 sectors, decode: 6,473,382 vectors per second 6.5M SEMQ orbit · scale 50 SEMQ orbit · scale 50, encode: 618,763 vectors per second 619k SEMQ orbit · scale 50, decode: 2,319,688 vectors per second 2.3M Encode Decode Vectors per second, 768 dimensions, one thread from Python
Table view
Format Encode, vectors/s Decode, vectors/s
SEMQ quant · 2 bits 875,934 1,165,874
SEMQ quant · 3 bits 798,074 902,340
SEMQ quant · 4 bits 795,248 712,510
SEMQ phase · 16 sectors 691,664 6,473,382
SEMQ orbit · scale 50 618,763 2,319,688
NumPy binary 6,536,037 3,060,038
NumPy fp16 22,920,225 25,037,820
NumPy int8 1,472,129 3,311,098

The table also lists plain NumPy casts to FP16, INT8 and binary for scale: they convert arrays and compute no identity.

Choose a configuration

Configuration Bits per dimension Bytes per vector (768) Search quality kept Encode, vectors/s
SEMQ quant · 2 bits 2 192 81.8–99.4% 876k
SEMQ quant · 3 bits 3 288 94.7–100.5% 798k
SEMQ quant · 4 bits 4 384 98.8–100.0% 795k
SEMQ phase · 16 sectors 2 192 75.5–99.6% 692k
SEMQ orbit · scale 50 8 768 97.4–100.2% 619k

Quality data · Storage data · Throughput data · Extended evaluation with uncertainty intervals · Methodology

Other measurements