Configuration Item | Description |
Database | RDS PostgreSQL 17 |
pgvector | 0.8.0 |
Instance Specifications | 16 vCPU / 64 GB |
Client | 4 vCPU / 8 GB (Same VPC) |
Test Tool | |
Stress Test Mode | Single-threaded |
Recall Metric | Recall@100 |
Dataset | Level | Vector Count | Distance Type | Typical Scenario |
dbpedia-openai-1M | 1536 | 1,000,000 | Cosine | OpenAI / Large Model Embedding |
NYTimes-256 | 256 | 290,000 | Cosine | Text similarity search |
git clone https://github.com/erikbern/ann-benchmarks.gitcd ann-benchmarkspip install -r requirements.txt
ann_benchmarks/algorithms/pgvector/module.py and add the connection information:import osos.environ['ANN_BENCHMARKS_PG_USER'] = '<username>'os.environ['ANN_BENCHMARKS_PG_PASSWORD'] = '<password>'os.environ['ANN_BENCHMARKS_PG_DBNAME'] = '<database name>'os.environ['ANN_BENCHMARKS_PG_HOST'] = '<instance private network address>'os.environ['ANN_BENCHMARKS_PG_PORT'] = '5432'os.environ['ANN_BENCHMARKS_PG_START_SERVICE'] = 'false'
ann_benchmarks/algorithms/pgvector/config.yml:float:any:- base_args: ['@metric']constructor: PGVectordisabled: falsedocker_tag: ann-benchmarks-pgvectormodule: ann_benchmarks.algorithms.pgvectorname: pgvectorrun_groups:M-16(200):arg_groups: [{M: 16, efConstruction: 200}]args: {}query_args: [[10, 20, 40, 80, 120, 200, 400, 800]]M-24(200):arg_groups: [{M: 24, efConstruction: 200}]args: {}query_args: [[10, 20, 40, 80, 120, 200, 400, 800]]M-48(256):arg_groups: [{M: 48, efConstruction: 256}]args: {}query_args: [[10, 20, 40, 80, 120, 200, 400, 800]]
python install.py --algorithm pgvector
# Testing with the dbpedia-openai-1M datasetpython run.py --dataset dbpedia-openai-1000k-angular -k 100 --algorithm pgvector --runs 1# Testing with the NYTimes-256 datasetpython run.py --dataset nytimes-256-angular -k 100 --algorithm pgvector --runs 1
# Generate visualization chartspython plot.py --dataset dbpedia-openai-1000k-angular --recompute# Export detailed CSV datapython data_export.py --out results.csv
Scenario | Index Configurations | ef_search | Recall@100 | QPS | P99(ms) |
High recall | m=24, ef_construction=200 | 120 | 99.2% | 225 | 7.2 |
Ultra-high recall | m=48, ef_construction=256 | 120 | 99.8% | 146 | 12.3 |
Scenario | Index Configurations | ef_search | Recall@100 | QPS | P99(ms) |
High recall | m=24, ef_construction=200 | 400 | 89.0% | 101 | 12.7 |
Ultra-high recall | m=24, ef_construction=200 | 800 | 93.9% | 52 | 23.6 |
피드백