Tune Vamana Search¶

Vamana는 L2 distance 기반 workload의 ANN 인덱스입니다.

이 예제에서는 vector_l2_ops를 사용하고, build-time parameter인 indexing.vamana.l_search와 search-time parameter인 vectoron.vamana_l_search를 별도로 설정합니다.

DROP TABLE IF EXISTS akasicdb_example_research_notes;

CREATE TABLE akasicdb_example_research_notes (
  id integer PRIMARY KEY,
  topic text NOT NULL,
  embedding vector(4) NOT NULL
);

INSERT INTO akasicdb_example_research_notes (id, topic, embedding) VALUES
  (1, 'vector index overview', ARRAY[0.10,0.80,0.05,0.00]::real[]::vector),
  (2, 'ann search tuning', ARRAY[0.12,0.78,0.06,0.01]::real[]::vector),
  (3, 'vamana graph notes', ARRAY[0.09,0.82,0.04,0.00]::real[]::vector),
  (4, 'query latency review', ARRAY[0.15,0.74,0.08,0.02]::real[]::vector),
  (5, 'recall measurement', ARRAY[0.18,0.70,0.07,0.02]::real[]::vector),
  (6, 'filter selectivity', ARRAY[0.20,0.65,0.10,0.03]::real[]::vector),
  (7, 'index operations', ARRAY[0.24,0.60,0.12,0.03]::real[]::vector),
  (8, 'tenant search design', ARRAY[0.26,0.56,0.16,0.04]::real[]::vector),
  (9, 'product taxonomy', ARRAY[0.30,0.50,0.20,0.05]::real[]::vector),
  (10, 'document ranking', ARRAY[0.32,0.48,0.22,0.05]::real[]::vector),
  (11, 'graph traversal', ARRAY[0.50,0.30,0.35,0.08]::real[]::vector),
  (12, 'storage checkpoint', ARRAY[0.55,0.25,0.40,0.10]::real[]::vector),
  (13, 'backup planning', ARRAY[0.60,0.20,0.42,0.12]::real[]::vector),
  (14, 'access control', ARRAY[0.65,0.18,0.44,0.14]::real[]::vector),
  (15, 'event ingestion', ARRAY[0.70,0.16,0.46,0.16]::real[]::vector),
  (16, 'analytics refresh', ARRAY[0.74,0.14,0.48,0.18]::real[]::vector),
  (17, 'search api design', ARRAY[0.16,0.76,0.10,0.02]::real[]::vector),
  (18, 'ranking eval', ARRAY[0.14,0.79,0.09,0.01]::real[]::vector),
  (19, 'vector schema', ARRAY[0.11,0.81,0.03,0.00]::real[]::vector),
  (20, 'index memory review', ARRAY[0.22,0.62,0.13,0.03]::real[]::vector),
  (21, 'content moderation', ARRAY[0.40,0.40,0.28,0.07]::real[]::vector),
  (22, 'stream processing', ARRAY[0.78,0.12,0.50,0.20]::real[]::vector),
  (23, 'catalog sync', ARRAY[0.34,0.46,0.24,0.06]::real[]::vector),
  (24, 'observability plan', ARRAY[0.28,0.58,0.18,0.04]::real[]::vector),
  (25, 'sql migration', ARRAY[0.62,0.22,0.43,0.11]::real[]::vector),
  (26, 'read replica notes', ARRAY[0.66,0.19,0.45,0.13]::real[]::vector),
  (27, 'feature flags', ARRAY[0.72,0.15,0.47,0.17]::real[]::vector),
  (28, 'dashboard refresh', ARRAY[0.76,0.13,0.49,0.19]::real[]::vector),
  (29, 'metadata filters', ARRAY[0.21,0.64,0.11,0.03]::real[]::vector),
  (30, 'distance operators', ARRAY[0.13,0.77,0.05,0.01]::real[]::vector),
  (31, 'index build queue', ARRAY[0.25,0.59,0.14,0.03]::real[]::vector),
  (32, 'service limits', ARRAY[0.68,0.17,0.46,0.15]::real[]::vector);

CREATE INDEX akasicdb_research_notes_vamana_idx
ON akasicdb_example_research_notes
USING vectoron (embedding vector_l2_ops)
WITH (options = $$
[indexing.vamana]
alpha = 1.2
r = 16
l_search = 32
$$);

SELECT fence_vector_index('akasicdb_research_notes_vamana_idx'::regclass);

BEGIN;
SET LOCAL vectoron.vamana_l_search = 80;

SELECT id, topic
FROM akasicdb_example_research_notes
ORDER BY embedding <-> ARRAY[0.10, 0.80, 0.05, 0.00]::real[]::vector
LIMIT 5;

COMMIT;

Settings¶

Setting	적용 시점	설명
`indexing.vamana.alpha`	Index build	graph pruning을 제어합니다. recall, build time, index size에 미치는 영향은 dataset에 따라 달라집니다.
`indexing.vamana.r`	Index build	각 node에 남기는 neighbor 수입니다. 값이 클수록 graph edge, memory use, build cost가 증가합니다.
`indexing.vamana.l_search`	Index build	index 구성 중 candidate search width입니다. 값이 클수록 후보 탐색량과 build cost가 증가합니다.
`vectoron.vamana_l_search`	Query execution	Vamana search 중 candidate exploration width입니다. 값이 클수록 더 많은 후보를 평가하므로 CPU 작업량과 latency가 증가할 수 있습니다.

Build-time option을 바꾸려면 인덱스를 다시 만들어야 합니다. SET LOCAL은 transaction 안에서만 적용됩니다.

vectoron.vamana_l_search 값을 조정할 때에는 recall과 latency 변화를 확인하며 적절한 값을 찾으시기 바랍니다.