In RDS for PostgreSQL 15.2, how do you set the number of probes for the pgvector extension?

0

For context see announcement of pgvector support in the RDS for PostgreSQL 15.2.

pgvector has a parameter called probes or ivfflat.probes. However when editing parameters for my database instance, I don't see a parameter containing "probes". The number of probes is critical for taking advantage of the ivfflat index using pgvector (see pgvector documentation for more detail). How do you set this parameter?

posta un anno fa631 visualizzazioni
1 Risposta
0

Hi there.

I'm reading the documentation and it seems this is set along with your code. It is a query option.

By default, pgvector performs exact nearest neighbor search, which provides perfect recall. You can add an index to use approximate nearest neighbor search, which trades some recall for performance. Unlike typical indexes, you will see different results for queries after adding an approximate index. Three keys to achieving good recall are:

  1. Create the index after the table has some data
  2. Choose an appropriate number of lists - a good place to start is rows / 1000 for up to 1M rows and sqrt(rows) for over 1M rows
  3. When querying, specify an appropriate number of probes (higher is better for recall, lower is better for speed) - a good place to start is sqrt(lists)

Add an index for each distance function you want to use.

This way, in your code you would just execute something like

BEGIN;
SET LOCAL ivfflat.probes = 10;
SELECT <your query>
COMMIT;

This doesn't look like a setting at the instance/database level but you set it on each query.

I hope this helps.

profile pictureAWS
ESPERTO
con risposta un anno fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande