Commit Graph
487 Commits
Author SHA1 Message Date
Andrew Kane 8b9dc3fcd2 Mark type-specific code 2024-03-29 14:01:48 -07:00
Andrew Kane b1a3a92017 Use variable for max dimemsions [skip ci] 2024-03-29 10:57:16 -07:00
Andrew Kane 6441604380 Added todo [skip ci] 2024-03-29 10:56:24 -07:00
Andrew Kane bf6efbc639 Removed vector-specific code from HNSW 2024-03-29 10:50:06 -07:00
Andrew Kane a2a56659dc Improved code [skip ci] 2024-03-27 21:38:22 -07:00
Andrew Kane 7070f9d8a2 Removed normvec from IVFFlat for simplicity (no difference in performance) 2024-03-27 16:41:17 -07:00
Andrew Kane 93db940d65 Removed normvec for simplicity (no difference in performance) 2024-03-27 16:33:11 -07:00
Andrew Kane 3190b6a9a4 Updated comment [skip ci] 2024-03-15 12:38:14 -07:00
Andrew Kane f2122604bd Updated comment [skip ci] 2024-03-15 12:37:47 -07:00
Heikki LinnakangasandGitHub 869c6df7aa Fix compiler warnings in strict C99 mode (#487)
Redefining a typedef is a C11 feature:

    In file included from src/hnsw.c:10:
    src/hnsw.h:147:5: warning: redefinition of typedef 'HnswElementData' is a C11 feature [-Wtypedef-redefinition]
    }                       HnswElementData;
                            ^
    src/hnsw.h:118:32: note: previous definition is here
    typedef struct HnswElementData HnswElementData;
                                   ^
    src/hnsw.h:163:5: warning: redefinition of typedef 'HnswNeighborArray' is a C11 feature [-Wtypedef-redefinition]
    }                       HnswNeighborArray;
                            ^
    src/hnsw.h:119:34: note: previous definition is here
    typedef struct HnswNeighborArray HnswNeighborArray;
                                     ^
    2 warnings generated.

I got these warnings when I built PostgreSQL with "CC=clang
CFLAGS=-std=gnu99"; other similar options would surely produce the
warnings too.
2024-03-12 02:02:33 -07:00
Andrew Kane 92a76df149 Reduced lock contention with parallel HNSW index builds 2024-03-11 20:16:55 -07:00
Andrew Kane 7b84dbb395 Fixed sort function for Postgres 12 2024-02-28 16:26:41 -08:00
Andrew KaneandHeikki Linnakangas ac280f83d2 Replaced pairing heap with array in SelectNeighbors - closes #447
Co-authored-by: Heikki Linnakangas <[email protected]>
2024-02-28 15:47:26 -08:00
Andrew Kane ae829a5b48 Fixed closer caching for Postgres 12 2024-02-28 15:44:38 -08:00
Andrew Kane f8bdb41964 Avoid base address for relptr for Postgres < 14.5 2024-02-28 14:10:14 -08:00
Andrew Kane d99252fec5 Revert "Replaced pairing heap with array in SelectNeighbors - closes #447"
This reverts commit 96764dbf79.
2024-02-28 11:33:14 -08:00
Andrew KaneandHeikki Linnakangas 96764dbf79 Replaced pairing heap with array in SelectNeighbors - closes #447
Co-authored-by: Heikki Linnakangas <[email protected]>
2024-02-28 11:25:12 -08:00
Andrew Kane 01ba9934c2 Revert "Remove offsethash"
This reverts commit e3b904587a.
2024-02-20 16:07:32 -08:00
Andrew Kane a37413b5f4 Revert "Eliminate a few HnswPtrAccess invocations"
This reverts commit dac698ac76.
2024-02-20 16:07:21 -08:00
Heikki Linnakangas dac698ac76 Eliminate a few HnswPtrAccess invocations
HnswPtrAccess() is pretty cheap, but it stills seems worthwhile to
avoid repeated calls in the hot paths when it can be easily avoided.
2024-02-19 14:13:18 +02:00
Heikki Linnakangas e3b904587a Remove offsethash
The original motivation was to eliminate the superfluous HnswPtrAccess
call from AddToVisited. The caller has to call HnswPtrAccess() anyway,
so it makes sense to pass the HnswElement rather than HnswElementPtr
to AddToVisited(). But then I realized that we can use the
pointer-variant even with shared memory, because the visited-hash is
backend-private, and the addresses where the elements are mapped to in
shared memory are stable within the backend.
2024-02-19 14:12:26 +02:00
Andrew Kane 8a4efc951a Fixed crash with shared_preload_libraries - fixes #460 2024-02-14 17:13:30 -08:00
Andrew Kane 415c286b6d Improved type [skip ci] 2024-02-05 19:38:52 -08:00
Andrew Kane dc3923c3a2 Fixed error with ANALYZE and vectors with different dimensions - fixes #451 2024-02-02 10:47:48 -08:00
Andrew Kane c5dcebff39 Removed unneeded item ids 2024-01-28 23:14:52 -08:00
Andrew Kane 94d29ba178 Ran pgindent [skip ci] 2024-01-28 23:03:48 -08:00
Andrew Kane f61ca86ba9 Updated variable name to be consistent with CreateGraphPages [skip ci] 2024-01-28 23:02:13 -08:00
Andrew Kane 04d861fb3e Zero memory for each element 2024-01-28 23:00:39 -08:00
Andrew Kane d4d42401b2 DRY HNSW tuple alloc size [skip ci] 2024-01-28 22:58:38 -08:00
Heikki LinnakangasandGitHub 0fb2113acc Remove unnecessary PageIndexTupleOverwrite calls that caused UB (#438)
These places called PageIndexTupleOverwrite(), with the new tuple
pointing directly to the original page. The PageIndexTupleOverwrite()
call is unnecessary in these cases, as we have already modified the
tuple on the page directly. Moreover, PageIndexTupleOverWrite() will
call memcpy with same src and dst arguments, which is undefined
behavior.

It's OK to modify the pages on disk directly, and we don't need
critical sections, because we either use the generic xlog functions
which create a temporary copy of the page, or we are building a new
index so if we crash the whole index is invisible and will be dropped
anyway.
2024-01-28 22:52:56 -08:00
Andrew Kane 0646799583 Revert "Update neighbor tuples in-place for HNSW index build"
This reverts commit d3d425f429.
2024-01-28 22:18:22 -08:00
Andrew Kane f0cdc6364b Improved function names 2024-01-28 21:00:12 -08:00
Andrew Kane d3d425f429 Update neighbor tuples in-place for HNSW index build 2024-01-28 20:53:56 -08:00
Andrew Kane 7cf767add7 Fixed Valgrind check for HNSW in-memory, parallel index builds - closes #441 2024-01-28 16:19:32 -08:00
Andrew Kane feee79486d Removed checking neighbors for cached distance 2024-01-28 02:11:24 -08:00
Andrew Kane 9859ea168e Fixed uninitialized memory in scan->xs_recheck [skip ci] 2024-01-27 19:54:16 -08:00
Andrew Kane 614368f543 Fixed possibly uninitialized memory in lists [skip ci] 2024-01-27 19:51:34 -08:00
Andrew Kane 1913881a2a Fixed uninitialized entry point level - fixes #439 and closes #440 2024-01-27 19:49:40 -08:00
Heikki LinnakangasandGitHub a1c670ffcc Remove unused heapRel arguments (#443) 2024-01-26 12:45:10 -08:00
Heikki LinnakangasandGitHub 74199312e8 Remove unnecessary UpdateProgress() wrappers (#433)
Now that we require PostgreSQL v12, we can use
pgstat_progress_update_param directly.
2024-01-25 00:07:57 -08:00
Andrew Kane 536239762d DRY neighbor array size [skip ci] 2024-01-24 17:54:33 -08:00
Andrew Kane 2b2cee365c Condensed code [skip ci] 2024-01-24 16:20:00 -08:00
Andrew Kane ed457dafd5 Moved allocating neighbor array to separate function [skip ci] 2024-01-24 16:17:34 -08:00
Heikki Linnakangas 1d3d77c541 Include generic_xlog.h directly in the .c files where it's needed
There are no references to anything that's in generic_xlog.h in the
header files.
2024-01-23 13:04:03 +02:00
Heikki Linnakangas 717a4c170b Include reloptions.h directly in the .c files where it's needed
There are no references to anything that's in reloptions.h in the
header files. They need to include genam.h instead, which defines
IndexScanDesc.
2024-01-23 13:02:24 +02:00
Heikki Linnakangas a29260d008 Add direct include to pairingheap.h in headers
ivfflat.h and hnsw.h have references to pairingheap_node, so they need
to include lib/pairingheap.h. It happened to work, because
lib/pairingheap.h was being included indirectly through
nodes/execnodes.h, but let's be explicit.

Remove the include from hnswbuild.c, because there are no calls to
pairingheap functions in that file. Instead, add the includes to
hnswutils.c and ivfscan.c, which do have such calls. They are not
strictly necessary again because of the indirect include from hnsw.h
and ivfflat.h, but let's be explicit while we're messing with this.
2024-01-23 12:53:22 +02:00
Heikki Linnakangas 38ba8f7071 Remove unused #include
pg_list.h has not been used in hnswbuild.c since commit 0a7fd04b5c.
2024-01-23 12:53:13 +02:00
Andrew Kane 145c3da384 Added validation for GUC parameters 2024-01-22 23:55:30 -08:00
Andrew Kane 43fb62b3d3 Dropped support for Postgres 11 2024-01-22 23:52:54 -08:00
Andrew Kane 03a72362a3 Leave more space for other shared memory 2024-01-22 23:31:55 -08:00