Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supabase CLI (Docker local) pg_vector with index crashes Postgres #1074

Open
2 tasks done
3dteemu opened this issue Jul 25, 2024 · 4 comments
Open
2 tasks done

Supabase CLI (Docker local) pg_vector with index crashes Postgres #1074

3dteemu opened this issue Jul 25, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@3dteemu
Copy link

3dteemu commented Jul 25, 2024

Bug report

  • I confirm this is a bug with Supabase, not with my own application.
  • I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

Supabase (or more specifically Postgres) crashes whenever an indexing happens on a vector column.

Error in the Docker logs: server process (PID 604) was terminated by signal 4: Illegal instruction

To Reproduce

Steps to reproduce the behavior:

  1. npx supabase init
  2. npx supabase start
  3. Run docker logs <db container> -f and observe the logs
  4. Open the Studio in the browser, navigate to SQL editor
  5. Execute the following:
create extension if not exists vector
with schema extensions;

create table
  vector_test (
    id bigint generated by default as identity primary key,
    content text,
    embedding vector (10)
  );

create index on vector_test using hnsw (embedding vector_ip_ops);

All should be ok now.

  1. Execute the following:
insert into vector_test (content, embedding) values ('sample text', '[1,2,3,4,5,6,7,8,9,10]');
  • Observe: Error: Connection terminated unexpectedly
  • The Docker logs will show something like this:
2024-07-25 12:24:48.727 UTC [10] LOG:  server process (PID 604) was terminated by signal 4: Illegal instruction
2024-07-25 12:24:48.727 UTC [10] DETAIL:  Failed process was running: 
	-- source: dashboard
	-- user: self host
	-- date: 2024-07-25T12:24:48.555Z
	
	insert into vector_test (content, embedding) values ('sample text', '[1,2,3,4,5,6,7,8,9,10]')

If you switch the order of the index creation and the insert, i.e. first insert data then create the index, the same error happens.
The same error happens also with the ivfflat index and all distance operators.

Expected behavior

An index is created successfully.

System information

  • OS: Ubuntu 24.04 LTS
  • Linux xx 6.8.0-38-generic #38-Ubuntu SMP PREEMPT_DYNAMIC Fri Jun 7 15:25:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
  • Intel® Core™ i7-3930K × 12
  • Docker version: 27.0.3, build 7d4bcd8 (Docker running in rootless mode)
  • Supabase CLI version: 1.187.3
  • Images
public.ecr.aws/supabase/studio:20240701-05dfbec
public.ecr.aws/supabase/postgres-meta:v0.83.2 
public.ecr.aws/supabase/edge-runtime:v1.55.0
public.ecr.aws/supabase/imgproxy:v3.8.0
public.ecr.aws/supabase/storage-api:v1.0.6
public.ecr.aws/supabase/postgrest:v12.2.0 
public.ecr.aws/supabase/realtime:v2.29.15
public.ecr.aws/supabase/inbucket:3.0.3
public.ecr.aws/supabase/gotrue:v2.151.0
public.ecr.aws/supabase/kong:2.8.1
public.ecr.aws/supabase/postgres:15.1.1.61

Additional context

The above works on hosted Supabase.

@3dteemu 3dteemu added the bug Something isn't working label Jul 25, 2024
@olirice
Copy link
Contributor

olirice commented Aug 7, 2024

I wasn't able to reproduce this locally with supabase/postgres:15.1.1.61

Could you please try

docker run -it --name supag -p 5535:5432 -d -e POSTGRES_DB=postgres -e POSTGRES_PASSWORD=local_pw -e POSTGRES_USER=postgres -d supabase/postgres:15.1.1.61

then connect on 5535 and try running your setup script

create extension if not exists vector
with schema extensions;

create table
  vector_test (
    id bigint generated by default as identity primary key,
    content text,
    embedding vector (10)
  );

create index on vector_test using hnsw (embedding vector_ip_ops);

insert into vector_test (content, embedding) values ('sample text', '[1,2,3,4,5,6,7,8,9,10]');

and let me know what result you get

@3dteemu
Copy link
Author

3dteemu commented Aug 12, 2024

I followed the instructions and it seems to crash there too:

postgres=> create extension if not exists vector
with schema extensions;
CREATE EXTENSION

postgres=> create table
  vector_test (
    id bigint generated by default as identity primary key,
    content text,
    embedding vector (10)
  );
CREATE TABLE

postgres=> create index on vector_test using hnsw (embedding vector_ip_ops);
CREATE INDEX

postgres=> insert into vector_test (content, embedding) values ('sample text', '[1,2,3,4,5,6,7,8,9,10]');
server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
The connection to the server was lost. Attempting reset: Failed.

This is what the docker logs show when inserting the data:

2024-08-12 05:52:55.278 UTC [1] LOG:  server process (PID 1687) was terminated by signal 4: Illegal instruction
 2024-08-12 05:52:55.278 UTC [1] DETAIL:  Failed process was running: insert into vector_test (content, embedding) values ('sample text', '[1,2,3,4,5,6,7,8,9,10]');
 2024-08-12 05:52:55.278 UTC [1] LOG:  terminating any other active server processes
172.17.0.1 2024-08-12 05:52:55.280 UTC [1777] postgres@postgres FATAL:  the database system is in recovery mode
 2024-08-12 05:52:55.283 UTC [1] LOG:  all server processes terminated; reinitializing
 2024-08-12 05:52:55.350 UTC [1778] LOG:  database system was interrupted; last known up at 2024-08-12 05:49:49 UTC
 2024-08-12 05:52:55.454 UTC [1778] LOG:  database system was not properly shut down; automatic recovery in progress
 2024-08-12 05:52:55.456 UTC [1778] LOG:  redo starts at 0/1910A48
 2024-08-12 05:52:55.456 UTC [1778] LOG:  invalid record length at 0/1910A80: wanted 24, got 0
 2024-08-12 05:52:55.456 UTC [1778] LOG:  redo done at 0/1910A48 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
 2024-08-12 05:52:55.461 UTC [1779] LOG:  checkpoint starting: end-of-recovery immediate wait
 2024-08-12 05:52:55.472 UTC [1779] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.003 s, sync=0.002 s, total=0.013 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB
 2024-08-12 05:52:55.479 UTC [1] LOG:  database system is ready to accept connections
 2024-08-12 05:52:55.479 UTC [1783] LOG:  TimescaleDB background worker launcher connected to shared catalogs
 2024-08-12 05:52:55.485 UTC [1785] LOG:  pg_cron scheduler started

Full log:

docker logs -f supag
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with this locale configuration:
  provider:    libc
  LC_COLLATE:  C.UTF-8
  LC_CTYPE:    C.UTF-8
  LC_MESSAGES: en_US.UTF-8
  LC_MONETARY: en_US.UTF-8
  LC_NUMERIC:  en_US.UTF-8
  LC_TIME:     en_US.UTF-8
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start.... 2024-08-12 05:45:57.952 UTC [56] LOG:  pgaudit extension initialized
 2024-08-12 05:45:57.985 UTC [56] LOG:  pgsodium primary server secret key loaded
 2024-08-12 05:45:58.039 UTC [56] LOG:  starting PostgreSQL 15.1 (Ubuntu 15.1-1.pgdg20.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0, 64-bit
 2024-08-12 05:45:58.041 UTC [56] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
 2024-08-12 05:45:58.049 UTC [65] LOG:  database system was shut down at 2024-08-12 05:45:56 UTC
 2024-08-12 05:45:58.060 UTC [56] LOG:  database system is ready to accept connections
 2024-08-12 05:45:58.060 UTC [68] LOG:  TimescaleDB background worker launcher connected to shared catalogs
 2024-08-12 05:45:58.065 UTC [70] LOG:  pg_cron scheduler started
 done
server started

/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/init-scripts

/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/migrate.sh
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/init-scripts/00-schema.sql
CREATE ROLE
REVOKE
CREATE SCHEMA
CREATE FUNCTION
REVOKE
GRANT
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/init-scripts/00000000000000-initial-schema.sql
CREATE PUBLICATION
CREATE ROLE
ALTER ROLE
CREATE ROLE
CREATE ROLE
GRANT ROLE
CREATE SCHEMA
CREATE EXTENSION
CREATE EXTENSION
CREATE EXTENSION
CREATE ROLE
CREATE ROLE
CREATE ROLE
CREATE ROLE
GRANT ROLE
GRANT ROLE
GRANT ROLE
GRANT ROLE
GRANT
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
GRANT
ALTER ROLE
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER ROLE
ALTER ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/init-scripts/00000000000001-auth-schema.sql
CREATE SCHEMA
CREATE TABLE
CREATE INDEX
CREATE INDEX
COMMENT
CREATE TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
COMMENT
CREATE TABLE
COMMENT
CREATE TABLE
CREATE INDEX
COMMENT
CREATE TABLE
COMMENT
INSERT 0 7
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
GRANT
CREATE ROLE
GRANT
GRANT
GRANT
ALTER ROLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/init-scripts/00000000000002-storage-schema.sql
CREATE SCHEMA
GRANT
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
CREATE TABLE
CREATE INDEX
CREATE TABLE
CREATE INDEX
CREATE INDEX
ALTER TABLE
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE TABLE
CREATE ROLE
GRANT
GRANT
GRANT
ALTER ROLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER FUNCTION
ALTER FUNCTION
ALTER FUNCTION
ALTER FUNCTION
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/init-scripts/00000000000003-post-setup.sql
ALTER ROLE
ALTER ROLE
CREATE FUNCTION
CREATE EVENT TRIGGER
COMMENT
CREATE FUNCTION
COMMENT
DO
CREATE ROLE
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
ALTER ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/00-extension.sql
psql:/docker-entrypoint-initdb.d/migrations/00-extension.sql:1: NOTICE:  schema "extensions" already exists, skipping
CREATE SCHEMA
CREATE EXTENSION
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/10000000000000_demote-postgres.sql
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
GRANT
ALTER ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20211115181400_update-auth-permissions.sql
GRANT
GRANT
GRANT
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
GRANT
GRANT
GRANT
GRANT
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20211118015519_create-realtime-schema.sql
CREATE SCHEMA
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20211122051245_update-realtime-permissions.sql
GRANT
GRANT
GRANT
GRANT
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20211124212715_update-auth-owner.sql
ALTER FUNCTION
ALTER FUNCTION
ALTER FUNCTION
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20211130151719_update-realtime-permissions.sql
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20220118070449_enable-safeupdate-postgrest.sql
ALTER ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20220126121436_finer-postgrest-triggers.sql
psql:/docker-entrypoint-initdb.d/migrations/20220126121436_finer-postgrest-triggers.sql:3: NOTICE:  event trigger "api_restart" does not exist, skipping
DROP EVENT TRIGGER
psql:/docker-entrypoint-initdb.d/migrations/20220126121436_finer-postgrest-triggers.sql:4: NOTICE:  function extensions.notify_api_restart() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
CREATE FUNCTION
psql:/docker-entrypoint-initdb.d/migrations/20220126121436_finer-postgrest-triggers.sql:59: NOTICE:  event trigger "pgrst_ddl_watch" does not exist, skipping
DROP EVENT TRIGGER
CREATE EVENT TRIGGER
psql:/docker-entrypoint-initdb.d/migrations/20220126121436_finer-postgrest-triggers.sql:64: NOTICE:  event trigger "pgrst_drop_watch" does not exist, skipping
DROP EVENT TRIGGER
CREATE EVENT TRIGGER
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20220224211803_fix-postgrest-supautils.sql
DO
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20220317095840_pg_graphql.sql
CREATE SCHEMA
psql:/docker-entrypoint-initdb.d/migrations/20220317095840_pg_graphql.sql:5: NOTICE:  function graphql_public.graphql(text,text,jsonb) does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
GRANT
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
CREATE FUNCTION
psql:/docker-entrypoint-initdb.d/migrations/20220317095840_pg_graphql.sql:90: NOTICE:  event trigger "issue_pg_graphql_access" does not exist, skipping
DROP EVENT TRIGGER
CREATE EVENT TRIGGER
COMMENT
CREATE FUNCTION
psql:/docker-entrypoint-initdb.d/migrations/20220317095840_pg_graphql.sql:143: NOTICE:  event trigger "issue_graphql_placeholder" does not exist, skipping
DROP EVENT TRIGGER
CREATE EVENT TRIGGER
COMMENT
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20220321174452_fix-postgrest-alter-type-event-trigger.sql
psql:/docker-entrypoint-initdb.d/migrations/20220321174452_fix-postgrest-alter-type-event-trigger.sql:3: NOTICE:  event trigger "api_restart" does not exist, skipping
DROP EVENT TRIGGER
psql:/docker-entrypoint-initdb.d/migrations/20220321174452_fix-postgrest-alter-type-event-trigger.sql:4: NOTICE:  function extensions.notify_api_restart() does not exist, skipping
DROP FUNCTION
CREATE FUNCTION
CREATE FUNCTION
DROP EVENT TRIGGER
CREATE EVENT TRIGGER
DROP EVENT TRIGGER
CREATE EVENT TRIGGER
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20220322085208_gotrue-session-limit.sql
ALTER ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20220404205710_pg_graphql-on-by-default.sql
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
psql:/docker-entrypoint-initdb.d/migrations/20220404205710_pg_graphql-on-by-default.sql:142: NOTICE:  extension "pg_graphql" does not exist, skipping
DROP EXTENSION
DO
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20220609081115_grant-supabase-auth-admin-and-supabase-storage-admin-to-postgres.sql
GRANT ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20220613123923_pg_graphql-pg-dump-perms.sql
CREATE FUNCTION
DROP EXTENSION
DO
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20220713082019_pg_cron-pg_net-temp-perms-fix.sql
DO
DO
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20221028101028_set_authenticator_timeout.sql
ALTER ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20221103090837_revoke_admin.sql
REVOKE ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20221207154255_create_pgsodium_and_vault.sql
DO
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20230201083204_grant_auth_roles_to_postgres.sql
GRANT ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20230224042246_grant_extensions_perms_for_postgres.sql
GRANT
GRANT
GRANT
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
ALTER DEFAULT PRIVILEGES
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20230306081037_grant_pg_monitor_to_postgres.sql
GRANT ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20230327032006_grant_auth_roles_to_supabase_storage_admin.sql
GRANT ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20230529180330_alter_api_roles_for_inherit.sql
ALTER ROLE
ALTER ROLE
ALTER ROLE
GRANT ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20231013070755_grant_authenticator_to_supabase_storage_admin.sql
GRANT ROLE
REVOKE ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20231017062225_grant_pg_graphql_permissions_for_custom_roles.sql
CREATE FUNCTION
DROP EXTENSION
DO
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20231020085357_revoke_writes_on_cron_job_from_postgres.sql
DO
CREATE FUNCTION
DROP EVENT TRIGGER
CREATE EVENT TRIGGER
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20231130133139_set_lock_timeout_to_authenticator_role.sql
ALTER ROLE
/docker-entrypoint-initdb.d/migrate.sh: running /docker-entrypoint-initdb.d/migrations/20240124080435_alter_lo_export_lo_import_owner.sql
ALTER FUNCTION
ALTER FUNCTION
ALTER FUNCTION
 pg_stat_statements_reset 
--------------------------
 
(1 row)

 pg_stat_reset 
---------------
 
(1 row)


/usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/migrations

 2024-08-12 05:46:00.967 UTC [56] LOG:  received fast shutdown request
waiting for server to shut down.... 2024-08-12 05:46:00.970 UTC [56] LOG:  aborting any active transactions
 2024-08-12 05:46:00.970 UTC [70] LOG:  pg_cron scheduler shutting down
 2024-08-12 05:46:00.970 UTC [68] FATAL:  terminating background worker "TimescaleDB Background Worker Launcher" due to administrator command
 2024-08-12 05:46:00.973 UTC [56] LOG:  background worker "TimescaleDB Background Worker Launcher" (PID 68) exited with exit code 1
 2024-08-12 05:46:00.973 UTC [56] LOG:  background worker "logical replication launcher" (PID 71) exited with exit code 1
 2024-08-12 05:46:00.973 UTC [56] LOG:  background worker "pg_cron launcher" (PID 70) exited with exit code 1
 2024-08-12 05:46:00.973 UTC [56] LOG:  background worker "pg_net 0.8.0 worker" (PID 69) exited with exit code 1
 2024-08-12 05:46:00.974 UTC [63] LOG:  shutting down
 2024-08-12 05:46:00.976 UTC [63] LOG:  checkpoint starting: shutdown immediate
 2024-08-12 05:46:01.124 UTC [63] LOG:  checkpoint complete: wrote 341 buffers (2.1%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.007 s, sync=0.129 s, total=0.150 s; sync files=119, longest=0.004 s, average=0.002 s; distance=2777 kB, estimate=2777 kB
 2024-08-12 05:46:01.167 UTC [56] LOG:  database system is shut down
 done
server stopped

PostgreSQL init process complete; ready for start up.

 2024-08-12 05:46:01.213 UTC [1] LOG:  pgaudit extension initialized
 2024-08-12 05:46:01.234 UTC [1] LOG:  pgsodium primary server secret key loaded
 2024-08-12 05:46:01.316 UTC [1] LOG:  starting PostgreSQL 15.1 (Ubuntu 15.1-1.pgdg20.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0, 64-bit
 2024-08-12 05:46:01.317 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
 2024-08-12 05:46:01.317 UTC [1] LOG:  listening on IPv6 address "::", port 5432
 2024-08-12 05:46:01.320 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
 2024-08-12 05:46:01.326 UTC [167] LOG:  database system was shut down at 2024-08-12 05:46:01 UTC
 2024-08-12 05:46:01.334 UTC [1] LOG:  database system is ready to accept connections
 2024-08-12 05:46:01.334 UTC [170] LOG:  TimescaleDB background worker launcher connected to shared catalogs
 2024-08-12 05:46:01.340 UTC [172] LOG:  pg_cron scheduler started
 2024-08-12 05:49:49.435 UTC [1] LOG:  server process (PID 781) was terminated by signal 4: Illegal instruction
 2024-08-12 05:49:49.435 UTC [1] DETAIL:  Failed process was running: insert into vector_test (content, embedding) values ('sample text', '[1,2,3,4,5,6,7,8,9,10]');
 2024-08-12 05:49:49.435 UTC [1] LOG:  terminating any other active server processes
172.17.0.1 2024-08-12 05:49:49.438 UTC [1056] postgres@postgres FATAL:  the database system is in recovery mode
 2024-08-12 05:49:49.441 UTC [1] LOG:  all server processes terminated; reinitializing
 2024-08-12 05:49:49.501 UTC [1057] LOG:  database system was interrupted; last known up at 2024-08-12 05:46:01 UTC
 2024-08-12 05:49:49.608 UTC [1057] LOG:  database system was not properly shut down; automatic recovery in progress
 2024-08-12 05:49:49.612 UTC [1057] LOG:  redo starts at 0/17B0380
 2024-08-12 05:49:49.620 UTC [1057] LOG:  invalid record length at 0/19109D0: wanted 24, got 0
 2024-08-12 05:49:49.620 UTC [1057] LOG:  redo done at 0/1910958 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
 2024-08-12 05:49:49.623 UTC [1058] LOG:  checkpoint starting: end-of-recovery immediate wait
 2024-08-12 05:49:49.701 UTC [1058] LOG:  checkpoint complete: wrote 201 buffers (1.2%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.008 s, sync=0.063 s, total=0.079 s; sync files=99, longest=0.003 s, average=0.001 s; distance=1409 kB, estimate=1409 kB
 2024-08-12 05:49:49.710 UTC [1] LOG:  database system is ready to accept connections
 2024-08-12 05:49:49.710 UTC [1062] LOG:  TimescaleDB background worker launcher connected to shared catalogs
 2024-08-12 05:49:49.715 UTC [1064] LOG:  pg_cron scheduler started
 2024-08-12 05:52:55.278 UTC [1] LOG:  server process (PID 1687) was terminated by signal 4: Illegal instruction
 2024-08-12 05:52:55.278 UTC [1] DETAIL:  Failed process was running: insert into vector_test (content, embedding) values ('sample text', '[1,2,3,4,5,6,7,8,9,10]');
 2024-08-12 05:52:55.278 UTC [1] LOG:  terminating any other active server processes
172.17.0.1 2024-08-12 05:52:55.280 UTC [1777] postgres@postgres FATAL:  the database system is in recovery mode
 2024-08-12 05:52:55.283 UTC [1] LOG:  all server processes terminated; reinitializing
 2024-08-12 05:52:55.350 UTC [1778] LOG:  database system was interrupted; last known up at 2024-08-12 05:49:49 UTC
 2024-08-12 05:52:55.454 UTC [1778] LOG:  database system was not properly shut down; automatic recovery in progress
 2024-08-12 05:52:55.456 UTC [1778] LOG:  redo starts at 0/1910A48
 2024-08-12 05:52:55.456 UTC [1778] LOG:  invalid record length at 0/1910A80: wanted 24, got 0
 2024-08-12 05:52:55.456 UTC [1778] LOG:  redo done at 0/1910A48 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s
 2024-08-12 05:52:55.461 UTC [1779] LOG:  checkpoint starting: end-of-recovery immediate wait
 2024-08-12 05:52:55.472 UTC [1779] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.003 s, sync=0.002 s, total=0.013 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB
 2024-08-12 05:52:55.479 UTC [1] LOG:  database system is ready to accept connections
 2024-08-12 05:52:55.479 UTC [1783] LOG:  TimescaleDB background worker launcher connected to shared catalogs
 2024-08-12 05:52:55.485 UTC [1785] LOG:  pg_cron scheduler started

I found a bug report that looks somewhat related: pgvector/pgvector#389
I'm not sure if this is relevant or something that can be fixed.

@olirice
Copy link
Contributor

olirice commented Aug 12, 2024

Similar to that thread, are you also running CentOS?

@3dteemu
Copy link
Author

3dteemu commented Aug 12, 2024

Similar to that thread, are you also running CentOS?

No. I'm running Ubuntu 24.04 LTS. Full system specs in the original bug report.

The similarities are that I use pg_vector and get the same error: "Illegal instruction". My googling only gives results related to the build flags that they discuss also in that related bug report. I wanted to link it if it gives any insight on what might be happening.

Unfortunately, I don't have the skills to asses if that is actually related in any way.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants