Database running stable on PostgreSQL 15.3.1, started importing large table into it from S3 using table_import_from_s3, import runs fine for a couple of minutes then the DB writer instance restarts with the following log.
I keep rerunning the import and it manages to resume from where it crashes BUT after a couple of mins it crashes again.
2023-11-06 13:14:33 UTC:100.72.181.179(39400):xxxx@xxxx:[3333]:CONTEXT: SQL function "table_import_from_s3" statement 1
2023-11-06 13:14:35 UTC:100.72.181.179(39400):xxxx@xxxx:[3333]:LOG: Attempting to COPY from S3. 3 attempt(s) remaining. CURL error code 23 with message Failure writing output to destination.
2023-11-06 13:14:35 UTC:100.72.181.179(39400):xxxx@xxxx:[3333]:HINT: Try obtaining a ROW EXCLUSIVE LOCK on the table before initiating the import.
2023-11-06 13:14:35 UTC:100.72.181.179(39400):xxxx@xxxx:[3333]:CONTEXT: SQL function "table_import_from_s3" statement 1
2023-11-06 13:14:35 UTC:100.72.181.179(39400):xxxx@xxxx:[3333]:STATEMENT: SELECT aws_s3.table_import_from_s3 ('test', 'id,data', '(FORMAT csv, DELIMITER e''?'', NULL ''\N'', ESCAPE ''"'')', aws_commons.create_s3_uri('xxxx','file.csv','eu-west-1'))
2023-11-06 13:14:35 UTC::@:[537]:LOG: server process (PID 3333) was terminated by signal 11: Segmentation fault
2023-11-06 13:14:35 UTC::@:[537]:DETAIL: Failed process was running: SELECT aws_s3.table_import_from_s3 ('test', 'id,data', '(FORMAT csv, DELIMITER e''?'', NULL ''\N'', ESCAPE ''"'')', aws_commons.create_s3_uri('xxxx','file.csv','eu-west-1'))
2023-11-06 13:14:35 UTC::@:[537]:LOG: terminating any other active server processes
2023-11-06 13:14:35 UTC::@:[537]:FATAL: Can't handle storage runtime process crash
2023-11-06 13:14:35 UTC::@:[537]:LOG: database system is shut down
AWS OFFICIALUpdated a year ago
Hi, We have encountered a similar issue on 15.4.1. For us the failure occurs after running an S3 export using the aws_s3 extension, and then running an S3 import. We have a support ticket open with AWS, but no clear conclusion yet. We haven't been able to reproduce the error on other Aurora Postgres databases we are running, so suspect it may only apply in particular circumstances. Have you identified a fix? Thanks
Hi, We have encountered a similar issue on 15.4.1. For us the failure occurs after running an S3 export using the aws_s3 extension, and then running an S3 import. We have a support ticket open with AWS, but no clear conclusion yet. We haven't been able to reproduce the error on other Aurora Postgres databases we are running, so suspect it may only apply in particular circumstances. Have you identified a fix? Thanks