Aurora postgres 18 upgrade failures 17.9 -> 18.3

You are right to point out the log limitation… Since we cannot access the pg_upgrade_utility.log …

You can try to stop all attempts to force the upgrade and switch to a purely diagnostic approach. The error could not open directory "pg_tblspc/16409/..." is a hard dependency violation. Before you or anyone else attempts any DROP operations, please collect the current state of the catalog. This data is the only thing an AWS support need to identify if this is a ghost entry or a corruption.

try to run these three read-only queries and document the output:

1. Check the existence of the OID in the catalog:

SELECT oid, spcname FROM pg_tablespace WHERE oid = 16409;

2. Check for associated relations:

SELECT relname, relkind FROM pg_class WHERE reltablespace = 16409;

3. Check for database-level assignment:

SELECT datname FROM pg_database WHERE dattablespace = 16409;

Why this is the safest path:

If these queries return any rows, you have isolated the object causing the pg_dumpall crash. If they return no results, you have definitive proof of catalog corruption (a dangling pointer without a catalog entry).

My advice: Do not run DROP commands yet. If these queries show results, the objects are 'orphaned' or 'stuck'. If you cannot remove them without errors, provide the output of these queries to AWS Support immediately. This allows them to perform a 'surgical' cleanup on their side without risking further damage to your data files.

In short: Do not attempt another upgrade until you have read the pg_upgrade_utility.log and resolved any catalog inconsistencies regarding OID 16409 on your source 17.9 cluster.

Based on the server.log output provided, your database successfully started during the upgrade process, but the upgrade subsequently failed during the global objects export phase. Specifically, the fatal error occurred when the upgrade engine attempted to run pg_dumpall --globals-only. This command exports database-wide objects such as roles, groups, and tablespace definitions.

To resolve this issue, I recommend the following troubleshooting steps:

1. Check the Utility Log (Crucial)

The server.log explicitly points to the exact cause of the failure at the very end:

"Consult the last few lines of "/rdsdbdata/db/pg_upgrade_output.d/20260629T164231.724/log/pg_upgrade_utility.log" for the probable cause of the failure."

You need to review this specific log file. You can access it via the RDS Console under the "Logs & events" tab of your cluster/instance, or by querying it through Amazon CloudWatch Logs if you have log exports enabled. The last few lines of this file will tell you exactly which global object caused pg_dumpall to fail.

2. Investigate the Missing Tablespace Directory

While the database successfully started and accepted connections despite the missing directory (could not open directory "pg_tblspc/16409/PG_17_202406281"), this missing directory is highly likely the reason pg_dumpall crashed. When pg_dumpall --globals-only runs, it queries the pg_tablespace catalog. If a tablespace is registered in the database catalog (OID 16409) but the underlying directory is completely missing, the dump will often fail. Connect to your source database (version 17.9) and run the following queries to identify if this tablespace OID is orphaned:

-- Check if the tablespace still exists in the catalog
SELECT oid, spcname, spcowner::regrole FROM pg_tablespace WHERE oid = 16409;

-- Check if any databases are set to use this tablespace as their default
SELECT datname FROM pg_database WHERE dattablespace = 16409;

-- Check if any specific tables or indexes are still assigned to this tablespace
SELECT relname, relkind FROM pg_class WHERE reltablespace = 16409;

3. Resolution Steps

Depending on what you find in step 2:

• If it is an orphaned custom tablespace: If the tablespace returns results in pg_tablespace but holds no databases or relations, you can attempt to drop it (DROP TABLESPACE <spcname>;) before retrying the upgrade.

Note: Aurora PostgreSQL generally does not support user-defined tablespaces, so this might be an artifact from a previous migration.

• If you receive permission errors: If you cannot drop the orphaned tablespace due to RDS permission restrictions (rdsadmin), or if the pg_upgrade_utility.log indicates system catalog corruption, you will need to open a support case with AWS Premium Support. They have the backend privileges required to clean up orphaned catalog entries safely.

Based on the error logs you've provided, the upgrade failure appears to be related to a missing tablespace directory issue during the pg_upgrade process. The key error message is:

could not open directory "pg_tblspc/16409/PG_17_202406281": No such file or directory

This error occurs when the upgrade process attempts to access a tablespace that either no longer exists or has been removed, but references to it still remain in the database catalog.

To resolve this issue, you should:

1. Identify orphaned tablespaces: Check if tablespace 16409 is still in use or if it's an orphaned reference. You can query your current database to see if this tablespace exists:

SELECT * FROM pg_tablespace WHERE oid = 16409;

2. Check for objects using the tablespace: If the tablespace exists in the catalog but the directory is missing, identify any objects that might be using it:

SELECT * FROM pg_class WHERE reltablespace = 16409;

3. Clean up the orphaned tablespace: If the tablespace is no longer needed and no objects are using it, you can drop it before attempting the upgrade again:

DROP TABLESPACE IF EXISTS <tablespace_name>;

4. Verify all tablespaces are accessible: Before retrying the upgrade, ensure all tablespaces referenced in your database have valid, accessible directories.

The "plans update worker" exit code 1 that follows is likely a cascading effect of the initial tablespace directory error. Once you resolve the tablespace issue, you should be able to retry the upgrade process. Make sure to address this in your source 17.9 cluster before attempting another upgrade to 18.3.
Sources
Troubleshoot Aurora MySQL upgrade issues | AWS re:Post
Finding the reasons for Aurora MySQL major version upgrade failures - Amazon Aurora