E.9.1. Migration to Version 9.6.16

A dump/restore is not required for those running 9.6.X.

However, if you use the contrib/intarray extension with a GiST index, and you rely on indexed searches for the <@ operator, see the entry below about that.

Also, if you are upgrading from a version earlier than 9.6.9, see Section E.16.

E.9.2. Changes

• Fix failure of ALTER TABLE SET with a custom relation option (Michael Paquier)

• Disallow changing a multiply-inherited column's type if not all parent tables were changed (Tom Lane)

  Previously, this was allowed, whereupon queries on the now-out-of-sync parent would fail.

• Prevent VACUUM from trying to freeze an old multixact ID involving a still-running transaction (Nathan Bossart, Jeremy Schneider)

  This case would lead to VACUUM failing until the old transaction terminates.

• Ensure that offset expressions in WINDOW clauses are processed when a query's expressions are manipulated (Andrew Gierth)

  This oversight could result in assorted failures when the offsets are nontrivial expressions. One example is that a function parameter reference in such an expression would fail if the function was inlined.

• Fix handling of whole-row variables in WITH CHECK OPTION expressions and row-level-security policy expressions (Andres Freund)

  Previously, such usage might result in bogus errors about row type mismatches.

• Avoid postmaster failure if a parallel query requests a background worker when no postmaster child process array slots remain free (Tom Lane)

• Prevent possible double-free if a BEFORE UPDATE trigger returns the old tuple as-is, and it is not the last such trigger (Thomas Munro)

• Provide a relevant error context line when an error occurs while setting GUC parameters during parallel worker startup (Thomas Munro)

• In serializable mode, ensure that row-level predicate locks are acquired on the correct version of the row (Thomas Munro, Heikki Linnakangas)

  If the visible version of the row is HOT-updated, the lock might be taken on its now-dead predecessor, resulting in subtle failures to guarantee serialization.

• Ensure that fsync() is applied only to files that are opened read/write (Andres Freund, Michael Paquier)

  Some code paths tried to do this after opening a file read-only, but on some platforms that causes "bad file descriptor" or similar errors.

• Allow encoding conversion to succeed on longer strings than before (Álvaro Herrera, Tom Lane)

  Previously, there was a hard limit of 0.25GB on the input string, but now it will work as long as the converted output is not over 1GB.

• Avoid creating unnecessarily-bulky tuple stores for window functions (Andrew Gierth)

  In some cases the tuple storage would include all columns of the source table(s), not just the ones that are needed by the query.

• Allow repalloc() to give back space when a large chunk is reduced in size (Tom Lane)

• Ensure that temporary WAL and history files are removed at the end of archive recovery (Sawada Masahiko)

• Avoid failure in archive recovery if recovery_min_apply_delay is enabled (Fujii Masao)

  recovery_min_apply_delay is not typically used in this configuration, but it should work.

• Avoid unwanted delay during shutdown of a logical replication walsender (Craig Ringer, Álvaro Herrera)

• Correctly time-stamp replication messages for logical decoding (Jeff Janes)

  This oversight resulted, for example, in pg_stat_subscription.last_msg_send_time usually reading as NULL.

• In logical decoding, ensure that sub-transactions are correctly accounted for when reconstructing a snapshot (Masahiko Sawada)

  This error leads to assertion failures; it's unclear whether any bad effects exist in production builds.

• Fix race condition during backend exit, when the backend process has previously waited for synchronous replication to occur (Dongming Liu)

• Fix ALTER SYSTEM to cope with duplicate entries in postgresql.auto.conf (Ian Barwick)

  ALTER SYSTEM itself will not generate such a state, but external tools that modify postgresql.auto.conf could do so. Duplicate entries for the target variable will now be removed, and then the new setting (if any) will be appended at the end.

• Reject include directives with empty file names in configuration files, and report include-file recursion more clearly (Ian Barwick, Tom Lane)

• Avoid logging complaints about abandoned connections when using PAM authentication (Tom Lane)

  libpq-based clients will typically make two connection attempts when a password is required, since they don't prompt their user for a password until their first connection attempt fails. Therefore the server is coded not to generate useless log spam when a client closes the connection upon being asked for a password. However, the PAM authentication code hadn't gotten that memo, and would generate several messages about a phantom authentication failure.

• Fix some cases where an incomplete date specification is not detected in time with time zone input (Alexander Lakhin)

  If a time zone with a time-varying UTC offset is specified, then a date must be as well, so that the offset can be resolved. Depending on the syntax used, this check was not enforced in some cases, allowing bogus output to be produced.

• Fix misbehavior of bitshiftright() (Tom Lane)

  The bitstring right shift operator failed to zero out padding space that exists in the last byte of the result when the bitstring length is not a multiple of 8. While invisible to most operations, any nonzero bits there would result in unexpected comparison behavior, since bitstring comparisons don't bother to ignore the extra bits, expecting them to always be zero.

  If you have inconsistent data as a result of saving the output of bitshiftright() in a table, it's possible to fix it with something like

  UPDATE mytab SET bitcol = ~(~bitcol) WHERE bitcol != ~(~bitcol);

• Fix detection of edge-case integer overflow in interval multiplication (Yuya Watari)

• Avoid crashes if ispell text search dictionaries contain wrong affix data (Arthur Zakirov)

• Fix incorrect compression logic for GIN posting lists (Heikki Linnakangas)

  A GIN posting list item can require 7 bytes if the distance between adjacent indexed TIDs exceeds 16TB. One step in the logic was out of sync with that, and might try to write the value into a 6-byte buffer. In principle this could cause a stack overrun, but on most architectures it's likely that the next byte would be unused alignment padding, making the bug harmless. In any case the bug would be very difficult to hit.

• Fix handling of infinity, NaN, and NULL values in KNN-GiST (Alexander Korotkov)

  The query's output order could be wrong (different from a plain sort's result) if some distances computed for non-null column values are infinity or NaN.

• Fix handling of searches for NULL in KNN-SP-GiST (Nikita Glukhov)

• On Windows, recognize additional spellings of the "Norwegian (Bokmål)" locale name (Tom Lane)

• Avoid compile failure if an ECPG client includes ecpglib.h while having ENABLE_NLS defined (Tom Lane)

  This risk was created by a misplaced declaration: ecpg_gettext() should not be visible to client code.

• In psql, resynchronize internal state about the server after an unexpected connection loss and successful reconnection (Peter Billen, Tom Lane)

  Ordinarily this is unnecessary since the state would be the same anyway. But it can matter in corner cases, such as where the connection might lead to one of several servers. This change causes psql to re-issue any interactive messages that it would have issued at startup, for example about whether SSL is in use.

• Avoid platform-specific null pointer dereference in psql (Quentin Rameau)

• Fix pg_dump's handling of circular dependencies in views (Tom Lane)

  In some cases a view may depend on an object that pg_dump needs to dump later than the view; the most common example is that a query using GROUP BY on a primary-key column may be semantically invalid without the primary key. This is now handled by emitting a dummy CREATE VIEW command that just establishes the view's column names and types, and then later emitting CREATE OR REPLACE VIEW with the full view definition. Previously, the dummy definition was actually a CREATE TABLE command, and this was automagically converted to a view by a later CREATE RULE command. The new approach has been used successfully in PostgreSQL version 10 and later. We are back-patching it into older releases now because of reports that the previous method causes bogus error messages about the view's replica identity status. This change also avoids problems when trying to use the --clean option during a restore involving such a view.

• In pg_dump, ensure stable output order for similarly-named triggers and row-level-security policy objects (Benjie Gillam)

  Previously, if two triggers on different tables had the same names, they would be sorted in OID-based order, which is less desirable than sorting them by table name. Likewise for RLS policies.

• Fix pg_dump to work again with pre-8.3 source servers (Tom Lane)

  A previous fix caused pg_dump to always try to query pg_opfamily, but that catalog doesn't exist before version 8.3.

• In pg_restore, treat -f - as meaning "output to stdout" (Álvaro Herrera)

  This synchronizes pg_restore's behavior with some other applications, and in particular makes pre-v12 branches act similarly to version 12's pg_restore, simplifying creation of dump/restore scripts that work across multiple PostgreSQL versions. Before this change, pg_restore interpreted such a switch as meaning "output to a file named -", but few people would want that.

• Improve pg_upgrade's checks for the use of a data type that has changed representation, such as line (Tomas Vondra)

  The previous coding could be fooled by cases where the data type of interest underlies a stored column of a domain or composite type.

• Detect file read errors during pg_basebackup (Jeevan Chalke)

• In pg_rewind with an online source cluster, disable timeouts, much as pg_dump does (Alexander Kukushkin)

• Fix failure in pg_waldump with the -s option, when a continuation WAL record ends exactly at a page boundary (Andrey Lepikhov)

• In pg_waldump, include the newitemoff field in btree page split records (Peter Geoghegan)

• In pg_waldump with the --bkp-details option, avoid emitting extra newlines for WAL records involving full-page writes (Andres Freund)

• Fix small memory leak in pg_waldump (Andres Freund)

• Fix vacuumdb with a high --jobs option to handle running out of file descriptors better (Michael Paquier)

• Fix contrib/intarray's GiST opclasses to not fail for empty arrays with <@ (Tom Lane)

  A clause like array_column <@ constant_array is considered indexable, but the index search may not find empty array values; of course, such entries should trivially match the search.

  The only practical back-patchable fix for this requires making <@ index searches scan the whole index, which is what this patch does. This is unfortunate: it means that the query performance is likely worse than a plain sequential scan would be.

  Applications whose performance is adversely impacted by this change have a couple of options. They could switch to a GIN index, which doesn't have this bug, or they could replace array_column <@ constant_array with array_column <@ constant_array AND array_column && constant_array. That will provide about the same performance as before, and it will find all non-empty subsets of the given constant array, which is all that could reliably be expected of the query before.

• Allow configure --with-python to succeed when only python3 or only python2 can be found (Peter Eisentraut, Tom Lane)

  Search for python, then python3, then python2, so that configure can succeed in the increasingly-more-common situation where there is no executable named simply python. It's still possible to override this choice by setting the PYTHON environment variable.

• Fix configure's test for presence of libperl so that it works on recent Red Hat releases (Tom Lane)

  Previously, it could fail if the user sets CFLAGS to -O0.

• Ensure correct code generation for spinlocks on PowerPC (Noah Misch)

  The previous spinlock coding allowed the compiler to select register zero for use with an assembly instruction that does not accept that register, causing a build failure. We have seen only one long-ago report that matches this bug, but it could cause problems for people trying to build modified PostgreSQL code or use atypical compiler options.

• On PowerPC, avoid depending on the xlc compiler's __fetch_and_add() function (Noah Misch)

  xlc 13 and newer interpret this function in a way incompatible with our usage, resulting in an unusable build of PostgreSQL. Fix by using custom assembly code instead.

• On AIX, don't use the compiler option -qsrcmsg (Noah Misch)

  This avoids an internal compiler error with xlc v16.1.0, with little consequence other than changing the format of compiler error messages.

• Fix MSVC build process to cope with spaces in the file path of OpenSSL (Andrew Dunstan)

• Update time zone data files to tzdata release 2019c for DST law changes in Fiji and Norfolk Island, plus historical corrections for Alberta, Austria, Belgium, British Columbia, Cambodia, Hong Kong, Indiana (Perry County), Kaliningrad, Kentucky, Michigan, Norfolk Island, South Korea, and Turkey.