release-3916

What's Changed

• Release 2023-09-15-1: Downgrade plv8 for Postgres 14/15 by @bayandin in #5318

Full Changelog: release-3913...release-3916

release-3913

What's Changed

• Release 2023-09-15 by @vipvap in #5313

Full Changelog: release-3909...release-3913

release-3909

What's Changed

• s3_scrubber: remove atty dependency by @jcsp in #5171
• fix deadlock around TENANTS by @problame in #5285
• Feat/postgres 16 by @MMeent in #4761
• revert recent VirtualFile asyncification changes by @problame in #5291
• Document how to run CI for external contributors by @bayandin in #5279
• approved-for-ci-run.yml: fix ci-run/pr-* branch deletion by @bayandin in #5278
• Use CI_ACCESS_TOKEN to create release PR by @bayandin in #5286
• Fixes for Pg16: by @MMeent in #5292
• proxy http: reproduce issue with transactions in pool by @bojanserafimov in #5293
• Fix lfc_ensure_function which now disables LFC by @knizhnik in #5294
• Release 2023-09-13 by @vipvap in #5296

New Contributors

• @vipvap made their first contribution in #5296

Full Changelog: release-3898...release-3909

release-3898

What's Changed

• Convert VirtualFile::{seek,metadata} to async by @arpad-m in #5195
• Merge custom extension CI jobs by @bayandin in #5194
• Fix corruption issue in Local File Cache by @MMeent in #5208
• Generate lcov coverage report by @bayandin in #4999
• Make CI more friendly for external contributors. Second try by @bayandin in #5210
• approved-for-ci-run.yml: trigger on pull_request_target by @bayandin in #5212
• Create Release PR at 7:00 UTC every Tuesday by @vadim2404 in #5213
• chore(pageserver): remove unnecessary logging in tenant task loops by @duguorong009 in #5188
• test_runner: add scale parameter to tpc-h tests by @bayandin in #5079
• scrubber: add scan-metadata and hook into integration tests by @jcsp in #5176
• approved-for-ci-run.yml: use our bot by @bayandin in #5216
• pageserver: generation number fetch on startup and use in /attach by @jcsp in #5163
• approved-for-ci-run.yml: use different tokens by @bayandin in #5218
• Add WriteBlobWriter buffering and make VirtualFile::{write,write_all} async by @arpad-m in #5203
• Notify Slack channel about upcoming releases by @bayandin in #5197
• Misc post-merge fixes by @bayandin in #5219
• remove the duplicate entries in postgresql.conf by @Reminiscent in #5090
• Complete the missing metrics for files_created/bytes_written by @Reminiscent in #5120
• refactor(pageserver): refactor pageserver router state creation by @duguorong009 in #5165
• doc: tests expect lsof by @koivunej in #5226
• Remove Virtual from FileBlockReaderVirtual variant name by @arpad-m in #5225
• Improve comment in page cache by @arpad-m in #5220
• fix(pageserver): add the walreceiver state to tenant timeline GET api endpoint by @duguorong009 in #5196
• Even better handling of approved-for-ci-run label by @bayandin in #5227
• Update plv8 to 3.1.8 by @bayandin in #5230
• Fix pg_config version parsing by @rmodpur in #5200
• Misc workflows: fixes to make CI friendly for external contributors by @bayandin in #5235
• Update PostGIS to version 3.3.3 by @hlinnaka in #5236
• Upgrade h3-pg to version 4.1.3. by @hlinnaka in #5237
• Make VirtualFile::{open, open_with_options, create,sync_all,with_file} async fn by @arpad-m in #5224
• Upgrade ip4r to version 2.4.2 by @hlinnaka in #5242
• Upgrade postgresql-hll to version 2.18. by @hlinnaka in #5241
• Upgrade prefix extension to version 1.2.10 by @hlinnaka in #5244
• Update hypopg extension to version 1.4.0 by @hlinnaka in #5245
• test_startup: increase timeout by @bayandin in #5238
• Update plpgsql_check extension to version v2.4.0 by @hlinnaka in #5249
• Update pg_cron to version 1.6.0 by @hlinnaka in #5252
• test: Remote storage refactorings by @koivunej in #5243
• rust-1.72.0 changes by @koivunej in #5255
• Perform throttling for concurrent build index which is done outside transaction by @knizhnik in #5048
• tests: enable multiple pageservers in neon_local and neon_fixture by @jcsp in #5231
• Miscellaneous fixes for tests-related things by @bayandin in #5259
• rust-toolchain: use 1.72.0, same as CI by @koivunej in #5256
• Update rdkit to version 2023_03_03. by @hlinnaka in #5260
• fix: LocalFs root in test_compatibility is PosixPath('...') by @koivunej in #5261
• Fix issues with reanabling LFC by @knizhnik in #5209
• Ingore DISK_FULL error when performing availability check for client by @knizhnik in #5010
• Create GitHub release from release tag by @bayandin in #5246
• vm-monitor: Don't include Args in top-level span by @sharnoff in #5264
• vm-monitor: Rate-limit upscale requests by @sharnoff in #5263
• GitHub Workflows: add actionlint by @bayandin in #5265
• Fix GitHub Autocomment for ci-run/prs by @bayandin in #5268
• approved-for-ci-run.yml: use token to checkout the repo by @bayandin in #5266
• Ack tenant detach before local files are deleted by @rmodpur in #5211
• Misc test flakyness fixes by @koivunej in #5233
• fix(pageserver): update the STORAGE_IO_TIME metrics to avoid expensive operations by @duguorong009 in #5273
• Measure pageserver wal recovery time and fix flush() method by @bojanserafimov in #5240
• Use tokio locks in VirtualFile and turn with_file into macro by @arpad-m in #5247
• Make File opening in VirtualFile async-compatible by @arpad-m in #5280
• Disallow block_in_place and Handle::block_on by @arpad-m in #5101
• Release 2023-09-12 by @github-actions in #5283

New Contributors

• @duguorong009 made their first contribution in #5188
• @rmodpur made their first contribution in #5200

Full Changelog: release-3836...release-3898

release-3836

What's Changed

• Proxy: proxy protocol v2 by @conradludgate in #5028
• proxy: locked is not retriable by @conradludgate in #5162
• s3_scrubber: import into the main neon repository by @jcsp in #5141
• tests: broaden a log allow regex in test_ignored_tenant_stays_broken_without_metadata by @jcsp in #5168
• Wait for custom extensions build before deploy by @bayandin in #5170
• remote_timeline_client: tests: run upload ops on the tokio::test runtime by @problame in #5177
• rfc: Crash-Consistent Layer Map Updates By Leveraging index_part.json by @problame in #5086
• FileBlockReader is never used by @problame in #5181
• pageserver: run all Rust tests with remote storage enabled by @problame in #5164
• proxy: error typo by @conradludgate in #5187
• tests: get test name automatically for remote storage by @jcsp in #5184
• VirtualFile: add crash-safe overwrite abstraction & use it by @problame in #5186
• pageserver: safe deletion for tenant directories by @jcsp in #5182
• Make VirtualFile::crashsafe_overwrite async fn by @arpad-m in #5189
• Convert many VirtualFile APIs to async by @arpad-m in #5190
• support deploying in IPv6-only environments by @nikitakalyanov in #4135
• Release 2023-09-05 by @github-actions in #5202

Full Changelog: release-3819...release-3836

release-3819

What's Changed

• Form archive_path for remote extensions on the fly by @lubennikovaav in #5142
• chore: remove webpki by @conradludgate in #5069
• Check if custiom extension is enabled. by @lubennikovaav in #5155
• pageserver: remove vestigial timeline_layers attribute by @jcsp in #5153
• proxy: password hack hack by @conradludgate in #5126
• [compute_ctl] Create check availability data during full configuration by @ololobus in #5084
• pageserver: generation number support in keys and indices by @jcsp in #5140
• pass JWT to management API by @nikitakalyanov in #5151
• pageserver: fix test v4_indexpart_is_parsed by @jcsp in #5157
• pageserver: fix flake in test_timeline_deletion_with_files_stuck_in_upload_queue by @jcsp in #5149
• Release 2023-08-31 by @lubennikovaav in #5161

Full Changelog: release-3808...release-3819

release-3808

What's Changed

• scripts/flaky_tests.py: Improve flaky tests detection by @bayandin in #5094
• test: enhanced logging for curious mock_s3 by @koivunej in #5134
• vm-monitor: Add flag for when file cache on disk by @sharnoff in #5130
• Safekeeper peer recovery preparatory patches by @arssher in #5118
• Make page cache and read_blk async by @arpad-m in #5023
• docs: add RFC for remote storage generation numbers by @jcsp in #4919
• fix: add context for ancestor lsn wait by @koivunej in #5143
• Release 2023-08-30 by @petuhovskiy in #5150

Full Changelog: release-3795...release-3808

release-3795

What's Changed

• Fix cargo deny errors by @awestover in #5068
• chore: add supported targets to deny by @conradludgate in #5070
• refactor: compaction becomes async again by @koivunej in #5058
• metrics: smgr query time: add a pre-aggregated histogram by @problame in #5064
• Trigger workflow in remote (private) repo to build and upload private extensions by @awestover in #4944
• refactor: pre-cleanup Layer, PersistentLayer and impls by @koivunej in #5059
• build: update tar, get rid of duplicate xattr by @koivunej in #5071
• refactor: remove is_incremental=true for ImageLayers footgun by @koivunej in #5061
• ci: fix upload-postgres-extensions-to-s3 job by @problame in #5063
• Track list of walreceivers and their voting/streaming state in shmem. by @arssher in #4977
• libs: make backoff::retry() take a cancellation token by @jcsp in #5065
• refactor: single phase Timeline::load_layer_map by @koivunej in #5074
• remove upload step from neon, it is in private repo now by @awestover in #5085
• internal-devx: future layers by @koivunej in #5092
• rename EphmeralFile::size to EphemeralFile::len by @Reminiscent in #5076
• Delete requests for non-existent images by @awestover in #5096
• Make EphemeralFile::write_blob function async by @arpad-m in #5056
• compute_ctl: add vm-monitor by @fprasx in #4946
• monitor: listen on correct port to accept agent connections by @fprasx in #5100
• Remove BlockReader::read_blk in favour of BlockCursor by @arpad-m in #5015
• proxy: dont return connection pending by @conradludgate in #5107
• remove obsolete need dependency by @awestover in #5087
• monitor: improve logging by @fprasx in #5099
• pageserver: refactor TimelineMetadata serialization in IndexPart by @jcsp in #5091
• Change codeowners by @vadim2404 in #5109
• monitor: fix filecache calculations by @fprasx in #5112
• compute_ctl: start pg in cgroup for vms by @fprasx in #4920
• load_layer_map: schedule deletions for any future layers by @koivunej in #5103
• compute_ctl: Fix switched vm-monitor args by @sharnoff in #5117
• page cache: don't proactively evict EphemeralFile pages by @problame in #5129
• remove unused file 'vm-cgconfig.conf' by @sharnoff in #5127
• monitor/compute_ctl: remove references to the informant by @fprasx in #5115
• page cache: small code cleanups by @arpad-m in #5125
• Upgrade pgvector to 0.5.0 by @vadim2404 in #5132
• Release 2023-08-29 by @github-actions in #5133

Full Changelog: release-3759...release-3795

release-3759

What's Changed

• Release 2023-08-25 by @conradludgate in #5113

Full Changelog: release-3756...release-3759

release-3756

What's Changed

• Added deferrable transaction option to http batch queries by @jawj in #4993
• Upload test results to DB right after generation by @bayandin in #4967
• fsync directory before mark file removal by @LizardWizzard in #4986
• Allow to enable http/pg/pg tenant only auth separately in safekeeper. by @arssher in #4988
• tests: extend timeout in timeline deletion test by @jcsp in #4992
• Disable neon-pool-opt-in by @petuhovskiy in #4995
• Fix test_s3_wal_replay flakiness. by @arssher in #4470
• Fix delete_objects in UnreliableWrapper by @LizardWizzard in #5002
• ephemeral_file: remove FileExt impl (was only used by tests) by @problame in #5003
• Correctly remove orphaned objects in RemoteTimelineClient::delete_all by @LizardWizzard in #5000
• Compaction: unify key and value reference vecs by @arpad-m in #4888
• proxy: reduce global conn pool contention by @conradludgate in #4747
• page cache: newtype the blob_io and ephemeral_file file ids by @problame in #5005
• delta_layer: Restore generic from last week by @koivunej in #5014
• WIP Get ext index from spec by @lubennikovaav in #4978
• ephemeral file: refactor write_blob impl to concentrate mutable state by @problame in #5004
• test_compatibility: run amcheck unconditionally by @bayandin in #4985
• Do not attach deleted tenants by @LizardWizzard in #5008
• proxy: add more sasl logs by @conradludgate in #5012
• Increase timeout once again. by @LizardWizzard in #5021
• fix: format context by @koivunej in #5022
• test: cleanup remote_timeline_client tests by @koivunej in #5013
• proxy: pool connection logs by @conradludgate in #5020
• Make the compaction warning more tolerant by @arpad-m in #5024
• Fix safekeeper recovery with auth by @petuhovskiy in #5035
• test: allow race with flush and stopped queue by @koivunej in #5027
• simplify page-caching of EphemeralFile by @problame in #4994
• rfcs: archive my MVCC S3 Notion Proposal by @problame in #5040
• resume timeline deletions on attach by @LizardWizzard in #5030
• Avoid flakiness in test_timeline_delete_fail_before_local_delete by @LizardWizzard in #5032
• bump VM_BUILDER_VERSION to v0.16.2 by @fprasx in #5031
• refactor: failpoint support by @koivunej in #5033
• Fix DeltaLayer dumping by @arpad-m in #5045
• Resume tenant deletion on attach by @LizardWizzard in #5039
• Remove initial timeline id troubles by @koivunej in #5044
• pageserver: do not read redundant timeline_layers from IndexPart, so that we can remove it later by @jcsp in #4972
• pageserver: refactor Tenant/Timeline args into structs by @jcsp in #5053
• bump vm-builder version to v0.16.3 by @fprasx in #5055
• proxy: remove connections on shutdown by @conradludgate in #5051
• Release 2023-08-22 by @github-actions in #5060
• Enable neon-pool-opt-in by @vadim2404 in #5062

Full Changelog: release-3714...release-3756