Releases: neondatabase/neon
Releases · neondatabase/neon
release-6616
What's Changed
- Lock the correct revision of rust-postgres crates by @hlinnaka in #8960
- Update toml and toml_edit crates by @hlinnaka in #8963
- Update strum by @hlinnaka in #8962
- Remove TEST_SHARED_FIXTURES by @hlinnaka in #8965
- feat(pageserver): split delta writer automatically determines key range by @skyzh in #8850
- Update parquet crate to a released version by @hlinnaka in #8961
- Specialize WalIngest on PostgreSQL version by @MMeent in #8904
- Remove async_trait from the Handler trait by @arpad-m in #8958
- reinstate write-path key check by @jcsp in #8973
- proxy: Handle all let underscore instances by @cloneable in #8898
- libs: tweak PageserverUtilization::is_overloaded by @jcsp in #8946
- Misc test cleanup by @hlinnaka in #8979
- Collect the last of on-demand WAL download in CreateReplicationSlot reverts by @tristan957 in #8982
- Prototype implementation for private access poc by @stradig in #8976
- [compute/postgres] feature: PostgreSQL 17 by @MMeent in #8573
- Fix broken image for PG17 by @MMeent in #8998
- CI(gather-rust-build-stats): set PQ_LIB_DIR to Postgres 17 by @bayandin in #9001
- Storage & Compute release 2024-09-16 by @vipvap in #9005
Full Changelog: release-6594...release-6616
release-6594
What's Changed
- storage controller: eliminate ensure_attached by @jcsp in #8875
- Implement archival_config timeline endpoint in the storage controller by @arpad-m in #8680
- bottommost-compaction: remove dead code / rectify cfg!()s by @problame in #8884
- fixup(#8839):
test_forward_compatibility
needs to allow lag warning as well by @problame in #8891 - page_cache/layer load: correctly classify layer summary block reads by @problame in #8885
- pageserver: reuse idempotency keys across metrics sinks by @erikgrinaker in #8876
- storcon: provide an az id in metadata.json from neon local by @VladLazar in #8897
- safekeeper: add endpoint resetting uploaded partial segment state. by @arssher in #8877
- safekeeper: check for non-consecutive writes in safekeeper.rs by @arssher in #8781
- pageserver: separate metadata and data pages in DatadirModification by @jcsp in #8621
- build(deps): bump flask-cors from 4.0.1 to 5.0.0 by @dependabot in #8899
- fix(pageserver): is_archived should be optional by @skyzh in #8902
- fix(storage-scrubber): wrong trial condition by @skyzh in #8905
- storcon: make hb interval an argument and speed up tests by @VladLazar in #8880
- storcon: boilerplate to upsert safekeeper records on deploy by @koivunej in #8879
- Fix submodule refs to point to the correct REL_X_STABLE_neon branches by @hlinnaka in #8910
- build(deps): bump cryptography from 42.0.4 to 43.0.1 by @dependabot in #8908
- storage controller: read from database in validate API by @jcsp in #8784
- Update submodule reference for vendor/postgres-v14 by @hlinnaka in #8913
- fix(pageserver): more information on aux v1 warnings by @skyzh in #8906
- storcon: handle fills including high tput tenants more gracefully by @VladLazar in #8865
- proxy: don't let one timeout eat entire retry budget by @cloneable in #8924
- refactor(pageserver): rely on serde derive for toml deserialization by @problame in #7656
- build: sync between benchies by @koivunej in #8919
- Fix precedence issue causing yielding loop to never yield. by @vladov3000 in #8922
- Add installation instructions for m4 on mac by @stradig in #8929
- storcon: make pageserver AZ id mandatory by @VladLazar in #8856
- update svg_fmt to 0.4.3 by @problame in #8930
- compact_level0_phase1: ignore access mode config, always do streaming-kmerge without validation by @problame in #8934
- Update Rust to 1.81.0 by @arpad-m in #8939
- storcon: track preferred AZ for each tenant shard by @VladLazar in #8937
- Fix 1.82 clippy lint too_long_first_doc_paragraph by @arpad-m in #8941
- safekeeper: fix endpoint restart immediately after xlog switch. by @arssher in #8943
- proxy: clean up code and lints for 1.81 and 1.82 by @cloneable in #8945
- safekeeper: add term_bump endpoint by @arssher in #8753
- safekeeper: fsync filesystem on start by @arssher in #8947
- CI(label-for-external-users): add retry logic for unexpected errors by @bayandin in #8938
- feat(storage-controller): add node shards api by @skyzh in #8896
- Address 1.82 clippy lints by @arpad-m in #8944
- feat(storcon): forward gc blocking and unblocking by @koivunej in #8956
- push images to prod ACR by @fcdm in #8940
- Fix rust warnings on macOS by @bayandin in #8955
- Storage release 2024-09-07 by @vipvap in #8959
New Contributors
- @erikgrinaker made their first contribution in #8876
- @vladov3000 made their first contribution in #8922
- @stradig made their first contribution in #8929
Full Changelog: release-6548...release-6594
release-6548
What's Changed
- safekeeper: reorder routes and their handlers. by @arssher in #8831
- Fix the pg_hintplan flakyness by @a-masterov in #8834
- proxy: remove dead code by @conradludgate in #8847
- proxy: Rename backend types and variants as prep for refactor by @cloneable in #8845
- fix(walproposer): Do not restart on safekeepers reordering by @ololobus in #8840
- Remove support for pageserver <-> compute protocol version 1 by @hlinnaka in #8774
- tests: Remove unnecessary entries from list of allowed errors by @hlinnaka in #8199
- .github: direct feature requests to the feedback form by @jcsp in #8849
- bench: flush before shutting down by @koivunej in #8844
- docs: rolling storage controller restarts RFC by @VladLazar in #8310
- pageserver: do vectored read on each dio-aligned section once by @yliang412 in #8763
- storcon: track pageserver availability zone by @VladLazar in #8852
- fix(storage-scrubber): make retry error into warnings by @skyzh in #8851
- bypass
PageCache
forInMemoryLayer
+ avoidValue::deser
on L0 flush by @problame in #8537 - feat(compute_ctl): add periodic
lease lsn
requests for static computes by @prepor in #7994 - Undo walloging replorgin file on checkpoint by @knizhnik in #8794
- ingest: rate-limited warning if WAL commit timestamps lags for > wait_lsn_timeout by @problame in #8839
- proxy+pageserver: shared leaky bucket impl by @conradludgate in #8539
- pageserver: default to 4MiB stack size and add env var to control it by @problame in #8862
- Don't unarchive a timeline if its ancestor is archived by @arpad-m in #8853
- tenant background loops: periodic log message if long-running iteration by @problame in #8832
- proxy: improve test performance by @conradludgate in #8863
- fix(pageserver): add dry-run to force compact API by @skyzh in #8859
- fix(pageserver): reject non-i128 key on the write path by @skyzh in #8648
- Remove async_trait usages from safekeeper and neon_local by @arpad-m in #8864
- proxy: fix password hash cancellation by @conradludgate in #8868
- chore: speed up testing by @conradludgate in #8874
- storage controller: sleep between compute notify retries by @jcsp in #8869
- CI(promote-compatibility-data): fix job by @bayandin in #8871
- feat(storage-scrubber): check layer map valid by @skyzh in #8867
- pageserver: set default io_buffer_alignment to 512 bytes by @yliang412 in #8878
- fix(pageserver): fire layer eviction alert only when it's visible by @skyzh in #8882
- Add safekeeper scrubber test by @arpad-m in #8785
- Storage & Compute release 2024-09-02 by @vipvap in #8883
Full Changelog: release-6514...release-6548
release-6514
What's Changed
- feat(compute): Introduce new compute_subscriptions_count metric by @ololobus in #8796
- proxy: clippy lints: handle some low hanging fruit by @cloneable in #8829
- Timeline archival test by @arpad-m in #8824
- pageserver: startup: ensure local disk state is durable by @problame in #8835
- feat(pageserver): use split layer writer in gc-compaction by @skyzh in #8608
- fix(sql-exporter): Fix typo in gauge metric type by @ololobus in #8842
- Compute release 2024-08-28 by @lubennikovaav in #8858
Full Changelog: release-6506...release-6514
release-6506
What's Changed
- proxy: Fix some warnings by extended clippy checks by @cloneable in #8748
- l0_flush: remove support for mode
page-cached
by @problame in #8739 - Migrate physical GC and scan_metadata to remote_storage by @arpad-m in #8673
- Update aws -> infra for repo rename by @arpad-m in #8755
- scrubber: remove _generic postfix and two unused functions by @arpad-m in #8761
- safekeeper: check for non-consecutive writes in safekeeper.rs by @arssher in #8640
- task_mgr::spawn: require a
TenantId
by @problame in #8462 - Make Postgres 16 default version by @bayandin in #8745
- Revert "safekeeper: check for non-consecutive writes in safekeeper.rs" by @jcsp in #8771
- storcon: run db migrations after step down sequence by @VladLazar in #8756
- proxy: make TLS and WS handling optional by @conradludgate in #8752
- storcon: add peer jwt token by @VladLazar in #8764
- pageserver: add metric pageserver_secondary_heatmap_total_size by @jcsp in #8768
- impr(pageserver): abort on fatal I/O writer error by @skyzh in #8777
- Benchmarking: need to checkout actions to download Neon artifacts by @Bodobolero in #8770
- pageserver: add counter for wait time on background loop semaphore by @problame in #8769
- storcon: handle heartbeater shutdown gracefully by @koivunej in #8767
- test: test_timeline_ancestor_detach_errors rare allowed_error by @koivunej in #8782
- CI(build-and-test): run regression tests on arm by @bayandin in #8552
- remove workspace hack from libs by @conradludgate in #8780
- Add compute_logical_snapshot_files metric by @tristan957 in #8762
- safekeeper: add listing timelines by @arssher in #8789
- test: fix more flaky due to graceful shutdown by @koivunej in #8787
- Add compute_receive_lsn metric by @tristan957 in #8750
- test: avoid some too long shutdowns by flushing before shutdown by @koivunej in #8772
- fix(pageserver): unify initdb optimization for sparse keyspaces; fix force img generation by @skyzh in #8776
- pageserver: batch InMemoryLayer
put
s, remove need to sort items by LSN during ingest by @jcsp in #8591 - Sleep in test_scrubber_physical_gc by @arpad-m in #8798
- Revert "Use sycnhronous commit for logical replicaiton worker (#8645)" by @ololobus in #8792
- proxy: Enable stricter/pedantic clippy checks by @cloneable in #8775
- test_compatibility: flush in the end by @koivunej in #8804
- Fix race condition in LRU list update in get_cached_relsize by @knizhnik in #8807
- feat(pageserver): warn on aux v1 tenants + default to v2 by @skyzh in #8625
- Upgrade Postgres to 14.13, 15.8, and 16.4 by @tristan957 in #8704
- Document how to use "git merge" for PostgreSQL minor version upgrades. by @hlinnaka in #8692
- Require poetry >=1.8 by @MMeent in #8812
- Update serde_json to 1.0.125 by @arpad-m in #8813
- CI(autocomment): add arch to build type by @bayandin in #8809
- feat(pageserver): remove features=testing restriction for compact by @skyzh in #8815
- fix(pageserver): l0 check criteria by @skyzh in #8797
- test: copy dict to avoid error on retry by @koivunej in #8811
- storcon_cli: add support for drain and fill operations by @VladLazar in #8791
- update diesel by @conradludgate in #8816
- impr(pageserver): error if keys are unordered in merge iter by @skyzh in #8818
- proxy: flesh out JWT verification code by @conradludgate in #8805
- controller: use PageserverUtilization for scheduling by @jcsp in #8711
- storage controller: enable timeline CRUD operations to run concurrently with reconciliation & make them safer by @jcsp in #8783
- proxy: local auth backend by @conradludgate in #8806
- proxy: new local-proxy application by @conradludgate in #8736
- chore: improve markdown formatting by @mrl5 in #8825
- Storage & Compute release 2024-08-26 by @vipvap in #8827
New Contributors
- @cloneable made their first contribution in #8748
- @mrl5 made their first contribution in #8825
Full Changelog: release-6450...release-6506
release-6450
What's Changed
- Adopt list_streaming in tenant deletion by @arpad-m in #8504
- Add a test for clickhouse as a logical replication consumer by @a-masterov in #8408
- test: deflake test_duplicate_creation by @koivunej in #8536
- pageserver: fix return code from secondary_download_handler by @jcsp in #8508
- pageserver: fail if
id
is present in pageserver.toml by @problame in #8489 - l0_flush: use mode=direct by default => coverage in automated tests by @problame in #8534
- scrubber: enable cleaning up garbage tenants from known deletion bugs, add object age safety check by @jcsp in #8461
- Add metrics for input data considered and taken for compression by @arpad-m in #8522
- scrubber: add remote_storage based listing APIs and use them in find-large-objects by @arpad-m in #8541
- CI(benchmarking): make neonvm default provisioner by @bayandin in #8538
- [neon/acr] impr: push to ACR while building images by @antonyc in #8545
- feat(storcon): store scrubber metadata scan result by @yliang412 in #8480
- feat(scrubber): post
scan_metadata
results to storage controller by @yliang412 in #8502 - cleanup(compact_level0_phase1): some commentary and wrapping into block expressions by @problame in #8544
- cicd: change Azure storage details [1/2] by @fcdm in #8553
- compaction_level0_phase1: bypass PS PageCache for data blocks by @problame in #8543
- CI(regress-tests): run less regression tests by @bayandin in #8561
- feat(pageserver): support btm-gc-compaction for child branches by @skyzh in #8519
- refactor(page_service): Timeline gate guard holding + cancellation + shutdown by @problame in #8339
- Fix negative replication delay metric by @tristan957 in #8520
- cicd: change Azure storage details [2/2] by @fcdm in #8562
- controller: simplify reconciler generation increment logic by @jcsp in #8560
- storage_scrubber: migrate FindGarbage to remote_storage by @arpad-m in #8548
- CI(create-test-report): fix missing benchmark results in Allure report by @bayandin in #8540
- test(pageserver): add test_gc_feedback_with_snapshots by @skyzh in #8474
- pageserver: shutdown all walredo managers 8s into shutdown by @problame in #8572
- Reduce linux-raw-sys duplication by @arpad-m in #8577
- pageserver: add layer visibility calculation by @jcsp in #8511
- storage-scrubber: log version on start by @skyzh in #8571
- fix(pageserver): skip existing layers for btm-gc-compaction by @skyzh in #8498
- test_runner: don't create artifacts if Allure is not enabled by @bayandin in #8580
- pageserver: refine how we delete timelines after shard split by @jcsp in #8436
- storage_controller: start adding chaos hooks by @jcsp in #7946
- Wait for completion of the upload queue in flush_frozen_layer by @arpad-m in #8550
- Improve safekeepers eviction rate limiting by @petuhovskiy in #8456
- tests: add test_historic_storage_formats by @jcsp in #8423
- fix(pageserver): deadlock in gc-compaction by @skyzh in #8590
- feat(pageserver): support auto split layers based on size by @skyzh in #8574
- safekeeper: remove unused partial_backup_enabled option by @dotdister in #8547
- feat(pageserver): support split delta layers by @skyzh in #8599
- Update Postgres versions to use smgrexists() instead of access() to check if Oid is used by @knizhnik in #8597
- CI(trigger-e2e-tests): wait for promote-images job from the last commit by @bayandin in #8592
- CI(trigger-e2e-tests): fix deadlock with Build and Test workflow by @bayandin in #8606
- fix: allow awaiting logical size for root timelines by @koivunej in #8604
- CI(benchmarking): set pub/sub projects for LR tests by @bayandin in #8483
- feat(pageserver): support dry-run for gc-compaction, add statistics by @skyzh in #8557
- fix: make Timeline::set_disk_consistent_lsn use fetch_max by @koivunej in #8311
- feat: persistent gc blocking by @koivunej in #8600
- pageserver: remove legacy read path by @VladLazar in #8601
- chore: bump index part version by @koivunej in #8611
- storage_scrubber: migrate scan_safekeeper_metadata to remote_storage by @arpad-m in #8595
- Add package-mode=false to poetry. by @arssher in #8609
- Add a test using Debezium as a client for the logical replication by @a-masterov in #8568
- enable rum test by @a-masterov in #8380
- tests: improve stability of
test_storage_controller_many_tenants
by @jcsp in #8607 - pageserver: fixes for layer visibility metric by @jcsp in #8603
- pageserver: use layer visibility when composing heatmap by @jcsp in #8616
- pageserver: add
bench_ingest
by @jcsp in #7409 - scrubber: clean up
scan_metadata
before prod by @yliang412 in #8565 - storage broker: only print one line for version and build tag in init by @arpad-m in #8624
- Lower level for timeline cancellations during gc by @arpad-m in #8626
- Add missing colon to ArchivalConfigRequest specification by @arpad-m in #8627
- proxy: random changes by @conradludgate in #8602
- fix: EphemeralFiles can outlive their Timeline via
enum LayerManager
by @koivunej in #8229 - fix(pageserver): dump the key when it's invalid by @skyzh in #8633
- refactor(timeline_detach_ancestor): replace ordered reparented with a hashset by @koivunej in #8629
- cicd: push build-tools image to ACR as well by @fcdm in #8638
- fix: drain completed page_service connections by @koivunej in #8632
- feat(pageserver): add direct io pageserver config by @yliang412 in #8622
- pageserver: make bench_ingest build (but panic) on macOS by @jcsp in #8641
- Use sycnhronous commit for logical replicaiton worker by @knizhnik in #8645
- fix(test): do not fail test for filesystem race by @koivunej in #8643
- fix: stop leaking BackgroundPurges by @koivunej in #8650
- pageserver: remove paranoia double-calculation of retain_lsns by @jcsp in #8617
- tests: don't require kafka client for regular tests by @jcsp in #8662
- CI(build-tools): update Rust, Python, Mold by @bayandin in #8667
- Dockerfiles: fix LegacyKeyValueFormat & JSONArgsRecommended by @bayandin in #8664
- Run a subset of benchmarking job steps on GitHub action runners in Azure - closer to the system under test by @Bodobolero in #8651
- Revert "proxy: update tokio-postgres to allow arbitrary config params (#8076)" by @conradludgate in #8654
- fix(neon): disable create tablespace stmt by @skyzh in #8657
- CI(pin-build-tools-image): fix permissions for Azure login by @bayandin in #8671
- pageserver: don't treat NotInitialized::Stopped as unexpected by @jcsp in #8675
- storcon: skip draining shard if it's secondary is lagging too much by @VladLazar in #8644
- Dockerfiles: remove cachepot by @bayandin in #86...
release-6299
What's Changed
Full Changelog: release-6250...release-6299
release-proxy-6107
Merge pull request #8647 from neondatabase/rc/proxy/2024-08-08 Proxy release 2024-08-08
release-6250
What's Changed
Full Changelog: release-6248...release-6250
release-6248
What's Changed
Full Changelog: release-6210...release-6248