Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Vfs object storage testing #32

Merged
merged 173 commits into from
Jan 29, 2024
Merged

Vfs object storage testing #32

merged 173 commits into from
Jan 29, 2024

Conversation

strtgbb
Copy link
Collaborator

@strtgbb strtgbb commented Jan 3, 2024 •
edited
Loading

Adds test suite dedicated to vfs and limited vfs support to other test suites.

Latest test run with 23.8 https://github.com/Altinity/clickhouse-regression/actions/runs/7697840221

Suite with minio has been added to runners:

Need to review:

  • How to handle vfs on the big s3 suites (tiered storage, s3)? How best to update actions workflow?
  • A few test suites that use s3 have had a module level added for normal|vfs, in some cases, modifying individual tests would be better

strtgbb added 24 commits December 28, 2023 12:26
@strtgbb
Hack zero copy tests to use allow_object_storage_vfs instead of allow…
ef77a51 
…_remote_fs_zero_copy_replication
@strtgbb
refatoring s3 regression: fix storage module call
3b314e1
@strtgbb
Add VFS S3 modules. Fix most XFails
a80fb75
@strtgbb
Clean up s3 feature names and fix xfails
0779219
@strtgbb
Update s3 zero copy tests to respect allow_object_storage_vfs
9e34652
@strtgbb
Fix more xfails
093149e
@strtgbb
Apply black to s3
1010d8e
@strtgbb
Add vfs toggle to s3, benchmark and tiered storage
ec72987
@strtgbb
Refactor order of s3 modules
80a50c2
@strtgbb
Fix location of add_vfs_config call
de01333
@strtgbb
Refactor storage_kwargs
fbcb37d
@strtgbb
Remove redundant context value
eb5060c
@strtgbb
Update s3 xfails for new module order
987dc0b
@strtgbb
Move add_vfs_config call inside relevant module context
b7dfa1b
@strtgbb
Add object_storage_mode feature level to benchmark and tiered storage
d974ce9
@strtgbb
Update tiered storage xfail paths
68d6aaa
@strtgbb
Fix zero copy benchmark with vfs
e4fa9a8
@strtgbb
Fix s3 disk cache xfail
ec91f42
@strtgbb
Fix s3 invalid disk cache xfail
baffe0b
@strtgbb
cleanup
48cbb46
@strtgbb
Fix benchmark vfs version check
8b36325
@strtgbb
rename with_vfs to allow_vfs to better match the config setting
7fe4bb0
@strtgbb
Move allow-vfs to top level arguments
605e109
@strtgbb
Add allow_vfs to all regression.py
9b83690
@strtgbb strtgbb force-pushed the vfs_object_storage_testing branch 4 times, most recently from 10e6565 to 87bbeae Compare January 8, 2024 15:20
strtgbb added 28 commits January 24, 2024 13:03
@strtgbb
add retries to vfs no duplication test
bb71bf6
@strtgbb
Add vfs optimize test
a7938a0
@strtgbb
Apply black
1d7ac66
@strtgbb
vfs fetch test
3fc9bfa
@strtgbb
Add stable bucket size checks to fix vfs no duplication test
c544404
@strtgbb
rearrange vfs requirements
d0e2264
@strtgbb
fix vfs ttl tests
bbe9579
@strtgbb
Add vfs settings reload requirement
699533d
@strtgbb
vfs alter column tests
4971498
@strtgbb
vfs order by test
34d111a
@strtgbb
vfs sample by test
f2c88a5
@strtgbb
Add exitcode checks to alter helpers
1f07f3c
@strtgbb
vfs alter update test
7739157
@strtgbb
vfs alter index test
3ea659e
@strtgbb
vfs check lightweight delete
92ef71c
@strtgbb
vfs alter projection test
a6eb8bc
@strtgbb
Update s3 regression.py for better vfs support
b3f958e
@strtgbb
rework 0-copy vfs support
133dd25
@strtgbb
update 0-copy vfs xfails
05a388f
@strtgbb
vfs for s3 sanity
3f2666d
@strtgbb
Update --allow-vfs help string
0d09055
@strtgbb
Add vfs support to s3 backup, disk, invalid disk
6aeaf4c
@strtgbb
Merge remote-tracking branch 'origin/main' into vfs_object_storage_te…
bf9bfa8 
…sting
@strtgbb
Fix lightweight delete xfails
cd5cd44
@strtgbb
Fix s3 xfail
6423731
@strtgbb
Ffail vfs benchmark
c2c4526
@strtgbb
Ffail vfs suite <24
26c4406
@strtgbb
Merge remote-tracking branch 'origin/main' into vfs_object_storage_te…
46704ae 
…sting
@strtgbb strtgbb merged commit 2732fd3 into main Jan 29, 2024
117 of 126 checks passed
@strtgbb strtgbb deleted the vfs_object_storage_testing branch January 29, 2024 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@strtgbb