Skip to content

Commit 7a25d08

Browse files
patrickbirchpatrickbirch
committed
PS-9765 - Document new features in percona-202401 to percona-202404 tags
 modified: docs/variables.md
1 parent c3bfdb0 commit 7a25d08

File tree

4 files changed

+398
-7
lines changed

4 files changed

+398
-7
lines changed

‎docs/variables.md

Lines changed: 88 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -21,26 +21,28 @@ Also, all variables can exist in one or both of the following scopes:
2121
| [`rocksdb_access_hint_on_compaction_start`](#rocksdb_access_hint_on_compaction_start) |
2222
| [`rocksdb_advise_random_on_open`](#rocksdb_advise_random_on_open) |
2323
| [`rocksdb_allow_concurrent_memtable_write`](#rocksdb_allow_concurrent_memtable_write) |
24-
| [`rocksdb_allow_to_start_after_corruption`](#rocksdb_allow_to_start_after_corruption) |
2524
| [`rocksdb_allow_mmap_reads`](#rocksdb_allow_mmap_reads) |
26-
| [`rocksdb_allow_mmap_writes`](#rocksdb_allow_mmap_writes) |
25+
| [`rocksdb_allow_mmap_writes`](#rocksdb_allow_mmap_writes) |
26+
| [`rocksdb_allow_to_start_after_corruption`](#rocksdb_allow_to_start_after_corruption) |
2727
| [`rocksdb_allow_unsafe_alter`](#rocksdb_allow_unsafe_alter) |
2828
| [`rocksdb_alter_column_default_inplace`](#rocksdb_alter_column_default_inplace) |
2929
| [`rocksdb_alter_table_comment_inplace`](#rocksdb_alter_table_comment_inplace) |
3030
| [`rocksdb_base_background_compactions`](#rocksdb_base_background_compactions) |
3131
| [`rocksdb_blind_delete_primary_key`](#rocksdb_blind_delete_primary_key) |
3232
| [`rocksdb_block_cache_numshardbits`](#rocksdb_block_cache_numshardbits) |
3333
| [`rocksdb_block_cache_size`](#rocksdb_block_cache_size) |
34-
| [`rocksdb_bulk_load_fail_if_not_bottommost_level`](#rocksdb_bulk_load_fail_if_not_bottommost_level) |
35-
| [`rocksdb_bulk_load_partial_index`](#rocksdb_bulk_load_partial_index) |
36-
| [`rocksdb_bulk_load_use_sst_partitioner`](#rocksdb_bulk_load_use_sst_partitioner) |
3734
| [`rocksdb_block_restart_interval`](#rocksdb_block_restart_interval) |
3835
| [`rocksdb_block_size`](#rocksdb_block_size) |
3936
| [`rocksdb_block_size_deviation`](#rocksdb_block_size_deviation) |
4037
| [`rocksdb_bulk_load`](#rocksdb_bulk_load) |
4138
| [`rocksdb_bulk_load_allow_sk`](#rocksdb_bulk_load_allow_sk) |
4239
| [`rocksdb_bulk_load_allow_unsorted`](#rocksdb_bulk_load_allow_unsorted) |
40+
| [`rocksdb_bulk_load_compression_parallel_threads`](#rocksdb_bulk_load_compression_parallel_threads) |
41+
| [`rocksdb_bulk_load_enable_unique_key_check`](#rocksdb_bulk_load_enable_unique_key_check) |
42+
| [`rocksdb_bulk_load_fail_if_not_bottommost_level`](#rocksdb_bulk_load_fail_if_not_bottommost_level) |
43+
| [`rocksdb_bulk_load_partial_index`](#rocksdb_bulk_load_partial_index) |
4344
| [`rocksdb_bulk_load_size`](#rocksdb_bulk_load_size) |
45+
| [`rocksdb_bulk_load_use_sst_partitioner`](#rocksdb_bulk_load_use_sst_partitioner) |
4446
| [`rocksdb_bytes_per_sync`](#rocksdb_bytes_per_sync) |
4547
| [`rocksdb_cache_dump`](#rocksdb_cache_dump) |
4648
| [`rocksdb_cache_high_pri_pool_ratio`](#rocksdb_cache_high_pri_pool_ratio) |
@@ -75,6 +77,7 @@ Also, all variables can exist in one or both of the following scopes:
7577
| [`rocksdb_debug_cardinality_multipler`](#rocksdb_debug_cardinality_multiplier) |
7678
| [`rocksdb_debug_manual_compaction_delay`](#rocksdb_debug_manual_compaction_delay) |
7779
| [`rocksdb_debug_optimizer_no_zero_cardinality`](#rocksdb_debug_optimizer_no_zero_cardinality) |
80+
| [`rocksdb_debug_skip_bloom_filter_check_on_iterator_bounds`](#rocksdb_debug_skip_bloom_filter_check_on_iterator_bounds) |
7881
| [`rocksdb_debug_ttl_ignore_pk`](#rocksdb_debug_ttl_ignore_pk) |
7982
| [`rocksdb_debug_ttl_read_filter_ts`](#rocksdb_debug_ttl_read_filter_ts) |
8083
| [`rocksdb_debug_ttl_rec_ts`](#rocksdb_debug_ttl_rec_ts) |
@@ -88,13 +91,19 @@ Also, all variables can exist in one or both of the following scopes:
8891
| [`rocksdb_enable_bulk_load_api`](#rocksdb_enable_bulk_load_api) |
8992
| [`rocksdb_enable_delete_range_for_drop_index`](#rocksdb_enable_delete_range_for_drop_index) |
9093
| [`rocksdb_enable_insert_with_update_caching`](#rocksdb_enable_insert_with_update_caching) |
94+
|[`rocksdb_enable_instant_ddl`](#rocksdb_enable_instant_ddl)|
95+
|[`rocksdb_enable_instant_ddl_for_append_column`](#rocksdb_enable_instant_ddl_for_append_column)|
96+
|[`rocksdb_enable_instant_ddl_for_column_default_changes`](#rocksdb_enable_instant_ddl_for_column_default_changes)|
97+
|[`rocksdb_enable_instant_ddl_for_drop_index_changes`](#rocksdb_enable_instant_ddl_for_drop_index_changes)|
98+
|[`rocksdb_enable_instant_ddl_for_table_comment_changes`](#rocksdb_enable_instant_ddl_for_table_comment_changes)|
9199
| [`rocksdb_enable_iterate_bounds`](#rocksdb_enable_iterate_bounds) |
92100
| [`rocksdb_enable_pipelined_write`](#rocksdb_enable_pipelined_write) |
93101
| [`rocksdb_enable_remove_orphaned_dropped_cfs`](#rocksdb_enable_remove_orphaned_dropped_cfs) |
94102
| [`rocksdb_enable_ttl`](#rocksdb_enable_ttl) |
95103
| [`rocksdb_enable_ttl_read_filtering`](#rocksdb_enable_ttl_read_filtering) |
96104
| [`rocksdb_enable_thread_tracking`](#rocksdb_enable_thread_tracking) |
97105
| [`rocksdb_enable_write_thread_adaptive_yield`](#rocksdb_enable_write_thread_adaptive_yield) |
106+
| [`rocksdb_enable_udt_in_mem`](#rocksdb_enable_udt_in_mem) |
98107
| [`rocksdb_error_if_exists`](#rocksdb_error_if_exists) |
99108
| [`rocksdb_error_on_suboptimal_collation`](#rocksdb_error_on_suboptimal_collation) |
100109
| [`rocksdb_file_checksums`](#rocksdb_file_checksums) |
@@ -109,6 +118,8 @@ Also, all variables can exist in one or both of the following scopes:
109118
| [`rocksdb_ignore_unknown_options`](#rocksdb_ignore_unknown_options) |
110119
| [`rocksdb_index_type`](#rocksdb_index_type) |
111120
| [`rocksdb_info_log_level`](#rocksdb_info_log_level) |
121+
|[`rocksdb_invalid_create_option_action`](#rocksdb_invalid_create_option_action)|
122+
|[`rocksdb_io_error_action`](#rocksdb_io_error_action)|
112123
| [`rocksdb_is_fd_close_on_exec`](#rocksdb_is_fd_close_on_exec) |
113124
| [`rocksdb_keep_log_file_num`](#rocksdb_keep_log_file_num) |
114125
| [`rocksdb_large_prefix`](#rocksdb_large_prefix) |
@@ -173,6 +184,7 @@ Also, all variables can exist in one or both of the following scopes:
173184
| [`rocksdb_strict_collation_check`](#rocksdb_strict_collation_check) |
174185
| [`rocksdb_strict_collation_exceptions`](#rocksdb_strict_collation_exceptions) |
175186
| [`rocksdb_table_cache_numshardbits`](#rocksdb_table_cache_numshardbits) |
187+
|[`rocksdb_table_stats_skip_system_cf`](#rocksdb_table_stats_skip_system_cf)|
176188
| [`rocksdb_table_stats_background_thread_nice_value`](#rocksdb_table_stats_background_thread_nice_value) |
177189
| [`rocksdb_table_stats_max_num_rows_scanned`](#rocksdb_table_stats_max_num_rows_scanned) |
178190
| [`rocksdb_table_stats_recalc_threshold_count`](#rocksdb_table_stats_recalc_threshold_count) |
@@ -193,6 +205,7 @@ Also, all variables can exist in one or both of the following scopes:
193205
| [`rocksdb_use_direct_reads`](#rocksdb_use_direct_reads) |
194206
| [`rocksdb_use_fsync`](#rocksdb_use_fsync) |
195207
| [`rocksdb_use_hyper_clock_cache`](#rocksdb_use_hyper_clock_cache) |
208+
|[`rocksdb_use_io_uring`](#rocksdb_use_io_uring) |
196209
| [`rocksdb_use_write_buffer_manager`](#rocksdb_use_write_buffer_manager) |
197210
| [`rocksdb_validate_tables`](#rocksdb_validate_tables) |
198211
| [`rocksdb_verify_row_debug_checksums`](#rocksdb_verify_row_debug_checksums) |
@@ -218,6 +231,8 @@ Also, all variables can exist in one or both of the following scopes:
218231
| Data type | String or numeric |
219232
| Default | NORMAL or 1 |
220233

234+
As of Percona Server for MySQL 8.0.42, the `--rocksdb-access-hint-on-compaction-start` variable has been removed.
235+
221236
Specifies the file access pattern once a compaction is started,
222237
applied to all input files of a compaction.
223238
Possible values are:
@@ -422,6 +437,32 @@ Default value is `536870912`.
422437

423438
Maximum value is `9223372036854775807`.
424439

440+
### `rocksdb_bulk_load_compression_parallel_threads`
441+
442+
| Option | Description |
443+
|--------------|----------------------------|
444+
| Command-line | --rocksdb_bulk_load_compression_parallel_threads |
445+
| Dynamic | Yes |
446+
| Scope | Global, Session |
447+
| Data type | Numeric |
448+
| Default | 1 |
449+
| Minimum | 1 |
450+
| Maximum | 1024 |
451+
452+
Percona Server for MySQL 8.0.42 introduces a new variable that controls the number of parallel worker threads. These worker threads actively compress Sorted Strings Table (SST) data blocks during a bulk load operation.
453+
454+
### `rocksdb_bulk_load_enable_unique_key_check`
455+
456+
| Option | Description |
457+
|--------------|----------------------------|
458+
| Command-line | --rocksdb_bulk_load_enable_unique_key_check |
459+
| Dynamic | Yes |
460+
| Scope | Global, Session |
461+
| Data type | Boolean |
462+
| Default | OFF |
463+
464+
Percona Server for MySQL 8.0.42 introduces a new variable that controls unique key constraint checking during bulk load operations. However, users can only modify this variable when a bulk load operation is not active.
465+
425466
### `rocksdb_bulk_load_fail_if_not_bottommost_level`
426467

427468
| Option | Description |
@@ -813,7 +854,9 @@ This variable acts as a trigger. Set the variable to `ON`, `rocksdb-compact-lzer
813854
| Dynamic | Yes |
814855
| Scope | Global |
815856
| Data type | Numeric |
816-
| Default | 0 |
857+
| Default | 2097152 |
858+
859+
As of Percona Server for MySQL 8.0.43, the `--rocksdb-compaction-readahead-size` default value is `2097152`. In previous versions, the default value was `0`.
817860

818861
Specifies the size of reads to perform ahead of compaction.
819862
Default value is `0`.
@@ -1116,6 +1159,18 @@ This variable specifies a sleep, in seconds, to simulate long-running compaction
11161159
Specifies whether MyRocks should prevent zero cardinality
11171160
by always overriding it with some value.
11181161

1162+
### `rocksdb_debug_skip_bloom_filter_check_on_iterator_bounds`
1163+
1164+
| Option | Description |
1165+
|--------------|----------------------------|
1166+
| Command-line | --rocksdb_debug_skip_bloom_filter_check_on_iterator_bounds |
1167+
| Dynamic | Yes |
1168+
| Scope | Global |
1169+
| Data type | Boolean |
1170+
| Default | OFF |
1171+
1172+
Percona Server for MySQL 8.0.42 introduces a new variable that allows users to set up iterator bounds for RocksDB. This functionality applies even when the query range conditions would typically allow the use of bloom filters. This variable provides more granular control over data retrieval, even in scenarios where bloom filters could optimize query performance.
1173+
11191174
### `rocksdb_debug_ttl_ignore_pk`
11201175

11211176
| Option | Description |
@@ -1271,7 +1326,12 @@ use or other misuse can have serious side effects to the server instance.
12711326
| Dynamic | Yes |
12721327
| Scope | Global |
12731328
| Data type | Boolean |
1274-
| Default | ON |
1329+
| Default | OFF |
1330+
1331+
1332+
As of Percona Server for MySQL 8.0.42, the `--rocksdb-disable-instant-ddl` variable has been deprecated and may be removed in a future release.
1333+
1334+
As of Percona Server for MySQL 8.0.43, the `--rocksdb-disable-instant-ddl` default value is `OFF`. In previous versions, the default value was `ON`.
12751335

12761336
The variable was implemented in [Percona Server for MySQL 8.0.33-25](release-notes/8.0.33-25.md).
12771337

@@ -1464,10 +1524,24 @@ insensitive). Enabled by default.
14641524
| Data type | Boolean |
14651525
| Default | OFF |
14661526

1527+
1528+
As of Percona Server for MySQL 8.0.43, the `--rocksdb-file-checksums` data type is `ENUM`. In previous versions, the data type was `Boolean`.
1529+
1530+
As of Percona Server for MySQL 8.0.43, the `--rocksdb-file-checksums` default value is `CHECKSUM_OFF`. In previous versions, the default value was `OFF`.
1531+
1532+
As of Percona Server for MySQL 8.0.43, the available values are:
1533+
1534+
* `CHECKSUM_OFF`: Nothing happens.
1535+
1536+
* `CHECKSUM_WRITE_ONLY`: Write checksums, but skip verification.
1537+
1538+
* `CHECKSUM_WRITE_AND_VERIFY`: Write checksums and verify on DB::open.
1539+
14671540
This variable has been implemented in [Percona Server for MySQL 8.0.36-28](.//release-notes/8.0.36-28.md).
14681541

14691542
This variable controls whether to write and check RocksDB file-level checksums. The default value is `OFF` which means the variable is disabled.
14701543

1544+
14711545
### `rocksdb_flush_log_at_trx_commit`
14721546

14731547
| Option | Description |
@@ -1676,6 +1750,8 @@ Enabled by default.
16761750
| Data type | Boolean |
16771751
| Default | ON |
16781752

1753+
As of Percona Server for MySQL 8.0.42, the `--rocksdb-large-prefix` variable has been removed.
1754+
16791755
This variable is deprecated in `Percona Server for MySQL 8.0.36-28` and will be removed in a future release.
16801756

16811757
When enabled, this option allows index key prefixes longer than 767 bytes (up to 3072 bytes). The values for `rocksdb_large_prefix` should be the same between source and replica.
@@ -2604,6 +2680,8 @@ Disabled by default.
26042680
| Data type | Boolean |
26052681
| Default | ON |
26062682

2683+
As of Percona Server for MySQL 8.0.42, the `--rocksdb-strict-collation-check` variable has been removed.
2684+
26072685
This variable is considered **deprecated** in version 8.0.23-14.
26082686

26092687
Specifies whether to check and verify
@@ -2620,6 +2698,9 @@ Enabled by default.
26202698
| Data type | String |
26212699
| Default | |
26222700

2701+
2702+
As of Percona Server for MySQL 8.0.42, the `--rocksdb-strict-collation-exceptions` variable has been removed.
2703+
26232704
This variable is considered **deprecated** in version 8.0.23-14.
26242705

26252706
Lists tables (as a regular expression) that should be excluded

‎generate_table.py

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
import yaml
2+
import re
3+
4+
def read_original_content(file_path):
5+
try:
6+
with open(file_path, "r") as f:
7+
return f.read()
8+
except FileNotFoundError:
9+
print(f"Warning: {file_path} not found. Creating a new file.")
10+
return ""
11+
12+
def find_insertion_point(content):
13+
# Find the position after the bullet points in the introduction
14+
# This pattern looks for the last line that starts with a bullet point (*)
15+
bullet_matches = list(re.finditer(r'^\s*\*.*$', content, re.MULTILINE))
16+
17+
if not bullet_matches:
18+
# If no bullet points found, look for the end of an introductory paragraph
19+
paragraph_ends = list(re.finditer(r'\n\n', content))
20+
if paragraph_ends:
21+
# Return the end of the first paragraph if no bullets found
22+
return paragraph_ends[0].end()
23+
else:
24+
# If no clear paragraphs, just append to the end
25+
return len(content)
26+
27+
# Return the position after the last bullet point plus a newline
28+
last_bullet = bullet_matches[-1]
29+
return last_bullet.end()
30+
31+
def generate_variable_table():
32+
# Load the YAML file
33+
with open("myrocks-variables.yaml", "r") as f:
34+
variables = yaml.safe_load(f)
35+
36+
# Sort the variable names alphabetically
37+
sorted_vars = sorted(variables.keys())
38+
39+
# Generate Markdown table
40+
table_lines = ["| Variable |", "|----------|"]
41+
for var in sorted_vars:
42+
table_lines.append(f"| [`{var}`](#{var}) |")
43+
44+
return "\n".join(table_lines)
45+
46+
def main():
47+
# Read the original content from variables.md.backup
48+
original_content = read_original_content("docs/variables.md.backup")
49+
50+
# Find the appropriate insertion point
51+
insertion_point = find_insertion_point(original_content)
52+
53+
# Generate the variable table
54+
variable_table = generate_variable_table()
55+
56+
# Combine content with the table inserted at the appropriate position
57+
new_content = (
58+
original_content[:insertion_point] +
59+
"\n\n" + variable_table + "\n\n" +
60+
original_content[insertion_point:]
61+
)
62+
63+
# Write the combined content back to variables.md
64+
with open("docs/variables.md", "w") as out:
65+
out.write(new_content)
66+
67+
print("✅ Markdown table inserted into docs/variables.md while preserving existing content")
68+
69+
if __name__ == "__main__":
70+
main()

‎generate_yaml.py

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
import re
2+
import yaml
3+
4+
def parse_variables_from_md(md_file):
5+
variables = {}
6+
7+
with open(md_file, "r") as file:
8+
md_content = file.read()
9+
10+
# Regex pattern to extract variable names and links
11+
pattern = r'\[`([a-zA-Z0-9_]+)`\]\(#([a-zA-Z0-9_]+)\)'
12+
13+
matches = re.findall(pattern, md_content)
14+
15+
# Loop through matches and format them into a dictionary
16+
for match in matches:
17+
variable_name, link = match
18+
# The link itself could be used as the description or you can format it
19+
description = f"Link: {link}" # Or customize description extraction logic as needed
20+
variables[variable_name] = description
21+
22+
return variables
23+
24+
def write_yaml(variables, yaml_file):
25+
with open(yaml_file, "w") as file:
26+
yaml.dump(variables, file, default_flow_style=False)
27+
28+
def main():
29+
md_file = "docs/variables.md" # Path to the markdown file
30+
yaml_file = "myrocks-variables.yaml" # Output YAML file
31+
32+
variables = parse_variables_from_md(md_file)
33+
write_yaml(variables, yaml_file)
34+
35+
print(f"YAML file '{yaml_file}' has been generated successfully.")
36+
37+
if __name__ == "__main__":
38+
main()

0 commit comments

Comments
 (0)