Skip to content
This repository was archived by the owner on Aug 9, 2026. It is now read-only.

README.md

Latest commit

 

History

History
143 lines (109 loc) · 6.57 KB

README.md

File metadata and controls

143 lines (109 loc) · 6.57 KB
Raw

OxCache Evaluation — SYSTOR '26 Reproduction Guide

This directory contains the complete pipeline used to produce the figures and tables in "When Do Zoned Namespaces SSDs Matter? A Comparative Study of Cache Workloads" (SYSTOR '26). Every script here was used for the published paper.

Terminology used throughout the pipeline and in run filenames:

  • promotional = Zone LRU eviction, chunk = Chunk LRU eviction.
  • nvme1n1 = block-interface SSD (WD Ultrastar DC SN540), nvme0n2 = ZNS SSD (WD Ultrastar DC ZN540).

What produces each paper artifact

Paper artifact Generated by Output
Architecture figure (Fig. 1) drawn by hand, not scripted oxcache-high-level.png
Hit-ratio bar figure hitratio_horizontal_bars_combined.py (via generate_all_plots.sh) plots/comparison/hitratio_bars_combined.png
Cache-throughput boxplot figure (Zipfian + Uniform panels) distribution_comparison_boxplots.py (via generate_all_plots.sh) plots/comparison/boxplot-nofill/{zipfian,uniform}_throughput.png
Table 2 — GET P99 latency matrix latency_table_matrix.py (via generate_all_plots.sh) P99 rows of plots/tables/get_total_{zone,chunk}_lru_matrix.tex
WiredTiger throughput boxplot figure boxplot_wt.py (via generate_all_plots_wt.sh) plots_wt/boxplot-nofill/wt_throughput.png
WiredTiger throughput timeline figure (Chunk LRU / Zone LRU panels) plot_throughput.py (via generate_all_plots_wt.sh) plots_wt/comparison/65536/*client_request_bytes_total_throughput.png
Table 3 — WiredTiger latency statistics latency_table.py (via generate_all_plots_wt.sh) plots_wt/tables/get_total_latency_table.tex
WiredTiger throughput comparison values throughput_table_wt.py (via generate_all_plots_wt.sh) plots_wt/tables/wt_throughput_table.tex
WiredTiger hit ratios quoted in prose plot_hitratio.py per-run *_hitratio.png time series
Measured WAF/RAF values gc_analysis.py gc_analysis_results-2026-07-14.csv
Appendix — object-store latency tables remotetransfer/pulltest.py statistics printed to stdout

Table 2 pairs the Zone-LRU-matrix ZNS values with the Chunk-LRU-matrix block-interface values (best-vs-best); both matrices are produced by the same driver run.

Environment

python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
source .venv/bin/activate

remotetransfer/ has its own requirements.txt (boto3); see remotetransfer/REMOTE_TRANSFER_EVAL.md.

Stage 1 — Generate workloads

Workloads are .bin files of little-endian i32 chunk indices, produced by the generator test classes in the vendor/workloadgen submodule (a YCSB fork) and consumed by the evaluationclient binary:

cd ../vendor/workloadgen/core
mvn -Dtest=site.ycsb.generator.TestZipfianGeneratorZNS test    # -> target/workloadszoned
mvn -Dtest=site.ycsb.generator.TestZipfianGeneratorBLOCK test  # -> target/workloadsblock

TestZipfianGeneratorZNSEvictTune (-> target/workloads) sweeps the eviction high/low-water (and clean-low) threshold matrix; it was used with scripts/run_cpu_bench_evict.sh to select the thresholds used in the paper's eviction-algorithm analysis.

Stage 2 — Run experiments

See docs/WORKLOADS.md for the benchmark-host (cortes) setup, device layout, scheduler requirements, and preconditioning. In short, each sweep is driven by:

sudo ./scripts/run_cpu_bench.sh <workload_dir> <cortes.server.*.toml> <device>

which starts oxcache, replays each workload with evaluationclient, records pidstat, and tars per-run logs into logs-compressed/.

The WiredTiger case study replays a trace captured with the vendor/OxCache-WiredTiger-Trace submodule (WiredTiger's wtperf YCSB-C runner, instrumented to log its block accesses). The trace is replayed in SPC format through the spclient binary via scripts/run_cpu_bench_spc.sh, with the eviction-parameter combinations in scripts/spcconfigspace.{block,zns}.sh.

Stage 3 — Pre-process logs

Move the .tar.gz run archives from logs-compressed/ into a per-device data directory, then:

# Extract
for f in *.tar.gz; do dir="${f%%.tar*}"; mkdir -p -- "$dir"; tar --force-local -xzf "$f" -C "$dir"; done

# Flatten into <dir>-consolidated (per-run .client/.json/.server/.pidstat)
python3 consolidate_metrics.py $DATA_DIR

# Split the combined per-event logs into one JSON per metric
#    (done on demand by the generate_all_plots* scripts, or manually:)
python3 split_data_fast.py ${DATA_DIR}-consolidated

cache_generator.py --dirs <consolidated_dir> --metrics all optionally pre-builds the .npz caches the plotting scripts use, in parallel; the plotting scripts build them lazily otherwise.

Stage 4 — Generate the figures and tables

Parameter sweep (hit-ratio bars, throughput boxplots, Table 2 matrices):

./generate_all_plots.sh data/logs/FINAL/PARAM/ZNS-consolidated data/logs/FINAL/PARAM/SSD-consolidated

WiredTiger case study (timelines, boxplot, Table 3, throughput table):

./generate_all_plots_wt.sh data/logs/FINAL/WTHIGHERHRATIO/ZNS-consolidated data/logs/FINAL/WTHIGHERHRATIO/SSD-consolidated

Stage 5 — WAF/RAF analysis

gc_analysis.py derives host-side write/read amplification from the per-event logs (no re-runs needed); its header documents the methodology and validation.

./gc_analysis.py --format csv data/logs/FINAL/PARAM/ZNS-consolidated/split_output/*-run

Script inventory

Script Role
generate_all_plots.sh Driver: parameter-sweep figures + Table 2 matrices
generate_all_plots_wt.sh Driver: WiredTiger figures + Table 3 + throughput table
data_cache.py Shared library: metric loading, .npz caching, binning
consolidate_metrics.py Flatten raw log trees into *-consolidated
split_data_fast.py Split combined logs into one JSON per metric
cache_generator.py Optional parallel pre-build of .npz caches
distribution_comparison_boxplots.py ZNS vs block throughput boxplots
hitratio_horizontal_bars_combined.py Combined hit-ratio bar chart
boxplot_wt.py WiredTiger throughput boxplot
plot_throughput.py Throughput timelines (WT figure panels)
plot_hitratio.py Hit-ratio time series
latency_table_matrix.py Table 2 latency matrices
latency_table.py Table 3 WiredTiger latency table
throughput_table_wt.py WiredTiger throughput comparison table
gc_analysis.py Host-side WAF/RAF/eviction-onset analysis
remotetransfer/ S3 transfer-latency benchmark (appendix tables)