Skip to content

Conversation

@FindHao
Copy link
Contributor

@FindHao FindHao commented Dec 22, 2025

Summary

Add CUTRACER_ZSTD_LEVEL environment variable to configure zstd compression level (1-22). This allows users to trade off compression speed vs compression ratio based on their use case.

Changes

  • include/env_config.h: Add extern declaration for zstd_compression_level
  • src/env_config.cu: Add environment variable reading logic with validation (range 1-22)
  • src/trace_writer.cpp: Use configurable compression level instead of hardcoded value 22
  • readme.md: Document new CUTRACER_ZSTD_LEVEL environment variable

Configuration

CUTRACER_ZSTD_LEVEL: Zstd compression level (1-22, default 22)

  • Lower values (1-3): Faster compression, slightly larger output
  • Higher values (19-22): Maximum compression, slower but smallest output
  • Default of 22 provides maximum compression for smallest output

Motivation

The default compression level of 22 (maximum) prioritizes compression ratio over speed. For use cases where compression speed is more important, users can set a lower level (e.g., 3) to get nearly the same compression ratio with significantly faster compression. This change allows users to choose the trade-off that best suits their workflow.

Example Usage

# Fast compression (level 1)
CUTRACER_ZSTD_LEVEL=1 CUDA_INJECTION64_PATH=~/CUTracer/lib/cutracer.so ./app

# Maximum compression (level 22, default)
CUDA_INJECTION64_PATH=~/CUTracer/lib/cutracer.so ./app

# Balanced compression (level 9)
CUTRACER_ZSTD_LEVEL=9 CUDA_INJECTION64_PATH=~/CUTracer/lib/cutracer.so ./app

Test Plan

  1. Build CUTracer: make -j$(nproc)
  2. Verify default level works: Run with TRACE_FORMAT_NDJSON=1 and check output
  3. Verify custom level: Set CUTRACER_ZSTD_LEVEL=1 and verify faster compression
  4. Verify validation: Set invalid value (e.g., 25) and verify warning + fallback to default

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Dec 22, 2025
Add environment variable CUTRACER_ZSTD_LEVEL to configure zstd compression
level (1-22, default 9). This allows users to trade off compression speed
vs ratio:
- Lower values (1-3): Faster compression, slightly larger output
- Higher values (19-22): Maximum compression, slower but smallest output
- Default of 9 provides good compression with reasonable speed

Changes:
- env_config.h: Add extern declaration for zstd_compression_level
- env_config.cu: Add environment variable reading and validation
- trace_writer.cpp: Use configurable level instead of hardcoded 22
- readme.md: Document new CUTRACER_ZSTD_LEVEL environment variable
Update the default CUTRACER_ZSTD_LEVEL from 9 to 22 (maximum compression).
This prioritizes compression ratio over speed by default, as the original
hardcoded value was 22 before making it configurable.

Changes:
- src/env_config.cu: Change default from 9 to 22, update warning messages
- include/trace_writer.h: Update comment to reflect new default
- readme.md: Update documentation to show default 22
- pr.md: Update PR summary to reflect new default
@FindHao FindHao force-pushed the findhao/configurable-zstd-level branch from 303910c to 0e88d7b Compare December 23, 2025 01:09
@FindHao FindHao marked this pull request as ready for review December 23, 2025 01:09
Copilot AI review requested due to automatic review settings December 23, 2025 01:09
@meta-codesync
Copy link

meta-codesync bot commented Dec 23, 2025

@FindHao has imported this pull request. If you are a Meta employee, you can view this in D89703353.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds configurability for the zstd compression level used in NDJSON+Zstd trace output by introducing the CUTRACER_ZSTD_LEVEL environment variable. Users can now adjust the compression level (1-22) to balance compression speed versus compression ratio based on their specific use case.

  • Adds CUTRACER_ZSTD_LEVEL environment variable with validation (range 1-22, default 22)
  • Updates TraceWriter to use the configurable compression level instead of a hardcoded value
  • Documents the new configuration option in readme.md

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
include/env_config.h Adds extern declaration for zstd_compression_level variable
src/env_config.cu Implements environment variable reading with range validation (1-22)
src/trace_writer.cpp Updates TraceWriter constructor to use configurable compression level
include/trace_writer.h Updates comment to reflect new default value (22)
readme.md Documents the new CUTRACER_ZSTD_LEVEL environment variable with usage guidance
pr.md Adds PR description file (should not be committed to repository)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

extern int trace_format_ndjson;

// Zstd compression level (1-22, higher = better compression but slower)
// Default: 9 (good compression with reasonable speed)
Copy link

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment states "Default: 9" but the actual default value is 22 as specified in src/env_config.cu line 233. This inconsistency should be corrected to reflect the actual default value of 22.

Suggested change
// Default: 9 (good compression with reasonable speed)
// Default: 22 (good compression with reasonable speed)

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +41
## Summary

Add `CUTRACER_ZSTD_LEVEL` environment variable to configure zstd compression level (1-22). This allows users to trade off compression speed vs compression ratio based on their use case.

## Changes

- `include/env_config.h`: Add extern declaration for `zstd_compression_level`
- `src/env_config.cu`: Add environment variable reading logic with validation (range 1-22)
- `src/trace_writer.cpp`: Use configurable compression level instead of hardcoded value 22
- `readme.md`: Document new `CUTRACER_ZSTD_LEVEL` environment variable

## Configuration

`CUTRACER_ZSTD_LEVEL`: Zstd compression level (1-22, default 22)
- Lower values (1-3): Faster compression, slightly larger output
- Higher values (19-22): Maximum compression, slower but smallest output
- Default of 22 provides maximum compression for smallest output

## Motivation

The default compression level of 22 (maximum) prioritizes compression ratio over speed. For use cases where compression speed is more important, users can set a lower level (e.g., 3) to get nearly the same compression ratio with significantly faster compression. This change allows users to choose the trade-off that best suits their workflow.

## Example Usage

```bash
# Fast compression (level 1)
CUTRACER_ZSTD_LEVEL=1 CUDA_INJECTION64_PATH=~/CUTracer/lib/cutracer.so ./app

# Maximum compression (level 22, default)
CUDA_INJECTION64_PATH=~/CUTracer/lib/cutracer.so ./app

# Balanced compression (level 9)
CUTRACER_ZSTD_LEVEL=9 CUDA_INJECTION64_PATH=~/CUTracer/lib/cutracer.so ./app
```

## Test Plan

1. Build CUTracer: `make -j$(nproc)`
2. Verify default level works: Run with `TRACE_FORMAT_NDJSON=1` and check output
3. Verify custom level: Set `CUTRACER_ZSTD_LEVEL=1` and verify faster compression
4. Verify validation: Set invalid value (e.g., 25) and verify warning + fallback to default
Copy link

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file appears to be a PR description file that was accidentally included in the commit. PR descriptions should be part of the pull request metadata, not committed to the repository. This file should be removed before merging.

Suggested change
## Summary
Add `CUTRACER_ZSTD_LEVEL` environment variable to configure zstd compression level (1-22). This allows users to trade off compression speed vs compression ratio based on their use case.
## Changes
- `include/env_config.h`: Add extern declaration for `zstd_compression_level`
- `src/env_config.cu`: Add environment variable reading logic with validation (range 1-22)
- `src/trace_writer.cpp`: Use configurable compression level instead of hardcoded value 22
- `readme.md`: Document new `CUTRACER_ZSTD_LEVEL` environment variable
## Configuration
`CUTRACER_ZSTD_LEVEL`: Zstd compression level (1-22, default 22)
- Lower values (1-3): Faster compression, slightly larger output
- Higher values (19-22): Maximum compression, slower but smallest output
- Default of 22 provides maximum compression for smallest output
## Motivation
The default compression level of 22 (maximum) prioritizes compression ratio over speed. For use cases where compression speed is more important, users can set a lower level (e.g., 3) to get nearly the same compression ratio with significantly faster compression. This change allows users to choose the trade-off that best suits their workflow.
## Example Usage
```bash
# Fast compression (level 1)
CUTRACER_ZSTD_LEVEL=1 CUDA_INJECTION64_PATH=~/CUTracer/lib/cutracer.so ./app
# Maximum compression (level 22, default)
CUDA_INJECTION64_PATH=~/CUTracer/lib/cutracer.so ./app
# Balanced compression (level 9)
CUTRACER_ZSTD_LEVEL=9 CUDA_INJECTION64_PATH=~/CUTracer/lib/cutracer.so ./app
```
## Test Plan
1. Build CUTracer: `make -j$(nproc)`
2. Verify default level works: Run with `TRACE_FORMAT_NDJSON=1` and check output
3. Verify custom level: Set `CUTRACER_ZSTD_LEVEL=1` and verify faster compression
4. Verify validation: Set invalid value (e.g., 25) and verify warning + fallback to default

Copilot uses AI. Check for mistakes.
@meta-codesync
Copy link

meta-codesync bot commented Dec 23, 2025

@FindHao merged this pull request in a294dd8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot. Merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants