Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bench: Document how to use cross platform Samply profiler #15481

Merged
merged 6 commits into from
Mar 30, 2025

Conversation

comphead
Copy link
Contributor

Which issue does this PR close?

  • Closes #.

Rationale for this change

Introduce cross platform Samply profiler for DataFusion and benchmarks

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added the documentation Improvements or additions to documentation label Mar 28, 2025
@parthchandra
Copy link

There is also some profiling information in https://datafusion.apache.org/comet/contributor-guide/profiling_native_code.html
I'm presuming this will replace that?
How does samply compare with cargo flamegraph ?

@comphead
Copy link
Contributor Author

There is also some profiling information in https://datafusion.apache.org/comet/contributor-guide/profiling_native_code.html I'm presuming this will replace that? How does samply compare with cargo flamegraph ?

the tool also builds a flame graph, it can be found in the separate tab.
For the Comet I think we can refer to this section to provide alternative profiling ways

@alamb alamb changed the title bench: Introduce cross platform Samply profiler bench: Document how to use cross platform Samply profiler Mar 30, 2025
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great @comphead -- I tried the instructions and it is awesome

@@ -82,6 +82,43 @@ CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --bench sql_planner --

[Video: how to CPU profile DataFusion with XCode Instruments](https://youtu.be/P3dXH61Kr5U)

## Linux
## Cross Platform
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Cross Platform
## Profiling using Samply


```shell
cargo build --profile profiling --bin tpch
samply record ./target/debug/tpch benchmark datafusion --iterations 5 --path datafusion/benchmarks/data/tpch_sf10 --prefer_hash_join true --format parquet -o datafusion/benchmarks/results/dev2/tpch_sf10.json --query 22
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps you want to change this to

Suggested change
samply record ./target/debug/tpch benchmark datafusion --iterations 5 --path datafusion/benchmarks/data/tpch_sf10 --prefer_hash_join true --format parquet -o datafusion/benchmarks/results/dev2/tpch_sf10.json --query 22
samply record ./target/profiling/tpch benchmark datafusion --iterations 5 --path datafusion/benchmarks/data/tpch_sf10 --prefer_hash_join true --format parquet -o datafusion/benchmarks/results/dev2/tpch_sf10.json --query 22

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is awesome catch

@comphead comphead merged commit f5d96b0 into apache:main Mar 30, 2025
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants