Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try to make pg_dump_splitsort.py compilable with nuitka #33

Open
akaihola opened this issue Apr 24, 2024 · 2 comments
Open

Try to make pg_dump_splitsort.py compilable with nuitka #33

akaihola opened this issue Apr 24, 2024 · 2 comments
Assignees
Milestone

Comments

@akaihola
Copy link
Owner

In #13, @oldcai writes:

I tested it with a 444G .sql file and it finished within 523m22.189s.

Let's see if we can compile pg_dump_splitsort.py with nuitka, and document the performance gain if it works.

@akaihola akaihola added this to the 1.1.1 milestone Apr 24, 2024
@akaihola akaihola self-assigned this Apr 24, 2024
@akaihola
Copy link
Owner Author

Benchmark on my Lenovo Carbon X1 Gen 9 with NixOS:

  • 18MB SQL, Python 3.11.4: 0.5 s
  • 18MB SQL, Nuitka 1.4.8: 0.5 s
  • 100MB SQL, Python 3.11.4: 11.6 s
  • 100MB SQL, Nuitka 1.4.8: 10.2 s

@akaihola
Copy link
Owner Author

$ nuitka3 pgtricks/pg_dump_splitsort.py
Nuitka-Options:INFO: Used command line options: pgtricks/pg_dump_splitsort.py
Nuitka-Options:WARNING: You did not specify to follow or include anything but main program. Check options and make sure that is intended.
Nuitka:WARNING: Using very slow fallback for ordered sets, please install 'ordered-set' PyPI package for best Python compile time performance.
Nuitka:INFO: Starting Python compilation with Nuitka '1.4.8' on Python '3.10' commercial grade 'not installed'.
Nuitka:INFO: Completed Python level compilation and optimization.
Nuitka:INFO: Generating source code for C backend compiler.
Nuitka:INFO: Running data composer tool for optimal constant value handling.
Nuitka:INFO: Running C compilation via Scons.
Nuitka-Scons:INFO: Backend C compiler: gcc (gcc).
Nuitka-Scons:INFO: Backend linking program with 7 files (no progress information available).
Nuitka-Scons:WARNING: You are not using ccache.
Nuitka:INFO: Keeping build directory 'pg_dump_splitsort.build'.
Nuitka:INFO: Successfully created 'pg_dump_splitsort.bin'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant