Skip to content

Improve write efficiency - #504

Merged
TimG1964 merged 6 commits into
JuliaData:masterfrom
TimG1964:Improve-write-efficiency
Nov 16, 2026
Merged

TimG1964 merged 6 commits into
JuliaData:masterfrom
TimG1964:Improve-write-efficiency

Conversation

@TimG1964

Copy link
Copy Markdown
Collaborator

No description provided.

- Rows hold their cells in a Vector in column order instead of a Dict;
  all access goes through src/rowcells.jl.
- Cell is immutable and stored inline; cells are changed by storing a
  changed copy (_updatecell!).
- The cache fill sizes each row as the previous one.

Cache 58-61 -> 26-30 B/cell; cached 1M x 20 write peaks at 1.7 GiB
(was 2.7); opening rw allocates 13-35% less.

Co-Authored-By: Claude Opus 5.5
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 94.97354% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 97.58%. Comparing base (a35969a) to head (f434b16).

Files with missing lines Patch % Lines
src/save.jl 90.83% 11 Missing ⚠️
src/writetable.jl 93.84% 4 Missing ⚠️
src/streamwrite.jl 95.52% 3 Missing ⚠️
src/rowcells.jl 97.43% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #504      +/-   ##
==========================================
- Coverage   97.66%   97.58%   -0.09%     
==========================================
  Files          84       85       +1     
  Lines       17714    17918     +204     
==========================================
+ Hits        17301    17485     +184     
- Misses        413      433      +20     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TimG1964
TimG1964 merged commit 53329ae into JuliaData:master Nov 16, 2026
16 checks passed
@TimG1964
TimG1964 deleted the Improve-write-efficiency branch November 18, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants