Skip to content

Feature/stream excel #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Feature/stream excel #22

wants to merge 11 commits into from

Conversation

rjsamra
Copy link
Member

@rjsamra rjsamra commented Apr 12, 2025

Add Streaming Support for Large Excel Files

Description

This PR adds streaming support for processing large Excel files efficiently, preventing memory exhaustion when handling massive datasets.

Features Added

  • New stream() method in ExcelTo class for processing large files
  • Chunk-based processing with configurable chunk size
  • Memory-efficient handling of large datasets
  • Consistent output format with existing methods
  • Comprehensive test coverage

Technical Details

  • Implemented ProcessExcelChunk job for handling data in chunks
  • Added streaming support to ExcelTo class
  • Maintained backward compatibility with existing methods
  • Added feature tests for streaming functionality
  • Updated documentation with streaming feature details

Testing

  • Added comprehensive feature tests
  • Tested with various file sizes and chunk configurations
  • Verified memory usage with large files
  • Tested single and multiple sheet scenarios

Documentation

  • Updated README with streaming feature details
  • Added usage examples
  • Documented response formats
  • Added memory efficiency notes

Breaking Changes

None. This is a backward-compatible feature addition.

Checklist

  • Code follows the project's coding standards
  • Tests have been added/updated
  • Documentation has been updated
  • All tests pass
  • No breaking changes introduced

@rjsamra rjsamra self-assigned this Apr 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant