Skip to content

Implement pagination helpers with stream methods for seamless data access#31

Merged
godely merged 3 commits intomainfrom
copilot/fix-14
Jul 2, 2025
Merged

Implement pagination helpers with stream methods for seamless data access#31
godely merged 3 commits intomainfrom
copilot/fix-14

Conversation

Copy link
Contributor

Copilot AI commented Jun 30, 2025

This PR implements high-level iterators and stream methods that automatically follow next_token to yield all items transparently, as requested in Task 7 of the Milestone 2 roadmap.

🚀 New Features

Seamless Pagination with .stream() Methods

All paginated endpoints now support automatic streaming without manual pagination logic:

# Before: Manual pagination (15+ lines of boilerplate)
next_token = None
all_sleep_data = []
while True:
    response = client.daily_sleep.get_daily_sleep_documents(
        start_date="2024-01-01", next_token=next_token
    )
    all_sleep_data.extend(response.data)
    if not response.next_token:
        break
    next_token = response.next_token

# After: Seamless streaming (1 line!)
for sleep_record in client.daily_sleep.stream(start_date="2024-01-01"):
    print(f"Sleep score: {sleep_record.score}")

Supported Endpoints

All major paginated endpoints now have .stream() methods:

  • client.daily_sleep.stream() - Stream sleep data
  • client.daily_activity.stream() - Stream activity data
  • client.heartrate.stream() - Stream heart rate samples
  • client.session.stream() - Stream workout/session data
  • client.daily_readiness.stream() - Stream readiness scores

🔧 Implementation Details

  • Core Utility: Added oura_api_client.utils.pagination.stream_paginated_data() for reusable pagination logic
  • BaseRouter Enhancement: Added _stream_documents() method for consistent pagination across all endpoints
  • Memory Efficient: Items are yielded one at a time, not loaded into memory all at once
  • Type Safe: Full type annotations with proper Iterator[ModelType] return types
  • No Breaking Changes: All existing functionality preserved

📚 Documentation Updates

  • Updated README with prominent examples showcasing seamless pagination
  • Added comprehensive docstrings with usage examples
  • Manual pagination examples moved to "Advanced Usage" section

🧪 Testing

  • Added 8 comprehensive test cases covering pagination utilities and stream methods
  • Tests include single page, multi-page, and error scenarios
  • All 120 tests passing (112 original + 8 new)
  • Zero linting errors

📋 Benefits

  • Pythonic: Simple for item in endpoint.stream(): syntax
  • Memory Efficient: Stream large datasets without memory issues
  • Developer Friendly: Eliminates pagination boilerplate code
  • Robust: Built on existing error handling and retry logic
  • Flexible: Works with date ranges and all existing parameters

Fixes #14.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits June 30, 2025 15:45
…cess

Co-authored-by: godely <3101049+godely@users.noreply.github.com>
Co-authored-by: godely <3101049+godely@users.noreply.github.com>
Copilot AI changed the title [WIP] Task 7: Implement pagination helpers Implement pagination helpers with stream methods for seamless data access Jun 30, 2025
Copilot AI requested a review from godely June 30, 2025 15:48
@godely godely marked this pull request as ready for review July 2, 2025 22:12
@godely godely merged commit 0d65ac1 into main Jul 2, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Task 7: Implement pagination helpers

2 participants