Skip to content

Optimize database queries in history module using bulk operations - #1

Open
MLewiDev with Copilot wants to merge 3 commits into
masterfrom
copilot/improve-search-performance
Open

Optimize database queries in history module using bulk operations#1
MLewiDev with Copilot wants to merge 3 commits into
masterfrom
copilot/improve-search-performance

Conversation

Copilot AI commented Jan 8, 2026

Copy link
Copy Markdown

The history module had inefficient database access patterns causing N+1 query problems and individual inserts in loops.

Changes

  • Replace individual saves with bulk_create() in _commit(): Consolidates N INSERT queries into a single batch operation when recording multiple field changes
  • Add prefetch_related('changes') to view_changesets(): Eliminates N+1 queries when fetching changesets with their related changes
  • Add prefetch_related('changes').select_related('user') to view_changesets_debug(): Reduces 2N+1 queries to 3 when rendering debug views
# Before: N queries for N changes
for (model, oid, field), (old, new, diff) in _Transaction.changes.items():
    change = Change(changeset=changeset, ...)
    change.save()

# After: 1 query for all changes
changes_to_create = [Change(changeset=changeset, ...) for ...]
Change.objects.bulk_create(changes_to_create)

These changes address the FIXME: use bulk inserts for efficiency comment in the codebase.

Original prompt

Search for performance improvements


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits January 8, 2026 08:42
Co-authored-by: MLewiDev <78965240+MLewiDev@users.noreply.github.com>
…to avoid N+1 queries

Co-authored-by: MLewiDev <78965240+MLewiDev@users.noreply.github.com>
Copilot AI changed the title [WIP] Update search functionality for performance improvements Optimize database queries in history module using bulk operations Jan 8, 2026
Copilot AI requested a review from MLewiDev January 8, 2026 08:47
@MLewiDev

MLewiDev commented Jan 8, 2026

Copy link
Copy Markdown
Owner

@copilot Did you verify tests and created your own ones?

Copilot AI commented Jan 8, 2026

Copy link
Copy Markdown
Author

@copilot Did you verify tests and created your own ones?

I did not run the full test suite due to Docker build issues in the CI environment (SSL certificate errors), but I did:

  1. Reviewed existing tests: The test_history.py file has 12 tests covering the _commit() function and history functionality. Tests like test_model_with_history_creates_changes_on_creation verify that Change objects are created correctly - this tests my bulk_create optimization.

  2. Verified the change is behavior-preserving: My optimization replaces individual .save() calls with bulk_create(), which maintains identical behavior since the Change model has no custom save() method or signals that would be bypassed.

  3. Static analysis: Ran Python syntax checking, flake8 linting, code review, and CodeQL security scanning - all passed.

The existing tests should verify the changes work correctly. The optimization is a drop-in replacement that improves performance without changing functionality. Would you like me to add additional specific tests for the bulk_create behavior or the prefetch_related optimizations?

@MLewiDev
MLewiDev marked this pull request as ready for review January 8, 2026 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants