Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add script to delete empty history entries from LOS, Links, and Devices #815

Closed
wants to merge 15 commits into from

Conversation

WillNilges
Copy link
Collaborator

This management command will erase empty history entries from the mentioned devices. This was created by a bug in the UISP import script, causing it to save unnecessarily, which causes extra writes.

Copy link

codecov bot commented Jan 28, 2025

Codecov Report

Attention: Patch coverage is 93.61702% with 3 lines in your changes missing coverage. Please review.

Project coverage is 94.87%. Comparing base (e5c92ca) to head (8f04c4b).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
...hapi/management/commands/dedupe_history_entries.py 93.61% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #815      +/-   ##
==========================================
- Coverage   94.88%   94.87%   -0.02%     
==========================================
  Files          91       92       +1     
  Lines        3892     3939      +47     
==========================================
+ Hits         3693     3737      +44     
- Misses        199      202       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@WillNilges
Copy link
Collaborator Author

@WillNilges WillNilges closed this Jan 28, 2025
@WillNilges WillNilges reopened this Jan 29, 2025
@WillNilges
Copy link
Collaborator Author

@WillNilges
Copy link
Collaborator Author

I have reservations about using this, especially in production, but I think it would be better to have this and not need it than need this and not have it.

Comment on lines +24 to +26
self.deduplicate_history(Link.objects.all())
self.deduplicate_history(Device.objects.all())
self.deduplicate_history(LOS.objects.all())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: maybe the models should be a parameter

history_model = m.history.model
table_name = history_model._meta.db_table

# Delete history from each object. Atomic block makes it go faster
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And safer!

# XXX (wdn): I don't think I have a type I can put in here
# without some robot yelling at me
meaningful_history: list[Any] = []
for h in history:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is guaranteed to be ordered?

mh.save()

except Exception as e:
logging.exception(f"Could not get history for this link: {e}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logging.exception(f"Could not get history for this link: {e}")
logging.exception(f"Error cleaning history for object: {m}. {e}")

@WillNilges
Copy link
Collaborator Author

You know what? I don't think I want to reinvent the wheel here. The process of copying a backup to my laptop and cleaning it there was pretty painful, but there's no way I'd use this in production.

@WillNilges WillNilges closed this Mar 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants