Duplicate IP addresses with the same UID found

[ndombroski]

While operating the netbox-ip-controller, our instance of NetBox experienced a period of downtime where API requests were intermittently failing. This resulted in a number of duplicated IP Address objects in NetBox. The duplicated objects had all fields identical, including UID. After a period of downtime, it is expected for their to be a brief delay in the IP addresses to become eventually consistent, but this should not result in duplicates.

This was discovered due to errors of the following form in the logs:

deleting IP: checking if IP exists: more than one IP with UID "1a9ef0ae-7f1b-47b3-973b-20a61b3d48c2" found

At first glance, this doesn't seem like it should be possible. The function for upserting an IP address is essentially the following logic:
1.) Get the IP address with the given UID
2.) If it does not exist, create it. Else do nothing

It only seems possible for duplicates to happen if the backing data source for NetBox does not have strong read-after-write consistency. In which case, if the function is run twice in rapid succession, then its possible that both times the IP with the given UID is not found, and both times it is created.