Inconsistent data being returned from new dcim.mac_addresses endpoint #19917
Description
Deployment Type
Self-hosted
NetBox Version
v4.3.4
Python Version
3.11
Steps to Reproduce
BUG: Total interfaces from dcim.mac_addresses endpoint (count 7978) does not match total interfaces from dcim.interfaces endpoint (count 8628)
After having stumbled upon a bug present in v4.1.5 with IPAM, which I successfully reproduced here https://github.com/jseifeddine/netbox-ipam-bug/
Realizing that it was fixed in latest version from v4.3.3 - we upgraded to the newest v4.3.4 - and now experiencing a very similar issue with the new MAC addresses objects on the new dcim.mac_addresses endpoint. (mac objects and endpoint introduced in v4.2.0)
We use pynetbox to maintain "synchronization" between devices and NetBox.
Syncing interface information (Status, Description, IPs, speed, type, mac address etc)
Our sync script only makes changes when it needs to, doing a delta check before hand to figure out what in NetBox needs updating/deleting/creating.
IP addresses were an issue, as per the bug repository attached above - causing unnecessary operations.
That was fixed with IP addresses, but MAC addresses now have similar issues. I haven't yet been able to reproduce it exactly how I did with the IPAM repo above, but in our dev environment, which is a clone of production - I can see inconsistent data being returned from the dcim.mac_addresses endpoint, here is the best way I can display that:
NetBox v4.3.4
Found 8748 MAC addresses in NetBox for device 13756 from dcim.mac_addresses endpoint.
Found 9262 interfaces in NetBox for device 13756 from dcim.interfaces endpoint.
Found 8748 MAC addresses on 7978 interfaces with from dcim.mac_addresses endpoint.
Found 8748 MAC addresses on 8628 interfaces from dcim.interfaces endpoint.
BUG: Total interfaces from dcim.mac_addresses endpoint (7978) does not match total interfaces from dcim.interfaces endpoint (8628)
Test completeRunning this script;
#!venv/bin/python
from dotenv import load_dotenv
import os
import pynetbox
load_dotenv()
nb = pynetbox.api(
url=os.getenv("NETBOX_API_URL"),
token=os.getenv("NETBOX_API_TOKEN"),
threading=True
)
DEVICE_ID = 13756
all_netbox_macs = list(nb.dcim.mac_addresses.filter(device_id=DEVICE_ID))
all_netbox_interfaces = list(nb.dcim.interfaces.filter(device_id=DEVICE_ID))
print(f"Found {len(all_netbox_macs)} MAC addresses in NetBox for device {DEVICE_ID} from dcim.mac_addresses endpoint.")
print(f"Found {len(all_netbox_interfaces)} interfaces in NetBox for device {DEVICE_ID} from dcim.interfaces endpoint.")
mac_interfaces = {}
interface_macs = {}
for mac in all_netbox_macs:
if mac.assigned_object and mac.assigned_object.name:
if not mac_interfaces.get(mac.assigned_object.name):
mac_interfaces[mac.assigned_object.name] = []
mac_interfaces[mac.assigned_object.name].append(mac)
total_macs = sum(len(macs) for macs in mac_interfaces.values())
print(f"Found {total_macs} MAC addresses on {len(mac_interfaces)} interfaces with from dcim.mac_addresses endpoint.")
for interface in all_netbox_interfaces:
if interface.mac_addresses:
if not interface_macs.get(interface.name):
interface_macs[interface.name] = []
for mac in interface.mac_addresses:
interface_macs[interface.name].append(mac)
total_macs_from_interface = sum(len(macs) for macs in interface_macs.values())
print(f"Found {total_macs_from_interface} MAC addresses on {len(interface_macs)} interfaces from dcim.interfaces endpoint.")
# Expose bug in mac / interface counts returned from different endpoints
# total_macs is the total number of MAC addresses on the device from the dcim.mac_addresses endpoint
# total_macs_from_interface is the total number of MAC addresses on all interfaces from the dcim.interfaces endpoint
if total_macs != total_macs_from_interface:
print(f"BUG: Total MAC addresses from dcim.mac_addresses endpoint ({total_macs}) does not match total MAC addresses from dcim.interfaces endpoint ({total_macs_from_interface})")
# mac_interfaces[mac.assigned_object.name] -> list of MAC addresses from the dcim.mac_addresses endpoint
# interface_macs[interface.name] -> list of MAC addresses from the dcim.interfaces endpoint
if len(mac_interfaces) != len(interface_macs):
print(f"BUG: Total interfaces from dcim.mac_addresses endpoint ({len(mac_interfaces)}) does not match total interfaces from dcim.interfaces endpoint ({len(interface_macs)})")
print()
print("Test complete")Expected Behavior
Consistent data from the new MAC Addresses endpoint
For example, correct interface counts derived from assigned_object
Observed Behavior
BUG: Total interfaces from dcim.mac_addresses endpoint (7978) does not match total interfaces from dcim.interfaces endpoint (8628)
# mac_interfaces[mac.assigned_object.name] -> list of MAC addresses from the dcim.mac_addresses endpoint
# interface_macs[interface.name] -> list of MAC addresses from the dcim.interfaces endpoint
if len(mac_interfaces) != len(interface_macs):
print(f"BUG: Total interfaces from dcim.mac_addresses endpoint ({len(mac_interfaces)}) does not match total interfaces from dcim.interfaces endpoint ({len(interface_macs)})")