You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In New Relic there are many repeated logs like Removed node *. and Registered new node *.. That indicates that either the API server or Redis was too slow to update at the expected frequency.
Most likely that is the reason why some probes are offline on the dashboard from time to time:
Worker can't sync with a external node -> removes it's nodes from fetchProbes -> AdoptedProbes syncs missing probes as offline.
The text was updated successfully, but these errors were encountered:
99% of the occurrences were caused by wrong redis configuration (memory overload), and the errors went close to zero since we fixed that.
The remaining occasional cases are caused on the node.js side by event loop blocking, which is hard to track down. It got better with #560 and #561, and there might be some other places in request handling causing the rest of the spikes, but since it hardly causes any issues at this point, I'm assigning low priority.
In New Relic there are many repeated logs like
Removed node *.
andRegistered new node *.
. That indicates that either the API server or Redis was too slow to update at the expected frequency.Most likely that is the reason why some probes are offline on the dashboard from time to time:
Worker can't sync with a external node -> removes it's nodes from fetchProbes -> AdoptedProbes syncs missing probes as offline.
The text was updated successfully, but these errors were encountered: