You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Sep 8, 2020. It is now read-only.
Currently, the data processor processes deals and cashback rewards one at a time. This takes around 2 minutes to process all data.
It would be significantly faster if this was multithreaded so it can process multiple items at once. Since the majority of time is spent in the Googling hostnames phase (where we have to wait for network responses), it could definitely help.
One concern is that Google might rate limit or even mark us as a bot if we were to increase the speed. Someone will have to try it out and see.
The text was updated successfully, but these errors were encountered:
Currently, the data processor processes deals and cashback rewards one at a time. This takes around 2 minutes to process all data.
It would be significantly faster if this was multithreaded so it can process multiple items at once. Since the majority of time is spent in the Googling hostnames phase (where we have to wait for network responses), it could definitely help.
One concern is that Google might rate limit or even mark us as a bot if we were to increase the speed. Someone will have to try it out and see.
The text was updated successfully, but these errors were encountered: