Question about Multi-threaded Link Checker task

[yesenarman]

Hi. I have a difficulty understanding the following subtask:

Use threads to check the links in parallel: send the URLs to be checked to a channel and let a few threads check the URLs in parallel.

Is the suggestion here to create a channel on the main thread, which will spawn a new thread whenever it receives a URL and then run a recursive link checker for this URL on the spawned thread? So for instance, a user could simultaneously run two parallel link checkers for site A and site B.

Or is it more about parallelizing the recursive checks of the same site? So for instance, you start with the start page of site A, extract a list of URLs from that page and send them to some queue from which several other threads can pick up URLs, process them and add new extracted URLs to the queue, and so on.

[mgeisler]

Or is it more about parallelizing the recursive checks of the same site?

That was my idea: let the checker use, say, 10 threads and let these 10 threads work through the URLs to be checked in parallel. You're exactly right that you'll want to build a queue for this 🙂

[yesenarman]

Got it. Thanks.

[mgeisler]

You're welcome! Please let us know if you run into other things that are not clear. We can also add a line about it to the exercise text itself.