In our company, we use testing pages to crawling. So, Don't use it on any web pages, and please read about robots.txt or Robots Exclusion Protocol first before use crawler.
Acet-Aminophen/WEB-CRAWLER
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
| Name | Name | Last commit date | ||
|---|---|---|---|---|