Stabilized v1.3.3

Author: OSINT-TECHNOLOGIES

docs/dpulse-docs/docs/config.md

@@ -6,6 +6,10 @@ As you can understand, configuration file is a file that contains certain parame
 
 Configuration file (config.ini) located in 'serivce' folder, which is located inside DPULSE root folder. Default config.ini file generated with your first DPULSE start and it looks like that:
 ```
+[HTML_REPORTING]
+template = modern
+delete_txt_files = n
+
 [LOGGING]
 log_level = info
 
@@ -16,6 +20,8 @@ font = slant
 [DORKING]
 dorking_delay (secs) = 2
 delay_step = 5
+full_path_to_browser = path\to\browser\for\dorking
+browser_mode = nonheadless
 
 [SNAPSHOTTING]
 installed_browser = firefox
@@ -51,17 +57,23 @@ proxies_file_path = NONE
 
 As you can see, config file built with sections, which represent separated DPULSE functions. Lets describe these sections and parameters:
 
-[LOGGING] is section for log file configuration, and it's only parameter - log_level - describes log level for Python's logging library. More information about logging levels you [can find here](https://docs.python.org/3/library/logging.html#logging-levels)
-
-[CLI VISUAL] is section for command line interface configuration. It's parameters are preview_color (determines color for big DPULSE logo on start) and font (determines font for big DPULSE logo on start). More about these parameters you [can find here](http://www.figlet.org/)
-
-[DORKING] is section for dorking process configuration. It's parameters are dorking_delay (secs) (determines delay between dorking steps) and delay_step (determines how many requests will be in one step)
-
-[SNAPSHOTTING] is section for snapshotting process configuration. It's parameters are installed_browser (determines your installed browser to be used in screenshot snapshotting), opera_browser_path (path to opera.exe, only in case if you want to use this browser for screenshot snapshotting), wayback_retries (determines amount of retries for getting exact Wayback snapshot in case of errors) and wayback_req_pause (determines pause in seconds between requests to Wayback archive)
-
-[USER-AGENTS] is section for user-agents list, which is used in Google Dorking (user-agents are randomly swapped in process)
-
-[PROXIES] is section for configuring proxies usage in Dorking, and it's only parameter - proxies_file_path - used to determine path to .txt file (path should be full with //-style slashes) which contains list of proxies (one proxy per row)
+| SECTION    | PARAMETER    | POSSIBLE VALUES     | COMMENT    |
+| ------------- | ------------- | ------------- | ------------- |
+| [HTML_REPORTING] | template | modern / legacy | Determines which HTML report template should be used while creating report itself. Modern contains more features like analytics, graphs and interactive features, while legacy is not supported anymore but may be convenient choice for someone |
+| [HTML_REPORTING] | delete_txt_files | y / n | Modern HTML report template contains text boxes for the content of robots.txt and sitemap.xml files, so maybe you won't need them as .txt files. Y parameter makes DPULSE delete this files from report folder, and N parameter leave everything as is |
+| [LOGGING] | log_level | See [here](https://docs.python.org/3/library/logging.html#logging-levels) | Determines how much technical info about program's execution will be logged in journal.log file |
+| [CLI_VISUAL] | preview_color | See [here](https://pypi.org/project/colorama/) | Determines the color of DPULSE ASCII art's color |
+| [CLI_VISUAL] | font | - | - |
+| [DORKING] | dorking_delay (secs) | Any integer value >=0 | Determines how much time browser will be on pause between dorks |
+| [DORKING] | delay_step | Any integer value >0 | Determines the amount of dorks browser should handle before activating delay |
+| [DORKING] | full_path_to_browser | Full path to your browser's executable file with \ symbol as a separator | Determines which browser will be used for Dorking |
+| [DORKING] | browser_mode | headless / nonheadless | Sets which browser mode will be used during Dorking process (headless means that browser window won't be opened, so nonheadless means that browser window will be opened every new dork, and actually it gives better results with TOS and Captcha bypassing) |
+| [SNAPSHOTTING] | installed_browser | Cell 1, Row 2 | Cell 1, Row 2 |
+| [SNAPSHOTTING] | opera_browser_path | Full path to your Opera.exe / None | Enter your Opera.exe path only if you decided to use Opera for screenshot snapshotting, in other cases - leave it as None |
+| [SNAPSHOTTING] | wayback_retries | Any integer value >0 | Determines how many retries DPULSE will make before abandoning inaccessible Wayback link |
+| [SNAPSHOTTING] | wayback_req_pause | Any integer value >0 | Determines how many seconds DPULSE will wait between different retries to an inaccessible Wayback link |
+| [USER-AGENTS] | agent_N | Any default user-agent string | User-agent are used to try bypassing TOS and Captcha when Dorking domain |
+| [PROXIES] | proxies_file_path | Full path to your proxies .txt file | Determines path to .txt file with "//" symbols as a separator which contains list of proxies (one proxy per row) |
 
 ## Editing configuration file
 

docs/dpulse-docs/docs/getting_started.md

@@ -99,6 +99,7 @@ After you've enter correct domain name of your target, you'll be asked to set so
 * PageSearch keywords (if previous modifier was set to Y): keywords which DPULSE will try to find in found PDF documents
 * Dorking mode: determines which Dorking database will be used during scan, but can be declined with N flag
 * Third-party APIs usage: determines which API will be used during scan, but can be declined with N flag
+* Snapshotting usage: determines which type of snapshotting will be used during scan, but can be declined with N flag
 
 Example of modifiers selection is shown below:
 

docs/dpulse-docs/docs/index.md

@@ -14,4 +14,6 @@ DPULSE is a software solution for conducting OSINT research in relation to a cer
 
 4. ***API scan:*** extended domain research function with prepared functions for 3rd party APIs usage. Currently DPULSE supports VirusTotal API (for brief domain information gathering) and SecurityTrails API (deep subdomains and DNS enumeration)
 
+5. ***Snapshotting:*** extended domain research function which copys domain's home page content in various ways: by screenshotting it (screenshot snapshot), by HTML copying (HTML snapshot) and by finding and downloading previous versions (Wayback Machine snapshot)
+
 Finally, DPULSE compiles all found data into an easy-to-read HTML or XLSX report by category. It also saves all information about scan in local report storage database, which can be restored later.

docs/dpulse-docs/docs/pagesearch.md

@@ -14,3 +14,7 @@ PageSearch returns extended information about found subdomains during basic scan
 6. Documents, config files, databases files (and PageSearch can download them!)
 7. Specified words by user in PDF files
 
+PageSearch scan example (this example is not so representative because scanned site is not an example of real domain):
+
+![pagesearch1](https://github.com/user-attachments/assets/ed91f37f-578f-462b-a464-5281dd06ba0c)
+

poetry.lock

@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
 
 [tool.poetry]
 name = "dpulse"
-version = "1.3.2"
+version = "1.3.3"
 description = "Convenient,fast and user-friendly collector of domain information from Open-Sources"
 authors = ["OSINT-TECHNOLOGIES <[email protected]>"]
 readme = "README.md"
@@ -21,7 +21,7 @@ classifiers = [
 ]
 
 [tool.poetry.dependencies]
-python = ">=3.10,<3.13"
+python = ">=3.10,<4.0"
 Jinja2 = "^3.1.6"
 beautifulsoup4 = "4.12.2"
 requests = "2.32.4"
@@ -36,6 +36,7 @@ PyMuPDF = "1.24.7"
 selenium = "4.28.1"
 webdriver-manager = "4.0.2"
 undetected_chromedriver = "3.5.5"
+setuptools = "80.9.0"
 
 [tool.poetry.scripts]
 dpulse = 'dpulse.dpulse:run'

pyproject.toml

@@ -31,7 +31,7 @@ def welcome_menu(self):
             Text.assemble(
                 (fig.renderText('DPULSE'), preview_style),
                 ("\n", ""),
-                ("DPULSE-CLI - v1.3.2 stable - OSINT-TECHNOLOGIES\n\n", "magenta bold"),
+                ("DPULSE-CLI - v1.3.3 rolling - OSINT-TECHNOLOGIES\n\n", "magenta bold"),
                 ("Visit our pages:\n", "white"),
                 ("GitHub: ", "white"), ("https://github.com/OSINT-TECHNOLOGIES\n", "blue underline"),
                 ("PyPi: ", "white"), ("https://pypi.org/project/dpulse/\n", "blue underline"),