Telegram Snowball Sampling Tool

Overview

The Telegram Snowball Sampling Tool is a Python-based utility for mapping Telegram channel networks through three main methods:

1. Forwarded Messages: Automatically discovers channels through message forwards.
2. Channel Recommendations: Collects Telegram's built-in channel recommendations.
3. URL Extraction: Maps external connections by extracting URLs from messages.

The tool creates detailed edge lists for network visualization and provides analysis outputs for downstream tools such as Gephi.

Summary of Network Analysis in Telegram

This project implements multiple discovery methods to map the network structure of Telegram channels.

Snowball Sampling Through Message Forwards

Snowball sampling discovers channels through forwarded messages, starting with a seed channel and expanding outward. This identifies both the origin and dissemination paths of information and creates a directed network structure.

Channel Recommendations

The tool leverages Telegram's built-in recommendation algorithm to discover topically related channels. This adds network coverage beyond forward relationships alone.

URL Extraction

By capturing external URLs shared in messages, the tool maps connections between Telegram channels and external websites, providing a broader view of the information ecosystem.

Important Warning: Runtime Expectations

Exponential Growth in Runtime

The Telegram Snowball Sampling Tool can take several days to complete because the sampling process expands rapidly. Each iteration can add a new set of channels, growing the search space exponentially.

Recommendations for Efficient Use

• Limit iterations to three or fewer to balance depth and runtime.
• Focus on channels with multiple mentions when using forwards.
• Set a reasonable maximum number of posts unless you explicitly want a full scrape.
• Enable recommendations and URL extraction selectively based on your goal.

Features

• Automated discovery of Telegram channels through:
  • forwarded message tracking
  • channel recommendation retrieval
  • URL extraction from messages
• Customizable parameters for depth, thresholds, and scope
• Comprehensive edge list creation for network analysis
• Network visualization-ready outputs
• Network metrics calculation and export
• Environment-based configuration
• Resume checkpoints and SQLite persistence
• Detailed logging for monitoring progress

Project Structure

telegram-snowball-sampling/
|-- src/
|   `-- telegram_snowball_sampling/
|       |-- __init__.py                  # Package exports
|       |-- check_api_credentials.py     # API credential checks
|       |-- config.py                    # Configuration manager
|       |-- csv_loader.py                # CSV seed loader
|       |-- database.py                  # SQLite persistence
|       |-- edge_list.py                 # Edge list helpers
|       |-- main.py                      # Main package CLI
|       |-- merge_csv_data.py            # CSV merging utility
|       |-- network_analysis.py          # Network analysis package module
|       |-- project_paths.py             # Repo-relative path helpers
|       |-- recommendations.py           # Channel recommendations module
|       `-- utils.py                     # Utility functions
|-- scripts/
|   |-- check_api_credentials.py         # CLI wrapper
|   |-- network_analysis.py              # CLI wrapper
|   `-- validate.py                      # Repo validation runner
|-- bootstrap.py                         # Shared compatibility bootstrap helper
|-- check_api_credentials.py             # Repo-root compatibility launcher
|-- example.env                          # Template environment variables
|-- .env                                 # Your environment variables
|-- main.py                              # Main application script
|-- network_analysis.py                  # Repo-root compatibility launcher
|-- pyproject.toml                       # Project metadata for uv and editable installs
|-- README.md                            # Project documentation
|-- requirements.txt                     # Python dependencies
|-- uv.lock                              # Locked uv dependency set
`-- var/
    |-- analysis/                        # Default network analysis outputs
    |-- backups/                         # Recovery exports created after critical failures
    |-- data/                            # SQLite database and runtime data
    |-- edge_lists/                      # Edge list exports
    |-- merged/                          # Merged channel exports
    |-- runs/                            # Per-run CSVs, URLs, and guides
    `-- sessions/                        # Telethon session files

Requirements

• Python 3.10 or higher
• Telethon
• NetworkX and Matplotlib for analysis and visualization
• A registered Telegram application for API credentials
• All dependencies listed in requirements.txt

Installation

1. Clone the repository:

git clone https://github.com/yourusername/telegram-snowball-sampling.git
cd telegram-snowball-sampling

2. Create a virtual environment, optionally:

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install the required dependencies with pip:

pip install -r requirements.txt

If you want to use the package-style python -m telegram_snowball_sampling... commands in a pip-managed environment, install the project itself as well:

pip install -e .

4. Or install with uv:

uv sync --extra dev

Configuration

The tool can create a .env file from the template and prompt for your Telegram API credentials on first run. You can also configure the following options manually:

Variable	Description	Default
TELEGRAM_API_ID	Your Telegram API ID	(required)
TELEGRAM_API_HASH	Your Telegram API Hash	(required)
TELEGRAM_SESSION_NAME	Path stem for the Telegram session file	var/sessions/session_name
DEFAULT_MIN_MENTIONS	Minimum mentions threshold	1
DEFAULT_ITERATIONS	Number of iterations	3
DEFAULT_MAX_POSTS	Maximum posts per channel (none for full scrape)	none
DEFAULT_INCLUDE_FORWARDS	Whether to include forwarded message scraping	True
CHECKPOINT_INTERVAL	Iterations between resume checkpoints	10
DEFAULT_INCLUDE_RECOMMENDATIONS	Whether to include recommendations	True
DEFAULT_RECOMMENDATIONS_DEPTH	Maximum depth for recommendations	2
DEFAULT_INCLUDE_URLS	Whether to extract URLs from messages	True
RESULTS_FOLDER	Directory for per-run results	var/runs
MERGED_FOLDER	Directory for merged results	var/merged
EDGE_LIST_FOLDER	Directory for edge list files	var/edge_lists
EDGE_LIST_FILENAME	Name of the edge list file	Edge_List.csv
MERGED_FILENAME	Name of the merged file	merged_channels.csv
API_DETAILS_FILE	Backup file for API details	var/api_values.txt
DATA_FOLDER	Directory for SQLite data	var/data
DATABASE_FILENAME	SQLite database filename	telegram_snowball.sqlite
NETWORK_ANALYSIS_FOLDER	Directory for network analysis outputs	var/analysis
DEBUG	Enable debug logging	False

By default the project now keeps generated state under var/ instead of splitting it across several top-level folders. If you prefer the older layout, you can still override any of these paths in .env.

Upgrading From Older Versions

If you are updating from an older checkout, update the path-related entries in your existing .env to the new defaults:

TELEGRAM_SESSION_NAME=var/sessions/session_name
RESULTS_FOLDER=var/runs
MERGED_FOLDER=var/merged
EDGE_LIST_FOLDER=var/edge_lists
API_DETAILS_FILE=var/api_values.txt
DATA_FOLDER=var/data
NETWORK_ANALYSIS_FOLDER=var/analysis

Newer CLI commands will warn if they detect the older values such as session_name, results, merged, EdgeList, data, or network_analysis.

Usage

Preferred command:

python -m telegram_snowball_sampling.main

Compatibility launcher:

python main.py

The script will:

1. Prompt for Telegram API credentials if not configured.
2. Ask for seed channels as a comma-separated list, text file, or CSV file.
3. Request parameters for iterations, minimum mentions, and other options.
4. Begin the data collection process using the enabled methods.
5. Save results to CSV and edge list files.
6. Offer to run network analysis on the collected data.

Validation

Run the full validation suite from the repository root:

python scripts/validate.py

Include the uv smoke test as well:

python scripts/validate.py --with-uv

If you want to smoke test the credential checker too, and you have a repo .env file present:

python scripts/validate.py --check-credentials

Beginner Guide

If you are new to Telegram scraping, follow this checklist to get a first run working quickly.

1. Create Telegram API credentials. Get them from https://my.telegram.org/auth and keep the API ID and API Hash ready.

2. Create your .env. Copy example.env to .env and paste in your API ID and API Hash, or just run python -m telegram_snowball_sampling.main once and follow the prompt.

3. Start the script.

python -m telegram_snowball_sampling.main

4. Choose your seed input. Pick one of:

• CSV file
• Text file (.txt)
• Comma-separated list

5. Choose crawl mode.

• Recommendations only: faster, no message scraping
• Recommendations plus forwards: slower, more comprehensive

6. Set basic limits. If you chose forwards, you will be asked for:

• Minimum mentions
• Max posts per channel, either none or a numeric cap such as 100

7. Optional settings. If recommendations are enabled, choose a depth such as 1 or 2. If forwards are enabled, choose whether to extract URLs.

8. Find your results. Look in:

• var/runs/ for run CSVs, URL lists, and visualization guides
• var/edge_lists/ for Edge_List.csv
• var/merged/ for consolidated channels
• var/analysis/ for metrics and visualizations, if analysis is run
• var/data/ for the SQLite database and related runtime state
• var/sessions/ for Telethon session files

If a run is interrupted, the tool can resume from the last checkpoint the next time you start it.

Data Collection Methods

1. Forward Detection

Analyzes messages in each channel to find forwards from other channels. This reveals information flow between channels.

2. Channel Recommendations

Retrieves Telegram's own channel recommendations for each discovered channel. These recommendations are based on Telegram's algorithm, which considers content similarity and user overlap.

3. URL Extraction

Extracts all URLs shared in messages across channels, creating connections between Telegram channels and external websites.

Output Files

The tool generates several outputs:

1. Individual Run Results in the var/runs folder:
   • CSV files containing channel IDs, names, and usernames
   • URL lists from message content
2. Edge List in the var/edge_lists folder:
   • CSV file with network connections, including forward relationships, recommendation relationships, and URL connections
   • Connection types and weights for advanced analysis
3. Merged Results in the var/merged folder:
   • Consolidated CSV with all unique channels found across multiple runs
4. Network Analysis in the var/analysis folder, when analysis is run:
   • Network metrics in Excel format
   • Gephi-compatible GEXF file for visualization
   • Basic network visualization image
5. Runtime State in the var/data and var/sessions folders:
   • SQLite database file
   • Telethon session files used for authenticated scraping

Network Analysis

The included network analysis tooling provides:

1. Basic Network Metrics:
   • Node and edge counts
   • Network density
   • Connected components
   • Average path length
2. Key Influencer Identification:
   • Top source channels with the most outgoing connections
   • Top receiver channels with the most incoming connections
3. Connection Type Analysis:
   • Distribution of connection types such as forwards, recommendations, and URLs
   • Weight distribution analysis
4. Visualization:
   • Gephi-compatible GEXF file
   • Basic visualization image
   • Network metrics in Excel format

Run network analysis separately from the repository root:

python -m telegram_snowball_sampling.network_analysis --edge-list var/edge_lists/Edge_List.csv --output-dir var/analysis
# Or:
python network_analysis.py --edge-list var/edge_lists/Edge_List.csv --output-dir var/analysis

If you prefer uv-managed commands, you can also run:

uv run python -m telegram_snowball_sampling.network_analysis --edge-list var/edge_lists/Edge_List.csv --output-dir var/analysis

The repo-root and scripts/ launchers add src/ to PYTHONPATH automatically, so they still work directly from a cloned repository without an editable install.

Network Visualization with Gephi

For advanced network visualization:

1. Download and install Gephi
2. Import the GEXF file from the var/analysis folder
3. Apply layouts such as ForceAtlas2 to organize the network
4. Style nodes based on metrics such as degree or betweenness
5. Run community detection algorithms to identify clusters

A detailed guide is created in the var/runs folder after each run.

Disclaimer

This tool is for educational and research purposes only. Ensure that you comply with Telegram's terms of service and respect privacy and ethical guidelines when using it.

Contributing

Contributions are welcome. Please feel free to submit a pull request.

Future Development

• Add language detection for message content filtering
• Implement community detection algorithms
• Add multi-API parallel processing for improved performance
• Create live network visualization capabilities