- Disclaimer
- Overview
- Architecture
- Installation
- Working with TCBee
- Accessing Recorded Data with Custom Scripts
- Preview of TCBee
This repository contains the first stable version of TCBee and will be improved/refined in the future. The current Todo-List includes
- Documentation for the tools and interfaces
- Merging tools into a single program
- Add plugins for the calculation of common TCP congestion metrics
- Implement InfluxDB interface for faster processing
- Test and benchmark bottlenecks (eBPF Ringbuf size, File writer, etc.)
- Cleanup of eBPF and user space code
- ...
The current version is tested for linux kernel 6.13.6 and may not work on older or newer kernel versions.
This repository contains the source code for a TCP flow analysis and visualization tool that can monitor any number of TCP flows with up to 1.4 Mpps in total. It uses the Rust programming languages and monitors both packet headers with XDP and TC, and kernel metrics using eBPF.
TCBee
- provides a command-line program to record flows and track current data rates
- monitors both packet headers for incoming and outgoing packets
- hooks onto the linux kernel functions
tcp_sendmsgandtcp_recvmsgto read kernel metrics - stores recorded data in a structured flow database
- provides a simple plugin interface to calculate metrics from recorded data and save the results
- comes with a visualization tool to analyse and compare TCP flow metrics
- provides a rust library to access flow data for custom visualization tools
The architecture of the TCP analysis tool focuses on achieving a high online processing speed while still being extensible. To that end, the structure of the tool consists of the three phases: record, process, and visualize.
The tool monitors incoming and outgoing TCP traffic, identifies flows and stores all available information in a database. For each flow, the TCP header of every single packet is collected over an eBPF XDP for incoming packets or TC hook for outgoing packets and stored with an associated timestamp. Further, the eBPF tracepoints monitor kernel metrics such as the congestion window size and store them in the same way.
Here, more complex metrics are extracted such as duplicate ACK events or retransmissions which would otherwise slow down the live recording. Further, TCBee provides a plugin system to define the calculation of new metrics. Writing such a plugin uses a simple interfaces and requires no knowledge about the code of TCBee.
The information from the database can be read by visualization tools that generate graphs or use a GUI to analyze the results. TCBee uses a strucutred format with SQLite or InfluxDB databases to simplify access for custom scripts and visualization tools.
Note: TCBee was developed on and is designed for linux systems only. It will not work on MacOS or Windows. This project was built using the aya rust template: https://github.com/aya-rs/aya-template. You can visit the project for more information on prerequisites and compiling the project for different architectures.
To compile and run the program, the following requirements need to be fulfilled:
- Clang and LLVM (e.g. for Ubuntu
sudo apt install -y llvm clang libelf-dev libclang-dev) - Rustup (> 1.28.1), install via rustup
- Stable Rust toolchain
rustup toolchain install stable - Nightly Rust toolchain
rustup toolchain install nightly --component rust-src - BPF linker
cargo install bpf-linker
For the visualization tool:
- Pkg-config and fontconfig (e.g. for Ubuntu
sudo apt install -y pkg-config fontconfig libfontconfig1-dev)
You can build the entire project using make, or build single components with make record, make process, make viz.
The resulting binaries will be copied into the install folder in the root directory.
Then, move these binaries to any directory that is in your PATH.
The tcbee script is used as main command and runs the other binaries depending on passed arguments.
Alternatively, you can also build the tool parts manually using cargo.
When working with TCBee, you can call all sub-programs through the tcbee script.
Use tcbee record [interface] to start recording data on the specified interface.
Available options are:
-q,--quietto start the program without the terminal UI-p,--portto filter for flows that have the specified port as source or destination--tui-update-msto set an alternative update interval of the UI. May help with tearing, default is 100ms.
Data recorded by this tool is written as bytes to *.tcp files in /tmp/.
Use tcbee process to read the recorded data and generate the flow database.
Currently, the flow database will always be created in the same directory as db.sqlite.
In the future, it will support an InfluxDB backend to speedup processing of large traces.
Use tcbee viz to start the visualization tool.
Once the tool opens, you can load an *.sqlite file to visualize.
You can navigate between plotting, multi-flow plotting, processing and settings via the navigation bar.
The visualization tool is still in development and you may need to resize the window if fields or buttons are missing.
If you dont want to use the visualization tool, you can access the recorded data directly from the flow database or directly after the recording.
ts-storage contains a database interface created for TCBee.
It uses an abstract TSDBInterface that provides the same interface independant of the used database systems.
For example code and usage, see ts-storage/README.md
You can generate custom graphs and visualization using your own tools and scripts by accessing the flow database directly. To that end, you either need to implement access over SQLite or InfluxDB depending on the storage format. For a guide on how to read flow data, see ts-storage/ACCESS.md.
TCBee stores the recorded data in raw byte files under /tmp/*.tcp.
If you want to read the raw bytes from your own program, take a look at tcbee-record/tcbee-common/src/bindings/ to find the appropriate structs (struct names that are written end with _entry).
