Earthworm Project

Overview

The Earthworm project is designed to monitor the heartbeat signals of Kubernetes clusters using eBPF (Extended Berkeley Packet Filter) technology. The project visualizes this heartbeat data in a web-based interface, resembling a cardiogram, allowing users to monitor the health of their Kubernetes environments effectively. The name "Earthworm" symbolizes the project's ability to manage multiple "hearts" (Kubernetes nodes ) simultaneously.

Kubernetes and Kernel version

This project is designed to support Kubernetes v1.20 and later, as these versions provide stable APIs and features required for heartbeat monitoring and eBPF integration. For optimal results, use a recent Kubernetes release and ensure your nodes run a Linux kernel version 5.4 or newer (for optimal eBPF support).

Project Structure

The project is organized into several directories, each serving a specific purpose:

• src/ebpf: Contains the eBPF program that intercepts heartbeat signals from Kubernetes clusters.
• src/kubernetes: Implements a Kubernetes client that listens for heartbeat events and forwards the data to the server.
• src/server: The Go server that receives heartbeat data and serves it to the UI.
• src/types: Defines TypeScript types and interfaces used throughout the project.
• src/heartbeat-visualizer: Contains the heartbeat visualization logic, including:
  • src/heartbeat-visualizer/src/heartbeat.js: Chart.js heartbeat visualizer.
  • src/heartbeat-visualizer/src/parseLeases.js: Script to parse leases.yaml and generate leases.json.
  • src/heartbeat-visualizer/src/leases.json: Parsed heartbeat data for visualization.
  • src/leases.yaml: Raw heartbeat lease data in YAML format.

earthworm/ ├── README.md ├── CHANGELOG.md ├── .gitignore ├── src/ │ ├── ebpf/ │ │ └── heartbeat.c # eBPF program for heartbeat interception │ ├── kubernetes/ │ │ └── monitor.go # Go client for Kubernetes heartbeat events │ ├── server/ │ │ └── main.go # Go server with in-memory heartbeat storage │ ├── types/ │ │ └── index.ts # Shared TypeScript types │ ├── heartbeat-visualizer/ │ │ ├── src/ │ │ │ ├── heartbeat.js # Chart.js heartbeat visualizer │ │ │ ├── parseLeases.js # YAML to JSON parser for heartbeat data │ │ │ └── leases.json # Parsed heartbeat data for visualization │ └── leases.yaml # Raw heartbeat lease data in YAML format ├── package.json # Node.js dependencies ├── tsconfig.json # TypeScript config ├── go.mod # Go module file ├── go.sum # Go dependencies checksum

Getting Started

Notice: At this moment we are using only mock data (leases.json) from both sources Otel and eBPF.

Prerequisites

• Go (version 1.16 or later)
• Node.js (version 14 or later)
• Kubernetes cluster (for testing)

Installation

1. Clone the repository:

   git clone https://github.com/yourusername/earthworm.git
   cd earthworm
   

2. Install Go dependencies:

   cd src/server
   go mod tidy
   

3. Install Node.js dependencies:

   cd src/ui
   npm install
   

Running the Project

1. Start the Go server:

   cd src/server
   go run main.go
   

2. Run the eBPF program:

   • Compile and load the eBPF program located in src/ebpf/heartbeat.c using the appropriate tools (e.g., clang, bpftool).

3. Start the React application:

   cd src/heart-visualizer
   npm start
   

Architecture

The Earthworm project consists of the following components:

• eBPF Program: Intercepts heartbeat signals from Kubernetes clusters and collects the data.
• Kubernetes Monitor: Listens for heartbeat events using the Kubernetes API and forwards the data to the Go server.
• Go Server: Receives heartbeat data from the Kubernetes monitor and serves it to the React UI.
  Note: Currently, the Go server uses in-memory storage for heartbeat events. In future versions, a real-time database (such as Redis, MongoDB with change streams, or Kafka) will be integrated to improve scalability and real-time data streaming.
• React UI: Visualizes the heartbeat data in a graph format, providing real-time updates.

Deployment

The delivery mechanism for the eBPF program will use a Kubernetes DaemonSet. This ensures that the eBPF loader and monitoring agent run on every node in the cluster, allowing Earthworm to intercept heartbeat signals from all nodes efficiently. The DaemonSet can be packaged and deployed using a Helm chart for easy installation and management.

Conclusion

The Earthworm project provides a comprehensive solution for monitoring Kubernetes cluster health through innovative use of eBPF technology. By visualizing heartbeat data, users can gain insights into the performance and reliability of their Kubernetes environments. This project serves as an educational resource for understanding eBPF, Kubernetes, and modern web application development.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Preview

Author

Alejandro Mercado Peña mercadoalex[at]gmail.com

Update

Anomalies detected

Anomalies detail

eBPF power unleashed