Welcome to the comprehensive documentation for IPFS Kit Python. This guide will help you find exactly what you need, whether you're just getting started or building advanced distributed systems.
New to IPFS Kit? Follow this path:
- Installation Guide - Get everything set up (10 minutes)
- Quick Reference - Learn basic operations (5 minutes)
- API Reference - Explore the full API (30 minutes)
- Examples - See code in action
Building something specific? Jump to:
- Cluster Setup - Multi-node deployments
- AI/ML Integration - Machine learning workloads
- MCP Server - Model Context Protocol
- Production Deployment - Docker, Kubernetes, CI/CD
Installation Guide - Start here if you're new
- System requirements and dependencies
- Installation methods (pip, source, Docker)
- Initial configuration
- Verification steps
- Answers: "How do I install?" "What do I need?"
Quick Reference - Your cheat sheet
- Common operations with examples
- CLI commands
- Python API quick start
- Troubleshooting tips
- Answers: "How do I...?" "What's the command for...?"
Validation Quick Start - Verify your setup
- Test installation
- Run example operations
- Check cluster connectivity
- Answers: "Is it working?" "How do I test?"
API Reference - Complete API documentation
- All classes and methods
- Parameter descriptions
- Return value documentation
- Usage examples
- Answers: "What methods are available?" "How do I use X?"
CLI Reference - Command-line interface
- All CLI commands
- Options and flags
- Examples for each command
- Answers: "What CLI commands exist?" "How do I use ipfs-kit?"
Core Concepts - Understanding the architecture
- System design overview
- Key abstractions
- Data flow
- Component interactions
- Answers: "How does it work?" "What's the architecture?"
High-Level API - Simplified interface
- Easy-to-use wrappers
- Common patterns
- Best practices
- Answers: "What's the easiest way?" "Are there shortcuts?"
Storage Backends - Multi-backend storage system
- Enhanced Analytics
- 6 integrated backends: IPFS, Filecoin, S3, Storacha, HuggingFace, Lassie
- Multi-tier storage strategy (memory → disk → network → cloud)
- Automatic content distribution across backends
- Answers: "What storage backends are available?" "How do I use S3/Filecoin?" "Multi-backend setup?"
Tiered Cache - Advanced multi-tier caching
- ARC (Adaptive Replacement Cache) algorithm
- Memory cache (100MB default) + Disk cache (1GB+ default)
- Heat-based eviction and automatic tier promotion
- Zero-copy memory-mapped access for large files
- Answers: "How does caching work?" "Cache configuration?" "Performance optimization?"
Replica Management - Content replication strategies
- Cluster-based replication with configurable factors
- Replication policies: distributed, local-first, geo-aware, cost-optimized
- Automatic repair and health monitoring
- Min/max replica constraints with auto-repair
- Answers: "How do replicas work?" "Replication strategies?" "High availability setup?"
Pin Management - Keep content available
- Pin Management Guide - Complete guide
- Quick Start - Get started fast
- Filecoin Integration - Filecoin pinning
- Dashboard Features - Web interface
- Answers: "How do I keep content?" "What's pinning?" "How does replication work?"
Auto-Healing - Automatic error recovery
- Auto-Healing Guide - System overview
- Quick Start - Setup in 5 minutes
- MCP Auto-Healing - MCP integration
- Answers: "Can it fix itself?" "How does error recovery work?"
MCP Server - Model Context Protocol server
- MCP tool integration
- Server configuration
- Custom tools development
- Answers: "What's MCP?" "How do I use it with AI?"
Dashboard - Web-based management
- Dashboard setup
- Monitoring and metrics
- Configuration management
- Answers: "Is there a GUI?" "How do I monitor?"
VFS (Virtual File System) - POSIX-like virtual filesystem on IPFS
- VFS Management - Complete VFS system
- Filesystem Journal - Change tracking and journaling
- POSIX-like operations: mkdir, ls, mv, rm, cp
- VFS buckets with quotas and policies
- Automatic metadata extraction and indexing
- Journal replication across nodes
- Answers: "How do I use IPFS like a filesystem?" "What are VFS buckets?" "Filesystem operations?"
GraphRAG - Knowledge graphs & intelligent search
- GraphRAG Documentation - Complete guide
- VFS Integration - Auto-indexing
- Knowledge Graph - Graph-based knowledge management
- Automatic entity extraction and relationship mapping
- 5 search methods: text, graph, vector, SPARQL, hybrid
- RDF triple store for structured knowledge
- Graph analytics (centrality, importance scoring)
- Answers: "What's GraphRAG?" "How do I search semantically?" "Knowledge graph setup?" "Vector search?"
Integration Overview - Third-party integrations
- Available integrations
- Integration patterns
- Best practices
- Answers: "What can I integrate?" "How do integrations work?"
Integration Quick Start - Get started with integrations
Integration Cheat Sheet - Quick reference
AI/ML Integration - Machine learning support
- AI/ML Integration Guide - Overview
- Distributed Training - Train models
- Metrics & Visualization - Track performance
- Answers: "Can I store models?" "How do I track metrics?"
LangChain - LangChain framework
- LangChain document loaders
- IPFS-backed chains
- Answers: "Does it work with LangChain?"
LlamaIndex - LlamaIndex framework
- Index storage on IPFS
- Query engines
- Answers: "Does it work with LlamaIndex?"
IPFS Datasets - Dataset management
- Large dataset handling
- Chunking and reassembly
- Answers: "How do I store big datasets?"
FSSpec - Filesystem specification
- Filesystem abstraction
- Pandas integration
- Answers: "Can I use it like a filesystem?"
IPLD - IPLD data structures
- IPLD DAGs
- Custom codecs
- Answers: "What's IPLD?" "How do I work with DAGs?"
LibP2P - P2P networking
- Implementation Plan
- Peer discovery
- Network configuration
- Answers: "How does P2P work?" "Can I customize networking?"
Cluster Management - Multi-node setup
- Cluster architecture
- Node roles (master/worker/leecher)
- Leader election
- Scaling strategies
- Answers: "How do I set up a cluster?" "What's leader election?"
Cluster Monitoring - Health & metrics
- Health checks
- Performance monitoring
- Alert configuration
- Answers: "How do I monitor my cluster?" "Is it healthy?"
Cluster State - State management
- State synchronization
- Consistency guarantees
- Answers: "How is state managed?" "What about consistency?"
Cluster Authentication - Security
- Authentication setup
- Authorization policies
- Answers: "How do I secure my cluster?"
Containerization - Docker & containers
- Docker images
- Container configuration
- Answers: "How do I use Docker?" "Is there an image?"
CI/CD Automation - Continuous deployment
- CI/CD Summary
- Quick Reference
- Integration Plan
- GitHub Runner Setup
- GitHub API Caching
- Answers: "How do I automate deployment?" "CI/CD setup?"
Docker Deployment - Docker-specific
ARM64 Support - ARM architecture
- ARM64 builds
- Raspberry Pi deployment
- Answers: "Does it run on ARM?" "Raspberry Pi support?"
Multi-Architecture - Multi-platform
Observability - Monitoring & logging
- Logging configuration
- Metrics collection
- Tracing setup
- Answers: "How do I debug?" "Where are the logs?"
Performance Metrics - Performance tuning
- Metrics Optimization
- Performance benchmarks
- Optimization tips
- Answers: "How fast is it?" "How do I optimize?"
Resource Management - Resource limits
- Memory management
- Disk usage
- Network bandwidth
- Answers: "How much memory does it use?" "Can I limit resources?"
Architecture - System design
- MCP Integration Architecture
- Backend Architecture
- Filesystem Backend
- Refactored Architecture
- Answers: "How is it designed?" "What's the architecture?"
Storage Backends - Storage options
- Enhanced Analytics
- Available backends
- Performance characteristics
- Answers: "Where is data stored?" "What backends exist?"
Metadata Index - Fast lookups
- Index structure
- Query performance
- Answers: "How does search work?" "Index internals?"
Write-Ahead Log - Data consistency
- WAL design
- Recovery procedures
- Answers: "How is data protected?" "What about crashes?"
Tiered Cache - Caching strategy
- Cache layers
- Eviction policies
- Answers: "How does caching work?" "Cache configuration?"
Protobuf - Protocol buffers
Telemetry - System telemetry
Testing Guide - Running tests
- Test suite organization
- Writing tests
- CI integration
- Answers: "How do I test?" "Where are the tests?"
Async Architecture - Async patterns
- Async/await usage
- Concurrency patterns
- Answers: "How does async work?" "Concurrency model?"
API Stability - API versioning
- Stability guarantees
- Breaking changes
- Answers: "Will APIs change?" "Backwards compatibility?"
User Guides - Step-by-step tutorials
- CLI Policy Usage
- Cluster Deployment
- Secure Credentials
- Config Fix Reference
- Multiprocessing
- Auto Update
- Answers: Step-by-step "how to" guides
Documentation Guide - Writing docs
Reorganization Guide - Project structure
Credential Management - Secure secrets storage
- Unified credential manager for all services
- S3, Storacha, Filecoin, HuggingFace credentials
- Secure storage with proper permissions
- Environment variable support
- Multiple named credential sets per service
- Answers: "How do I store API keys?" "Credential management?" "Secrets security?"
Configuration - System configuration
- YAML/JSON configuration files
- Environment variable override
- Storage backend configuration
- Cache settings and policies
- Cluster configuration
- VFS bucket policies
- Feature flags and toggles
- Answers: "How do I configure the system?" "Config file format?" "Environment variables?"
Secure Credentials Guide - Security best practices
- Credential storage security
- File permissions and access control
- CI/CD secrets management
- Production security practices
- Credential rotation strategies
- Answers: "How do I secure credentials?" "Production security?" "Best practices?"
Filesystem Journal - Filesystem journaling
Knowledge Graph - Knowledge graph integration
IPFS Dataloader - Data loading utilities
Metadata Replication - Cross-node replication
Advanced Prefetching - Predictive loading
Probabilistic Data Structures - Bloom filters, etc.
Lotus Daemon Management - Filecoin Lotus
Simplified Bucket Architecture - Bucket design
Credential Management - Secrets handling
Extensions - Plugin system
Integrated Search - Search capabilities
Documentation Plan - Doc strategy
Performance Optimization - Optimization roadmap
Telemetry API - Telemetry endpoints
PyPI Release - Package release
Index - Documentation index
Testing - Test documentation
- 100% Coverage Initiative
- Test Health Matrix
- Test reports and summaries
ARCHIVE - Historical documentation
- Previous implementations
- Old status reports
- Deprecated features
- Migration guides
- Installation Guide
- Quick Reference - basic operations
- Examples - run sample code
- API Reference - explore methods
- Quick Reference
- AI/ML Integration
- LangChain Integration
- LlamaIndex Integration
- Examples - ML examples
- Core Concepts
- Architecture
- Storage Backends
- Async Architecture
- Testing Guide
- Source code exploration
"How do I install/setup?" → Installation Guide
"How do I use X feature?" → Quick Reference → API Reference
"How does X work internally?" → Core Concepts → Architecture
"How do I deploy to production?" → Containerization → CI/CD
"How do I integrate with Y?" → Integration Overview → Specific integration
"What can I build with this?" → Examples → Use Cases in main README
"Something's not working" → Auto-Healing → Observability
"How do I contribute?" → Testing Guide → GitHub
Application Developers
Data Scientists
DevOps/SRE
System Architects
Contributors
UPPERCASE.md- Major guides and documentationlowercase.md- Technical references and specifications
Each document includes:
- Overview - What it covers
- Prerequisites - What you need first
- Examples - Code samples
- Reference - Detailed information
- See Also - Related documents
- ✅ Production Ready - Stable and tested
- 🚧 Beta - Usable but evolving
- 📋 Planned - Future feature
- 🗄️ Archived - Historical reference
Found an issue or want to help?
- Report Issues - Open an issue for errors or confusion
- Suggest Improvements - PRs welcome for clarity, examples, fixes
- Add Examples - Share your use cases
- Fill Gaps - Help document undocumented features
See CONTRIBUTING.md for guidelines.
- Version: 0.3.0
- Last Updated: February 2, 2026
- Python: 3.12+ required
- Main README - Project overview
- GitHub Repository
- Issue Tracker
- Examples - Code examples
Need help? Start with the Quick Reference or open an issue.
Can't find something? Use GitHub's search or open a discussion.