|
| 1 | +# Standard DevOps Framework |
| 2 | + |
| 3 | +Standard is a Nix Flakes-based DevOps framework that organizes your development lifecycle into Cells (folders) and Cell Blocks (Nix files). It provides a CLI/TUI for discovering and running targets across the entire SDLC. |
| 4 | + |
| 5 | +**ALWAYS follow these instructions first and fallback to search or exploration commands only when you encounter unexpected information that does not match the information here.** |
| 6 | + |
| 7 | +## Working Effectively |
| 8 | + |
| 9 | +### Prerequisites and Installation |
| 10 | +- **CRITICAL**: Install Nix package manager first: |
| 11 | + ```bash |
| 12 | + curl -L https://nixos.org/nix/install | sh |
| 13 | + ``` |
| 14 | + or use the Determinate Systems installer: |
| 15 | + ```bash |
| 16 | + curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install |
| 17 | + ``` |
| 18 | +- Install direnv for automatic environment loading: |
| 19 | + ```bash |
| 20 | + # See https://direnv.net/docs/installation.html for your platform |
| 21 | + curl -sfL https://direnv.net/install.sh | bash |
| 22 | + ``` |
| 23 | +- Configure your shell for direnv (add to `.bashrc`/`.zshrc`): |
| 24 | + ```bash |
| 25 | + eval "$(direnv hook bash)" # or zsh |
| 26 | + ``` |
| 27 | + |
| 28 | +### Environment Setup and Entry |
| 29 | +- **ALWAYS** start by entering the development environment: |
| 30 | + ```bash |
| 31 | + direnv allow |
| 32 | + ``` |
| 33 | + **First time takes 5-15 minutes** - NEVER CANCEL. This downloads and builds the development dependencies. |
| 34 | +- Alternative environment entry (if direnv fails): |
| 35 | + ```bash |
| 36 | + direnv allow || nix develop -c "$SHELL" |
| 37 | + ``` |
| 38 | + **Takes 5-15 minutes** - NEVER CANCEL. |
| 39 | +- **After making changes to Nix files**, reload the environment: |
| 40 | + ```bash |
| 41 | + direnv reload |
| 42 | + ``` |
| 43 | +- **Verify environment loaded correctly**: |
| 44 | + ```bash |
| 45 | + std --help # Should show Standard CLI help |
| 46 | + ``` |
| 47 | + |
| 48 | +### Build Commands |
| 49 | +- **Build the entire project** - NEVER CANCEL: Build takes 10-30 minutes. Set timeout to 45+ minutes: |
| 50 | + ```bash |
| 51 | + nix build |
| 52 | + ``` |
| 53 | +- **Build and check all flake outputs** (includes tests): |
| 54 | + ```bash |
| 55 | + nix flake check |
| 56 | + ``` |
| 57 | + **Takes 15-45 minutes** - NEVER CANCEL. Set timeout to 60+ minutes. |
| 58 | +- **Build specific targets** using the Standard CLI: |
| 59 | + ```bash |
| 60 | + std # Launch the TUI to see available targets |
| 61 | + std //local/shells:default # Build specific target |
| 62 | + ``` |
| 63 | +- **Verify build without execution**: |
| 64 | + ```bash |
| 65 | + nix build --dry-run |
| 66 | + ``` |
| 67 | + |
| 68 | +### Testing |
| 69 | +- **Run snapshot tests** - NEVER CANCEL: Tests take 5-15 minutes. Set timeout to 30+ minutes: |
| 70 | + ```bash |
| 71 | + namaka check |
| 72 | + # or via Standard CLI: |
| 73 | + std //tests/checks/snapshots:eval |
| 74 | + ``` |
| 75 | +- **Run all CI checks** as they would run in GitHub Actions: |
| 76 | + ```bash |
| 77 | + nix flake check |
| 78 | + ``` |
| 79 | + **Takes 15-45 minutes** - NEVER CANCEL. Set timeout to 60+ minutes. |
| 80 | + |
| 81 | +### Formatting and Linting |
| 82 | +- **ALWAYS run formatting before committing** (CI will fail otherwise): |
| 83 | + ```bash |
| 84 | + treefmt |
| 85 | + ``` |
| 86 | +- **Format only changed files**: |
| 87 | + ```bash |
| 88 | + treefmt $(git diff --name-only --cached) |
| 89 | + ``` |
| 90 | +- **License compliance check** (required for CI): |
| 91 | + ```bash |
| 92 | + reuse lint |
| 93 | + ``` |
| 94 | + |
| 95 | +## Validation |
| 96 | + |
| 97 | +### Manual Validation Requirements |
| 98 | +**ALWAYS run these validation steps after making changes:** |
| 99 | + |
| 100 | +1. **Environment validation**: Verify the development environment loads correctly: |
| 101 | + ```bash |
| 102 | + direnv reload && echo "Environment loaded successfully" |
| 103 | + ``` |
| 104 | + |
| 105 | +2. **Build validation**: Ensure the project builds successfully: |
| 106 | + ```bash |
| 107 | + nix build && echo "Build completed successfully" |
| 108 | + ``` |
| 109 | + |
| 110 | +3. **Standard CLI validation**: Test the CLI functionality: |
| 111 | + ```bash |
| 112 | + std --help # Should show help without errors |
| 113 | + std # Should launch TUI - exit with 'q' |
| 114 | + ``` |
| 115 | + |
| 116 | +4. **Test validation**: Run the test suite: |
| 117 | + ```bash |
| 118 | + namaka check && echo "All tests passed" |
| 119 | + ``` |
| 120 | + |
| 121 | +5. **Documentation validation**: If docs were modified, build them: |
| 122 | + ```bash |
| 123 | + mdbook build # Builds to docs/book/ |
| 124 | + ``` |
| 125 | + **Takes 2-5 minutes** - do not cancel. |
| 126 | + |
| 127 | +### Critical Validation Scenarios |
| 128 | +After making code changes, ALWAYS test these complete workflows: |
| 129 | + |
| 130 | +1. **Fresh environment setup** (simulates new contributor): |
| 131 | + ```bash |
| 132 | + direnv disallow && direnv allow |
| 133 | + # Should complete without errors in 5-15 minutes |
| 134 | + ``` |
| 135 | + |
| 136 | +2. **Full development cycle**: |
| 137 | + ```bash |
| 138 | + # Make a trivial change, then: |
| 139 | + treefmt # Format code |
| 140 | + nix build # Build project |
| 141 | + namaka check # Run tests |
| 142 | + reuse lint # Check licenses |
| 143 | + ``` |
| 144 | + |
| 145 | +3. **Template functionality** (if templates were modified): |
| 146 | + ```bash |
| 147 | + nix flake init -t .#minimal /tmp/test-project |
| 148 | + cd /tmp/test-project && direnv allow |
| 149 | + # Should work without errors - takes 5-15 minutes first time |
| 150 | + ``` |
| 151 | + |
| 152 | +4. **Subflake lock file updates** (for maintainers): |
| 153 | + ```bash |
| 154 | + # When updating Standard framework itself: |
| 155 | + ./.github/workflows/update-subflake.sh |
| 156 | + # This updates src/local/flake.lock and src/tests/flake.lock |
| 157 | + ``` |
| 158 | + |
| 159 | +## Common Tasks |
| 160 | + |
| 161 | +### Repository Structure |
| 162 | +``` |
| 163 | +. |
| 164 | +├── .envrc # direnv configuration - auto-loads dev environment |
| 165 | +├── flake.nix # Main Nix flake definition - project entry point |
| 166 | +├── dogfood.nix # Project self-configuration - how std uses itself |
| 167 | +├── src/ |
| 168 | +│ ├── local/ # Local development environment |
| 169 | +│ │ ├── shells.nix # Development shells - FREQUENTLY MODIFIED |
| 170 | +│ │ ├── configs.nix # Tool configurations (treefmt, nixago, etc.) |
| 171 | +│ │ └── tasks.nix # Automation tasks |
| 172 | +│ ├── std/ # Standard framework core - MAIN CODEBASE |
| 173 | +│ │ ├── fwlib/ # Framework library - core abstractions |
| 174 | +│ │ │ ├── actions/ # Build/run actions for targets |
| 175 | +│ │ │ └── blockTypes/ # Cell Block type definitions |
| 176 | +│ │ ├── cli.nix # CLI/TUI implementation |
| 177 | +│ │ └── templates/ # Project templates |
| 178 | +│ └── tests/ # Test suites - snapshot tests |
| 179 | +├── docs/ # Documentation source (mdbook) |
| 180 | +│ ├── explain/ # Why Nix? Why Standard? |
| 181 | +│ ├── tutorials/ # Step-by-step guides |
| 182 | +│ ├── guides/ # How-to guides |
| 183 | +│ └── reference/ # API documentation |
| 184 | +└── .github/workflows/ # CI/CD pipelines - uses std-action |
| 185 | +``` |
| 186 | + |
| 187 | +### Frequently Modified Files |
| 188 | +- `src/local/shells.nix` - Add tools, modify dev environment |
| 189 | +- `src/local/configs.nix` - Configure treefmt, linting, nixago tools |
| 190 | +- `src/std/fwlib/blockTypes/` - Extend Standard with new block types |
| 191 | +- `src/std/templates/` - Add/modify project templates |
| 192 | +- `docs/` - Documentation updates |
| 193 | +- `flake.nix` - Project inputs and core configuration |
| 194 | + |
| 195 | +### Standard Framework Concepts |
| 196 | +- **Cells**: Folders that group related functionality (e.g., `local`, `std`, `tests`) |
| 197 | +- **Cell Blocks**: Nix files that define outputs (e.g., `shells.nix`, `packages.nix`) |
| 198 | +- **Targets**: Individual functions/packages within Cell Blocks |
| 199 | +- **Actions**: Operations you can perform on targets (build, run, etc.) |
| 200 | + |
| 201 | +### Key Commands Reference |
| 202 | +```bash |
| 203 | +# Environment |
| 204 | +direnv allow # Enter dev environment (5-15 min first time) |
| 205 | +direnv reload # Reload after changes |
| 206 | +nix develop # Alternative environment entry |
| 207 | + |
| 208 | +# Building |
| 209 | +nix build # Build everything (10-30 min, NEVER CANCEL) |
| 210 | +std # Launch TUI to see targets |
| 211 | +std //local/shells:default # Build specific target |
| 212 | + |
| 213 | +# Testing |
| 214 | +namaka check # Run snapshot tests (5-15 min, NEVER CANCEL) |
| 215 | +nix flake check # Run all checks (15-45 min, NEVER CANCEL) |
| 216 | + |
| 217 | +# Formatting & Linting |
| 218 | +treefmt # Format all files (REQUIRED before commit) |
| 219 | +reuse lint # Check license compliance |
| 220 | + |
| 221 | +# Documentation |
| 222 | +mdbook build # Build documentation (2-5 min) |
| 223 | +mdbook serve # Serve docs locally at localhost:3000 |
| 224 | +``` |
| 225 | + |
| 226 | +### File Watching and Live Reload |
| 227 | +- The `.envrc` file automatically watches relevant Nix files for changes |
| 228 | +- When you modify shell configurations, direnv will prompt to reload |
| 229 | +- Documentation can be live-reloaded: `mdbook serve` |
| 230 | + |
| 231 | +### Troubleshooting |
| 232 | +- **"command not found: std"**: Run `direnv reload` or `nix develop -c "$SHELL"` |
| 233 | +- **Build failures**: Check `nix log` output for specific errors; try `nix build --show-trace` for more details |
| 234 | +- **Slow builds**: This is normal for Nix - do not cancel, builds are cached after first success |
| 235 | +- **direnv not working**: Ensure direnv is installed and shell hook is configured |
| 236 | +- **Network errors during setup**: Some dependencies require internet access; check firewall/proxy settings |
| 237 | +- **Flake lock issues**: In subflakes, run `./.github/workflows/update-subflake.sh` to update lock files |
| 238 | +- **"error: getting status of '/nix/store/...'**: Usually means incomplete download; try `nix build` again |
| 239 | +- **Out of disk space**: Nix store can get large; run `nix-collect-garbage` to clean up |
| 240 | + |
| 241 | +### Performance Notes |
| 242 | +- **First-time setup**: Expect 5-15 minutes for initial environment build |
| 243 | +- **Subsequent environment loads**: Usually under 30 seconds due to Nix caching |
| 244 | +- **Full builds**: 10-30 minutes depending on what changed |
| 245 | +- **Tests**: 5-15 minutes for complete test suite |
| 246 | +- **CI builds**: 15-45 minutes for full CI pipeline |
| 247 | + |
| 248 | +**CRITICAL TIMING REMINDERS:** |
| 249 | +- **NEVER CANCEL** any nix build, test, or environment setup commands |
| 250 | +- Always set timeouts to at least 2x the expected time |
| 251 | +- Use `timeout` values of 60+ minutes for builds and 30+ minutes for tests |
| 252 | +- If a command appears stuck, wait at least 15 minutes before investigating |
| 253 | + |
| 254 | +### Integration with CI |
| 255 | +- All formatting and linting must pass for CI to succeed |
| 256 | +- CI uses the same Standard framework via `divnix/std-action` |
| 257 | +- Local validation with `nix flake check` mirrors CI exactly |
| 258 | +- Always run `treefmt` and `reuse lint` before pushing |
| 259 | +- **CI timing expectations**: Full CI takes 15-45 minutes across multiple platforms (Linux/macOS) |
| 260 | + |
| 261 | +### Common Development Patterns |
| 262 | + |
| 263 | +#### Adding a new tool to the development environment: |
| 264 | +1. Edit `src/local/shells.nix` |
| 265 | +2. Add package to `commands` list with appropriate category |
| 266 | +3. Run `direnv reload` to update environment |
| 267 | +4. Test with the new tool available |
| 268 | + |
| 269 | +#### Modifying Standard framework behavior: |
| 270 | +1. Core logic in `src/std/fwlib/` |
| 271 | +2. Block types in `src/std/fwlib/blockTypes/` |
| 272 | +3. Actions in `src/std/fwlib/actions/` |
| 273 | +4. Always run full test suite after changes |
| 274 | + |
| 275 | +#### Working with documentation: |
| 276 | +1. Source in `docs/` using mdbook format |
| 277 | +2. Live preview: `mdbook serve` at localhost:3000 |
| 278 | +3. Build: `mdbook build` outputs to `docs/book/` |
| 279 | +4. Documentation includes auto-generated API docs via paisano-preprocessor |
| 280 | + |
| 281 | +#### Template development: |
| 282 | +1. Templates in `src/std/templates/` |
| 283 | +2. Test with: `nix flake init -t .#<template-name> /tmp/test` |
| 284 | +3. Verify template works: `cd /tmp/test && direnv allow` |
0 commit comments