Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
80e74c1
Sohail/add 3rd party licenses (#1289)
sosahi Jan 13, 2026
2640917
Rm llama core (#1291)
jperez999 Jan 13, 2026
20e2dca
Bump ffmpeg to 8.0.1 and add nasm build dependency (#1294)
edknv Jan 13, 2026
51b9f0d
Prepare for 26.1.1 release
jdye64 Jan 13, 2026
f8097a6
Remove huggingface access token and llama tokenizer download from con…
jdye64 Jan 14, 2026
a1f6989
remove conda llama package ref (#1302)
jperez999 Jan 14, 2026
a2fb0b6
Kheiss/helm by default (#1303)
kheiss-uwzoo Jan 14, 2026
509e77b
Update version number (26.1.1) (#1298)
nkmcalli Jan 13, 2026
bf5071b
docs: update requirements file for doc build (26.1.1) (#1300)
nkmcalli Jan 14, 2026
c3db0bf
Update version references from 26.1.0 to 26.1.1 (#1310)
sosahi Jan 16, 2026
79194d7
Merge remote-tracking branch 'upstream/release/26.1.1'
sosahi Jan 16, 2026
41f55a9
Merge remote-tracking branch 'upstream/main'
sosahi Jan 21, 2026
cac0c45
Merge remote-tracking branch 'upstream/main'
sosahi Jan 21, 2026
a953255
Merge upstream/main, accepting all upstream changes
sosahi Jan 23, 2026
e0f4c14
Merge remote-tracking branch 'upstream/main'
sosahi Feb 6, 2026
555f3f6
Merge remote-tracking branch 'upstream/main'
sosahi Feb 10, 2026
85fe95b
Merge remote-tracking branch 'upstream/main'
sosahi Feb 19, 2026
c182c2b
Merge remote-tracking branch 'upstream/main'
sosahi Feb 26, 2026
f01d8a2
Merge remote-tracking branch 'upstream/main'
sosahi Feb 27, 2026
365bfd7
Merge remote-tracking branch 'upstream/main'
sosahi Feb 27, 2026
8b003b6
Fix GitHub Pages docs workflow and Dockerfile docs stage
sosahi Feb 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
223 changes: 223 additions & 0 deletions .github/ACTIONS_README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,223 @@
# GitHub Actions Structure

This directory contains all GitHub Actions workflows, reusable components, and documentation for the nv-ingest CI/CD pipeline.

## 📁 Directory Structure

```
.github/
├── workflows/ # Workflows (including reusable workflows)
├── actions/ # Composite actions (3 actions)
├── ISSUE_TEMPLATE/ # Issue templates
├── CODEOWNERS # Code ownership
├── PULL_REQUEST_TEMPLATE.md
├── copy-pr-bot.yaml
└── Documentation:
├── README.md (this file) # Overview and quick reference
├── WORKFLOWS_REFERENCE.md # Complete technical reference
├── WORKFLOWS_QUICKSTART.md # Quick start guide
└── ARCHITECTURE.md # System architecture
```

## 🚀 Quick Start

### For Developers
Read: [`WORKFLOWS_QUICKSTART.md`](./WORKFLOWS_QUICKSTART.md)

### For Complete Reference
Read: [`WORKFLOWS_REFERENCE.md`](./WORKFLOWS_REFERENCE.md)

### For Architecture Details
Read: [`ARCHITECTURE.md`](./ARCHITECTURE.md)

## 🎯 Workflow Overview

### Continuous Integration

| Workflow | File | Trigger | Purpose |
|----------|------|---------|---------|
| PR Validation | `ci-pull-request.yml` | Pull requests | Pre-commit, build, test |
| Main CI | `ci-main.yml` | Push to main | Full validation + multi-platform |

### Nightly & Scheduled

| Workflow | File | Schedule | Purpose |
|----------|------|----------|---------|
| Nightly Builds | `scheduled-nightly.yml` | Daily 23:30 UTC | Docker + Conda + PyPI |

### Release Management

| Workflow | File | Trigger | Purpose |
|----------|------|---------|---------|
| Docker Release | `release-docker.yml` | release/* branch OR manual | Publish Docker images |
| Conda Release | `release-conda.yml` | release/* branch OR manual | Publish Conda packages |
| PyPI Release | `release-pypi.yml` | release/* branch OR manual | Publish Python wheels |

### Documentation

| Workflow | File | Trigger | Purpose |
|----------|------|---------|---------|
| Build & Deploy Docs | `build-docs.yml` | Push to main OR manual | Build docs in Docker, deploy to GitHub Pages (https://nvidia.github.io/nv-ingest/) |
| Docs Deploy (manual) | `docs-deploy.yml` | Manual only | Same as above; use for one-off deploys without pushing to main |

## 🔧 Reusable Components

### Workflows (in `workflows/`, prefixed with `reusable-`)

- `reusable-docker-build.yml` - Flexible Docker image building
- `reusable-docker-test.yml` - Container-based testing
- `reusable-conda-build.yml` - Conda package building
- `reusable-conda-publish.yml` - Conda package publishing
- `reusable-pypi-build.yml` - Python wheel building
- `reusable-pypi-publish.yml` - PyPI publishing
- `reusable-integration-test.yml` - Library mode testing

### Actions (in `actions/`)

- `setup-docker-buildx/` - Docker Buildx + QEMU setup
- `docker-login-ngc/` - NGC registry authentication
- `determine-version/` - Smart version determination

## 📊 Workflow Architecture

```
┌─────────────────────┐
│ Main Workflows │ (Triggered by events)
│ - ci-pull-request │
│ - ci-main │
│ - scheduled-* │
│ - release-* │
└──────┬──────────────┘
│ calls
┌─────────────────────┐
│ Reusable Workflows │ (Business logic)
│ - docker-build │
│ - conda-publish │
│ - pypi-build │
└──────┬──────────────┘
│ uses
┌─────────────────────┐
│ Composite Actions │ (Common operations)
│ - setup-buildx │
│ - docker-login │
│ - determine-ver │
└─────────────────────┘
```

## 🎯 Key Features

### Reusable Workflows
- Docker build logic defined once, used everywhere
- Consistent patterns across all workflows
- Type-safe interfaces with validation

### Flexible Configuration
- Reusable workflows accept inputs
- Composite actions are parameterized
- Easy to customize per use case

### Clear Separation
- Main workflows = triggers + orchestration
- Reusable workflows = business logic
- Composite actions = common operations

### ✅ Type Safety
- Inputs/outputs explicitly defined
- Required vs optional parameters
- Validation built-in

### ✅ Better Testing
- Reusable components can be tested independently
- workflow_dispatch for manual testing
- Clear job dependencies

## 🔐 Required Secrets

### Docker/NGC
- `DOCKER_PASSWORD` - NGC API token
- `DOCKER_REGISTRY` - Registry URL (e.g., nvcr.io)
- `HF_ACCESS_TOKEN` - Hugging Face token

### Conda
- `NVIDIA_CONDA_TOKEN` - Anaconda.org token

### PyPI
- `ARTIFACTORY_URL` - PyPI repository URL
- `ARTIFACTORY_USERNAME` - Username
- `ARTIFACTORY_PASSWORD` - Password

### Integration Tests
- `NGC_API_KEY` / `NVIDIA_API_KEY`
- `AUDIO_FUNCTION_ID`
- `EMBEDDING_NIM_MODEL_NAME`
- `NEMOTRON_PARSE_MODEL_NAME`
- `PADDLE_HTTP_ENDPOINT`
- `VLM_CAPTION_ENDPOINT`
- `VLM_CAPTION_MODEL_NAME`
- `YOLOX_*_HTTP_ENDPOINT` (multiple)

## 📝 Common Tasks

### Run PR checks locally
```bash
pre-commit run --all-files
docker build --target runtime -t nv-ingest:test .
docker run nv-ingest:test pytest -m "not integration"
```

### Trigger nightly build manually
```
Actions → "Nightly Builds & Publishing" → Run workflow
```

### Create a release
```bash
# Automatic - All three artifact types (recommended)
git checkout -b release/25.4.0
git push origin release/25.4.0
# → Triggers Docker, Conda, AND PyPI releases automatically

# Manual (for custom options)
Actions → Release - Docker/Conda/PyPI → Run workflow
```

### Debug workflows
```
Actions → Select workflow → View logs → Expand steps
```

## 🐛 Troubleshooting

| Issue | Solution |
|-------|----------|
| Workflow not found | Check path: `.github/workflows/` (reusable workflows are `reusable-*.yml`) |
| Secret not available | Verify in Settings → Secrets → Actions |
| Build timeout | Use `linux-large-disk` runner |
| Integration tests fail | Check NIM endpoints and credentials |

## 📚 Documentation

- **Quick Start**: [`WORKFLOWS_QUICKSTART.md`](./WORKFLOWS_QUICKSTART.md)
- **Complete Reference**: [`WORKFLOWS_REFERENCE.md`](./WORKFLOWS_REFERENCE.md)
- **Architecture**: [`ARCHITECTURE.md`](./ARCHITECTURE.md)

## 🆘 Getting Help

1. Check workflow logs in Actions tab
2. Review documentation in this folder
3. Search for similar issues
4. Contact DevOps team
5. Open an issue with details

## 📞 Maintainers

See [`CODEOWNERS`](./CODEOWNERS) for ownership information.

---

**Architecture**: Reusable workflows + Composite actions
**Documentation**: 4 comprehensive guides
**Total Components**: Workflows + reusable workflows + composite actions
Loading
Loading