Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/workflows/build-docs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: "Build documentation"

concurrency:
group: "pages"
cancel-in-progress: false

on:
# Runs on pushes targeting the default branch
push:
branches: ["master"]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Pages
uses: actions/configure-pages@v5
- name: Run sphinx
uses: ammaraskar/sphinx-action@master
with:
pre-build-command: "pip install -U sphinx>8 kentigern pyyaml"
docs-folder: "docs/"
- uses: actions/upload-pages-artifact@v3
with:
path: docs/_build/html/
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
42 changes: 42 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg

# Virtual environments
venv/
ENV/
env/

# IDE
.vscode/
.idea/
*.swp
*.swo
*~

# OS
.DS_Store
Thumbs.db

# Documentation
docs/_build/
docs/_static/
docs/_templates/
245 changes: 245 additions & 0 deletions DOCUMENTATION_SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,245 @@
# Gravitic Documentation and Asimov Integration Summary

## Overview

This document summarizes the enhancements made to gravitic in response to the request for Sphinx documentation, GitHub Pages deployment, and asimov integration.

## Changes Made (Commit f419d1b)

### 1. Sphinx Documentation

Created comprehensive documentation using the kentigern theme:

#### Documentation Pages

- **index.rst** - Main documentation hub with project overview
- **quickstart.rst** - Getting started guide with installation and basic usage
- **blueprints.rst** - Complete blueprint format reference with examples
- **asimov.rst** - Asimov integration guide with usage examples
- **api.rst** - API reference using autodoc

#### Configuration Updates

- **conf.py** - Added extensions:
- `sphinx.ext.autodoc` - Automatic API documentation
- `sphinx.ext.viewcode` - Links to source code
- `sphinx.ext.intersphinx` - Cross-references to Python/asimov docs
- Set up path for module imports
- Configured kentigern theme

### 2. GitHub Pages Deployment

Created automated documentation deployment workflow:

#### Workflow File

`.github/workflows/build-docs.yaml`:
- Triggers on push to master branch
- Also supports manual workflow dispatch
- Uses sphinx-action for building
- Deploys to GitHub Pages automatically
- Installs required dependencies (sphinx>8, kentigern, pyyaml)

#### Features

- Automatic build on every push to master
- GitHub Pages deployment with proper permissions
- Artifact upload and deployment steps
- Consistent with asimov-cogwheel pattern

### 3. Asimov Integration

Implemented complete asimov pipeline integration:

#### GraviticPipeline Class

`gravitic/asimov.py` - Full asimov Pipeline implementation:

**Key Features:**
- Inherits from `asimov.pipeline.Pipeline`
- HTCondor DAG construction
- Job submission and monitoring
- Completion detection via asset files
- Template rendering support
- Graceful degradation when asimov not installed

**Methods:**
- `build_dag()` - Constructs HTCondor DAG for pipeline execution
- `submit_dag()` - Submits DAG to HTCondor scheduler
- `detect_completion()` - Checks for asset files to determine completion
- `after_completion()` - Post-processing actions
- `samples()` - Returns paths to posterior sample files

#### Template File

`gravitic/asimov_template.yaml`:
- Blueprint template for asimov workflows
- Supports Jinja2 template variables from asimov
- Access to production metadata
- Customizable for different analyses

#### Package Updates

- **setup.py** - Updated to include template as package data
- **MANIFEST.in** - Created to ensure template file is included in distribution
- **__init__.py** - Exports GraviticPipeline (with graceful import handling)

## Usage Examples

### Documentation Build

Local build:
```bash
cd docs
pip install sphinx kentigern
make html
```

View at: `docs/_build/html/index.html`

Deployed version will be available at: `https://transientlunatic.github.io/gravitic/`

### Asimov Integration

#### Production Configuration

```yaml
events:
- name: GW150914
repository: git@github.com:myorg/analyses
productions:
- name: gravitic_test
pipeline: gravitic
comment: Test analysis
status: ready
meta:
event time: 1126259462.4
interferometers: [H1, L1]
scheduler:
accounting group: ligo.dev.o4.cbc.pe.lalinference
```

#### Blueprint Template

Create `gravitic_test.yml`:
```yaml
kind: analysis
pipeline: gravitic
comment: {{ production.comment }}

steps:
- name: Setup
script: |
echo "Event: {{ production.event.name }}"
echo "GPS: {{ production.meta['event time'] }}"

- name: Analysis
shell: /usr/bin/python3
script: |
# Your analysis code
import h5py
# ...

assets:
posterior:
- posterior_samples.h5
```

#### Python API

```python
from gravitic.asimov import GraviticPipeline

# Create pipeline instance
pipeline = GraviticPipeline(production)

# Build and submit
pipeline.build_dag()
cluster_id = pipeline.submit_dag()

# Check completion
if pipeline.detect_completion():
pipeline.after_completion()
```

## Integration Pattern

The asimov integration follows the same pattern as other asimov pipelines (e.g., asimov-cogwheel):

1. **Pipeline class** inherits from `asimov.pipeline.Pipeline`
2. **Template file** provides default blueprint structure
3. **DAG construction** creates HTCondor workflow
4. **Completion detection** based on output files
5. **Post-processing** hooks for additional steps

## Benefits

### For Users

- **Easy documentation access** via GitHub Pages
- **Automated deployment** keeps docs up-to-date
- **Asimov compatibility** enables workflow automation
- **Template system** simplifies pipeline creation

### For Developers

- **Comprehensive API docs** with autodoc
- **Cross-referenced documentation** with intersphinx
- **Standard asimov interface** for pipeline integration
- **Modular design** allows customization

## Testing

All existing tests still pass (9/9):
```bash
python -m unittest discover -s tests -p "test_*.py"
```

Documentation builds without errors:
```bash
cd docs && make html
```

Asimov integration imports correctly (requires asimov):
```python
from gravitic.asimov import GraviticPipeline
```

## Files Changed

### New Files

- `.github/workflows/build-docs.yaml` - GitHub Pages deployment
- `MANIFEST.in` - Package manifest
- `docs/api.rst` - API reference
- `docs/asimov.rst` - Asimov integration guide
- `docs/blueprints.rst` - Blueprint format reference
- `docs/quickstart.rst` - Getting started guide
- `gravitic/asimov.py` - Asimov integration module
- `gravitic/asimov_template.yaml` - Blueprint template

### Modified Files

- `.gitignore` - Exclude docs build artifacts
- `docs/conf.py` - Enable autodoc and other extensions
- `docs/index.rst` - Updated main documentation page
- `gravitic/__init__.py` - Export GraviticPipeline
- `setup.py` - Include package data

## Future Enhancements

Potential improvements for future iterations:

1. **Multi-step DAG support** - More complex dependency graphs
2. **Parallel execution** - Run independent steps concurrently
3. **Container integration** - Built-in Docker/Apptainer support
4. **PESummary integration** - Automatic post-processing
5. **Enhanced templates** - More sophisticated Jinja2 templates
6. **Result validation** - Automated checks for output quality

## References

- Asimov: https://github.com/etive-io/asimov
- Asimov-Cogwheel example: https://github.com/etive-io/asimov-cogwheel
- Kentigern theme: https://github.com/transientlunatic/kentigern
- Sphinx documentation: https://www.sphinx-doc.org/
Loading
Loading