Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 146 additions & 0 deletions docs/docs/git-integration/develop-changes.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
---
title: Develop changes from a Git repository
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Develop changes from a Git repository

Change the schemas, objects, Transformations, Generators, and other [`.infrahub.yml`](./infrahub-yml.mdx) content an instance imports from a [read-only repository](./overview.mdx#read-only-vs-core), and land it on the instance's `main` safely. This is the recommended workflow for a single instance; promoting a change across environments builds on it — see [promote changes between environments](./promote-between-environments.mdx).

The instance's `main` tracks a Git branch through the read-only repository's `ref`, and you never import onto `main` directly. Instead you develop on an Infrahub branch and merge it into `main` through a proposed change. This worked example adds a `color` attribute to the built-in Tag.

## Prerequisites

- An Infrahub instance with the repository connected as a **read-only repository** (see [connect a repository](./connect-repository.mdx)). Its `ref` is the Git branch it tracks — `main` in this example.
- A [`.infrahub.yml`](./overview.mdx#infrahub-yaml) file on that branch.
- A local checkout of the repository and permission to open a pull request.

:::note How imports are triggered

A read-only repository does not poll for new commits — there is no background sync. An import runs only when you trigger one: the **Import latest commit** action imports the current `ref`'s latest commit, and changing the repository's `ref` dispatches an import on its own (so you do not also run **Import latest commit**). A mutation that returns `ok: true` means the import was dispatched, not finished — confirm a change landed by checking the repository's recorded commit on the instance.

:::

## Step 1: Create the change in Git

Work from a feature branch cut from the tracked branch (`main`).

```bash
git switch main
git switch -c feature/add-tag-color
```

Add a schema file that extends the built-in Tag with a `color` attribute:

```yaml
# schemas/tags.yml
---
version: '1.0'
extensions:
nodes:
- kind: BuiltinTag
attributes:
- name: color
kind: Text
optional: true
```

Make sure `.infrahub.yml` loads it, then commit and push:

```yaml
# .infrahub.yml
---
schemas:
- schemas/tags.yml
```

```bash
git add schemas/tags.yml .infrahub.yml
git commit -m "Add color attribute to Tag"
git push -u origin feature/add-tag-color
```

## Step 2: Test it on an Infrahub branch

Import your feature branch onto an Infrahub branch and check it there before it reaches `main`.

1. On the instance, create an Infrahub branch, for example `test-tag-color`.
2. On `test-tag-color`, set the repository's `ref` to your `feature/add-tag-color` Git branch. Changing the `ref` triggers the import on its own. Because `ref` is branch-aware, only this branch is affected — `main` is untouched.

```graphql
# against the Infrahub branch, e.g. https://<host>/graphql/test-tag-color
mutation {
CoreReadOnlyRepositoryUpdate(data: {
id: "<repository-id>"
ref: { value: "feature/add-tag-color" }
}) { ok }
}
```

3. Test the change on `test-tag-color` — confirm the schema has the `color` attribute and behaves as you expect, for example by creating a Tag with a color on this branch. There is no proposed change here; you work on the branch directly and never merge it.

This is an **iterative loop**: refine the schema on your Git feature branch, push, run **Import latest commit** on `test-tag-color`, and re-check. Repeat until the feature is ready — delete the branch when you are done.

:::tip Faster iteration with infrahubctl

While developing, you can load a schema file straight onto the branch with [`infrahubctl`]($(base_url)infrahubctl/infrahubctl) instead of importing it from Git — no commit or push required:

```shell
infrahubctl schema load schemas/tags.yml --branch test-tag-color
```

This shortens the inner loop. The Git import is what makes the change reproducible and promotable, so land the final version through Git as below.

:::

## Step 3: Land the change on `main`

1. Merge the Git pull request into `main`:

```bash
gh pr create --base main --head feature/add-tag-color --title "Add color attribute to Tag" --fill
gh pr merge --merge
```

2. On the instance, create an Infrahub branch, set its `ref` to `main` (which now holds the change), and open a proposed change from it to `main`. The `ref` change re-imports on its own.

```graphql
# against the Infrahub branch, e.g. https://<host>/graphql/land-tag-color
mutation {
CoreReadOnlyRepositoryUpdate(data: {
id: "<repository-id>"
ref: { value: "main" }
}) { ok }
}
```

3. Review the proposed change and merge it. The instance's `main` now has the `color` attribute.

:::caution Do not import onto `main` directly

You can import onto `main` — run **Import latest commit** or set its `ref` there — to skip the Infrahub branch and proposed change. It is faster but not recommended: an import can fail, especially while iterating on a schema, and there is no clean way to undo a bad import on `main`, whereas an Infrahub branch can be discarded.

:::

## Step 4: Verify

Create a Tag with a color to confirm the attribute is on `main`:

```graphql
# against https://<host>/graphql/main
mutation {
BuiltinTagCreate(data: {
name: { value: "ready" }
color: { value: "#2ecc71" }
}) { ok object { id color { value } } }
}
```

## Related

- [Promote changes between environments](./promote-between-environments.mdx) — repeat this workflow across dev / staging / production
- [Connect a repository](./connect-repository.mdx) — repository registration
- [Schema extensions](../schema/extensions.mdx) — extending existing nodes like Tag
- [Proposed changes](../proposed-changes/overview.mdx) — the review workflow used to merge into `main`
91 changes: 91 additions & 0 deletions docs/docs/git-integration/multi-environment.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
title: Multiple environments from a single repository
---

Run a separate Infrahub instance for each environment — development, staging, production — from one Git repository. Each instance reads its schemas, objects, and automation from a [read-only repository](./overview.mdx#read-only-vs-core) pinned to one long-lived branch. To move a change from one environment to the next, merge it into the next branch on your Git host, then trigger an import on the target instance.

Use this pattern when you want environment isolation with a Git-native promotion path: each environment is independent, every promotion is a pull request on your Git host, and a change is reviewed on the Git side and again inside Infrahub before it reaches production.

## What moves through Git

A read-only repository imports what you declare in [`.infrahub.yml`](./infrahub-yml.mdx):

- Schemas
- Objects defined in object files
- GraphQL queries
- Jinja2 and Python Transformations
- Artifact definitions
- Generators
- Checks
- Menus

These are the changes you promote between environments. Data created directly in an instance through the UI or API — the objects your users edit day to day — is not part of this. It lives in each instance and never moves through Git.

## Git and Infrahub terminology

A promotion touches two systems that share vocabulary — both Git and Infrahub have branches, a `main`, and a merge step. The terms below keep them apart:

- **Git branch**, **Git pull request** — on your Git host. The long-lived per-environment branches (`develop`, `staging`, `main`) are Git branches; moving a change from one to the next is a pull request.
- **Infrahub branch**, **proposed change** — inside a single instance. You import into an Infrahub branch, review it as a proposed change, and merge it into that instance's `main`.

Production's Git branch is also named `main`, but it is unrelated to any instance's internal `main`.

## The model

One repository holds one long-lived Git branch per environment. Each branch maps to its own instance through that instance's read-only repository `ref`:

| Git branch | Infrahub instance | Read-only repository `ref` |
| --- | --- | --- |
| `develop` | Development | `develop` |
| `staging` | Staging | `staging` |
| `main` | Production | `main` |

![Multi-environment topology: one Git repository with a branch per environment, each feeding its own instance through a read-only ref, with changes authored on a feature branch and promoted along the chain](../media/multi-environment-topology.excalidraw.svg)

This example uses three environments, but the count is arbitrary — two (`develop` → `main`) or five work the same way, and the steps for each promotion hop are identical. Add or remove an environment by adding or removing a Git branch and the instance that tracks it.

Every environment is a **consumer**: its files come from a read-only repository, and no instance holds a read-write [`Repository`](./overview.mdx#read-only-vs-core), so nothing is ever pushed back to Git. You author changes the plain Git way — edit files locally and open a pull request into an environment's branch.

:::note Use a read-only repository, not a read-write one

Read-write `Repository` connections are not recommended for multi-environment setups: they push merges back to Git and track every branch, which breaks the per-environment isolation this pattern depends on. If you currently connect the repository as a read-write `Repository` on a single instance and want to adopt this pattern, connect it as a read-only repository instead.

:::

A read-only repository does not poll the remote for new commits — there is no background sync, unlike a read-write repository. It imports only in response to an explicit action: running the **Import latest commit** action, or changing its `ref` (which dispatches an import on its own). Either way, an import is a deliberate step, not an automatic sync of every new commit on the branch.

## Promoting a change

Each instance's `main` tracks its environment branch through the read-only repository's `ref` — the branch name you set. (Infrahub records which commit it imported internally; that is not something you set.)

You never import onto `main` directly. Land a change through an Infrahub branch: create the branch, import the change onto it by pointing `ref` at the branch that holds it, review the result as a [proposed change](../proposed-changes/overview.mdx), and merge — which advances `main`. Because `ref` is branch-aware, working on the Infrahub branch never disturbs `main`.

Authoring and promotion differ only in where the change comes from: a Git feature branch you are working on (authoring), or the environment branch after the previous environment merged into it (promotion). In both, `main`'s `ref` ends up on the environment branch. The [guide](./promote-between-environments.mdx) walks both, including how `ref` moves to a feature branch during authoring and returns to the environment branch before the merge.

:::caution Do not import onto `main` directly

Pointing `main`'s `ref` at a branch and importing there skips the Infrahub branch and its proposed change. It is faster but not recommended: an import can fail — especially while a schema is being developed and iterated — and there is no clean way to undo a bad import on `main`, whereas an Infrahub branch can be discarded.

:::

:::note Retiring schema is explicit, not automatic

Removing a field or node from the schema files does not remove it from an instance. To retire schema, set `state: absent` on those nodes in the schema files yourself — the same as with `infrahubctl schema load`. It is possible, but not declarative.

:::

## Reaching production safely

Lower environments make promotion safer, but they are not proof. Staging drifts from production over time — different data, a different history of applied changes — so a change that imported cleanly on staging can still behave differently against production's actual state.

Importing into an Infrahub branch and merging through a proposed change removes most of that risk: you review the exact diff before it touches production `main`. But the merge of that proposed change can itself fail partway and leave `main` in an unexpected state, and there is no clean rollback — an import applies forward, and reverting the `ref` restores the files, not the instance state the import produced.

To be certain a promotion applies cleanly, test it against production's real state first. Provision a pre-production instance on demand from a current production [database backup](../deploy-manage/maintain-upgrade/database-backup/backup-and-restore.mdx), run the promotion there, and confirm it applies cleanly before promoting production itself. Because it starts from a fresh backup, it matches production exactly, with none of the drift a long-lived staging environment accumulates. This rehearsal can be added as a stage in the pipeline that merges the promotion's proposed change, so production is touched only after the same change has applied cleanly on a copy of it.

## Further reading

- [Develop changes from a Git repository](./develop-changes.mdx) — the single-instance workflow promotion builds on
- [Promote changes between environments](./promote-between-environments.mdx) — the cross-environment how-to
- [Git integration](./overview.mdx) — repository types, `ref` tracking, and the import model
- [Proposed changes](../proposed-changes/overview.mdx) — the review and merge workflow used inside each instance
- [Database backup and restore](../deploy-manage/maintain-upgrade/database-backup/backup-and-restore.mdx) — spinning up a pre-production copy of production
86 changes: 86 additions & 0 deletions docs/docs/git-integration/promote-between-environments.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
title: Promote changes between environments
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Promote changes between environments

Run a separate Infrahub instance for each environment from one Git repository, and move changes from one environment to the next. This guide covers the **promotion** — taking a change that is already developed on one environment and landing it on the next. To develop and land a change on a single instance, see [develop changes from a Git repository](./develop-changes.mdx); for the concepts, see [multiple environments from a single repository](./multi-environment.mdx).

The example uses two environments — **development** (imports the `develop` branch) and **production** (imports the `main` branch) — but the steps repeat for any number.

![The develop-and-promote flow: author and test a change on development, then promote it to production, landing on each instance through a proposed change](../media/develop-promote-flow.excalidraw.svg)

## Prerequisites

- One Infrahub instance per environment, each with the repository connected as a **read-only repository** pinned to that environment's Git branch — `develop` on development, `main` on production (see [connect a repository](./connect-repository.mdx)).
- A change already developed and merged onto the development instance's `main` (see [develop changes from a Git repository](./develop-changes.mdx)).

## Register the repository

Connect the repository on each instance as a read-only repository, setting `ref` to that environment's Git branch.

```graphql
mutation {
CoreReadOnlyRepositoryCreate(data: {
name: { value: "infra" }
location: { value: "https://github.com/<org>/<repo>.git" }
ref: { value: "develop" } # "main" on the production instance
credential: { id: "<credential-id>" }
}) { ok }
}
```

Each instance's `main` keeps its `ref` on its environment branch from here on; promotion advances the imported commit, never the `ref`.

## Promote a change

A promotion is a Git merge into the next environment's branch, then an import on that environment's instance. The instance-side steps are the same ones you use to [develop a change](./develop-changes.mdx) — the change comes from the upstream branch instead of a feature branch.

1. On your Git host, open a pull request from the source branch to the next (for example `develop` → `main`) and merge it. The target branch's head now holds the change.
2. On the target instance, land the merged change through an Infrahub branch, exactly as in [Develop changes, Step 3](./develop-changes.mdx#step-3-land-the-change-on-main): create a branch, run **Import latest commit** (its `ref` is already the target branch), open a proposed change to `main`, review, and merge.

To catch problems before the Git pull request merges, validate first: import the source branch onto a throwaway Infrahub branch (set its `ref` to the source branch) and check it there — see [Develop changes, Step 2](./develop-changes.mdx#step-2-test-it-on-an-infrahub-branch).

Repeat the hop for each environment in the chain (for example `develop` → `staging` → `main`).

:::note Rehearse a production promotion first

Lower environments drift from production over time, so a clean import there does not prove the change applies cleanly to production — and a proposed-change merge that fails partway leaves `main` in an unexpected state, with no clean rollback. Before promoting production, rehearse the promotion against a pre-production instance restored from a current [database backup](../deploy-manage/maintain-upgrade/database-backup/backup-and-restore.mdx). See [reaching production safely](./multi-environment.mdx#reaching-production-safely).

:::

## Automate the promotion (optional)

:::note Experimental — a starting point, not a finished solution

The manual steps above are the supported path. Automating the Infrahub side is possible, but treat what follows as inspiration to adapt, not a turnkey pipeline, and keep production promotions manual until you have validated your own automation end to end.

:::

An automated gate follows the same pattern: on a Git pull request into an environment branch, CI creates an Infrahub branch, imports the change onto it, opens a proposed change, and blocks the pull request — as a **required status check** — until the proposed change's checks pass. Checking those from CI uses the proposed change's `validations`, where each reports a `state` (`queued`, `in_progress`, `completed`) and a `conclusion` (`unknown`, `failure`, `success`):

```graphql
query {
CoreProposedChange(ids: ["<proposed-change-id>"]) {
edges { node {
validations { edges { node {
display_label
state { value }
conclusion { value }
} } }
} }
}
}
```

Poll until every validation's `state` is `completed`, then fail the CI job unless every `conclusion` is `success`. Store each instance's address and an [API token](../deploy-manage/user-management/managing-api-tokens.mdx) as CI secrets, and authenticate with the `X-INFRAHUB-KEY` header.

## Related

- [Develop changes from a Git repository](./develop-changes.mdx) — the single-instance workflow this builds on
- [Multiple environments from a single repository](./multi-environment.mdx) — the concepts behind this pattern
- [Connect a repository](./connect-repository.mdx) — repository registration in depth
- [Proposed changes](../proposed-changes/overview.mdx) — the review workflow used inside each instance
Loading
Loading