diff --git a/Docs/Images/ci-cd-app-registrations-federated-credentials-configs.png b/Docs/Images/ci-cd-app-registrations-federated-credentials-configs.png new file mode 100644 index 00000000..dcb9c66e Binary files /dev/null and b/Docs/Images/ci-cd-app-registrations-federated-credentials-configs.png differ diff --git a/Docs/Images/ci-cd-app-registrations-federated-credentials.png b/Docs/Images/ci-cd-app-registrations-federated-credentials.png new file mode 100644 index 00000000..27e98a1b Binary files /dev/null and b/Docs/Images/ci-cd-app-registrations-federated-credentials.png differ diff --git a/Docs/Images/ci-cd-mg.png b/Docs/Images/ci-cd-mg.png index bc8e14a8..ffd971f6 100644 Binary files a/Docs/Images/ci-cd-mg.png and b/Docs/Images/ci-cd-mg.png differ diff --git a/Docs/ci-cd-ado-pipelines.md b/Docs/ci-cd-ado-pipelines.md index 7e43b7c3..94e88e2c 100644 --- a/Docs/ci-cd-ado-pipelines.md +++ b/Docs/ci-cd-ado-pipelines.md @@ -1,123 +1,30 @@ # Azure DevOps Pipelines -This page covers the specifics for Azure DevOps (ADO) pipelines. It si based on a simplified GitHub Flow as documented in [CI/CD Overview](ci-cd-overview.md) +This page covers the specifics for the Azure DevOps (ADO) pipelines created by using the Starter Kit. Pipelines can be further customized based on requirements. Guidance provided is for the simplified GitHub Flow as documented in the [branching flows](ci-cd-branching-flows.md). Documentation on the Release Flow pipelines will be made available in a future release. -Previously [setup App Registrations](ci-cd-app-registrations.md) are a pre-requisite. +> [!Note] +> [App Registration Setup](ci-cd-app-registrations.md) is a pre-requisite. -This repository contains starter pipelines +## Service connections for the Service Principals -* Azure DevOps (Single Tenant) -* Azure DevOps (Multi Tenant) -* Azure DevOps (Simplified) +Create ADO service connections for each of the previously created [App Registrations](ci-cd-app-registrations.md). You will need to retrieve the credential for the Service Principal that Azure Devops will use for Authentication. This can be either a Client Secret, a X509 certificate, or a Federated Credential. For more information on these options, refer to the [Application Credentials](ci-cd-app-registrations.md/#application-credentials) -## Service connections for Azure DevOps CI/CD +## Pipeline Templates -Create ADO service connections for each of the previously created [App Registrations](ci-cd-app-registrations.md). You will need to retrieve the client id and create a client secret or authenticate with a X509 certificate configured for the SPN. +The provided Azure DevOps pipelines utilize the template functionality to create re-usable components that are shared between pipeline files. More details on Azure DevOps Pipelines Templates can be found in the [Azure DevOps Documentation](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops&pivots=templates-includes) -When creating a Service Connection in Azure DevOps you can set up the service connections on a Subscription or a Management Group scope level. If you are using subscriptions to simulate a hierarchy during EPAC development, configure the service connection(s) scope level as **Subscription**. When creating a Service Connections for management groups (any EPAC environments) Deployment and EPAC Role Assignment the service connection scope level is **Management Group**. +## GitHub Flow Pipeline -Subscription scope level | Management Group scope level -:-----------:|:----------------: -![image](Images/azdoServiceConnectionSubConf.png) | ![image](Images/azdoServiceConnectionMGConf.png) +If utilizing the GitHub flow branching strategy, three pipeline files are created: +- epac-dev-pipeline +- epac-tenant-pipeline +- epac-remediation-pipeline -## Single Tenant Pipeline +### epac-dev-pipeline +This represents the Develop Policy Resources in a Feature Branch flow as described in [Branching Flows](ci-cd-branching-flows.md/#develop-policy-resources-in-a-feature-branch). In general, The EPAC-Dev pipeline is configured to run when any change is pushed to a `feature/*` branch. It runs across three (3) stages: Plan, Deploy & Tenant Plan. -### Single Tenant Stages +### epac-tenant-pipeline +This represents the Simplified `GitHub Flow` for Deployment as described in [Branching Flows](ci-cd-branching-flows.md/#simplified-`github-flow`-for-deployment). In general, The epac-tenant-pipeline is configured to run when any change is pushed to main and runs across three (3) stages: Plan, Deploy Policy & Deploy Roles. The Deploy stages utilize [Azure DevOps environments](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/environments?view=azure-devops) to configure approval gates -| Stage | Purpose | Trigger | Scripts | -|-------|---------|---------|---------| -| devStage | Feature branch DEV environment build, deploy and test | CI, Manual | Build-DeploymentPlans
Deploy-PolicyPlan
Deploy-RolesPlan | -| tenantPlanFeatureStage | Feature branch based plan for prod deployment | CI, Manual | Build-DeploymentPlans | -| tenantPlanMainStage | Main branch based plan for prod deployment | PR Merged, Manual | Build-DeploymentPlans | -| tenantDeployStage | Deploy Policies defined by Main branch based plan | Prod stage approved | Deploy-PolicyPlan | -| tenantRolesStage | Assign roles defined by Main branch based plan | Role stage approved | Deploy-RolesPlan | - -### Single Tenant Service Connections and Roles - -Create Service Principals and associated service connections in Azure DevOps or the equivalent in your CI/CD tool. The SPNs require the following roles to adhere to the least privilege principle. If you have a single tenant, remove the last column and rows with connections ending in "-2". If a pacEnvironment in any of these stages is deploying to a lighthouse managed tenant and additionalRoleAssignemnts are to be used, ABAC assignments will need to be granted to the service principal at all remote scopes granting User Access Administrator for any roles that may need to be granted via additionalRoleAssignments. - -| Connection | Stages | MG: epac-dev-mg | MG: Tenant Root | -| :--- | :--- | :--- | :--- | -| sc-pac-dev | devStage | Owner
MS Graph Permissions |||| -| sc-pac-plan | tenantPlanFeatureStage
tenantPlanMainStage || EPAC Policy Reader
MS Graph Permissions | -| sc-pac-prod | tenantDeployStage || Policy Contributor | -| sc-pac-roles | tenantRolesStage-1 || User Access Administrator
MS Graph Permissions | - -## Multi Tenant Pipeline - -### Multi Tenant Stages - -| Stage | Purpose | Trigger | Scripts | -|-------|---------|---------|---------| -| devStage | Feature branch EPAC DEV environment build, deploy and test | CI, Manual | Build-DeploymentPlans
Deploy-PolicyPlan
Deploy-RolesPlan | -| tenantPlanFeatureStage-1 | Feature branch based plan for prod deployment (tenant 1) | CI, Manual | Build-DeploymentPlans | -| tenantPlanFeatureStage-2 | Feature branch based plan for prod deployment (tenant 2) | CI, Manual | Build-DeploymentPlans | -| completedFeature | Empty stage to complete feature branch | None | None | -| tenantPlanMainStage-1 | Main branch based plan for prod deployment (tenant 1) | PR Merged, Manual | Build-DeploymentPlans | -| tenantDeployStage-1 | Deploy Policies defined by Main branch based plan (tenant 1) | Prod stage approved | Deploy-PolicyPlan | -| tenantRolesStage-1 | Assign roles defined by Main branch based plan (tenant 1) | Role stage approved | Deploy-RolesPlan | -| tenantPlanMainStage-2 | Main branch based plan for prod deployment (tenant 2) | PR Merged, Manual | Build-DeploymentPlans | -| tenantDeployStage-2 | Deploy Policies defined by Main branch based plan (tenant 2) | Prod stage approved | Deploy-PolicyPlan | -| tenantRolesStage-2 | Assign roles defined by Main branch based plan (tenant 2) | Role stage approved | Deploy-RolesPlan | - -### Multi Tenant Service Connections and Roles - -Create Service Principals and associated service connections in Azure DevOps or the equivalent in your CI/CD tool. The SPNs require the following roles to adhere to the least privilege principle. If you have a single tenant, remove the last column and rows with connections ending in "-2". If a pacEnvironment in any of these stages is deploying to a lighthouse managed tenant and additionalRoleAssignemnts are to be used, ABAC assignments will need to be granted to the service principal at all remote scopes granting User Access Administrator for any roles that may need to be granted via additionalRoleAssignments. - -| Connection | Stages | MG: epac-dev-mg | MG: Tenant 1 Root | MG: Tenant 2 Root | -| :--- | :--- | :--- | :--- | :--- | -| sc-pac-dev | devStage | Owner
MS Graph Permissions |||| -| sc-pac-plan-1 | tenantPlanFeatureStage-1
tenantPlanMainStage-1 || EPAC Policy Reader
MS Graph Permissions || -| sc-pac-plan-2 | tenantPlanFeatureStage-2
tenantPlanMainStage-2 ||| EPAC Policy Reader
MS Graph Permissions | -| sc-pac-prod-1 | tenantDeployStage-1 || Policy Contributor || -| sc-pac-prod-2 | tenantDeployStage-2 ||| Policy Contributor | -| sc-pac-roles-1 | tenantRolesStage-1 || User Access Administrator || -| sc-pac-roles-2 | tenantRolesStage-2 ||| User Access Administrator | -| none | completedPlanFeatureStage |||| - - -## Azure DevOps (Simplified Pipeline) - -If you have less complex requirements for a pipeline deployment using Azure DevOps you can utilize the ```simplified-pipeline.yaml``` file and the ```templates``` folder in the ```StarterKit``` folder to quickly get started in Azure Pipelines. - -This template requires the creation of two environments in Azure Pipelines and can easily have approvals added for deployment control. - -## Deployment Environments - -Create distinct ADO environment to configure approval gates. Refer to the following documentation: - - -## Pipeline Execution - -In Azure Devops pipelines the following happens. Your CI/CD tools will display progress differently. - -### `Commit` to a feature branch or a manual pipeline run - -* Stage devStage to deploy Policies, Policy Sets and Policy Assignments to the PAC DEV environment. -* Calculates the plan for PROD environment deployment based on the Feature branch. - * This plan is never executed. Instead the logs and if desired the artifact generated are used by the developer to verify the definition files and to determine if the code is ready for a Pull Request. - * The PR approver(s) will use the same input plus the source code changes to decide the PR approval or rejection. - -![image.png](Images/feature-run.png) - -Detail view: - -![image.png](Images/feature-run-details.png) - -### `Pull Request` is approved and branch merged into main - -* Calculates the plan for PROD environment deployment based on the merged Main branch. -* The pipeline stops for PROD gate(s) approval at this time. - * The logs and if desired the artifacts generated are used by the PROD gate(s) approver(s) to decide on the PROD stage approval(s) or rejection(s). -* ![image.png](Images/prod-approval.png) -* ![image.png](Images/prod-approval-dialog.png) -* After the approval deployments to PROD will begin. -* Optional a second approval before role assignments is required. -* ![image.png](Images/prod-complete.png) -* After the ntire run the overview page looks like this: -* ![image.png](Images/pipeline-runs.png) - -### No changes - -* Deployment steps and stages are skipped. Skipped stages do not need approvals. -* ![image.png](Images/prod-no-changes.png) +### epac-remediation-pipeline +This pipeline runs on a schedule to automatically start remediation tasks for each environment. \ No newline at end of file diff --git a/Docs/ci-cd-app-registrations.md b/Docs/ci-cd-app-registrations.md index c7bede03..003e697a 100644 --- a/Docs/ci-cd-app-registrations.md +++ b/Docs/ci-cd-app-registrations.md @@ -1,138 +1,94 @@ -# App Registrations Setup +# App Registration & Service Principal Setup -CI/CD pipelines/workflows require the creation of App Registrations in your Entra ID (Azure AD) tenants. The App Registrations are used by the EPAC pipeline to deploy the EPAC Management Group and the EPAC Management Group Policy Definitions. +The EPAC CI/CD pipelines/workflows utilize Entra ID (Azure AD) Service Principals as the identity to interact with your Azure environment. This section describes the recommended approach for utilizing service principals with EPAC. For general information on Service Principals, please review the [Microsoft Documentation](https://learn.microsoft.com/en-us/entra/identity-platform/app-objects-and-service-principals?tabs=browser) -The following screenshot shows the Management Group hierarchy that used for the App Registrations. +> [!IMPORTANT] +> Please review the [EPAC Deployment Concepts](start-implementing.md/#epac-concepts-and-environments) before proceeding as the following guidance builds upon the EPAC Environment Construct -![Management Group hierarchy](Images/ci-cd-mg.png) - -## Custom `EPAC Resource Policy Reader Role` - -EPAC uses a set of Entra ID App Registrations (Service Principals). To build the deployment plan and adhere to the least-privilege-principle, a Resource Policy Reader role is required. This role is not built-in. EPAC contains script `New-AzPolicyReaderRole` to create this role or you can use the below JSON in Azure Portal. - -```json -{ - "properties": { - "roleName": "EPAC Resource Policy Reader", - "description": "Provides read access to all Policy resources for the purpose of planning the EPAC deployments.", - "assignableScopes": [ - "/" - ], - "permissions": [ - { - "actions": [ - "Microsoft.Authorization/policyassignments/read", - "Microsoft.Authorization/policydefinitions/read", - "Microsoft.Authorization/policyexemptions/read", - "Microsoft.Authorization/policysetdefinitions/read", - "Microsoft.Authorization/roleAssignments/read", - "Microsoft.PolicyInsights/*", - "Microsoft.Management/register/action", - "Microsoft.Management/managementGroups/read", - "Microsoft.Resources/subscriptions/resourceGroups/read" - ], - "notActions": [], - "dataActions": [], - "notDataActions": [] - } - ] - } -} -``` - -## Create single App Registration and Role assignments for `epac-dev` - -Create the App Registrations for: - -- epac-dev environment with Owner rights to the epac-dev Management Group -- Optional: epac-test environment with Owner rights to the epac-test Management Group (repeat the steps below for epac-test) - -### Create the App Registration for `epac-dev` environment - -![App Registration 1](Images/ci-cd-app-reg-perm-1.png) - -### Grant the App Registration the necessary Microsoft Graph permissions - -![App Registration 2](Images/ci-cd-app-reg-perm-2.png) - -![App Registration 3](Images/ci-cd-app-reg-perm-3.png) - -![App Registration 4](Images/ci-cd-app-reg-perm-4.png) +## Recommended Service Principals -![App Registration 5](Images/ci-cd-app-reg-perm-5.png) +To help maintain a segmentation of duties and a least-privilege approach, it is recommended to create separate service principals for each of the [general deployment steps](ci-cd-overview.md/#general-deployment-flow). -![App Registration 6](Images/ci-cd-app-reg-perm-6.png) +- Build Deployment Plans + - A single service principal should be created for the plan steps across all EPAC environments. This Service Principal should be assigned the `Reader` Azure RBAC role at the Tenant Root. + - **Note:** For Multi-Tenant environments, each tenant will need a Plan Service Principal. +- Policy Deployment (Per EPAC Environment) + - Each EPAC Environment should have a separate service principal with the Azure RBAC Role `Resource Policy Contributor` assigned at the *EPAC environment root*. This Service Principal will be used for the policy deployment phase. +- Role Deployment (Per EPAC Environment) + - Each EPAC Environment should have a separate service principal with the Azure RBAC Role `Role Based Access Control Administrator` assigned at the *EPAC environment root*. This service Principal will be used for the Azure RBAC role assignment phase. -![App Registration 7](Images/ci-cd-app-reg-perm-7.png) +> [!TIP] +> For the EPAC Development Environment, a single service principal can be used for both the Policy Deployment & Role Deployment to simplify management. While it is recommended to separate these to maintain a separation of duties and enable additional security controls, the nature and isolation of the EPAC Development environment does not create the need for separation. Note: If you wish to use a single Service Principal for EPAC Development, both role assignments are still required. -![App Registration 8](Images/ci-cd-app-reg-perm-8.png) +> [!TIP] +> To further improve security posture, conditions can be used for the RBAC assignment for each role deployment service principal to restrict the assignment of privileged roles such as `Owner`, `User Access Admin` and `Role Based Access Control Administrator`. In general, policy assignments should never require the use of an additional role assignment that will create other role assignments. For more information on conditions for role assignments, please refer to the [Azure RBAC Documentation](https://learn.microsoft.com/en-us/azure/role-based-access-control/conditions-format) -![App Registration 9](Images/ci-cd-app-reg-perm-9.png) +### Create Service Principals -![App Registration 10](Images/ci-cd-app-reg-perm-a.png) +For guidance creating Service Principals in Entra ID, please refer to the [Entra ID Documentation](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#register-an-application-with-microsoft-entra-id-and-create-a-service-principal -### Grant the App Registration the necessary Azure `Owner` permissions for the epac Management Group +### Assign Service Principals Permissions in Azure -![App Registration 11](Images/ci-cd-app-reg-perm-b.png) +For guidance assigning Service Principals Azure RBAC roles, please refer to the [Azure RBAC Documentation](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#assign-a-role-to-the-application) -![App Registration 12](Images/ci-cd-app-reg-perm-c.png) +## Example Setup +Given the following screenshot of a Management Group hierarchy for Contoso. -![App Registration 13](Images/ci-cd-app-reg-perm-d1.png) - -![App Registration 14](Images/ci-cd-app-reg-perm-d2.png) - -![App Registration 15](Images/ci-cd-app-reg-perm-d3.png) - -![App Registration 16](Images/ci-cd-app-reg-perm-d4.png) - -![App Registration 17](Images/ci-cd-app-reg-perm-d5.png) - -## Create App Registrations and Role assignments for prod environments (per tenant) - -### App Registration with permissions to read Policy resources and Azure roles - -#### Create the App Registration the same as above with the same Microsoft Graph permissions - -![App Registration](Images/ci-cd-app-reg-root-reader.png) - -#### Create custom Azure role with permissions to read Policy resources - -![Reader Role 18](Images/ci-cd-role-policy-reader-1.png) - -![Reader Role 19](Images/ci-cd-role-policy-reader-2.png) - -![Reader Role 20](Images/ci-cd-role-policy-reader-3.png) +![Management Group hierarchy](Images/ci-cd-mg.png) -![Reader Role 21](Images/ci-cd-role-policy-reader-4.png) +Contoso has decided to utilize two EPAC Environments. One for EPAC Development (EPAC-Dev) and one for the remainder of their environment (Tenant), which has resulted the following global settings file: -![Reader Role 22](Images/ci-cd-role-policy-reader-5.png) +```json +{ + "$schema": "https://raw.githubusercontent.com/Azure/enterprise-azure-policy-as-code/main/Schemas/global-settings-schema.json", + "pacOwnerId": "{{guid}}", + "pacEnvironments": [ + { + "pacSelector": "epac-dev", + "cloud": "AzureCloud", + "tenantId": "{{tenant-id}}", + "deploymentRootScope": "/providers/Microsoft.Management/managementGroups/epac-contoso" + }, + { + "pacSelector": "tenant", + "cloud": "AzureCloud", + "tenantId": "{{tenant-id}}", + "deploymentRootScope": "/providers/Microsoft.Management/managementGroups/contoso" + } + ] +} +``` -#### Grant the App Registration the custom Azure role at the root Management Group +The following Service Principals & Role assignments would be created to support this structure: -![App Registration 23](Images/ci-cd-app-reg-root-reader-perm-1.png) +| Service Principal | Azure Role Assignment | Assignment Scope | +| :--- | :--- | :--- | +| spn-epac-plan | Reader | Tenant Root Group | +| spn-epac-dev | Resource Policy Contributor
Role Based Access Control Administrator | epac-contoso | +| spn-epac-tenant-deploy | Resource Policy Contributor | Contoso | +| spn-epac-tenant-roles | Role Based Access Control Administrator | Contoso | -![App Registration 24](Images/ci-cd-app-reg-root-reader-perm-2.png) +## Application Credentials -### App Registration with permissions to deploy Policy resources +Credentials will need to be created for each Service Principal to be used in the CI/CD process. Traditionally, this is accomplished by creating a Client Secret on the associated Entra ID Application, and providing the CI/CD tool with the Application's ID and Secret. Secrets present an automation challenge as they need to be managed, secured, and rotated as they eventually expire. To solve this, some tools, including Azure DevOps, now support the use of Federated Credentials as described below. -### Create the App Registration ***without*** Microsoft Graph permissions +### Alternative: `Azure Federated Identity Credentials` -![App Registration 25](Images/ci-cd-app-reg-root-contributor.png) +Federated identity credentials are a new type of credential that enables workload identity federation for software workloads. Workload identity federation allows you to access Microsoft Entra protected resources without needing to manage secrets (for supported scenarios). -#### Grant the App Registration the `ResourcePolicy Contributor` role at the root Management Group +Within your Registered App create a `Federated Credential` -![App Registration 26](Images/ci-cd-app-reg-root-contributor-perm-1.png) -![App Registration 27](Images/ci-cd-app-reg-root-contributor-perm-2.png) +![App Registration 30](Images/ci-cd-app-registrations-federated-credentials.png) -### App Registration with permissions to assign Roles at root Management Group -#### Create the App Registration the same as above with the same Microsoft Graph permissions +(Example below is for a GitLab Implementation) +![App Registration 30](Images/ci-cd-app-registrations-federated-credentials-configs.png) -![App Registration 28](Images/ci-cd-app-reg-root-roles.png) +`Federated credential scenario` — List of supported Scenarios for leveraging Federated Credentials. -#### Grant the App Registration the `User Access Administrator` role at the root Management Group +`Audience` — The audience that can appear in the external token. This field is mandatory and should be set to `api://AzureADTokenExchange` for Microsoft Entra ID. It says what Microsoft identity platform should accept in the aud claim in the incoming token. This value represents Microsoft Entra ID in your external identity provider and has no fixed value across identity providers - you might need to create a new application registration in your IdP to serve as the audience of this token. -![App Registration 29](Images/ci-cd-app-reg-root-role-assignments-perm-1.png) +`Issuer` — The URL of the external identity provider. Must match the issuer claim of the external token being exchanged. -![App Registration 30](Images/ci-cd-app-reg-root-role-assignments-perm-2.png) +`Subject identifier` — The identifier of the external software workload within the external identity provider. Like the audience value, it has no fixed format, as each IdP uses their own - sometimes a GUID, sometimes a colon delimited identifier, sometimes arbitrary strings. The value here must match the sub claim within the token presented to Microsoft Entra ID. diff --git a/Docs/ci-cd-branching-flows.md b/Docs/ci-cd-branching-flows.md new file mode 100644 index 00000000..f61e7837 --- /dev/null +++ b/Docs/ci-cd-branching-flows.md @@ -0,0 +1,82 @@ +# Branching Overview +The following section covers two branching strategies for EPAC that are supported by the starter kits - `GitHub Flow` and `Release Flow`. Both strategies follow their respective general guidance as documented for [GitHub Flow](https://docs.github.com/en/get-started/using-github/github-flow), and [Release Flow](https://learn.microsoft.com/en-us/devops/develop/how-microsoft-develops-devops), however, the method you choose may be dependent on the EPAC environment structure you choose to use. In general, `GitHub Flow` is recommended for simple EPAC deployments that contain an EPAC Development Environment and a Main/Tenant Environment. If you wish to utilize multiple additional environments, and/or deploy to those environments in a staged/ring-based fashion (Deploy to Environment A then Deploy to Environment B), the `Release Flow` model may provide greater flexibility. + +> [!IMPORTANT] +> Please review the [App Registrations Setup](ci-cd-app-registrations.md) for information on the permissions and Service Principals (SPNs) recommended for deployments. + +## Develop Policy Resources in a Feature Branch + +Developing Policy resources is the same for `GitHub Flow` and `Release Flow`. The following steps are recommended: + +1. Developers create feature branches from `main` branch with a name `feature/*user-id*/*feature-name*`. +2. Developers create or update Policy definitions, Policy Set definitions, Policy Assignment, and Policy Exemptions files in the `Definitions` folder. Developers push changes to the feature branch. +3. A "Development" CI/CD pipeline/action is triggered from the push to the feature branch. This pipeline will perform the following steps: + - Build Deployment Plans for the EPAC Development Environment (e.g. `EPAC-Dev`), showing any changes made. + - Deploy Policy Plans to the EPAC Development Environment (e.g. `EPAC-Dev`), providing an end-to-end test in development and allowing developers to validate the changes. + - Deploy Policy Roles to the EPAC Development Environment (e.g. `EPAC-Dev`) to create the role assignments for the Managed Identities required for any `DeployIfNotExists` and `Modify` Policies. + - Build Deployment Plans for **all** remaining environments (e.g. `Prod`). This accounts for a "Shift-Left" deployment mindset and helps validate changes against the other environments to surface any potential issues as the change is rolled out past the EPAC Development environment. The Plans are stored in the `Output` folder. +4. When the feature is ready, the developer creates a Pull Request (PR) to merge the feature branch into the `main` branch. +5. Set the pull request to be auto completed. Ensure that you select `Delete feature/*user-id*/*feature-name* after merging`. + +![image.png](Images/ci-cd-set-auto-complete.png) + +6. After the merge completes, cleanup your local clone by: + - Switching the branch to main + - Pull the latest changes from main + - Delete the feature branch + - Run `git remote prune origin` to remove the remote tracking branch. + +Steps 1 to 3 are repeated during the development process. In both models, this helps ensure short lived feature branches, with a constant push to `main`, while providing a baseline validation of the change against all environments. + +## Simplified `GitHub Flow` for Deployment + +The diagram below shows the use of GitHub Flow in Policy as Code. The diagram uses GitHub workflow terminology; however, the concepts apply equally to other CI/CD technologies. + +Once Development is completed, as noted above, the merge of the PR into the `main` branch triggers the CI/CD pipeline/action to deploy the changes to the main environment. The following steps are recommended: + +- Build Deployment Plans for the Prod Environment (e.g. `Prod`), showing any changes made. +- Approval gate for Policy resources deployment. +- Deploy Policy Plans to the Prod Environment (e.g. `Prod`) +- Approval gate for Role assignments deployment. +- Deploy Policy Roles to the Prod Environment (e.g. `Prod`) to create the role assignments for the Managed Identities required for any `DeployIfNotExists` and `Modify` Policies. + +![image.png](Images/epac-github-flow.png) + +### `GitHub Flow` Variations + +EPAC can handle any flow you like. For `GitHub Flow`, the following variations are possible. + +- Adding a deployment plan from the feature branch to the production environment in step 3 above during the development process (see steps 1 through 3 in the diagram above) by adding a step using Build-DeploymentPlans.ps1. This is useful to test the deployment plan in the production environment before creating a PR. We recommend using a separate SPN for this step (job). +- PR creation trigger for a CI/CD pipeline/action deploying the changes to an `epac-test` environment with the same steps as the deployment to `epac-dev` environment in steps 3 above. + +## Advanced Deployment with Release Flow + +In some cases, especially when using multiple EPAC Environments, it may be desirable to deploy the Policy changes in a ring-based model, deploying to one environment before deploying to all remaining environments. This can help limit risk and impact should an update cause unintended issues. While there are many ways to implement a ring-based deployment, the Release Flow branching strategy as described below, can be an effective model with EPAC. For the following, lets assume Contoso has three (3) EPAC environments, `EPAC-Dev`, `nonprod`, and `prod`. They wish to deploy all policy changes as soon as they are ready into the `nonprod` environment, while waiting to deploy to `prod` and batching deployments into larger "releases" once they are sure there are no impacts. + +- Initial policy development follows the same process as outlined in [Develop Policy Resources in a Feature Branch](#develop-policy-resources-in-a-feature-branch) above. +- The merge of the PR into the `main` branch triggers a NonProd specific CI/CD pipeline/action to deploy the changes to the EPAC `nonprod` environment. This pipeline would follow the general deployment steps, but targeted only to the `nonprod` EPAC environment. + - Build Deployment Plans for the nonprod Environment (e.g. `nonprod`), showing any changes made. + - Approval gate for Policy resources deployment. + - Deploy Policy Plans to the nonprod Environment (e.g. `nonprod`) + - Approval gate for Role assignments deployment. + - Deploy Policy Roles to the nonprod Environment (e.g. `nonprod`) to create the role assignments for the Managed Identities required for any `DeployIfNotExists` and `Modify` Policies. +- Wait to verify that the Policies in the EPAC `nonprod` environment are working as expected, and/or wait for additional policies/development cycles to finish. + - Create a `releases-prod` branch to trigger a Prod specific CI/CD pipeline/action to deploy the changes to the EPAC `prod` environment. The pipeline would follow the same general deployment steps as the nonprod pipeline, but targeted to the `prod` EPAC environment. + - Keep n-1 `releases-prod` branches to allow for quick rollback in case of issues. +- Sometimes, Exemptions need to be granted while keeping a regular lifecycle for Definitions and Assignments. To accomplish this, follow the general steps below: + - The exemption should be committed to `main` through the standard development process. This is critical as it provides the standard development plans to run, ensures a code review, and guarantees the change is in `main` so it does not re-occur in future deployments + - Once merged to `main` use the PR page to cherry-pick the changes into the active `releases-prod` branch, which creates a new pull request. This ensures traceability and allows for approval before the change is deployed. + - This process is documented in further detail on the [Microsoft Release Flow Documentation](https://learn.microsoft.com/en-us/devops/develop/how-microsoft-develops-devops#release-hotfixes) + +![image.png](Images/epac-release-flow.png) + +### `Release Flow` Variations + +EPAC can handle any flow you like. For `Release Flow`, the following variations are possible. + +- Adding a deployment plan from the feature branch to the production environment in step 3 above during the development process (see steps 1 through 3 in the diagram above) by adding a step using Build-DeploymentPlans.ps1. This is useful to test the deployment plan in the production environment before creating a PR. We recommend using a separate SPN for this step (job). +- PR creation trigger for a CI/CD pipeline/action deploying the changes to an `epac-test` environment with the same steps as the deployment to `epac-dev` environment in steps 3 above. + +## Multiple Tenants + +For multiple tenants simply apply each of the flows (except for the `feature` branch) above to each tenant's EPAC environments. This works for both simplified GitHub flow and Microsoft Release flow. diff --git a/Docs/ci-cd-overview.md b/Docs/ci-cd-overview.md index 42aceadb..d493bd63 100644 --- a/Docs/ci-cd-overview.md +++ b/Docs/ci-cd-overview.md @@ -9,175 +9,113 @@ This repository contains starter pipelines and instructions for can be found her - [Azure DevOps Pipelines](ci-cd-ado-pipelines.md) - [GitHub Actions](ci-cd-github-actions.md) -## Create Azure DevOps Pipelines or GitHub Workflows +## General EPAC Deployment Steps -The scripts `New-PipelinesFromStarterKit` create [Azure DevOps Pipelines or GitHub Workflows from the starter kit](operational-scripts-hydration-kit.md#create-azure-devops-pipeline-or-github-workflow). You select the type of pipeline to create, the branching flow to implement, and the type of script to use. +EPAC has three major steps in the Deployment process for each environment. +- Build Deployment Plans +- Policy Deployment +- Role Deployment -### Azure DevOps Pipelines - -The following commands create Azure DevOps Pipelines from the starter kit; use one of the commands: - -```ps1 -New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\pipelines -PipelineType AzureDevOps -BranchingFlow GitHub -ScriptType script -New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\pipelines -PipelineType AzureDevOps -BranchingFlow Release -ScriptType script -New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\pipelines -PipelineType AzureDevOps -BranchingFlow GitHub -ScriptType module -New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\pipelines -PipelineType AzureDevOps -BranchingFlow Release -ScriptType module -``` - -### GitHub Workflows - -The following commands create GitHub Workflows from the starter kit; use one of the commands: - -```ps1 -New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\.github/workflows -PipelineType GitHubActions -BranchingFlow GitHub -ScriptType script -New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\.github/workflows -PipelineType GitHubActions -BranchingFlow Release -ScriptType script -New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\.github/workflows -PipelineType GitHubActions -BranchingFlow GitHub -ScriptType module -New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\.github/workflows -PipelineType GitHubActions -BranchingFlow Release -ScriptType module -``` - - - -## Developing Policy Resources in a Feature Branch - -Developing Policy resources is the same for `GitHub Flow` and `Release Flow`. The following steps are recommended: - -1. Developers create feature branches from `main` branch with a name `feature/*user-id*/*feature-name*`. -2. Developers create or update Policy definitions, Policy Set definitions, Policy Assignment, amd Policy Exemptions files in the `Definitions` folder. Developers push changes to the feature branch. -3. The CI/CD pipeline/action is triggered from the push to the feature branch. We recommend to use a single App Registration (SPN) to execute pipeline/action during development. The SPN must have `Owner` rights to the `epac-dev` Management Group and the Microsoft Graph permissions described below. The steps are: - - Build-DeploymentPlans.ps1 to calculate the deployment plan - - Deploy-PolicyPlan.ps1 to deploy the plan's Policy Resources - - Deploy-RolesPlan.ps1 to create the role assignments for the Managed Identities required for `DeployIfNotExists` and `Modify` Policies. - - The starter pipelines calculate the Plans for tenant(s). The Plans are stored in the `Output` folder. -4. When the feature is ready, the developer creates a Pull Request (PR) to merge the feature branch into the `main` branch. -5. Set the pull request to be auto-completed. Ensure that you select `Delete feature/*user-id*/*feature-name* after merging`. - -![image.png](Images/ci-cd-set-auto-complete.png) +Each step can be called by using the `EnterprisePolicyAsCode` PowerShell module (recommended), or calling the script directly. For more details on EPAC installation options, please refer to the [Start Implementation](start-implementing.md/#install-powershell-and-epac) section. -6. After the merge completes, cleanup your local clone by: - - Switching the branch to main - - Pull the latest changes from main - - Delete the feature branch - - Run `git remote prune origin` to remove the remote tracking branch. +> [!TIP] +> EPAC is **declarative** and **idempotent**: this means, that regardless how many times it is run, EPAC will always push all changes that were implemented in the JSON files to the Azure environment, i.e. if a JSON file is newly created/updated/deleted, EPAC will create/update/delete the Policy and/or Policy Set and/or Policy Assignments definition in Azure. If there are no changes, EPAC can be run any number of times, as it won't make any changes to Azure. -Steps 1 to 3 are repeated during the development process. +### Build Deployment Plans +Analyzes changes in Policy definition, Policy Set definition, Policy Assignment & Policy Exemption files for a given environment. It calculates and displays any deltas, while creating the deployment plan(s) to apply any changes. A "Policy Plan" will be created for use by the Policy Deployment step if any changes are found to the policy objects, assignments, or exemptions while a "Role Plan" will be created for use by the Role deployment step should there be any changes to role assignments for the deployed policies. If no changes are found, no plans are created. -The resulting pipeline CI/CD trigger depends on the type of flow used. The following sections describe the simplified `GitHub Flow` and the more advanced `Release Flow`. +For saving the output related to ```Build-DeploymentPlans``` there is global variable called ```$epacInfoStream``` which captures all output from the commands. If required, this can be used as a PR message or to present a summary of the plan. -## General Hardening Guidelines - -- **Least Privilege**: Use the least privilege principle when assigning roles to the SPNs used in the CI/CD pipeline. The roles should be assigned at the root or pseudo-root management group level. -- Require a Pull Request for changes to the `main` branch. This ensures that changes are reviewed before deployment. -- Require additional reviewers for yml pipeline and script changes. -- Require branches to be in a folder `feature` to prevent accidental deployment of branches. -- Require an approval step between the Plan stage/job and the Deploy stage/job. This ensures that the changes are reviewed before deployment. -- [Optional] Require an approval step between the Deploy stage/job and the Role Assignments stage/job. This ensures that the role assignments are reviewed before deployment. -- For `Release Flow` only: allow only privileged users to create `releases-prod` and `releases-exemptions-only` branches and require those branches to be created from the main branch only. - -## Simplified `GitHub Flow` for Policy as Code - -The diagram below shows the use of GitHub Flow in Policy as Code. The diagram uses GitHub workflow terminology; however, the concepts apply equally to other CI/CD technologies. - -The merge of the PR into the `main` branch triggers the CI/CD pipeline/action to deploy the changes to the `tenant` environment. - -Since these deployments are most often deployed at the pseudo root of the tenant, we recommend creating a separate App Registration (SPN) for each of the 3 steps with roles assigned in line with the least privilege principle. The steps are: - -- Build-DeploymentPlans.ps1 to calculate the deployment plan. SPN must have `EPAC Resource Policy Reader` custom role on the root or pseudo-root management group and the Microsoft Graph permissions described below. -- Approval gate for Policy resources deployment. -- Deploy-PolicyPlan.ps1 to deploy the plan's Policy Resources. SPN must have `Resource Policy Contributor` built-in role on the root or pseudo-root management group. Microsoft Graph permissions are not required. -- Approval gate for Role assignments deployment. -- Deploy-RolesPlan.ps1 to create the role assignments for the Managed Identities required for `DeployIfNotExists` and `Modify` Policies. SPN must have `User Access Administrator` built-in role on the root or pseudo-root management group and the Microsoft Graph permissions described below. - -![image.png](Images/epac-github-flow.png) +**Deployment Mechanism** -### `GitHub Flow` Variations - -EPAC can handle any flow you like. For `GitHub Flow`, the following variations are possible. - -- Adding a deployment plan from the feature branch to the production environment in step 3 above during the development process (see steps 1 through 3 in the diagram above) by adding a step using Build-DeploymentPlans.ps1. This is useful to test the deployment plan in the production environment before creating a PR. We recommend using a separate SPN for this step (job). -- PR creation trigger for a CI/CD pipeline/action deploying the changes to an `epac-test` environment with the same steps as the deployment to `epac-dev` environment in steps 3 above. - -## Advanced CI/CD with Release Flow - -Testing the Policy changes against the IaC `nonprod` environment is often desirable to prevent surprises when deploying against the IaC `prod` environment. IaC environments can either be separate tenants or management groups within a single tenant (recommended). - -[Release Flow](https://devblogs.microsoft.com/devops/release-flow-how-we-do-branching-on-the-vsts-team/) is a more advanced CI/CD process that allows for testing the Policy changes against the IaC `nonprod` environment before deploying to the IaC `prod` environment as shown in the diagram below. - -- The merge of the PR into the `main` branch triggers the CI/CD pipeline/action to deploy the changes to the IaC `nonprod` environment. -- Wait a few days to verify that the Policies in the IaC `nonprod` environment are working as expected. - - Creating a `releases-prod` branch triggers a pipeline deploying Policy resources to the IaC `prod` environment. - - Keep n-1 `releases-prod` branches to allow for quick rollback in case of issues. -- Sometimes, Exemptions need to be granted while keeping a regular lifecycle for Definitions and Assignments. The script `Build-DeploymentPlans` has a parameter `BuildExemptionsOnly` to deploy only Exemptions. - - Creating a `releases-exemptions-only` branch triggers a pipeline deploying Exemptions only to the IaC `prod` environment. - - Keep n-1 `releases-exemptions-only` branches to allow for quick rollback in case of issues. - -If necessary, you can also branch of the `releases-prod` branch to create a `hotfix` branch to fix issues in the `prod` environment. Similar to the development process, the `hotfix` branch is merged into the `releases-prod` branch with a Pull request. - -![image.png](Images/epac-release-flow.png) - -### `Release Flow` Variations - -EPAC can handle any flow you like. For `Release Flow`, the following variations are possible. - -- Adding a deployment plan from the feature branch to the production environment in step 3 above during the development process (see steps 1 through 3 in the diagram above) by adding a step using Build-DeploymentPlans.ps1. This is useful to test the deployment plan in the production environment before creating a PR. We recommend using a separate SPN for this step (job). -- PR creation trigger for a CI/CD pipeline/action deploying the changes to an `epac-test` environment with the same steps as the deployment to `epac-dev` environment in steps 3 above. +|Deployment Mode | Command/Script | +|----------|-------------| +| Module (Recommended) | Build-DeploymentPlans | +| Script | Build-DeploymentPlans.ps1 | -## Multiple Tenants +**Parameters** -For multiple tenants simply apply each of the flows (except for the `feature` branch) above to each tenant's IaC environments. This works for both simplified GitHub flow and Microsoft Release flow. +|Parameter | Explanation | +|----------|-------------| +| `PacEnvironmentSelector` | Selects the EPAC environment for this plan. If omitted, interactively prompts for the value. | +| `DefinitionsRootFolder` | Definitions folder path. Defaults to environment variable `$env:PAC_DEFINITIONS_FOLDER` or `./Definitions`. It must contain the file `global-settings.jsonc`. | +| `Interactive` | Defaults to `$false`. | +| `OutputFolder` | Output folder path for plan files. Defaults to environment variable `$env:PAC_OUTPUT_FOLDER` or `./Output`. | +| `DevOpsType` | If set, outputs variables consumable by conditions in a DevOps pipeline. Default: not set. | +| `BuildExemptionsOnly` | If set, only builds the Exemptions plan. This useful to fast-track Exemption when utilizing [Release Flow](#advanced-cicd-with-release-flow) Default: not set. | -## Deployment Scripts +### Policy Deployment +Deploys Policies, Policy Sets, Policy Assignments, and Policy Exemptions at their desired scope based on the plan. -While the scripts are intended to be used in CI/CD, they can be run manually to create a semi-automated EPAC solution. This is useful: +**Deployment Mechanism** -- CI/CD environment is not yet available. -- Debugging the scripts from Visual Studio Code. +|Deployment Mode | Command/Script | +|----------|-------------| +| Module (Recommended) | Deploy-PolicyPlan | +| Script | Deploy-PolicyPlan.ps1 | -Deployment scripts require permissions to the Azure environment and Microsoft Graph API. In a CI/CD scenario, App Registration (SPNs) are used to execute the scripts. These identities must be granted the necessay permissions as documented in [App Registrations Setup](ci-cd-app-registrations.md). In a semi-automated scenario, the user executing the scripts must have the necessary permissions. The scripts will prompt for the necessary permissions. +**Parameters** -The image below shows the scripts and the roles required for their execution. +|Parameter | Explanation | +|----------|-------------| +| `PacEnvironmentSelector` | Selects the EPAC environment for this plan. If omitted, interactively prompts for the value. | +| `DefinitionsRootFolder` | Definitions folder path. Defaults to environment variable `$env:PAC_DEFINITIONS_FOLDER` or `./Definitions`. It must contain the file `global-settings.jsonc`. | +| `Interactive` | Defaults to `$false`. | +| `InputFolder` | Input folder path for plan files. Defaults to environment variable `$env:PAC_INPUT_FOLDER`, `$env:PAC_OUTPUT_FOLDER` or `./Output`. | -- `Build-DeploymentPlans.ps1` requires `EPAC Resource Policy Reader` custom role on the root or pseudo-root management group and the Microsoft Graph permissions described below. -- `Deploy-PolicyPlan.ps1` requires `Resource Policy Contributor` built-in role on the root or pseudo-root management group. Microsoft Graph permissions are not required. -- `Deploy-RolesPlan.ps1` requires `User Access Administrator` built-in role on the root or pseudo-root management group and the Microsoft Graph permissions described below. +### Role Deployment +Creates the role assignments for the Managed Identities required for `DeployIfNotExists` and `Modify` Policies. -Furthermore, it shows the consumption of the `Definitions` files by script Build-DeploymentPlans.ps1 and output of two plan files (Policy and Roles). The plan files are subsequently used by the deployment scripts `Deploy-PolicyPlan.ps1` and `Deploy-RolesPlan.ps1`. +**Deployment Mechanism** -![image.pmg](Images/epac-deployment-scripts.png) +|Deployment Mode | Command/Script | +|----------|-------------| +| Module (Recommended) | Deploy-RolesPlan | +| Script | Deploy-RolesPlan.ps1 | -### Common Script Parameters +**Parameters** |Parameter | Explanation | |----------|-------------| | `PacEnvironmentSelector` | Selects the EPAC environment for this plan. If omitted, interactively prompts for the value. | -| `DefinitionsRootFolder` | Definitions folder path. Defaults to environment variable `$env:PAC_DEFINITIONS_FOLDER` or `./Definitions`. It must contain file `global-settings.jsonc`. | +| `DefinitionsRootFolder` | Definitions folder path. Defaults to environment variable `$env:PAC_DEFINITIONS_FOLDER` or `./Definitions`. It must contain the file `global-settings.jsonc`. | | `Interactive` | Defaults to `$false`. | +| `InputFolder` | Input folder path for plan files. Defaults to environment variable `$env:PAC_INPUT_FOLDER`, `$env:PAC_OUTPUT_FOLDER` or `./Output`. | -### Build-DeploymentPlans.ps1 - -Analyzes changes in Policy definition, Policy Set definition, and Policy Assignment files. It calculates a plan to apply deltas. The deployment scripts are **declarative** and **idempotent**: this means, that regardless how many times they are run, they always push all changes that were implemented in the JSON files to the Azure environment, i.e. if a JSON file is newly created/updated/deleted, the pipeline will create/update/delete the Policy and/or Policy Set and/or Policy Assignments definition in Azure. If there are no changes, the pipeline can be run any number of times, as it won't make any changes to Azure. +## Create Azure DevOps Pipelines or GitHub Workflows from Starter Pipelines. -For saving the output related to ```Build-DeploymentPlans``` there is global variable called ```$epacInfoStream``` which capture all output from the commands. If required this can be used as a PR message or to present a summary of the plan. - -|Parameter | Explanation | -|----------|-------------| -| `OutputFolder` | Output folder path for plan files. Defaults to environment variable `$env:PAC_OUTPUT_FOLDER` or `./Output`. | -| `DevOpsType` | If set, outputs variables consumable by conditions in a DevOps pipeline. Default: not set. | -| `BuildExemptionsOnly` | If set, only builds the Exemptions plan. This useful to fast-track Exemption when utilizing [Release Flow](#advanced-cicd-with-release-flow) Default: not set. | +Starter Pipelines have been created to orchestrate the EPAC deployment steps listed above. The scripts `New-PipelinesFromStarterKit` create [Azure DevOps Pipelines or GitHub Workflows from the starter kit](operational-scripts-hydration-kit.md#create-azure-devops-pipeline-or-github-workflow). You select the type of pipeline to create, the branching flow to implement, and the ScriptType to use. +- The starter kits support two branching/release strategies (`GitHub` and `Release`). More details on these branching flows refer to the [Branching Flow Guidance](ci-cd-branching-flows.md). +- The recommended `ScriptType` is `module`, which utilizes the `EnterprisePolicyAsCode` Powershell module. For more details on EPAC installation options, please refer to the [Start Implementation](start-implementing.md/#install-powershell-and-epac) section. +### Azure DevOps Pipelines -### Deploy-PolicyPlan.ps1 +The following commands create Azure DevOps Pipelines from the starter kit; use one of the commands: -Deploys Policies, Policy Sets, Policy Assignments, and Policy Exemptions at their desired scope based on the plan. +```ps1 +New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\pipelines -PipelineType AzureDevOps -BranchingFlow GitHub -ScriptType script +New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\pipelines -PipelineType AzureDevOps -BranchingFlow Release -ScriptType script +New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\pipelines -PipelineType AzureDevOps -BranchingFlow GitHub -ScriptType module +New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\pipelines -PipelineType AzureDevOps -BranchingFlow Release -ScriptType module +``` -|Parameter | Explanation | -|----------|-------------| -| `InputFolder` | Input folder path for plan files. Defaults to environment variable `$env:PAC_INPUT_FOLDER`, `$env:PAC_OUTPUT_FOLDER` or `./Output`. | +### GitHub Workflows -### Deploy-RolesPlan.ps1 +The following commands create GitHub Workflows from the starter kit; use one of the commands: -Creates the role assignments for the Managed Identities required for `DeployIfNotExists` and `Modify` Policies. +```ps1 +New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\.github/workflows -PipelineType GitHubActions -BranchingFlow GitHub -ScriptType script +New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\.github/workflows -PipelineType GitHubActions -BranchingFlow Release -ScriptType script +New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\.github/workflows -PipelineType GitHubActions -BranchingFlow GitHub -ScriptType module +New-PipelinesFromStarterKit -StarterKitFolder .\StarterKit -PipelinesFolder .\.github/workflows -PipelineType GitHubActions -BranchingFlow Release -ScriptType module +``` -|Parameter | Explanation | -|----------|-------------| -| `InputFolder` | Input folder path for plan files. Defaults to environment variable `$env:PAC_INPUT_FOLDER`, `$env:PAC_OUTPUT_FOLDER` or `./Output`. | +## General Hardening Guidelines +- **Least Privilege**: Use the least privilege principle when assigning roles to the SPNs used in the CI/CD pipeline. The roles should be assigned at the root or pseudo-root management group level. For more details on the SPNs to use and required permissions refer to [App Registrations Setup](ci-cd-app-registrations.md) +- Require a Pull Request for changes to the `main` branch. This ensures that changes are reviewed before deployment. +- Require additional reviewers for yml pipeline and script changes. +- Require branches to be in a folder `feature` to prevent accidental deployment of branches. +- Require an approval step between the Plan stage/job and the Deploy stage/job. This ensures that the changes are reviewed before deployment. +- [Optional] Require an approval step between the Deploy stage/job and the Role Assignments stage/job. This ensures that the role assignments are reviewed before deployment. +- For `Release Flow` only: allow only privileged users to create `releases-prod` and `releases-exemptions-only` branches and require those branches to be created from the main branch only. diff --git a/Docs/index.md b/Docs/index.md index fe1418e0..0016d4ff 100644 --- a/Docs/index.md +++ b/Docs/index.md @@ -48,7 +48,7 @@ EPAC is designed for medium and large organizations with a larger number of Poli ## Deployment Scripts -Three deployment scripts plan a deployment, deploy Policy resources, and Role Assignments respectively as shown in the following diagram. The solution consumes definition files (JSON and/or CSV files). The planning script (`Build-DeploymentPlan`) creates plan files (`policy-plan.json` and `roles-plan.json`) to be consumed by the two deployment scripts (`Deploy-PolicyPlan` and `Deploy-RolesPlan`). The scripts require `Reader`, `Contributor` and `User Access Administrator` privileges respectively as indicated in blue text in the diagram. The diagram also shows the usual approval gates after each step/script for prod deployments. +Three deployment scripts plan a deployment, deploy Policy resources, and Role Assignments respectively as shown in the following diagram. The solution consumes definition files (JSON and/or CSV files). The planning script (`Build-DeploymentPlan`) creates plan files (`policy-plan.json` and `roles-plan.json`) to be consumed by the two deployment scripts (`Deploy-PolicyPlan` and `Deploy-RolesPlan`). The scripts require `Reader`, `Resource Policy Contributor` and `Role Based Access Administrator` privileges respectively as indicated in blue text in the diagram. The diagram also shows the usual approval gates after each step/script for prod deployments. ![image.png](Images/epac-deployment-scripts.png) diff --git a/Docs/operational-scripts-documenting-policy.md b/Docs/operational-scripts-documenting-policy.md index 26b9c1d7..1bb0bff5 100644 --- a/Docs/operational-scripts-documenting-policy.md +++ b/Docs/operational-scripts-documenting-policy.md @@ -132,6 +132,28 @@ Each file must contain one or both documentation topics. This example file in th } ``` +## Modifying the Markdown Output + +Markdown processors vary slightly. This shipt has settings to tune the output to match the Markdown processor you are using. + +Azure DevOps Wikis (and maybe others) recognize `[[_TOC_]]` to insert a table of contents. Setting to `addMarkdownAdoWikiToc` to true enables generating the table of content. + +```jsonc +"addMarkdownAdoWikiToc": true, // default is false, set to true to add markdown ADO Wiki TOC +``` + +SharePoint (and maybe others) do not recognize embedded HTML, such as line braeks (`
`) within a Markdown table. Setting `noMarkdownInTableLineBreaks` to true emits commas instead of the HTML tag. + +```jsonc +"noMarkdownInTableLineBreaks": true, // default is false, set to true to remove markdown in table line breaks +``` + +Policy definition group names are not included in Markdown to reduce clutter. You can include a column by setting `includeComplianceGroupNamesInMarkdown` to true, + +```jsonc +"includeComplianceGroupNamesInMarkdown": true, // default is false, set to true to include compliance group names +``` + ## Assignment Documentation ### Element `environmentCategories` diff --git a/Schemas/policy-documentation-schema.json b/Schemas/policy-documentation-schema.json index f790d3eb..6cb610c5 100644 --- a/Schemas/policy-documentation-schema.json +++ b/Schemas/policy-documentation-schema.json @@ -79,6 +79,15 @@ }, "title": { "type": "string" + }, + "addMarkdownAdoWikiToc": { + "type": "boolean" + }, + "noMarkdownInTableLineBreaks": { + "type": "boolean" + }, + "includeComplianceGroupNamesInMarkdown": { + "type": "boolean" } }, "required": [ diff --git a/Scripts/CloudAdoptionFramework/policyAssignments/ALZ-Platform-Default.jsonc b/Scripts/CloudAdoptionFramework/policyAssignments/ALZ-Platform-Default.jsonc index 48ce6ca3..a22777ad 100644 --- a/Scripts/CloudAdoptionFramework/policyAssignments/ALZ-Platform-Default.jsonc +++ b/Scripts/CloudAdoptionFramework/policyAssignments/ALZ-Platform-Default.jsonc @@ -11,7 +11,7 @@ "parameters": { "userAssignedManagedIdentityName": "", // Replace with the name of the user assigned managed identity "userAssignedIdentityName": "", // Replace with the name of the user assigned managed identity - "bringYourOwnUserAssignedManagedIdentity": true, + "bringYourOwnUserAssignedManagedIdentity": "true", "enableProcessesAndDependencies": true, "userAssignedManagedIdentityResourceGroup": "", //Replace with the name of the resource group where the user assigned managed identity is deployed "identityResourceGroup": "", // Replace with the name of the resource group where the user assigned managed identity is deployed @@ -314,4 +314,4 @@ ] } ] -} +} \ No newline at end of file diff --git a/Scripts/Helpers/Add-HelperScripts.ps1 b/Scripts/Helpers/Add-HelperScripts.ps1 index ebe9e756..bbac2f80 100644 --- a/Scripts/Helpers/Add-HelperScripts.ps1 +++ b/Scripts/Helpers/Add-HelperScripts.ps1 @@ -77,10 +77,10 @@ . "$PSScriptRoot/New-ExportNode.ps1" . "$PSScriptRoot/Out-PolicyAssignmentFile.ps1" -. "$PSScriptRoot/Out-PolicyAssignmentDocumentationToFile.ps1" +. "$PSScriptRoot/Out-DocumentationForPolicyAssignments.ps1" . "$PSScriptRoot/Out-PolicyDefinition.ps1" . "$PSScriptRoot/Out-PolicyExemptions.ps1" -. "$PSScriptRoot/Out-PolicySetsDocumentationToFile.ps1" +. "$PSScriptRoot/Out-DocumentationForPolicySets.ps1" . "$PSScriptRoot/Remove-NullFields.ps1" . "$PSScriptRoot/Remove-GlobalNotScopes.ps1" diff --git a/Scripts/Helpers/Convert-EffectToMarkdownString.ps1 b/Scripts/Helpers/Convert-EffectToMarkdownString.ps1 index 7478c9fe..cbfc9047 100644 --- a/Scripts/Helpers/Convert-EffectToMarkdownString.ps1 +++ b/Scripts/Helpers/Convert-EffectToMarkdownString.ps1 @@ -1,20 +1,16 @@ function Convert-EffectToMarkdownString { param ( [string] $Effect, - [array] $AllowedValues + [array] $AllowedValues, + [string] $InTableBreak = "
" ) [string] $text = "" if ($null -ne $Effect) { - if ($AllowedValues.Count -eq 1) { - $text = "***$Effect***" - } - else { - $text = "**$Effect**" - } + $text = "**$Effect**" foreach ($allowed in $AllowedValues) { if ($allowed -cne $Effect) { - $text += "
*$allowed*" + $text += "$($InTableBreak)$($allowed)" } } } diff --git a/Scripts/Helpers/Convert-ParametersToString.ps1 b/Scripts/Helpers/Convert-ParametersToString.ps1 index be2ad977..bb98549b 100644 --- a/Scripts/Helpers/Convert-ParametersToString.ps1 +++ b/Scripts/Helpers/Convert-ParametersToString.ps1 @@ -24,34 +24,6 @@ function Convert-ParametersToString { $value = $defaultValue } switch ($OutputType) { - markdown { - if ($value -is [string]) { - $text += "
      *$parameterName = ``$value``*" - } - else { - $json = ConvertTo-Json $value -Depth 100 -Compress - $jsonTruncated = $json - if ($json.length -gt 40) { - $jsonTruncated = $json.substring(0, 40) + "..." - } - $text += "
      *$parameterName = ``$jsonTruncated``*" - } - } - markdownAssignment { - if (-not $isEffect) { - if ($value -is [string]) { - $text += "
      *$parameterName = ``$value``*" - } - else { - $json = ConvertTo-Json $value -Depth 100 -Compress - $jsonTruncated = $json - if ($json.length -gt 40) { - $jsonTruncated = $json.substring(0, 40) + "..." - } - $text += "
      *$parameterName = ``$jsonTruncated``*" - } - } - } csvValues { if (-not ($multiUse -or $isEffect)) { $null = $csvParametersHt.Add($parameterName, $value) diff --git a/Scripts/Helpers/Out-PolicyAssignmentDocumentationToFile.ps1 b/Scripts/Helpers/Out-DocumentationForPolicyAssignments.ps1 similarity index 75% rename from Scripts/Helpers/Out-PolicyAssignmentDocumentationToFile.ps1 rename to Scripts/Helpers/Out-DocumentationForPolicyAssignments.ps1 index 670b4f3e..e74d0c7a 100644 --- a/Scripts/Helpers/Out-PolicyAssignmentDocumentationToFile.ps1 +++ b/Scripts/Helpers/Out-DocumentationForPolicyAssignments.ps1 @@ -1,4 +1,4 @@ -function Out-PolicyAssignmentDocumentationToFile { +function Out-DocumentationForPolicyAssignments { [CmdletBinding()] param ( [string] $OutputPath, @@ -42,6 +42,8 @@ function Out-PolicyAssignmentDocumentationToFile { $flatPolicyEntry = $flatPolicyList.$policyTableId $isEffectParameterized = $flatPolicyEntry.isEffectParameterized + $policyDisplayName = $flatPolicyEntry.displayName -replace "\n\r", " " -replace "\n", " " -replace "\r", " " + $policyDescription = $flatPolicyEntry.description -replace "\n\r", " " -replace "\n", " " -replace "\r", " " $effectValue = "Unknown" if ($null -ne $flatPolicyEntry.effectValue) { $effectValue = $flatPolicyEntry.effectValue @@ -64,8 +66,8 @@ function Out-PolicyAssignmentDocumentationToFile { policyTableId = $policyTableId name = $flatPolicyEntry.name referencePath = $flatPolicyEntry.ReferencePath - displayName = $flatPolicyEntry.displayName - description = $flatPolicyEntry.description + displayName = $policyDisplayName + description = $policyDescription policyType = $flatPolicyEntry.policyType category = $flatPolicyEntry.category isEffectParameterized = $isEffectParameterized @@ -120,12 +122,14 @@ function Out-PolicyAssignmentDocumentationToFile { foreach ($shortName in $flatPolicyEntryPolicySetList.Keys) { $policySetInfo = $flatPolicyEntryPolicySetList.$shortName if (-not $policySetList.ContainsKey($shortName)) { + $policySetDisplayName = $policySetInfo.displayName -replace "\n\r", " " -replace "\n", " " -replace "\r", " " + $policySetDescription = $policySetInfo.description -replace "\n\r", " " -replace "\n", " " -replace "\r", " " $policySetEntry = @{ shortName = $shortName id = $policySetInfo.id name = $policySetInfo.name - displayName = $policySetInfo.displayName - description = $policySetInfo.description + displayName = $policySetDisplayName + description = $policySetDescription policyType = $policySetInfo.policyType effectParameterName = $policySetInfo.effectParameterName effectDefault = $policySetInfo.effectDefault @@ -151,29 +155,77 @@ function Out-PolicyAssignmentDocumentationToFile { [System.Collections.Generic.List[string]] $allLines = [System.Collections.Generic.List[string]]::new() $null = $allLines.Add("# $title`n") - $null = $allLines.Add("Auto-generated Policy effect documentation across environments '$($environmentCategories -join "', '")' sorted by Policy category and Policy display name.`n") - $null = $allLines.Add("## Table of Contents`n") - $null = $allLines.Add("- [Policy Effects](#policy-effects)") + $null = $allLines.Add("Auto-generated Policy effect documentation across environments '$($environmentCategories -join "', '")' sorted by Policy category and Policy display name.") + if ($DocumentationSpecification.addMarkdownAdoWikiToc) { + $null = $allLines.Add("`n[[_TOC_]]") + } + + #region Environment Categories + + foreach ($environmentCategory in $environmentCategories) { + $perEnvironment = $AssignmentsByEnvironment.$environmentCategory + $itemList = $perEnvironment.itemList + $assignmentsDetails = $perEnvironment.assignmentsDetails + $scopes = $perEnvironment.scopes + $null = $allLines.Add("`n## Environment Category ``$environmentCategory``") + + $null = $allLines.Add("`n### Scopes`n") + foreach ($scope in $scopes) { + $null = $allLines.Add("- $scope") + } + + foreach ($item in $itemList) { + $assignmentId = $item.assignmentId + if ($assignmentsDetails.ContainsKey($assignmentId)) { + # should always be true + $assignmentsDetail = $assignmentsDetails.$assignmentId + $null = $allLines.Add("`n### Assignment: ``$($assignmentsDetail.assignment.properties.displayName)```n") + $null = $allLines.Add("| Property | Value |") + $null = $allLines.Add("| :------- | :---- |") + $null = $allLines.Add("| Assignment Id | $($assignmentId) |") + $null = $allLines.Add("| Policy Set | ``$($assignmentsDetail.displayName)`` |") + $null = $allLines.Add("| Policy Set Id | $($assignmentsDetail.policySetId) |") + $null = $allLines.Add("| Type | $($assignmentsDetail.policyType) |") + $null = $allLines.Add("| Category | ``$($assignmentsDetail.category)`` |") + $null = $allLines.Add("| Description | $($assignmentsDetail.description) |") + } + } + } + + #endregion Environment Categories #region Policy Effects $addedTableHeader = "" $addedTableDivider = "" foreach ($environmentCategory in $environmentCategories) { - $null = $allLines.Add("- [Environment Category ``$environmentCategory``](#environment-category-$($environmentCategory.ToLower()))") # Calculate environment columns $addedTableHeader += " $environmentCategory |" $addedTableDivider += " :-----: |" } - $null = $allLines.Add("`n## Policy Effects`n") - $null = $allLines.Add("| Category | Policy |$addedTableHeader") - $null = $allLines.Add("| :------- | :----- |$addedTableDivider") + + if ($DocumentationSpecification.includeComplianceGroupNamesInMarkdown) { + $null = $allLines.Add("`n## Policy Effects by Policy`n") + $null = $allLines.Add("| Category | Policy | Compliance |$addedTableHeader") + $null = $allLines.Add("| :------- | :----- | :--------- |$addedTableDivider") + } + else { + $null = $allLines.Add("`n## Policy Effects by Policy`n") + $null = $allLines.Add("| Category | Policy |$addedTableHeader") + $null = $allLines.Add("| :------- | :----- |$addedTableDivider") + } + + $inTableAfterDisplayNameBreak = "
" + $inTableBreak = "
" + if ($DocumentationSpecification.noMarkdownInTableLineBreaks) { + $inTableAfterDisplayNameBreak = ": " + $inTableBreak = ", " + } $flatPolicyListAcrossEnvironments.Values | Sort-Object -Property { $_.category }, { $_.displayName } | ForEach-Object -Process { # Build additional columns $addedEffectColumns = "" $environmentList = $_.environmentList - $additionalInfoFragment = "" foreach ($environmentCategory in $environmentCategories) { if ($environmentList.ContainsKey($environmentCategory)) { $environmentCategoryValues = $environmentList.$environmentCategory @@ -181,85 +233,90 @@ function Out-PolicyAssignmentDocumentationToFile { $effectAllowedValues = $_.effectAllowedValues $text = Convert-EffectToMarkdownString ` -Effect $effectValue ` - -AllowedValues $effectAllowedValues.Keys + -AllowedValues $effectAllowedValues.Keys ` + -InTableBreak $inTableBreak $addedEffectColumns += " $text |" - - # $parameters = $environmentCategoryValues.parameters - # $hasParameters = $false - # if ($null -ne $parameters -and $parameters.psbase.Count -gt 0) { - # foreach ($parameterName in $parameters.Keys) { - # $parameter = $parameters.$parameterName - # if (-not $parameter.isEffect) { - # $hasParameters = $true - # break - # } - # } - # } - - # $additionalInfoFragment += "
***$($environmentCategory)*** *environment:*" - # $policySetList = $environmentCategoryValues.policySetList - # foreach ($shortName in $policySetList.Keys) { - # $perPolicySet = $policySetList.$shortName - # # $policySetDisplayName = $perPolicySet.displayName - # $effectString = $perPolicySet.effectString - # $additionalInfoFragment += "
    $($shortName): ``$($effectString)``" - # } - - # if ($hasParameters) { - # $text = Convert-ParametersToString -Parameters $parameters -OutputType "markdownAssignment" - # $additionalInfoFragment += $text - # } } else { $addedEffectColumns += " |" } } - $groupNames = $_.groupNames - if ($groupNames.Count -gt 0) { - $separator = "
    " - $additionalInfoFragment += "
*Compliance:*$separator" - $sortedGroupNames = $groupNames | Sort-Object -Unique - $additionalInfoFragment += ($sortedGroupNames -join $separator) + $groupNamesText = "" + if ($DocumentationSpecification.includeComplianceGroupNamesInMarkdown) { + $groupNames = $_.groupNames + if ($groupNames.Count -gt 0) { + $sortedGroupNames = $groupNames | Sort-Object -Unique + $groupNamesText = "| $($sortedGroupNames -join $inTableBreak) " + } + else { + $groupNamesText = "| " + } } - $null = $allLines.Add("| $($_.category) | **$($_.displayName)**
$($_.description)$($additionalInfoFragment) | $($addedEffectColumns)") + $null = $allLines.Add("| $($_.category) | **$($_.displayName)**$($inTableAfterDisplayNameBreak)$($_.description) $($groupNamesText)|$($addedEffectColumns)") } #endregion Policy Effects - #region Environment Categories - - foreach ($environmentCategory in $environmentCategories) { - $perEnvironment = $AssignmentsByEnvironment.$environmentCategory - $itemList = $perEnvironment.itemList - $assignmentsDetails = $perEnvironment.assignmentsDetails - $scopes = $perEnvironment.scopes - $null = $allLines.Add("`n## Environment Category ``$environmentCategory``") + #region Parameters - $null = $allLines.Add("`n### Scopes`n") - foreach ($scope in $scopes) { - $null = $allLines.Add("- $scope") - } + $null = $allLines.Add("`n## Policy Parameters by Policy`n") + $null = $allLines.Add("| Category | Policy |$addedTableHeader") + $null = $allLines.Add("| :------- | :----- |$addedTableDivider") - foreach ($item in $itemList) { - $assignmentId = $item.assignmentId - if ($assignmentsDetails.ContainsKey($assignmentId)) { - # should always be true - $assignmentsDetail = $assignmentsDetails.$assignmentId - $null = $allLines.Add("`n### Assignment: ``$($assignmentsDetail.assignment.properties.displayName)```n") - $null = $allLines.Add("| Property | Value |") - $null = $allLines.Add("| :------- | :---- |") - $null = $allLines.Add("| Assignment Id | $($assignmentId) |") - $null = $allLines.Add("| Policy Set | ``$($assignmentsDetail.displayName)`` |") - $null = $allLines.Add("| Policy Set Id | $($assignmentsDetail.policySetId) |") - $null = $allLines.Add("| Type | $($assignmentsDetail.policyType) |") - $null = $allLines.Add("| Category | ``$($assignmentsDetail.category)`` |") - $null = $allLines.Add("| Description | $($assignmentsDetail.description) |") + $flatPolicyListAcrossEnvironments.Values | Sort-Object -Property { $_.category }, { $_.displayName } | ForEach-Object -Process { + # Build additional columns + $addedParametersColumns = "" + $environmentList = $_.environmentList + $hasParameters = $false + foreach ($environmentCategory in $environmentCategories) { + if ($environmentList.ContainsKey($environmentCategory)) { + $environmentCategoryValues = $environmentList.$environmentCategory + $text = "" + $parameters = $environmentCategoryValues.parameters + $notFirst = $false + foreach ($parameterName in $parameters.Keys) { + $parameter = $parameters.$parameterName + if (-not $parameter.isEffect) { + $hasParameters = $true + $value = $parameter.value + if ($notFirst) { + $text += $inTableBreak + } + else { + $notFirst = $true + } + if ($null -eq $value) { + $value = $parameter.defaultValue + if ($null -eq $value) { + $value = "null" + } + } + if ($value -is [string]) { + $text += "$($parameterName) = **```"$value`"``**" + } + else { + $json = ConvertTo-Json $value -Depth 100 -Compress + $jsonTruncated = $json + if ($json.length -gt 40) { + $jsonTruncated = $json.substring(0, 37) + "..." + } + $text += "$($parameterName) = **``$jsonTruncated``**" + } + } + } + $addedParametersColumns += " $text |" + } + else { + $addedParametersColumns += " |" } } + if ($hasParameters) { + $null = $allLines.Add("| $($_.category) | **$($_.displayName)**$($inTableAfterDisplayNameBreak)$($_.description) |$($addedParametersColumns)") + } } - #endregion Environment Categories + #endregion Parameters # Output file $outputFilePath = "$($OutputPath -replace '[/\\]$', '')/$($fileNameStem).md" diff --git a/Scripts/Helpers/Out-PolicySetsDocumentationToFile.ps1 b/Scripts/Helpers/Out-DocumentationForPolicySets.ps1 similarity index 65% rename from Scripts/Helpers/Out-PolicySetsDocumentationToFile.ps1 rename to Scripts/Helpers/Out-DocumentationForPolicySets.ps1 index e651a655..8510bcde 100644 --- a/Scripts/Helpers/Out-PolicySetsDocumentationToFile.ps1 +++ b/Scripts/Helpers/Out-DocumentationForPolicySets.ps1 @@ -1,10 +1,9 @@ -function Out-PolicySetsDocumentationToFile { +function Out-DocumentationForPolicySets { [CmdletBinding()] param ( [string] $OutputPath, - [string] $FileNameStem, [switch] $WindowsNewLineCells, - [string] $Title, + $DocumentationSpecification, [array] $ItemList, [array] $EnvironmentColumnsInCsv, [hashtable] $PolicySetDetails, @@ -12,44 +11,66 @@ function Out-PolicySetsDocumentationToFile { [switch] $IncludeManualPolicies ) - Write-Information "Generating Policy Set documentation for '$Title', files '$FileNameStem'." + $fileNameStem = $DocumentationSpecification.fileNameStem + $title = $DocumentationSpecification.title + $environmentColumnsInCsv = $DocumentationSpecification.environmentColumnsInCsv + + + Write-Information "Generating Policy Set documentation for '$title', files '$FileNameStem'." #region Markdown [System.Collections.Generic.List[string]] $allLines = [System.Collections.Generic.List[string]]::new() - [System.Collections.Generic.List[string]] $headerAndToc = [System.Collections.Generic.List[string]]::new() - [System.Collections.Generic.List[string]] $body = [System.Collections.Generic.List[string]]::new() - $null = $headerAndToc.Add("# $Title`n") - $null = $headerAndToc.Add("Auto-generated Policy effect documentation for PolicySets grouped by Effect and sorted by Policy category and Policy display name.`n") - $null = $headerAndToc.Add("## Table of contents`n") + $null = $allLines.Add("# $title`n") + $null = $allLines.Add("Auto-generated Policy effect documentation for PolicySets grouped by Effect and sorted by Policy category and Policy display name.") + if ($DocumentationSpecification.addMarkdownAdoWikiToc) { + $null = $allLines.Add("`n[[_TOC_]]") + } - $null = $headerAndToc.Add("- [PolicySets](#policySets)") - $null = $body.Add("`n## PolicySets`n") + #region Policy Set List $addedTableHeader = "" $addedTableDivider = "" + $null = $allLines.Add("`n## Policy Set (Initiative) List`n") foreach ($item in $ItemList) { $shortName = $item.shortName $policySetId = $item.policySetId $policySetDetail = $PolicySetDetails.$policySetId - $null = $body.Add("### $($shortName)`n") - $null = $body.Add("- Display name: $($policySetDetail.displayName)") - $null = $body.Add("- Type: $($policySetDetail.policyType)") - $null = $body.Add("- Category: $($policySetDetail.category)`n") - $null = $body.Add("$($policySetDetail.description)`n") + $policySetDisplayName = $policySetDetail.displayName -replace "\n\r", " " -replace "\n", " " -replace "\r", " " + $policySetDescription = $policySetDetail.description -replace "\n\r", " " -replace "\n", " " -replace "\r", " " + $null = $allLines.Add("### $($shortName)`n") + $null = $allLines.Add("- Display name: $($policySetDisplayName)`n") + $null = $allLines.Add("- Type: $($policySetDetail.policyType)") + $null = $allLines.Add("- Category: $($policySetDetail.category)`n") + $null = $allLines.Add("$($policySetDescription)`n") $addedTableHeader += " $shortName |" $addedTableDivider += " :-------- |" } - $null = $headerAndToc.Add("- [Policies](#policies)") - $null = $body.Add("`n
`n`n## Policies`n`n
`n") - $null = $body.Add("| Category | Policy |$addedTableHeader") - $null = $body.Add("| :------- | :----- |$addedTableDivider") + #endregion Policy Set List + + $inTableAfterDisplayNameBreak = "
" + $inTableBreak = "
" + if ($DocumentationSpecification.noMarkdownInTableLineBreaks) { + $inTableAfterDisplayNameBreak = ": " + $inTableBreak = ", " + } + + #region Policy Effects + if ($DocumentationSpecification.includeComplianceGroupNamesInMarkdown) { + $null = $allLines.Add("`n## Policy Effects by Policy`n") + $null = $allLines.Add("| Category | Policy | Compliance |$addedTableHeader") + $null = $allLines.Add("| :------- | :----- | :----------|$addedTableDivider") + } + else { + $null = $allLines.Add("`n## Policy Effects`n") + $null = $allLines.Add("| Category | Policy |$addedTableHeader") + $null = $allLines.Add("| :------- | :----- |$addedTableDivider") + } $FlatPolicyList.Values | Sort-Object -Property { $_.category }, { $_.displayName } | ForEach-Object -Process { $policySetList = $_.policySetList $addedEffectColumns = "" - $addedRows = "" $effectValue = "Unknown" if ($null -ne $_.effectValue) { $effectValue = $_.effectValue @@ -59,7 +80,7 @@ function Out-PolicySetsDocumentationToFile { } if ($effectValue -ne "Manual" -or $IncludeManualPolicies) { - + $groupNamesList = [System.Collections.ArrayList]::new() foreach ($item in $ItemList) { $shortName = $item.shortName if ($policySetList.ContainsKey($shortName)) { @@ -68,40 +89,108 @@ function Out-PolicySetsDocumentationToFile { $effectAllowedValues = $perPolicySet.effectAllowedValues $text = Convert-EffectToMarkdownString ` -Effect $effectValue ` - -AllowedValues $effectAllowedValues + -AllowedValues $effectAllowedValues 1 ` + -inTableBreak $inTableBreak $addedEffectColumns += " $text |" [array] $groupNames = $perPolicySet.groupNames - $parameters = $perPolicySet.parameters - if ($parameters.psbase.Count -gt 0 -or $groupNames.Count -gt 0) { - $addedRows += "
*$($perPolicySet.displayName):*" - $text = Convert-ParametersToString -Parameters $parameters -OutputType "markdown" - $addedRows += $text - foreach ($groupName in $groupNames) { - $addedRows += "
      $groupName" - } + if ($groupNames.Count -gt 0) { + $groupNamesList.AddRange($groupNames) } } else { $addedEffectColumns += " |" } } - $referencePathString = "" - if ($_.referencePath -ne "") { - $referencePathString = "      referencePath: ``$($_.referencePath)``
" + $complianceText = "" + if ($DocumentationSpecification.includeComplianceGroupNamesInMarkdown) { + if ($groupNamesList.Count -gt 0) { + $groupNamesList = $groupNamesList | Sort-Object -Unique + $complianceText = "| $($groupNamesList -join $inTableBreak) " + } + else { + $complianceText = "| " + } } - $null = $body.Add("| $($_.category) | **$($_.displayName)**
$($referencePathString)$($_.description)$($addedRows) |$addedEffectColumns") + $policyDisplayName = $_.displayName -replace "\n\r", " " -replace "\n", " " -replace "\r", " " + $policyDescription = $_.description -replace "\n\r", " " -replace "\n", " " -replace "\r", " " + $null = $allLines.Add("| $($_.category) | **$($policyDisplayName)**$($inTableAfterDisplayNameBreak)$($policyDescription) $complianceText|$addedEffectColumns") } else { Write-Verbose "Skipping manual policy: $($_.name)" } } - $null = $headerAndToc.Add("`n
") - $null = $allLines.AddRange($headerAndToc) - $null = $allLines.AddRange($body) + #endregion Policy Effects + + #region Policy Parameters + $null = $allLines.Add("`n## Policy Parameters by Policy`n") + $null = $allLines.Add("| Category | Policy |$addedTableHeader") + $null = $allLines.Add("| :------- | :----- |$addedTableDivider") + + $FlatPolicyList.Values | Sort-Object -Property { $_.category }, { $_.displayName } | ForEach-Object -Process { + $policySetList = $_.policySetList + $addedParametersColumns = "" + $effectValue = "Unknown" + if ($null -ne $_.effectValue) { + $effectValue = $_.effectValue + } + else { + $effectValue = $_.effectDefault + } + if ($effectValue -ne "Manual" -or $IncludeManualPolicies) { + $hasParameters = $false + foreach ($item in $ItemList) { + $shortName = $item.shortName + if ($policySetList.ContainsKey($shortName)) { + $perPolicySet = $policySetList.$shortName + $parameters = $perPolicySet.parameters + $text = "" + $notFirst = $false + foreach ($parameterName in $parameters.Keys) { + $parameter = $parameters.$parameterName + if (-not $parameter.isEffect) { + $hasParameters = $true + $value = $parameter.defaultValue + if ($notFirst) { + $text += $inTableBreak + } + else { + $notFirst = $true + } + if ($value -is [string]) { + $text += "$($parameterName) = **```"$value`"``**" + } + else { + $json = ConvertTo-Json $value -Depth 100 -Compress + $jsonTruncated = $json + if ($json.length -gt 40) { + $jsonTruncated = $json.substring(0, 37) + "..." + } + $text += "$($parameterName) = **``$jsonTruncated``**" + } + } + } + $addedParametersColumns += " $text |" + } + else { + $addedParametersColumns += " |" + } + } + if ($hasParameters) { + $policyDisplayName = $_.displayName -replace "\n\r", " " -replace "\n", " " -replace "\r", " " + $policyDescription = $_.description -replace "\n\r", " " -replace "\n", " " -replace "\r", " " + $null = $allLines.Add("| $($_.category) | **$($policyDisplayName)**$($inTableAfterDisplayNameBreak)$($policyDescription) |$addedParametersColumns") + } + } + else { + Write-Verbose "Skipping manual policy: $($_.name)" + } + } + #endregion Policy Parameters + # Output file - $outputFilePath = "$($OutputPath -replace '[/\\]$','')/$FileNameStem.md" + $outputFilePath = "$($OutputPath -replace '[/\\]$','')/$fileNameStem.md" $allLines | Out-File $outputFilePath -Force #endregion Markdown diff --git a/Scripts/Operations/Build-PolicyDocumentation.ps1 b/Scripts/Operations/Build-PolicyDocumentation.ps1 index 9abf18cd..51d46000 100644 --- a/Scripts/Operations/Build-PolicyDocumentation.ps1 +++ b/Scripts/Operations/Build-PolicyDocumentation.ps1 @@ -181,7 +181,6 @@ foreach ($file in $files) { if (-not $policySets -or $policySets.Count -eq 0) { Write-Error "documentPolicySet entry does not specify required policySets array entry." -ErrorAction Stop } - $environmentColumnsInCsv = $documentPolicySetEntry.environmentColumnsInCsv # Load pacEnvironment if not already loaded if (-not $cachedPolicyResourceDetails.ContainsKey($pacEnvironmentSelector)) { @@ -237,11 +236,10 @@ foreach ($file in $files) { -Details $policySetDetails # Print documentation - Out-PolicySetsDocumentationToFile ` + Out-DocumentationForPolicySets ` -OutputPath $outputPath ` - -FileNameStem $fileNameStem ` -WindowsNewLineCells:$WindowsNewLineCells ` - -Title $title ` + -DocumentationSpecification $documentPolicySetEntry ` -ItemList $itemList ` -EnvironmentColumnsInCsv $environmentColumnsInCsv ` -PolicySetDetails $policySetDetails ` @@ -309,13 +307,13 @@ foreach ($file in $files) { if ($null -ne $documentationType) { Write-Information "Field documentationType ($($documentationType)) is deprecated" } - Out-PolicyAssignmentDocumentationToFile ` + Out-DocumentationForPolicyAssignments ` -OutputPath $outputPath ` -WindowsNewLineCells:$WindowsNewLineCells ` -DocumentationSpecification $documentationSpecification ` -AssignmentsByEnvironment $assignmentsByEnvironment ` -IncludeManualPolicies:$IncludeManualPolicies - # Out-PolicyAssignmentDocumentationToFile ` + # Out-DocumentationForPolicyAssignments ` # -OutputPath $outputPath ` # -WindowsNewLineCells:$true ` # -DocumentationSpecification $documentationSpecification ` diff --git a/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-dev-pipeline.yml b/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-dev-pipeline.yml index 2210e8c0..1d611cd1 100644 --- a/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-dev-pipeline.yml +++ b/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-dev-pipeline.yml @@ -4,7 +4,11 @@ variables: PAC_DEFINITIONS_FOLDER: ./Definitions # Use the plain text name of each service connection as a reference + planServiceConnection: "sc-epac-plan" devServiceConnection: "sc-epac-dev" + + # set the environment selector + pacEnvironmentSelector: epac-dev # what to build trigger trigger: @@ -23,21 +27,21 @@ pool: stages: - stage: Plan - displayName: "Plan epac-dev" + displayName: "Plan ${{ variables.pacEnvironmentSelector }}" jobs: - job: Plan steps: - template: templates/plan.yml parameters: - serviceConnection: $(devServiceConnection) - pacEnvironmentSelector: epac-dev + serviceConnection: $(planServiceConnection) + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - stage: Deploy - displayName: "Deploy epac-dev" + displayName: "Deploy ${{ variables.pacEnvironmentSelector }}" dependsOn: Plan condition: and(not(failed()), not(canceled()), or(eq(dependencies.Plan.outputs['Plan.Plan.deployPolicyChanges'], 'yes'), eq(dependencies.Plan.outputs['Plan.Plan.deployRoleChanges'], 'yes'))) variables: - PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-epac-dev" + PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-${{ variables.pacEnvironmentSelector }}" localDeployPolicyChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployPolicyChanges']] localDeployRoleChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployRoleChanges']] jobs: @@ -52,7 +56,7 @@ stages: - template: templates/deploy-policy.yml parameters: serviceConnection: $(devServiceConnection) - pacEnvironmentSelector: epac-dev + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - deployment: DeployRoles displayName: "Deploy Role Changes" dependsOn: DeployPolicy @@ -65,7 +69,7 @@ stages: - template: templates/deploy-roles.yml parameters: serviceConnection: $(devServiceConnection) - pacEnvironmentSelector: epac-dev + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - stage: tenantPlan displayName: "Plan tenant" @@ -77,5 +81,5 @@ stages: steps: - template: templates/plan.yml parameters: - serviceConnection: $(devServiceConnection) + serviceConnection: $(planServiceConnection) pacEnvironmentSelector: tenant diff --git a/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-remediation-pipeline.yml b/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-remediation-pipeline.yml index 913e907a..2c38baf1 100644 --- a/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-remediation-pipeline.yml +++ b/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-remediation-pipeline.yml @@ -1,3 +1,10 @@ +parameters: + - name: pacEnvironmentsToRemediate + type: object + default: + - epac-dev + - tenant + variables: # This pipeline is used to auto remdiate Azure policy that are non-compliant. PAC_OUTPUT_FOLDER: ./Output @@ -25,26 +32,15 @@ schedules: always: true stages: - - stage: epacDev - dependsOn: [] - displayName: "Remediate epac-dev environment" - jobs: - - job: remediation - displayName: "Remediation Job" - steps: - - template: templates/remediate.yml - parameters: - serviceConnection: $(remediationServiceConnection) - pacEnvironmentSelector: epac-dev - - - stage: tenant - dependsOn: [] - displayName: "Remediation tenant environment" - jobs: - - job: remediation - steps: - - template: templates/remediate.yml - parameters: - serviceConnection: $(remediationServiceConnection) - pacEnvironmentSelector: tenant - + - ${{ each pacEnvironment in parameters.pacEnvironmentsToRemediate }}: + - stage: Remediate${{ pacEnvironment }} + dependsOn: [] + displayName: "Remediate ${{ pacEnvironment }} environment" + jobs: + - job: remediation + displayName: "Remediation Job" + steps: + - template: templates/remediate.yml + parameters: + serviceConnection: $(remediationServiceConnection) + pacEnvironmentSelector: ${{ pacEnvironment }} diff --git a/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-tenant-pipeline.yml b/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-tenant-pipeline.yml index fe6e45ba..e5c02742 100644 --- a/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-tenant-pipeline.yml +++ b/StarterKit/Pipelines/AzureDevOps/GitHub-Flow/epac-tenant-pipeline.yml @@ -4,9 +4,12 @@ variables: PAC_DEFINITIONS_FOLDER: ./Definitions # Use the plain text name of each service connection as a reference - planServiceConnection: "sc-epac-tenant-plan" + planServiceConnection: "sc-epac-plan" deployServiceConnection: "sc-epac-tenant-deploy" rolesServiceConnection: "sc-epac-tenant-roles" + + # set the environment selector + pacEnvironmentSelector: tenant # what to build trigger trigger: none @@ -17,21 +20,21 @@ pool: stages: - stage: Plan - displayName: "Plan tenant" + displayName: "Plan ${{ variables.pacEnvironmentSelector }}" jobs: - job: Plan steps: - template: templates/plan.yml parameters: serviceConnection: $(planServiceConnection) - pacEnvironmentSelector: tenant + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - stage: Deploy - displayName: "Deploy tenant" + displayName: "Deploy ${{ variables.pacEnvironmentSelector }}" dependsOn: Plan condition: and(not(failed()), not(canceled()), or(eq(dependencies.Plan.outputs['Plan.Plan.deployPolicyChanges'], 'yes'), eq(dependencies.Plan.outputs['Plan.Plan.deployRoleChanges'], 'yes')), contains(variables['Build.SourceBranch'], 'refs/heads/main')) variables: - PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-tenant" + PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-${{ variables.pacEnvironmentSelector }}" localDeployPolicyChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployPolicyChanges']] localDeployRoleChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployRoleChanges']] jobs: @@ -45,8 +48,8 @@ stages: steps: - template: templates/deploy-policy.yml parameters: - serviceConnection: $(deployPolicyServiceConnection) - pacEnvironmentSelector: tenant + serviceConnection: $(deployServiceConnection) + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - deployment: DeployRoles displayName: "Deploy Role Changes" dependsOn: DeployPolicy @@ -58,6 +61,6 @@ stages: steps: - template: templates/deploy-roles.yml parameters: - serviceConnection: $(deployRolesServiceConnection) - pacEnvironmentSelector: tenant + serviceConnection: $(rolesServiceConnection) + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} diff --git a/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-dev-pipeline.yml b/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-dev-pipeline.yml index 4847b783..f89fc55e 100644 --- a/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-dev-pipeline.yml +++ b/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-dev-pipeline.yml @@ -1,10 +1,21 @@ +parameters: + - name: additionalPacEnvironmentsToPlan + type: object + default: + - nonprod + - prod + variables: # This pipeline is used to deploy Policies, Initiative definitions and Assignments into Azure. PAC_OUTPUT_FOLDER: ./Output - PAC_DEFINITIONS_FOLDER: ./Test/Pipelines/Definitions + PAC_DEFINITIONS_FOLDER: ./Definitions # Use the plain text name of each service connection as a reference + planServiceConnection: "sc-epac-plan" devServiceConnection: "sc-epac-dev" + + # set the environment selector + pacEnvironmentSelector: epac-dev # what to build trigger trigger: @@ -23,21 +34,21 @@ pool: stages: - stage: Plan - displayName: "Plan epac-dev" + displayName: "Plan ${{ variables.pacEnvironmentSelector }}" jobs: - job: Plan steps: - template: templates/plan.yml parameters: - serviceConnection: $(devServiceConnection) - pacEnvironmentSelector: epac-dev + serviceConnection: $(planServiceConnection) + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - stage: Deploy - displayName: "Deploy epac-dev" + displayName: "Deploy ${{ variables.pacEnvironmentSelector }}" dependsOn: Plan condition: and(not(failed()), not(canceled()), or(eq(dependencies.Plan.outputs['Plan.Plan.deployPolicyChanges'], 'yes'), eq(dependencies.Plan.outputs['Plan.Plan.deployRoleChanges'], 'yes'))) variables: - PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-epac-dev" + PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-${{ variables.pacEnvironmentSelector }}" localDeployPolicyChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployPolicyChanges']] localDeployRoleChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployRoleChanges']] jobs: @@ -52,7 +63,7 @@ stages: - template: templates/deploy-policy.yml parameters: serviceConnection: $(devServiceConnection) - pacEnvironmentSelector: epac-dev + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - deployment: DeployRoles displayName: "Deploy Role Changes" dependsOn: DeployPolicy @@ -65,31 +76,19 @@ stages: - template: templates/deploy-roles.yml parameters: serviceConnection: $(devServiceConnection) - pacEnvironmentSelector: epac-dev - + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - - stage: nonprodPlan - displayName: "Plan nonprod" + - stage: PlanStageTenant + displayName: "Plan Tenant" dependsOn: - Deploy condition: and(not(failed()), not(canceled())) jobs: - - job: Plan - steps: - - template: templates/plan.yml - parameters: - serviceConnection: $(devServiceConnection) - pacEnvironmentSelector: nonprod - - - stage: prodPlan - displayName: "Plan prod" - dependsOn: - - Deploy - condition: and(not(failed()), not(canceled())) - jobs: - - job: Plan - steps: - - template: templates/plan.yml - parameters: - serviceConnection: $(devServiceConnection) - pacEnvironmentSelector: prod + - ${{ each pacEnvironment in parameters.additionalPacEnvironmentsToPlan }}: + - job: Plan${{ pacEnvironment }} + displayName: "Plan ${{ pacEnvironment }}" + steps: + - template: templates/plan.yml + parameters: + serviceConnection: $(planServiceConnection) + pacEnvironmentSelector: ${{ pacEnvironment }} \ No newline at end of file diff --git a/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-nonprod-pipeline.yml b/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-nonprod-pipeline.yml index 0bbb393f..66276b79 100644 --- a/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-nonprod-pipeline.yml +++ b/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-nonprod-pipeline.yml @@ -4,7 +4,7 @@ variables: PAC_DEFINITIONS_FOLDER: ./Definitions # Use the plain text name of each service connection as a reference - planServiceConnection: "sc-epac-tenant-plan" + planServiceConnection: "sc-epac-plan" deployServiceConnection: "sc-epac-nonprod-deploy" rolesServiceConnection: "sc-epac-nonprod-roles" @@ -22,21 +22,21 @@ pool: stages: - stage: Plan - displayName: "Plan nonprod" + displayName: "Plan ${{ variables.pacEnvironmentSelector }}" jobs: - job: Plan steps: - template: templates/plan.yml parameters: serviceConnection: $(planServiceConnection) - pacEnvironmentSelector: nonprod + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - stage: Deploy - displayName: "Deploy nonprod" + displayName: "Deploy ${{ variables.pacEnvironmentSelector }}" dependsOn: Plan condition: and(not(failed()), not(canceled()), or(eq(dependencies.Plan.outputs['Plan.Plan.deployPolicyChanges'], 'yes'), eq(dependencies.Plan.outputs['Plan.Plan.deployRoleChanges'], 'yes')), contains(variables['Build.SourceBranch'], 'refs/heads/main')) variables: - PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-nonprod" + PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-${{ variables.pacEnvironmentSelector }}" localDeployPolicyChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployPolicyChanges']] localDeployRoleChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployRoleChanges']] jobs: @@ -50,8 +50,8 @@ stages: steps: - template: templates/deploy-policy.yml parameters: - serviceConnection: $(deployPolicyServiceConnection) - pacEnvironmentSelector: nonprod + serviceConnection: $(rolesServiceConnection) + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - deployment: DeployRoles displayName: "Deploy Role Changes" dependsOn: DeployPolicy @@ -63,6 +63,6 @@ stages: steps: - template: templates/deploy-roles.yml parameters: - serviceConnection: $(deployRolesServiceConnection) - pacEnvironmentSelector: nonprod + serviceConnection: $(rolesServiceConnection) + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} diff --git a/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-prod-exemptions-only-pipeline.yml b/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-prod-exemptions-only-pipeline.yml index 615734e6..e4a23b10 100644 --- a/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-prod-exemptions-only-pipeline.yml +++ b/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-prod-exemptions-only-pipeline.yml @@ -4,9 +4,12 @@ variables: PAC_DEFINITIONS_FOLDER: ./Definitions # Use the plain text name of each service connection as a reference - tenantPlanServiceConnection: "epac-mg-prod" - tenantDeployServiceConnection: "epac-mg-prod" - tenantRolesServiceConnection: "epac-mg-prod" + planServiceConnection: "sc-epac-plan" + deployServiceConnection: "sc-epac-prod-deploy" + rolesServiceConnection: "sc-epac-prod-roles" + + # set the environment selector + pacEnvironmentSelector: prod # System.Debug: true @@ -21,21 +24,21 @@ pool: stages: - stage: Plan - displayName: "Plan prod Exemptions Only" + displayName: "Plan ${{ variables.pacEnvironmentSelector }} Exemptions Only" jobs: - job: Plan steps: - template: templates/plan-exemptions-only.yml parameters: serviceConnection: $(planServiceConnection) - pacEnvironmentSelector: prod + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - stage: Deploy - displayName: "Deploy prod Exemptions Only" + displayName: "Deploy ${{ variables.pacEnvironmentSelector }} Exemptions Only" dependsOn: Plan condition: and(not(failed()), not(canceled()), eq(dependencies.Plan.outputs['Plan.Plan.deployPolicyChanges'], 'yes'), contains(variables['Build.SourceBranch'], 'releases-exemptions-only/')) variables: - PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-prod" + PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-${{ variables.pacEnvironmentSelector }}" localDeployPolicyChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployPolicyChanges']] localDeployRoleChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployRoleChanges']] jobs: @@ -49,5 +52,5 @@ stages: steps: - template: templates/deploy-policy.yml parameters: - serviceConnection: $(deployPolicyServiceConnection) - pacEnvironmentSelector: prod + serviceConnection: $(deployServiceConnection) + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} diff --git a/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-prod-pipeline.yml b/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-prod-pipeline.yml index 3fea46c0..c48628c7 100644 --- a/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-prod-pipeline.yml +++ b/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-prod-pipeline.yml @@ -4,7 +4,7 @@ variables: PAC_DEFINITIONS_FOLDER: ./Definitions # Use the plain text name of each service connection as a reference - planServiceConnection: "sc-epac-tenant-plan" + planServiceConnection: "sc-epac-plan" deployServiceConnection: "sc-epac-prod-deploy" rolesServiceConnection: "sc-epac-prod-roles" @@ -22,21 +22,21 @@ pool: stages: - stage: Plan - displayName: "Plan prod" + displayName: "Plan ${{ variables.pacEnvironmentSelector }}" jobs: - job: Plan steps: - template: templates/plan.yml parameters: serviceConnection: $(planServiceConnection) - pacEnvironmentSelector: prod + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - stage: Deploy - displayName: "Deploy prod" + displayName: "Deploy ${{ variables.pacEnvironmentSelector }}" dependsOn: Plan - condition: and(not(failed()), not(canceled()), or(eq(dependencies.Plan.outputs['Plan.Plan.deployPolicyChanges'], 'yes'), eq(dependencies.Plan.outputs['Plan.Plan.deployRoleChanges'], 'yes')), contains(variables['Build.SourceBranch'], 'releases-prod/') + condition: and(not(failed()), not(canceled()), or(eq(dependencies.Plan.outputs['Plan.Plan.deployPolicyChanges'], 'yes'), eq(dependencies.Plan.outputs['Plan.Plan.deployRoleChanges'], 'yes')), contains(variables['Build.SourceBranch'], 'releases-prod/')) variables: - PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-prod" + PAC_INPUT_FOLDER: "$(Pipeline.Workspace)/plans-${{ variables.pacEnvironmentSelector }}" localDeployPolicyChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployPolicyChanges']] localDeployRoleChanges: $[stageDependencies.Plan.Plan.outputs['Plan.deployRoleChanges']] jobs: @@ -50,8 +50,8 @@ stages: steps: - template: templates/deploy-policy.yml parameters: - serviceConnection: $(deployPolicyServiceConnection) - pacEnvironmentSelector: prod + serviceConnection: $(deployServiceConnection) + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} - deployment: DeployRoles displayName: "Deploy Role Changes" dependsOn: DeployPolicy @@ -63,6 +63,6 @@ stages: steps: - template: templates/deploy-roles.yml parameters: - serviceConnection: $(deployRolesServiceConnection) - pacEnvironmentSelector: prod + serviceConnection: $(rolesServiceConnection) + pacEnvironmentSelector: ${{ variables.pacEnvironmentSelector }} diff --git a/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-remediation-pipeline.yml b/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-remediation-pipeline.yml index d4c5e933..2fd6f2ba 100644 --- a/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-remediation-pipeline.yml +++ b/StarterKit/Pipelines/AzureDevOps/Release-Flow/epac-remediation-pipeline.yml @@ -1,3 +1,11 @@ +parameters: + - name: pacEnvironmentsToRemediate + type: object + default: + - epac-dev + - nonprod + - prod + variables: # This pipeline is used to auto remdiate Azure policy that are non-compliant. PAC_OUTPUT_FOLDER: ./Output @@ -25,36 +33,15 @@ schedules: always: true stages: - - stage: epacDev - dependsOn: [] - displayName: "Remediate epac-dev environment" - jobs: - - job: remediation - displayName: "Remediation Job" - steps: - - template: templates/remediate.yml - parameters: - serviceConnection: $(remediationServiceConnection) - pacEnvironmentSelector: epac-dev - - - stage: nonprod - dependsOn: [] - displayName: "Remediation nonprod environment" - jobs: - - job: remediation - steps: - - template: templates/remediate.yml - parameters: - serviceConnection: $(remediationServiceConnection) - pacEnvironmentSelector: nonprod - - - stage: prod - dependsOn: [] - displayName: "Remediation prod environment" - jobs: - - job: remediation - steps: - - template: templates/remediate.yml - parameters: - serviceConnection: $(remediationServiceConnection) - pacEnvironmentSelector: prod + - ${{ each pacEnvironment in parameters.pacEnvironmentsToRemediate }}: + - stage: Remediate${{ pacEnvironment }} + dependsOn: [] + displayName: "Remediate ${{ pacEnvironment }} environment" + jobs: + - job: remediation + displayName: "Remediation Job" + steps: + - template: templates/remediate.yml + parameters: + serviceConnection: $(remediationServiceConnection) + pacEnvironmentSelector: ${{ pacEnvironment }} \ No newline at end of file diff --git a/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/deploy-policy.yml b/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/deploy-policy.yml index 7ec9b9d6..1e810223 100644 --- a/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/deploy-policy.yml +++ b/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/deploy-policy.yml @@ -10,7 +10,6 @@ steps: artifact: "plans-${{ parameters.pacEnvironmentSelector }}" - pwsh: | Install-Module EnterprisePolicyAsCode -Force - - task: AzurePowerShell@5 displayName: Deploy Policy inputs: diff --git a/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/deploy-roles.yml b/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/deploy-roles.yml index 23a34e7e..a48ac176 100644 --- a/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/deploy-roles.yml +++ b/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/deploy-roles.yml @@ -10,7 +10,6 @@ steps: artifact: "plans-${{ parameters.pacEnvironmentSelector }}" - pwsh: | Install-Module EnterprisePolicyAsCode -Force - - task: AzurePowerShell@5 name: Deploy displayName: Deploy Roles diff --git a/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/plan-exemptions-only.yml b/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/plan-exemptions-only.yml index d9653bdb..7b7d19f0 100644 --- a/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/plan-exemptions-only.yml +++ b/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/plan-exemptions-only.yml @@ -8,7 +8,6 @@ steps: - checkout: self - pwsh: | Install-Module EnterprisePolicyAsCode -Force - - task: AzurePowerShell@5 name: Plan inputs: diff --git a/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/plan.yml b/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/plan.yml index e701c810..5705c3b6 100644 --- a/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/plan.yml +++ b/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/plan.yml @@ -8,7 +8,6 @@ steps: - checkout: self - pwsh: | Install-Module EnterprisePolicyAsCode -Force - - task: AzurePowerShell@5 name: Plan inputs: diff --git a/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/remediate.yml b/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/remediate.yml index 0dc127a0..d7b1161d 100644 --- a/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/remediate.yml +++ b/StarterKit/Pipelines/AzureDevOps/templates-ps1-module/remediate.yml @@ -8,7 +8,6 @@ steps: - checkout: self - pwsh: | Install-Module EnterprisePolicyAsCode -Force - - task: AzurePowerShell@5 name: PolicyRemediation displayName: Policy Remediation diff --git a/StarterKit/Pipelines/GitHubActions/templates-ps1-module/deploy-policy.yml b/StarterKit/Pipelines/GitHubActions/templates-ps1-module/deploy-policy.yml index e90887e8..33b3f26f 100644 --- a/StarterKit/Pipelines/GitHubActions/templates-ps1-module/deploy-policy.yml +++ b/StarterKit/Pipelines/GitHubActions/templates-ps1-module/deploy-policy.yml @@ -38,7 +38,6 @@ jobs: uses: Azure/powershell@v2 with: inlineScript: | - Install-Module EnterprisePolicyAsCode -Force azPSVersion: "latest" diff --git a/StarterKit/Pipelines/GitHubActions/templates-ps1-module/deploy-roles.yml b/StarterKit/Pipelines/GitHubActions/templates-ps1-module/deploy-roles.yml index dc46cdfb..0bc10ffd 100644 --- a/StarterKit/Pipelines/GitHubActions/templates-ps1-module/deploy-roles.yml +++ b/StarterKit/Pipelines/GitHubActions/templates-ps1-module/deploy-roles.yml @@ -38,7 +38,6 @@ jobs: uses: Azure/powershell@v2 with: inlineScript: | - Install-Module EnterprisePolicyAsCode -Force azPSVersion: "latest" diff --git a/StarterKit/Pipelines/GitHubActions/templates-ps1-module/plan-exemptions-only.yml b/StarterKit/Pipelines/GitHubActions/templates-ps1-module/plan-exemptions-only.yml index 1016a237..7fa5ae1d 100644 --- a/StarterKit/Pipelines/GitHubActions/templates-ps1-module/plan-exemptions-only.yml +++ b/StarterKit/Pipelines/GitHubActions/templates-ps1-module/plan-exemptions-only.yml @@ -44,7 +44,6 @@ jobs: uses: Azure/powershell@v2 with: inlineScript: | - Install-Module EnterprisePolicyAsCode -Force azPSVersion: "latest" diff --git a/StarterKit/Pipelines/GitHubActions/templates-ps1-module/plan.yml b/StarterKit/Pipelines/GitHubActions/templates-ps1-module/plan.yml index dd48b3fe..29a0865b 100644 --- a/StarterKit/Pipelines/GitHubActions/templates-ps1-module/plan.yml +++ b/StarterKit/Pipelines/GitHubActions/templates-ps1-module/plan.yml @@ -43,7 +43,6 @@ jobs: uses: Azure/powershell@v2 with: inlineScript: | - Install-Module EnterprisePolicyAsCode -Force azPSVersion: "latest" diff --git a/StarterKit/Pipelines/GitHubActions/templates-ps1-module/remediate.yml b/StarterKit/Pipelines/GitHubActions/templates-ps1-module/remediate.yml index 2495352b..932dd529 100644 --- a/StarterKit/Pipelines/GitHubActions/templates-ps1-module/remediate.yml +++ b/StarterKit/Pipelines/GitHubActions/templates-ps1-module/remediate.yml @@ -30,7 +30,6 @@ jobs: uses: Azure/powershell@v2 with: inlineScript: | - Install-Module EnterprisePolicyAsCode -Force azPSVersion: "latest" diff --git a/StarterKit/Pipelines/GitLab/GitHub-Flow/.gitlab-ci.yml b/StarterKit/Pipelines/GitLab/GitHub-Flow/.gitlab-ci.yml new file mode 100644 index 00000000..16b5eb9d --- /dev/null +++ b/StarterKit/Pipelines/GitLab/GitHub-Flow/.gitlab-ci.yml @@ -0,0 +1,77 @@ +# Example pipeline for deploying policies and roles using Federated Login +# The variables $AZURE_TENANT_ID and $AZURE_CLIENT_ID are set in the GitLab CI/CD settings + +image: mcr.microsoft.com/powershell:latest + +stages: +- plan +- deployPolicies +- deployRoles + +variables: + PAC_OUTPUT_FOLDER: ./Output + PAC_DEFINITIONS_FOLDER: ./Definitions + ENVIRONMENT: "EPAC-PROD" + +plan: + stage: plan + id_tokens: + GITLAB_OIDC_TOKEN: + aud: api://AzureADTokenExchange + script: + - | + pwsh -c ' + Install-Module -Name Az.Accounts -Force; + Install-Module -Name EnterprisePolicyAsCode -Force; + Install-Module -Name Az.ResourceGraph -Force; + Install-Module -Name Az.Resources -Force -AllowClobber;' + - pwsh -c "Connect-AzAccount -Tenant $AZURE_TENANT_ID -ApplicationId $AZURE_CLIENT_ID -FederatedToken $GITLAB_OIDC_TOKEN > \$null 2>&1" + - pwsh -c "Build-DeploymentPlans -PacEnvironmentSelector $ENVIRONMENT -DefinitionsRootFolder Definitions -devOpsType gitlab -outputFolder $PAC_OUTPUT_FOLDER" + artifacts: + paths: + - $PAC_OUTPUT_FOLDER + +deployPolicies: + stage: deployPolicies + id_tokens: + GITLAB_OIDC_TOKEN: + aud: api://AzureADTokenExchange + dependencies: + - plan + script: + - | + if [ ! -f "./Output/plans-$ENVIRONMENT/policy-plan.json" ]; then + echo "File not found, skipping the job." + exit 0 + fi + - | + pwsh -c ' + Install-Module -Name Az.Accounts -Force; + Install-Module -Name EnterprisePolicyAsCode -Force; + Install-Module -Name Az.ResourceGraph -Force; + Install-Module -Name Az.Resources -Force -AllowClobber;' + - pwsh -c "Connect-AzAccount -Tenant $AZURE_TENANT_ID -ApplicationId $AZURE_CLIENT_ID -FederatedToken $GITLAB_OIDC_TOKEN > \$null 2>&1" + - pwsh -c "Deploy-PolicyPlan -PacEnvironmentSelector $ENVIRONMENT -DefinitionsRootFolder Definitions -inputFolder $PAC_OUTPUT_FOLDER" + + +deployRoles: + stage: deployRoles + id_tokens: + GITLAB_OIDC_TOKEN: + aud: api://AzureADTokenExchange + dependencies: + - plan + script: + - | + if [ ! -f "./Output/plans-$ENVIRONMENT/roles-plan.json" ]; then + echo "File not found, skipping the job." + exit 0 + fi + - | + pwsh -c ' + Install-Module -Name Az.Accounts -Force; + Install-Module -Name EnterprisePolicyAsCode -Force; + Install-Module -Name Az.ResourceGraph -Force; + Install-Module -Name Az.Resources -Force -AllowClobber;' + - pwsh -c "Connect-AzAccount -Tenant $AZURE_TENANT_ID -ApplicationId $AZURE_CLIENT_ID -FederatedToken $GITLAB_OIDC_TOKEN > \$null 2>&1" + - pwsh -c "Deploy-RolesPlan -PacEnvironmentSelector $ENVIRONMENT -DefinitionsRootFolder Definitions -inputFolder $PAC_OUTPUT_FOLDER" diff --git a/mkdocs.yml b/mkdocs.yml index 94e0721a..85ae46b0 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -47,6 +47,7 @@ nav: - CI/CD Integration: - CI/CD Overview: ci-cd-overview.md - App Registrations Setup: ci-cd-app-registrations.md + - Branching Flows: ci-cd-branching-flows.md - Azure DevOps Pipelines: ci-cd-ado-pipelines.md - GitHub Actions: ci-cd-github-actions.md - Operational Scripts: