Update documentation with general architecture and concepts (#143)

* Update documentation with general architecture and concepts

* Update modules/scanning-delegate-role/README.md

Co-authored-by: Diogo Pereira <[email protected]>

* Apply suggestions from code review

Co-authored-by: Diogo Pereira <[email protected]>

* Apply suggestions from code review

Co-authored-by: Alicia Scott <[email protected]>

---------

Co-authored-by: Diogo Pereira <[email protected]>
Co-authored-by: Alicia Scott <[email protected]>

Author: 3 people

README.md

@@ -12,20 +12,34 @@ Before using this module, make sure you have the following:
 ## Usage
 
 To use this module in your Terraform configuration, add the following code in your existing Terraform code:
+
 ```hcl
+# First we need to define the proper roles for our scanners. It consists of two different modules that have a two way bindings between them.
+
+# 1. The "scanning delegate role" defines all the policies and IAM roles necessary for the scanner to interact and scan some specific account resources.
+# It shall be created for every account that the agentless scanner will be able scan. These roles are meant to be assumed by the "agentless scanner role".
+module "delegate_role" {
+  source = "git::https://github.com/DataDog/terraform-module-datadog-agentless-scanner//modules/scanning-delegate-role"
+
+  scanner_roles = [module.scanner_role.role.arn]
+}
+
+# 2. The "agentless scanner role" creates an EC2 instance profile along with an IAM role allowing the EC2 instance scanner to assume the scanning delegate role(s).
+# It shall be created in the same account as the agentless scanner instance.
 module "scanner_role" {
   source = "git::https://github.com/DataDog/terraform-module-datadog-agentless-scanner//modules/agentless-scanner-role"
 
   account_roles       = [module.delegate_role.role.arn]
   api_key_secret_arns = [module.agentless_scanner.api_key_secret_arn]
 }
 
-module "delegate_role" {
-  source = "git::https://github.com/DataDog/terraform-module-datadog-agentless-scanner//modules/scanning-delegate-role"
-
-  scanner_roles = [module.scanner_role.role.arn]
-}
+# As you can see there is a two way binding between these two "role" modules.
+# - the scanner role requires the list of delegate role ARNs for the scanner to assume.
+# - the delegate role(s) require the scanner role ARN as input in order to define the trust relationship between the EC2 scanner role and the delegate role to be assumed.
 
+# Finally we can create the agentless scanner instance. It requires the instance profile name that was created by the scanner_role.
+# This module will define the VPC, subnets, network and compute resources required for the agentless scanner.
+# See the documentation of each module for more information or our examples for a complete setup.
 module "agentless_scanner" {
   source = "git::https://github.com/DataDog/terraform-module-datadog-agentless-scanner"
 
@@ -54,6 +68,44 @@ To uninstall, remove the Agentless scanner module from your Terraform code. Remo
 > [!WARNING]
 > Exercise caution when deleting Terraform resources. Review the plan carefully to ensure everything is in order.
 
+## Architecture
+
+The Agentless Scanner deployment is split into different modules to allow for more flexibility and customization. The following modules are available:
+
+- [scanning-delegate-role](./modules/scanning-delegate-role/): Creates the necessary IAM role and policies for the scanning delegate. It creates an IAM role in a specific account that the scanner can then assume to scan the account. This role allows read access to many different resources (EBS snapshots, Lambdas etc.) in the account to be able to scan them.
+- [agentless-scanner-role](./modules/agentless-scanner-role/): Creates the necessary IAM role and policies for the agentless scanner instance. It creates an IAM role that allows the scanner to assume the role of the scanning delegate.
+- [instance](./modules/instance/): Creates the EC2 instance that runs the agentless scanner. This instance is launched as part of an Auto Scaling group to ensure high availability.
+- [user_data](./modules/user_data/): Creates the user data script that installs and configures the agentless scanner on the EC2 instance.
+- [vpc](./modules/vpc/): Creates the VPC, subnets and all network resources required for the agentless scanner.
+
+The main module provided at the root of this repository is a thin wrapper around the vpc, user_data and instance modules, with simplified inputs. The scanning-delegate-role and agentless-scanner-role modules are intended to be used in conjunction with this module, as they define the proper IAM permissions for the scanner.
+
+```mermaid
+flowchart TD
+    subgraph "Account A"
+      subgraph "Main module"
+          UD[user_data]
+          VPC[vpc]
+          I[instance]
+          UD-->I
+          VPC-->I
+        end
+
+        SR[agentless-scanner-role]
+        SR-->I
+
+        DRA[scanning-delegate-role A]
+        DRA-- trusts -->SR
+        SR-- assumes -->DRA
+    end
+
+    subgraph "Account B"
+      DRB[scanning-delegate-role B]
+      DRB-- trusts -->SR
+      SR-- assumes -->DRB
+    end
+```
+
 ## Examples
 
 For complete examples, refer to the [examples](./examples/) directory in this repository.

examples/README.md

@@ -20,3 +20,7 @@ To scan across accounts, see the [example](cross_account/README.md)
 # Custom VPC Example
 
 If you want to avoid creating a new VPC for the Agentless scanners, and you want to reuse one of your own, see the [example](custom_vpc/README.md)
+
+# Custom Agent Configurations Example
+
+If you want to add custom configurations for our Datadog Agent running alongside our scanners, see the [example](custom_agent_configurations/README.md)

modules/agentless-scanner-role/README.md

@@ -1,3 +1,9 @@
+## Description
+
+The agentless-scanner-role module creates the proper role and policies for the agentless scanner instance to be able to assume the scanning delegate roles.
+
+It exports the role and instance profile that should be attached to the agentless scanner instance.
+
 <!-- BEGIN_TF_DOCS -->
 ## Requirements
 

modules/instance/README.md

@@ -1,3 +1,9 @@
+## Description
+
+The instance module can be used to spawn a dedicated instance specifically crafted for running the agentless scanner.
+
+It creates an auto scaling group with a launch template that installs the agentless scanner on the instance.
+
 <!-- BEGIN_TF_DOCS -->
 ## Requirements
 
@@ -45,4 +51,4 @@ No modules.
 ## Outputs
 
 No outputs.
-<!-- END_TF_DOCS -->
+<!-- END_TF_DOCS -->

modules/scanning-delegate-role/README.md

@@ -1,3 +1,9 @@
+## Description
+
+The scanning-delegate-role module creates the proper role and policies to allow the agentless scanner to interact with AWS services of a specific account.
+
+It should be installed in every account that the agentless scanner will scan.
+
 <!-- BEGIN_TF_DOCS -->
 ## Requirements
 

modules/user_data/README.md

@@ -1,3 +1,9 @@
+## Description
+
+The user_data module exports a script that can be used to install the Datadog agentless scanner on a target instance. This script can be configured to install the scanner.
+
+It is then used to create the launch template for the agentless scanner instance.
+
 <!-- BEGIN_TF_DOCS -->
 ## Requirements
 

modules/vpc/README.md

@@ -1,3 +1,9 @@
+## Description
+
+The vpc module can be used to spawn a dedicated VPC specifically crafted for running the agentless scanner instance.
+
+It provides dedicated private subnet, associated with the proper private endpoints to AWS services in order to reduce the network cost induced by scanning resources.
+
 <!-- BEGIN_TF_DOCS -->
 ## Requirements
 
@@ -61,4 +67,4 @@ No modules.
 | <a name="output_public_subnets"></a> [public\_subnets](#output\_public\_subnets) | The public subnets of the created VPC |
 | <a name="output_routing_ready"></a> [routing\_ready](#output\_routing\_ready) | Indicates whether routing resources have been successfully provisioned |
 | <a name="output_vpc"></a> [vpc](#output\_vpc) | The VPC created for the Datadog agentless scanner |
-<!-- END_TF_DOCS -->
+<!-- END_TF_DOCS -->