Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Node metrics #166

Merged
merged 20 commits into from
Jan 24, 2025
Merged

Node metrics #166

merged 20 commits into from
Jan 24, 2025
+140 −115

Conversation

hellais
Copy link
Member

@hellais hellais commented Jan 24, 2025 •
edited
Loading

This improves the way in which labels are getting set on hosts and is related to: #157

More changes are:

  • Make the name of the notebook host consistent with the rest of our naming patterns
  • Fix the monitoring host role
  • Improve the prometheus_node_exporter role

Note: job names and labels will present some inconsistencies until we have rescraped all the data for the recent time period.

hellais added 20 commits January 24, 2025 14:13
@hellais
Refactor the prometheus labels and scraping config for clickhouse clu…
bd99902 
…ster

* Improve consistency of namings
* Perform node_exporter scraping over HTTPS
* Set custom label on test helper node scrapes
@hellais
Delete discontinued host vars
dd5b0e9
@hellais
Drop unnecessary nftables rules
664a305
@hellais
Add notebook host to monitoring
b48180f
@hellais
Fix label remapping
6020128
@hellais
Improve the labelling of the node_exporter metrics
5dd1cbf
@hellais
Change server name specificity
173c9e2
@hellais
Fix regex?
9166a88
@hellais
Fix password injection
7d27b85
@hellais
Update grafana gpg keys
2940b62
@hellais
Fix order of prometheus scraping
0159dac
@hellais
Fix extension to be .asc
6f74412
@hellais
Fix path
6e6ea98
@hellais
Add A record for notebook host
06644a9
@hellais
Fix consistency of naming of notebook server
a136e00
@hellais
Fix inventory hostname
285b2d1
@hellais
Fix typo
53518a3
@hellais
Fix to instance relabelling
d221d9b
@hellais
Fix to ssl domains recursion
1d7ad59
@hellais
Fix fqdn of notebook host in clickhouse deploy
4003d2e
@hellais
Copy link
Member Author

hellais commented Jan 24, 2025

There is still some remaining work to be done on #157 but we can merge this in the meantime.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 24, 2025 •
edited
Loading

Terraform Run Output 🤖

Format and Style 🖌failure

Initialization ⚙️success

Validation 🤖success

Validation Output 

$ terraform validate
Success! The configuration is valid.

Plan 📖success

  • Plan: 0 to add, 5 to change, 0 to destroy.
Show Plan 

$ terraform plan
module.ooni_clickhouse_proxy.data.cloudinit_config.ooni_ec2: Reading...
random_id.artifact_id: Refreshing state... [id=8Ujqew]
module.adm_iam_roles.tls_private_key.oonidevops: Refreshing state... [id=b49a9fdb9f720320340226016efe24808dd68203]
random_password.prometheus_metrics_password: Refreshing state... [id=none]
module.ansible_inventory.local_file.ansible_inventory: Refreshing state... [id=b6de844ed8d384f890fa6f467502390de843f758]
module.ooni_clickhouse_proxy.data.cloudinit_config.ooni_ec2: Read complete after 0s [id=2022394177]
module.ansible_inventory.null_resource.ansible_update_known_hosts: Refreshing state... [id=236461505953331670]
module.adm_iam_roles.data.aws_iam_policy_document.assume_role: Reading...
module.ooniapi_ooniauth.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.adm_iam_roles.aws_key_pair.oonidevops: Refreshing state... [id=oonidevops]
data.aws_ssm_parameter.clickhouse_readonly_url: Reading...
module.ooniapi_ooniauth.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-ooniauth]
module.ooniapi_oonirun.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonirun-task-role]
module.ooniapi_reverseproxy_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-reverseproxy]
module.oonidevops_github_user.aws_iam_policy.oonidevops_github: Refreshing state... [id=arn:aws:iam::905418398257:policy/oonidevops-github-policy]
module.adm_iam_roles.data.aws_iam_policy_document.assume_role: Read complete after 0s [id=367960279]
module.ooniapi_oonimeasurements.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonimeasurements-task-role]
aws_s3_bucket.oonith_codepipeline_bucket: Refreshing state... [id=codepipeline-oonith-eu-central-1-f148ea7b]
module.ooniapi_user.aws_secretsmanager_secret.aws_access_key_id: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_access_key_id-EcXOBx]
module.adm_iam_roles.aws_secretsmanager_secret.oonidevops_deploy_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe]
aws_acm_certificate.ooniapi_frontend: Refreshing state... [id=arn:aws:acm:eu-central-1:905418398257:certificate/190205f1-392d-425c-a059-7006ca8c8c46]
data.aws_ssm_parameter.oonipg_url: Reading...
module.ooniapi_oonirun_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonirun]
module.ooni_clickhouse_proxy.data.aws_ssm_parameter.ubuntu_22_ami: Reading...
module.ooniapi_ooniauth.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-ooniauth-td:91/ooniapi-service-ooniauth]
module.ooniapi_reverseproxy_deployer.data.aws_caller_identity.current: Reading...
data.aws_ssm_parameter.clickhouse_readonly_url: Read complete after 0s [id=/oonidevops/secrets/clickhouse_readonly_url]
module.ooniapi_ooniauth.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniauth-task-role]
module.ooniapi_oonimeasurements_deployer.data.aws_caller_identity.current: Reading...
data.aws_availability_zones.available: Reading...
module.ooniapi_reverseproxy_deployer.data.aws_caller_identity.current: Read complete after 1s [id=905418398257]
module.ooniapi_oonirun.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_oonirun.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonirun]
data.aws_ssm_parameter.oonipg_url: Read complete after 1s [id=/oonidevops/secrets/ooni-tier0-postgres/postgresql_write_url]
module.ooniapi_ooniprobe.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_oonifindings.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonifindings-task-role]
module.ooniapi_oonimeasurements_deployer.data.aws_caller_identity.current: Read complete after 1s [id=905418398257]
data.aws_ssm_parameter.jwt_secret: Reading...
module.ooniapi_oonifindings.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_oonirun.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-oonirun-td:86/ooniapi-service-oonirun]
module.ooniapi_oonifindings_deployer.data.aws_caller_identity.current: Reading...
data.aws_availability_zones.available: Read complete after 0s [id=eu-central-1]
module.ooniapi_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Reading...
module.ooni_clickhouse_proxy.data.aws_ssm_parameter.ubuntu_22_ami: Read complete after 1s [id=/aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id]
module.ooniapi_reverseproxy.data.aws_ecs_container_definition.ooniapi_service_current[0]: Reading...
module.ooniapi_oonifindings_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_ooniauth_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-ooniauth]
module.adm_iam_roles.aws_iam_policy.oonidevops: Refreshing state... [id=arn:aws:iam::905418398257:policy/OONIDevopsPolicy]
module.ooniapi_ooniprobe.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-ooniprobe-td:84/ooniapi-service-ooniprobe]
module.ooniapi_oonifindings_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonifindings]
module.ooniapi_ooniprobe_deployer.data.aws_caller_identity.current: Reading...
data.aws_ssm_parameter.jwt_secret: Read complete after 0s [id=/oonidevops/secrets/ooni_services/jwt_secret]
module.ooniapi_ooniauth_deployer.data.aws_caller_identity.current: Reading...
module.ooniapi_reverseproxy.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-reverseproxy]
module.ooniapi_user.aws_iam_user.ooniapi: Refreshing state... [id=oonidevops-ooniapi]
module.ooniapi_ooniprobe.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniprobe-task-role]
module.ooniapi_ooniprobe_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_oonifindings.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-oonifindings-td:35/ooniapi-service-oonifindings]
module.oonidevops_github_user.aws_secretsmanager_secret.oonidevops_github: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/github_user/access_key_json-9JTJgd]
module.ooniapi_cluster.aws_iam_role.container_host: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role]
module.ooniapi_cluster.aws_cloudwatch_log_group.ooniapi_services: Refreshing state... [id=ooni-ecs-group/ooniapi-ecs-cluster]
module.ooniapi_cluster.data.aws_ssm_parameter.ecs_optimized_ami: Read complete after 0s [id=/aws/service/ecs/optimized-ami/amazon-linux-2/recommended]
module.ooniapi_oonimeasurements.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonimeasurements]
aws_secretsmanager_secret.prometheus_metrics_password: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/prometheus_metrics_password-M8BbRw]
module.ooniapi_ooniauth_deployer.data.aws_caller_identity.current: Read complete after 0s [id=905418398257]
module.ooniapi_reverseproxy.aws_iam_role.ooniapi_service_task: Refreshing state... [id=ooniapi-service-reverseproxy-task-role]
module.ooniapi_reverseproxy.data.aws_ecs_container_definition.ooniapi_service_current[0]: Read complete after 0s [id=arn:aws:ecs:eu-central-1:905418398257:task-definition/ooniapi-service-reverseproxy-td:26/ooniapi-service-reverseproxy]
aws_secretsmanager_secret.oonipg_url: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/postgresql_url-w62CTZ]
module.ooniapi_oonifindings.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-oonifindings]
module.ooniapi_user.aws_ses_email_identity.ooniapi: Refreshing state... [[email protected]]
module.ooniapi_ooniprobe.aws_cloudwatch_log_group.ooniapi_service: Refreshing state... [id=ooni-ecs-group/ooniapi-service-ooniprobe]
data.aws_ssm_parameter.do_token: Reading...
module.ooniapi_ooniprobe_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-ooniprobe]
module.oonidevops_github_user.aws_iam_user.oonidevops_github: Refreshing state... [id=oonidevops-github]
module.ooniapi_user.aws_secretsmanager_secret.aws_secret_access_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_secret_access_key-L0DQDr]
module.ooniapi_oonirun_deployer.data.aws_caller_identity.current: Reading...
data.aws_ssm_parameter.do_token: Read complete after 0s [id=/oonidevops/secrets/digitalocean_access_token]
module.ooniapi_oonimeasurements_deployer.aws_iam_policy.codepipeline: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codepipeline-ooniapi-oonimeasurements]
aws_s3_bucket.ooniapi_codepipeline_bucket: Refreshing state... [id=codepipeline-ooniapi-eu-central-1-f148ea7b]
module.ooniapi_reverseproxy_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-reverseproxy]
module.ooniapi_oonirun_deployer.data.aws_caller_identity.current: Read complete after 1s [id=905418398257]
module.ooniapi_oonimeasurements.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonimeasurements-task-role:ooniapi-service-oonimeasurements-task-role]
module.ooniapi_oonirun.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonirun-task-role:ooniapi-service-oonirun-task-role]
module.ooniapi_oonirun_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonirun]
module.adm_iam_roles.aws_secretsmanager_secret_version.oonidevops_deploy_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe|terraform-20240925140131946100000002]
module.ooniapi_ooniauth.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniauth-task-role:ooniapi-service-ooniauth-task-role]
module.ooniapi_oonifindings.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-oonifindings-task-role:ooniapi-service-oonifindings-task-role]
module.adm_iam_roles.aws_iam_role.oonidevops: Refreshing state... [id=oonidevops]
module.ooniapi_ooniauth_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-ooniauth]
aws_route53_record.ooniapi_frontend_cert_validation["ooniprobe.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__a064be8aa084a037ff9fa5e3e541c87d.ooniprobe.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["ooniauth.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__48cd4e71cee9930614228176b7deefb9.ooniauth.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["api.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__cd4729fc0c282e771d056e719a7bdf4f.api.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["8.th.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__ef17825e5fd9713f596344bdd9626f5e.8.th.dev.ooni.io._CNAME]
aws_route53_record.ooniapi_frontend_cert_validation["oonirun.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL__05c891caeb4509d4cd7f9c24d8b6dbd0.oonirun.dev.ooni.io._CNAME]
module.ooniapi_oonifindings_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonifindings]
module.ooniapi_user.aws_iam_access_key.ooniapi: Refreshing state... [id=AKIA5FTZELIYSK2XEVOT]
module.ooniapi_user.aws_iam_user_policy.ooniapi: Refreshing state... [id=oonidevops-ooniapi:oonidevops-ooniapi-policy]
module.ooniapi_cluster.aws_iam_role_policy.container_host: Refreshing state... [id=ooniapi-ecs-cluster-container-host-role:ooniapi-ecs-cluster-instance-role-policy]
module.ooniapi_cluster.aws_iam_instance_profile.container_host: Refreshing state... [id=ooniapi-ecs-cluster]
module.ooniapi_ooniprobe.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-ooniprobe-task-role:ooniapi-service-ooniprobe-task-role]
module.ooniapi_reverseproxy.aws_iam_role_policy.ooniapi_service_task: Refreshing state... [id=ooniapi-service-reverseproxy-task-role:ooniapi-service-reverseproxy-task-role]
data.aws_secretsmanager_secret_version.prometheus_metrics_password: Reading...
aws_secretsmanager_secret_version.prometheus_metrics_password: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/prometheus_metrics_password-M8BbRw|terraform-20240314200140936700000008]
module.oonidevops_github_user.aws_iam_user_policy_attachment.oonidevops_github: Refreshing state... [id=oonidevops-github-20240313195612421500000001]
module.oonidevops_github_user.aws_iam_access_key.oonidevops_github: Refreshing state... [id=AKIA5FTZELIYXDN55SMS]
module.ooniapi_ooniprobe_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-ooniprobe]
module.ooniapi_cluster.aws_ecs_cluster.main: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:cluster/ooniapi-ecs-cluster]
module.ooniapi_oonimeasurements_deployer.aws_iam_role.codepipeline: Refreshing state... [id=codepipeline-ooniapi-oonimeasurements]
module.ooniapi_user.aws_secretsmanager_secret_version.aws_secret_access_key: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_secret_access_key-L0DQDr|terraform-20240314200140914600000006]
module.ooniapi_user.aws_secretsmanager_secret_version.aws_access_key_id: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooniapi_user/aws_access_key_id-EcXOBx|terraform-20240314200140918400000007]
aws_acm_certificate_validation.ooniapi_frontend: Refreshing state... [id=0001-01-01 00:00:00 +0000 UTC]
module.oonidevops_github_user.aws_secretsmanager_secret_version.oonidevops_github: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/github_user/access_key_json-9JTJgd|terraform-20240519071250187000000004]
data.aws_secretsmanager_secret_version.prometheus_metrics_password: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni_services/prometheus_metrics_password-M8BbRw|AWSCURRENT]
aws_codestarconnections_connection.oonidevops: Refreshing state... [id=arn:aws:codestar-connections:eu-central-1:905418398257:connection/6bd492f6-c11d-43ec-92b0-24c47700d528]
data.aws_secretsmanager_secret_version.deploy_key: Reading...
module.ooniapi_ooniauth.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-ooniauth-td]
module.ooniapi_oonirun.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonirun-td]
module.ooniapi_oonifindings.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonifindings-td]
module.ooniapi_reverseproxy.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-reverseproxy-td]
module.ooniapi_oonimeasurements.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-oonimeasurements-td]
module.ooniapi_ooniprobe.aws_ecs_task_definition.ooniapi_service: Refreshing state... [id=ooniapi-service-ooniprobe-td]
module.terraform_state_backend.data.aws_region.current: Reading...
module.terraform_state_backend.data.aws_region.current: Read complete after 0s [id=eu-central-1]
data.aws_secretsmanager_secret_version.deploy_key: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/deploy_key-2ebqSe|AWSCURRENT]
module.network.aws_vpc.main: Refreshing state... [id=vpc-0e382f3ad89286de9]
module.ooni_th_droplet.data.cloudinit_config.ooni_th_docker: Reading...
module.ooni_th_droplet.data.cloudinit_config.ooni_th_docker: Read complete after 0s [id=1719060339]
module.ooni_th_droplet.digitalocean_droplet.ooni_th_docker[0]: Refreshing state... [id=459912318]
module.terraform_state_backend.aws_s3_bucket.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Reading...
module.terraform_state_backend.data.aws_iam_policy_document.bucket_policy[0]: Read complete after 0s [id=2666303363]
module.terraform_state_backend.aws_dynamodb_table.with_server_side_encryption[0]: Refreshing state... [id=oonidevops-dev-terraform-state-lock]
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Reading...
module.terraform_state_backend.data.aws_iam_policy_document.aggregated_policy[0]: Read complete after 0s [id=2666303363]
module.ooniapi_ooniprobe_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-ooniprobe-eu-central-1]
module.ooniapi_oonifindings_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonifindings-eu-central-1]
module.ooniapi_oonimeasurements_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonimeasurements-eu-central-1]
module.ooniapi_oonirun_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-oonirun-eu-central-1]
module.ooniapi_ooniauth_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-ooniauth-eu-central-1]
module.ooniapi_reverseproxy_deployer.aws_iam_policy.codebuild: Refreshing state... [id=arn:aws:iam::905418398257:policy/service-role/codebuild-reverseproxy-eu-central-1]
module.ooniapi_oonifindings_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonifindings]
module.ooniapi_ooniauth_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-ooniauth]
module.ooniapi_oonirun_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonirun]
module.ooniapi_oonimeasurements_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-oonimeasurements]
module.ooniapi_ooniprobe_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-ooniprobe]
module.ooniapi_reverseproxy_deployer.aws_iam_role.codebuild: Refreshing state... [id=codebuild-ooniapi-reverseproxy]
module.ooni_th_droplet.aws_route53_record.ooni_th["0"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_0.do.th.dev.ooni.io_A]
module.ooniapi_oonifindings_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonifindings]
module.ooniapi_oonirun_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonirun]
module.ooniapi_oonimeasurements_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-oonimeasurements]
module.ooniapi_ooniauth_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-ooniauth]
module.ooniapi_ooniprobe_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-ooniprobe]
module.ooniapi_reverseproxy_deployer.aws_codebuild_project.ooniapi: Refreshing state... [id=arn:aws:codebuild:eu-central-1:905418398257:project/ooniapi-reverseproxy]
module.network.aws_internet_gateway.gw: Refreshing state... [id=igw-0c080e9b235ed29d1]
module.ooniapi_oonimeasurements.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OmeaM-20250116160254864500000001/4d88cb32eb2f381c]
module.ooniapi_ooniprobe.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OproM-20250115122624346700000001/9f9264a4e53931d3]
module.ooni_clickhouse_proxy.aws_security_group.ec2_sg: Refreshing state... [id=sg-0903c108a44c922a5]
module.ooniapi_oonirun.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OrunM-20250115122624347100000003/17e1664b99b708a5]
module.ooni_clickhouse_proxy.aws_alb_target_group.ooni_ec2: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oockpr20250116192249626700000002/2e9dada4dd22c268]
module.ooniapi_ooniauth.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OautM-20250115122624347200000004/6e746a968782a49f]
module.ooniapi_reverseproxy.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OrevM-20250115122624347000000002/32c2f9b4e4d3b8c4]
module.ooniapi_oonifindings.aws_alb_target_group.ooniapi_service: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/OfinM-20250115122624350600000005/ad715c6e26dd616c]
module.ooniapi_cluster.aws_security_group.web: Refreshing state... [id=sg-0187eedfe39538357]
module.network.aws_route_table.public: Refreshing state... [id=rtb-0ccb0852e6a365a95]
module.network.aws_route_table.private: Refreshing state... [id=rtb-011463437da96c77b]
module.network.aws_subnet.public[1]: Refreshing state... [id=subnet-0b18966cccfc9d5ef]
module.network.aws_subnet.public[0]: Refreshing state... [id=subnet-0e7a4478be988463f]
module.network.aws_subnet.private[0]: Refreshing state... [id=subnet-09314a43ec89d6331]
module.network.aws_subnet.private[1]: Refreshing state... [id=subnet-0b899a7ad10406d06]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_egress[1]: Refreshing state... [id=sgrule-1281654482]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_egress[0]: Refreshing state... [id=sgrule-1099643652]
module.ooniapi_cluster.aws_security_group.container_host: Refreshing state... [id=sg-0aa6a97400b619de3]
module.terraform_state_backend.aws_s3_bucket_versioning.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.aws_s3_bucket_server_side_encryption_configuration.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.terraform_state_backend.aws_s3_bucket_public_access_block.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooniapi_ooniprobe.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniprobe]
module.ooniapi_oonimeasurements.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonimeasurements]
module.ooniapi_reverseproxy.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-reverseproxy]
module.ooniapi_oonirun.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonirun]
module.ooniapi_oonifindings.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-oonifindings]
module.ooniapi_ooniauth.aws_ecs_service.ooniapi_service: Refreshing state... [id=arn:aws:ecs:eu-central-1:905418398257:service/ooniapi-ecs-cluster/ooniapi-service-ooniauth]
module.network.aws_route_table_association.public[0]: Refreshing state... [id=rtbassoc-0dbd7fb16801ee049]
module.network.aws_route_table_association.public[1]: Refreshing state... [id=rtbassoc-08ab18165bf481054]
module.terraform_state_backend.aws_s3_bucket_policy.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.network.aws_route_table_association.private[0]: Refreshing state... [id=rtbassoc-0e7933e6b804ff2c1]
module.network.aws_route_table_association.private[1]: Refreshing state... [id=rtbassoc-0c9cc0f117ef15fe7]
module.ooniapi_cluster.aws_launch_template.container_host: Refreshing state... [id=lt-0e328a8671f870c64]
module.ooniapi_frontend.aws_alb.ooniapi: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:loadbalancer/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6]
module.ooni_clickhouse_proxy.aws_launch_template.ooni_ec2: Refreshing state... [id=lt-0855bc6373ff4c75b]
module.oonipg.aws_db_subnet_group.pg: Refreshing state... [id=ooni-tier0-postgres-dbsng]
module.terraform_state_backend.time_sleep.wait_for_aws_s3_bucket_settings[0]: Refreshing state... [id=2024-03-10T15:06:17Z]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[0]: Refreshing state... [id=sgrule-1921217342]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[1]: Refreshing state... [id=sgrule-3288936075]
module.oonipg.aws_security_group.pg: Refreshing state... [id=sg-005ca579eb9c08cda]
module.ooni_clickhouse_proxy.aws_security_group_rule.ec2_sg_ingress[2]: Refreshing state... [id=sgrule-3890696530]
module.ooniapi_ooniprobe_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-ooniprobe]
module.terraform_state_backend.aws_s3_bucket_ownership_controls.default[0]: Refreshing state... [id=oonidevops-dev-terraform-state]
module.ooniapi_oonimeasurements_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonimeasurements]
module.ooniapi_reverseproxy_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-reverseproxy]
module.ooniapi_oonirun_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonirun]
module.ooniapi_cluster.aws_autoscaling_group.container_host: Refreshing state... [id=ooniapi-ecs-cluster20240310192644083800000003]
module.ooni_clickhouse_proxy.aws_instance.ooni_ec2: Refreshing state... [id=i-01f355c588856e904]
module.ooniapi_oonifindings_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-oonifindings]
module.ooniapi_frontend.aws_alb_listener.ooniapi_listener_http: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/d9b2448464179cd1]
module.ooniapi_frontend.aws_alb_listener.ooniapi_listener_https: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd]
module.ooniapi_ooniauth_deployer.aws_codepipeline.ooniapi: Refreshing state... [id=ooniapi-ooniauth]
aws_route53_record.ooniapi_frontend_main: Refreshing state... [id=Z055356431RGCLK3JXZDL_api.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["ooniauth.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_ooniauth.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["8.th.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_8.th.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["oonirun.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_oonirun.dev.ooni.io_A]
aws_route53_record.ooniapi_frontend_alt["ooniprobe.dev.ooni.io"]: Refreshing state... [id=Z055356431RGCLK3JXZDL_ooniprobe.dev.ooni.io_A]
module.oonipg.aws_db_instance.pg: Refreshing state... [id=db-27N7Q6XIBNASFCOXN4N7C762L4]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonirun_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/65e6f5e3aca0a4e5]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonimeasurements_rule_2[0]: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/49f4e3329b9b4665]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonifindings_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/cdc4e8e8eabb56f2]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniauth_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/128c53ea760208fc]
module.ooniapi_frontend.aws_alb_listener_rule.ooniapi_th: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/01ee7503374be8ff]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/2b09ed268181ba4f]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonimeasurements_rule_1[0]: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/cd947c873f9a1408]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniprobe_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/b436b91883ae7c86]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonimeasurements_rule_host[0]: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/f3142766eb744476]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_ooniauth_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/cefeff0d8aa3118a]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonirun_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/551c4128bb282fa4]
module.ooniapi_frontend.aws_lb_listener_rule.ooniapi_oonifindings_rule_host: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:listener-rule/app/ooni-tier0-api-frontend/52df1e7ac0eb1ea6/2f500e01e10ba5cd/5a872da4cc6b9135]
aws_route53_record.postgres_dns: Refreshing state... [id=Z091407123AEJO90Z3H6D_postgres.dev.ooni.nu_CNAME]
data.aws_secretsmanager_secret_version.pg_login: Reading...
data.aws_secretsmanager_secret_version.pg_login: Read complete after 0s [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:rds!db-5fe27151-3a37-44e0-a5bd-3517363fa2e8-BDI0KI|AWSCURRENT]
aws_secretsmanager_secret_version.oonipg_url: Refreshing state... [id=arn:aws:secretsmanager:eu-central-1:905418398257:secret:oonidevops/ooni-tier0-postgres/postgresql_url-w62CTZ|terraform-20250122123819245300000001]
module.ooni_clickhouse_proxy.aws_lb_target_group_attachment.oonibackend_proxy: Refreshing state... [id=arn:aws:elasticloadbalancing:eu-central-1:905418398257:targetgroup/oockpr20250116192249626700000002/2e9dada4dd22c268-20250116192309877300000006]
aws_route53_record.clickhouse_proxy_alias: Refreshing state... [id=Z055356431RGCLK3JXZDL_clickhouseproxy.dev.ooni.io_CNAME]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.ooni_clickhouse_proxy.aws_instance.ooni_ec2 will be updated in-place
  ~ resource "aws_instance" "ooni_ec2" {
        id                                   = "i-01f355c588856e904"
        tags                                 = {
            "Environment" = "dev"
            "Name"        = "ooni-tier0-clickhouseproxy"
            "Repository"  = "https://github.com/ooni/devops"
        }
      ~ user_data                            = "bd31c4f073ccfc1109babe7ffeb3ecc1983bd61a" -> "93eac876eb1214179891aaf309d7d19fd41e0c7f"
        # (39 unchanged attributes hidden)

        # (9 unchanged blocks hidden)
    }

  # module.ooniapi_oonifindings_deployer.aws_codepipeline.ooniapi will be updated in-place
  ~ resource "aws_codepipeline" "ooniapi" {
        id             = "ooniapi-oonifindings"
        name           = "ooniapi-oonifindings"
        tags           = {}
        # (5 unchanged attributes hidden)

      - trigger {
          - provider_type = "CodeStarSourceConnection" -> null

          - git_configuration {
              - source_action_name = "Source" -> null

              - push {
                  - branches {
                      - excludes = [] -> null
                      - includes = [
                          - "master",
                        ] -> null
                    }
                }
            }
        }

        # (4 unchanged blocks hidden)
    }

  # module.ooniapi_oonimeasurements_deployer.aws_codepipeline.ooniapi will be updated in-place
  ~ resource "aws_codepipeline" "ooniapi" {
        id             = "ooniapi-oonimeasurements"
        name           = "ooniapi-oonimeasurements"
        tags           = {}
        # (5 unchanged attributes hidden)

      - trigger {
          - provider_type = "CodeStarSourceConnection" -> null

          - git_configuration {
              - source_action_name = "Source" -> null

              - push {
                  - branches {
                      - excludes = [] -> null
                      - includes = [
                          - "richer-analysis",
                        ] -> null
                    }
                }
            }
        }

        # (4 unchanged blocks hidden)
    }

  # module.ooniapi_ooniprobe_deployer.aws_codepipeline.ooniapi will be updated in-place
  ~ resource "aws_codepipeline" "ooniapi" {
        id             = "ooniapi-ooniprobe"
        name           = "ooniapi-ooniprobe"
        tags           = {}
        # (5 unchanged attributes hidden)

      ~ stage {
            name = "Source"

          ~ action {
              ~ configuration    = {
                  ~ "BranchName"           = "luis/ams-probe-services-port" -> "master"
                    # (4 unchanged elements hidden)
                }
                name             = "Source"
                # (10 unchanged attributes hidden)
            }
        }

      - trigger {
          - provider_type = "CodeStarSourceConnection" -> null

          - git_configuration {
              - source_action_name = "Source" -> null

              - push {
                  - branches {
                      - excludes = [] -> null
                      - includes = [
                          - "luis/ams-probe-services-port",
                        ] -> null
                    }
                }
            }
        }

        # (3 unchanged blocks hidden)
    }

  # module.ooniapi_reverseproxy_deployer.aws_codepipeline.ooniapi will be updated in-place
  ~ resource "aws_codepipeline" "ooniapi" {
        id             = "ooniapi-reverseproxy"
        name           = "ooniapi-reverseproxy"
        tags           = {}
        # (5 unchanged attributes hidden)

      - trigger {
          - provider_type = "CodeStarSourceConnection" -> null

          - git_configuration {
              - source_action_name = "Source" -> null

              - push {
                  - branches {
                      - excludes = [] -> null
                      - includes = [
                          - "master",
                        ] -> null
                    }
                }
            }
        }

        # (4 unchanged blocks hidden)
    }

Plan: 0 to add, 5 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't
guarantee to take exactly these actions if you run "terraform apply" now.
Pusher @hellais
Action pull_request
Environment dev
Workflow .github/workflows/check_terraform.yml
Last updated Fri, 24 Jan 2025 15:44:05 GMT

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 24, 2025 •
edited
Loading

Ansible Run Output 🤖

Ansible Playbook Recap 🔍

Ansible playbook output 📖success

Show Execution 

$ ansible-playbook playbook.yml --check --diff -i ../tf/modules/ansible_inventory/inventories/inventory-dev.ini
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'
[WARNING]: Could not match supplied host pattern, ignoring: monitoring.ooni.org
[WARNING]: Could not match supplied host pattern, ignoring: backend-
hel.ooni.org
[WARNING]: Could not match supplied host pattern, ignoring:
clickhouseproxy.dev.ooni.io
[WARNING]: Could not match supplied host pattern, ignoring: notebook1.htz-
fsn.prod.ooni.nu
[WARNING]: Could not match supplied host pattern, ignoring: data1.htz-
fsn.prod.ooni.nu
[WARNING]: Could not match supplied host pattern, ignoring: data3.htz-
fsn.prod.ooni.nu
[WARNING]: Could not match supplied host pattern, ignoring: openvpn-
server1.ooni.io

PLAY [Ensure all hosts are bootstrapped correctly] *****************************
skipping: no hosts matched

PLAY [Deploy monitoring host] **************************************************
skipping: no hosts matched

PLAY [Update monitoring config] ************************************************
skipping: no hosts matched

PLAY [Deploy ooni backend services] ********************************************
skipping: no hosts matched

PLAY [Deploy clickhouse proxy] *************************************************
skipping: no hosts matched

PLAY [Deploy oonidata clickhouse hosts] ****************************************
skipping: no hosts matched

PLAY [Deploy airflow frontend host] ********************************************
skipping: no hosts matched

PLAY [Setup OpenVPN server] ****************************************************
skipping: no hosts matched

PLAY [Deploy notebook host] ****************************************************
skipping: no hosts matched

PLAY RECAP *********************************************************************
Pusher @hellais
Action pull_request
Working Directory
Workflow .github/workflows/check_ansible.yml
Last updated Fri, 24 Jan 2025 15:44:42 GMT

@hellais hellais merged commit 54271ca into main Jan 24, 2025
6 checks passed
@hellais hellais deleted the node-metrics branch January 24, 2025 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
Roadmap
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@hellais