Skip to content

Make examples/ uptestable and E2E test them - #206

Merged
fernandezcuesta merged 5 commits into
crossplane-contrib:mainfrom
jonasz-lasut:uptestable-examples
Oct 15, 2026
Merged

fernandezcuesta merged 5 commits into
crossplane-contrib:mainfrom
jonasz-lasut:uptestable-examples

Conversation

@jonasz-lasut

@jonasz-lasut jonasz-lasut commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Description of your changes

Makes the examples in examples/ runnable with uptest, alone or in one batch, and fixes the provider issues the e2e runs uncovered.

Builds on #201, cherry-picked here as b7c3f2c: merge #201 first or supersede it with this PR
Fixes #196

Provider

  • Adds references for fields whose Terraform examples used expressions: service account clientId, IAM role IDs, cluster names, teamId, stream processor workspaceName, x509 username and query limit tenantName. Fixes the OutageSimulation clusterName reference, which
    resolved to <projectId>-<name>.
  • References to service accounts, database users and federated database instances read the value from the target's status or spec. Their external name starts as metadata.name, and a selector that resolved early cached that name. The outage simulation waits for the cluster's
    observed name: a simulation started during creation keeps the cluster UPDATING, and its create never finishes.
  • Runs upjet's cmd/resolver for both scopes (internal/apis), so cross-group references no longer form import cycles.
  • Federated database instances no longer loop on unknown cloud_provider_config and data_process_region diffs.
  • The third-party integration uses the Atlas integration ID as its external name, which ends a diff on id at every observe.
  • Fix: a new providerAssignedID() external name (in config/resources/external_name.go) for Project, ResourcePolicy and the encryption-at-rest PrivateEndpoint, which had the same problem. The external name now stays empty until Create, so references wait for the real ID. Reads with an empty ID count as "not found", so Create runs.

Examples

  • Project, organization and user IDs come from the uptest datasource, as do the AWS container region and the Datadog credentials. The keys are listed in the Makefile.
  • Every selector points to an object of the right kind in the same file. Names and labels are prefixed with the file name, so a batch run cannot collide. Clusters use the cheapest tier the resource supports, random names and uptest timeouts.
  • Test hooks load the Atlas sample datasets (Atlas CLI) for OnlineArchive and search Index, and delete the search Deployment before its cluster.
  • 23 examples that need infrastructure outside Atlas, or cannot be cleaned up, carry upjet.upbound.io/manual-intervention with the reason.
  • make generate copies only new generated examples, so it no longer overwrites examples/.
  • Bumps golangci-lint to v2.14.0, which reads Go 1.27 export data.

I have:

  • Read and followed Crossplane's contribution process.
  • Run make reviewable test to ensure this PR is ready for review.

How has this code been tested

  • make generate, make lint and go test ./config/... ./internal/....
  • uptest e2e of the cluster-scoped examples against a test Atlas project:

Batch 1 (no clusters):

  • ai/v1alpha1/modelapikey.yaml
  • ai/v1alpha1/modelratelimit.yaml
  • alert/v1alpha1/configuration.yaml
  • cloud/v1alpha1/provideraccesssetup.yaml
  • cloud/v1alpha1/userprojectassignment.yaml
  • database/v1alpha1/customrole.yaml
  • database/v1alpha1/x509userauthentication.yaml
  • database/v1alpha3/user.yaml
  • federated/v1alpha1/databaseinstance.yaml
  • federated/v1alpha1/querylimit.yaml
  • mongodbatlas/v1alpha1/auditing.yaml
  • mongodbatlas/v1alpha1/customdnsconfigurationclusteraws.yaml
  • mongodbatlas/v1alpha1/maintenancewindow.yaml
  • network/v1alpha1/container.yaml
  • privateendpoint/v1alpha1/resource.yaml
  • project/v1alpha1/invitation.yaml
  • project/v1alpha1/ipaccesslist.yaml
  • project/v1alpha1/serviceaccount.yaml
  • project/v1alpha1/serviceaccountaccesslistentry.yaml
  • project/v1alpha1/serviceaccountsecret.yaml
  • stream/v1alpha1/instance.yaml
  • stream/v1alpha1/processor.yaml
  • stream/v1alpha1/workspace.yaml

Batch 2 (clusters and datadog):

  • cloud/v1alpha1/backupschedule.yaml
  • cloud/v1alpha1/backupsnapshot.yaml
  • cloud/v1alpha1/backupsnapshotrestorejob.yaml
  • cluster/v1alpha1/outagesimulation.yaml
  • mongodbatlas/v1alpha1/employeeaccessgrant.yaml
  • mongodbatlas/v1alpha1/flexcluster.yaml
  • mongodbatlas/v1alpha1/globalclusterconfig.yaml
  • mongodbatlas/v1alpha1/metricintegration.yaml
  • mongodbatlas/v1alpha1/onlinearchive.yaml
  • mongodbatlas/v1alpha1/partyintegration.yaml
  • mongodbatlas/v1alpha2/cluster.yaml
  • mongodbatlas/v1alpha3/advancedcluster.yaml
  • search/v1alpha1/deployment.yaml
  • search/v1alpha1/index.yaml
  • stream/v1alpha1/connection.yaml

Batch 3 (organization level MRs):

  • cloud/v1alpha1/userorgassignment.yaml
  • cloud/v1alpha1/userteamassignment.yaml
  • mongodbatlas/v1alpha1/accesslistapikey.yaml
  • mongodbatlas/v1alpha1/apikey.yaml
  • mongodbatlas/v1alpha1/apikeyprojectassignment.yaml
  • mongodbatlas/v1alpha1/project.yaml
  • mongodbatlas/v1alpha1/resourcepolicy.yaml
  • mongodbatlas/v1alpha1/serviceaccount.yaml
  • mongodbatlas/v1alpha1/serviceaccountaccesslistentry.yaml
  • mongodbatlas/v1alpha1/serviceaccountprojectassignment.yaml
  • mongodbatlas/v1alpha1/serviceaccountsecret.yaml
  • mongodbatlas/v1alpha1/team.yaml
  • org/v1alpha1/invitation.yaml
  • project/v1alpha1/apikey.yaml
  • team/v1alpha1/projectassignment.yaml

Omitted the following due to lack of required integrations:

  • cloud/v1alpha1/backupcollectionrestorejob.yaml: The source cluster must hold the restored collections (sample_mflix) in a completed snapshot, which this provider cannot load.
  • cloud/v1alpha1/backupcompliancepolicy.yaml: Once enabled, a Backup Compliance Policy can only be disabled by MongoDB Support, so the test project could not be cleaned up.
  • cloud/v1alpha1/backupsnapshotexportbucket.yaml: Needs an existing S3 bucket and an AWS IAM role that trusts the Atlas AWS account.
  • cloud/v1alpha1/backupsnapshotexportjob.yaml: Needs an existing S3 bucket and an AWS IAM role that trusts the Atlas AWS account.
  • cloud/v1alpha1/provideraccessauthorization.yaml: Needs an AWS IAM role that trusts the Atlas AWS account with the external ID from ProviderAccessSetup.
  • encryptionatrest/v1alpha1/privateendpoint.yaml: Needs an Azure Key Vault with a key and an Azure service principal in the referenced Secrets.
  • federated/v1alpha1/orgconfigsettings.yaml: Can only be imported: the organization must already be connected to the federation with an identity provider.
  • federated/v1alpha1/privatelinkendpointservice.yaml: Needs an AWS interface VPC endpoint created against the Atlas endpoint service.
  • federated/v1alpha1/rolemapping.yaml: Needs an organization connected to federation settings with an identity provider.
  • federated/v1alpha1/settingsidentityprovider.yaml: SAML identity providers can only be imported, not created.
  • ldap/v1alpha1/configuration.yaml: Needs a reachable LDAP server and bind credentials in the referenced Secret.
  • ldap/v1alpha1/verify.yaml: Needs a reachable LDAP server and bind credentials in the referenced Secret.
  • mongodbatlas/v1alpha1/encryptionatrest.yaml: Needs an AWS KMS key and an AWS IAM role that trusts the Atlas AWS account.
  • mongodbatlas/v1alpha1/eventtrigger.yaml: Needs an App Services app, function and linked data source.
  • mongodbatlas/v1alpha1/logintegration.yaml: Needs an existing S3 bucket and an AWS IAM role that trusts the Atlas AWS account.
  • mongodbatlas/v1alpha1/organization.yaml: Creating organizations needs a paying organization with cross-organization billing and an API key allowed to create organizations.
  • mongodbatlas/v1alpha1/pushbasedlogexport.yaml: Needs an existing S3 bucket and an AWS IAM role that trusts the Atlas AWS account.
  • network/v1alpha1/peering.yaml: Needs an AWS VPC and account to peer with.
  • privateendpoint/v1alpha1/regionalmode.yaml: Needs an AWS interface VPC endpoint created against the Atlas endpoint service.
  • privateendpoint/v1alpha1/service.yaml: Needs an AWS interface VPC endpoint created against the Atlas endpoint service.
  • serverless/v1alpha1/instance.yaml: Atlas no longer creates Serverless instances; use FlexCluster instead.
  • stream/v1alpha1/connectionfailover.yaml: Needs reachable Kafka brokers and credentials in the referenced Secret.
  • stream/v1alpha1/privatelinkendpoint.yaml: Needs a Confluent Cloud PrivateLink network.

jonasz-lasut and others added 3 commits October 13, 2026 10:11
The Terraform-converted examples pointed projectIdSelector and
orgIdSelector at labels that no Project or Organization in the same
file carries, or hardcoded Atlas IDs, so uptest could not run them.
Take the project and organization from the uptest datasource keys
mongodbatlas.project_id and mongodbatlas.org_id instead, in both
scopes, and document the keys in the Makefile and CLAUDE.md.

Signed-off-by: Jonasz Małecki <jonasz@upbound.io>
…import IDs

- project_api_key: use api_key_id only (project IDs are only in project_assignment)
- cloud_user_team_assignment: template uses user_id; add userId reference to UserOrgAssignment
- resource_policy, encryption_at_rest_private_endpoint: use the plain provider-assigned ID
- ai_model_api_key, cloud_backup_collection_restore_job, metric_integration,
  stream_connection_failover, log_integration: new computedKeyID constructor for
  plugin-framework resources without an id attribute
- ai_model_rate_limit: read the external name from model_group_name
- x509_authentication_database_user: add username to the state ID
- cluster: add provider_name to the state ID
- encodedStateGetIDFn: decode external names written as full state IDs by v1.1.x
- remove GetImportIDFn and the separator argument; the no-fork connectors never run terraform import
- refs.ExtractParamPath: rename the second parameter to isObservation

Signed-off-by: Jesús Fernández <7312236+fernandezcuesta@users.noreply.github.com>
(cherry picked from commit 89194cc)
Signed-off-by: Jonasz Małecki <jonasz@upbound.io>
Make every example runnable with uptest, alone or in one batch:

- Add the missing cross-resource references the Terraform examples
  expressed as expressions (service account client_id, IAM role IDs,
  cluster names, team_id, processor workspace_name, x509 username,
  query limit tenant_name) and fix the OutageSimulation cluster_name
  reference, which resolved to "<projectId>-<name>".
- Resolve references through a runtime scheme (upjet cmd/resolver,
  internal/apis) so cross API-group references cannot form import
  cycles.
- make generate now copies only new generated examples, so it no
  longer overwrites the hand-maintained examples/.
- Rewrite the examples: datasource project, org and user IDs, the
  cheapest cluster tier each resource supports, random cluster names
  and emails, in-file targets for every selector, file-prefixed names
  and labels, uptest timeouts, and manual-intervention on examples
  that need infrastructure outside Atlas or cannot be cleaned up.
- Add a pre-assert hook that loads the Atlas sample datasets for the
  OnlineArchive and search Index examples, and drop the stale metric
  integration example.
- Bump golangci-lint to v2.14.0, which reads Go 1.27.2 export data.

Signed-off-by: Jonasz Małecki <jonasz@upbound.io>
@jonasz-lasut
jonasz-lasut marked this pull request as ready for review October 13, 2026 18:48
Selectors resolved before their target was created picked up the
target's metadata.name, because service accounts, database users and
federated database instances start with the name initializer's
external name, and the value is cached once resolved. Read
service-account client_id from status.atProvider and the x509 username
and query limit tenant_name from the target's spec instead. The outage
simulation reads its cluster name from the cluster's observation, so
the simulation starts only after the cluster is created: a simulation
started earlier keeps the cluster UPDATING and its create never ends.

Project, resource policy and encryption at rest private endpoint
external names stay empty until Create instead of starting as
metadata.name. References to a Project cached that placeholder as the
project ID, and Atlas answers a resource policy read with it with 400
instead of 404, so the policy was never created. The Atlas SDK rejects
the empty ID client-side, which now counts as not found.

Federated database instances never became up to date: Terraform Read
sets cloud_provider_config and data_process_region only when Atlas
returns them, so every plan reported them as unknown. Drop those
unknown-only diffs while the blocks are unset in config and state.

The third-party integration uses the provider-assigned integration ID
as external name. Terraform Read sets the ID to that value, so the
templated <project_id>-<type> name caused a diff on every observe.

Example fixes from the e2e runs:
- The network container takes its region from the new
  mongodbatlas.aws_container_region datasource key, so it does not
  collide with an existing AWS container in the test project.
- The global cluster config uses an M30 cluster because Atlas does not
  shard M10 clusters.
- A pre-delete hook deletes the search Deployment before its cluster:
  once the cluster is gone, Atlas answers search deployment calls with
  400 ATLAS_SEARCH_DEPLOYMENT_INVALID_ATTRIBUTE and the Deployment
  never finalizes.
- The outage simulation runs on a two-region cluster, as Atlas no
  longer supports simulating an outage on a single-region cluster, and
  uptest asserts only Synced for that cluster while it is UPDATING.
- The snapshot restore job uses download delivery, so it does not
  overwrite its cluster, and the snapshot example no longer repeats it.
- The sample data hook reads the load job ID from _id, the field the
  Atlas CLI returns.
- The Datadog metric and third-party integrations take the API key,
  region and OTLP endpoint from new datadog.* datasource keys and
  create their own Secret, so they no longer need manual intervention.

Signed-off-by: Jonasz Małecki <jonasz@upbound.io>
Comment thread docs/import.md Resolved
…able

Signed-off-by: Jonasz Małecki <jonasz@upbound.io>
@fernandezcuesta

Copy link
Copy Markdown
Collaborator

Thanks for your huge effort here 🥇

@fernandezcuesta
fernandezcuesta merged commit a5aaa05 into crossplane-contrib:main Oct 15, 2026
9 checks passed
@jonasz-lasut
jonasz-lasut deleted the uptestable-examples branch October 15, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

(bug): Examples in examples/ are not uptestable

2 participants