Failed to properly parse terraform dependency in terragrunt.hcl

[mblum14]

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

CLI 39.86.4

Please tell us more about your question or problem

https://github.com/mblum14/renovate-reproduction-33370

I have the following terragrunt file:

# terragrunt.hcl
terraform {
  source = "git::ssh://[email protected]/myorg/my-terraform-module.git//sub/directory?rev=v2.0.0"
}

The problem is the parsed packageName does not contain the GitHub domain, so git tags cannot be looked up to determine if newer version of myorg/my-terraform-module are available. The parsed package name is null/myorg/my-terraform-module.git when I would expect it to have been ssh://[email protected]/myorg/my-terraform-module.git.

Abbreviated renovate output from scanning the project file:

{
  "repositories": {
    "myorg/my-terragrunt-repo": {
      "problems": [
        {
          "name": "renovate",
          "level": 40,
          "url": "null/myorg/my-terraform-module.git",
          "msg": "Attempting to use non-git url for git operations"
        },
        {
          "name": "renovate",
          "level": 40,
          "warnings": [
            "Failed to look up git-tags package null/myorg/my-terraform-module.git"
          ],
          "files": ["terragrunt.hcl"],
          "msg": "Package lookup failures"
        }
      ],
      "branches": [],
      "packageFiles": {
        "terragrunt": [
          {
            "deps": [
              {
                "depType": "gitTags",
                "depName": "my.enterprise.github/myorg/my-terraform-module",
                "currentValue": "v2.0.0",
                "datasource": "git-tags",
                "packageName": "null/myorg/my-terraform-module.git",
                "update": [],
                "versioning": "server-coerced",
                "warnings": [
                  {
                    "topic": "null/myorg/my-terraform-module.git",
                    "message": "Failed to look up git-tags package null/myorg/my-terraform-module.git"
                  }
                ]
              }
            ]
          }
        ]
      }
    }
  }
}

Logs (if relevant)

[!NOTE]
The system I'm working on is on an air-gapped network, so copying the entire log isn't exactly do-able but I'll try to include the relevant bits

Logs

DEBUG: Matched 1 file(s) for manager terragrunt: terragrunt.hcl
DEBUG: Found terragrunt package files (repository=myorg/my-terragrunt-repo)
DEBUG: Found terragrunt package files (repository=myorg/my-terragrunt-repo)
 INFO: Dependency extraction complete (repository=myorg/my-terragrunt-repo, baseBranch=master)
       "stats": {
         "managers": {
           "terraform-version": {"fileCount": 1, "depCount": 1},
           "terragrunt": {"fileCount": 1, "depCount": 1},
           "terragrunt-version": {"fileCount": 1, "depCount": 1}
         },
         "total": {"fileCOunt": 3, "depCount": 3}
       }
DEBUG: Datasource unknown error (repository=myorg/my-terragrunt-repo)
       "datasource": "git-tags",
       "packageName": "null/myorg/my-terraform-module.git",
       "err": {
         "task": {
           "commands": [
             "ls-remote",
             "null/myorg/my-terraform-module.git"
           ],
           "format": "utf-8",
           "parser": "[function]"
         },
         "message": "fatal: 'null/myorg/my-terraform-module.git' does not appear to be a git repository\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n",
         "stack": "Error: fatal: 'null/myorg/my-terraform-module.git' does not appear to be a git repository\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\n\n.   at Object.action (/alt/.volta/tools/image/packages/renovate/lib/node_modules/renovate/node_modules/simple-git/src/lib/plugins/error-detection.plugin.ts:42:29)\n.   at PluginStore.exec (/alt/.volta/tools/image/packages/renovate/lib/node_modules/renovate/node_modules/simple-git/src/lib/plugins/plugin-store.ts:54:29)\n.   at /alt/.volta/tools/image/packages/renovate/lib/node_modules/renovate/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:124:42\n.   at new Promise (<anonymous>)\n.   at GitExecutorChain.handleTaskData (/alt/.volta/tools/image/packages/renovate/lib/node_modules/renovate/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:121:14)\n.   at GitExecutorChain.<anonymous> (/alt/.volta/tools/image/packages/renovate/lib/node_modules/renovate/node_modules/simple-git/src/lib/runners/git-executor-chain.ts:97:40)\n.   at Generator.newx (<anonymous>)\n    at fulfilled (/alt/.volta/tools/image/packages/renovate/lib/node_modules/renovate/node_modules/simple-git/dist/cjs/index.js:52:24)"
DEBUG: Failed to look up git-tags package null/myorg/my-terraform-module.git (repository=myorg/my-terragrunt-repo, packageFile=terragrunt.hcl, dependency=null/myorg/my-terraform-module.git)

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

Please follow these steps:

1. Read our guide on creating a minimal reproduction.
2. Go to our minimal reproduction template repository.
3. Select the Use this template button to create a new repository based on the template.
4. Work on the minimal reproduction in your own repository.
5. Fill out the information in your repository's README.md.
6. Add the link to your reproduction to the first post of your Discussion. If you are not the original author, you can post a new comment with the link.

If you need help with running renovate on your minimal reproduction repository, please refer to our Running Renovate guide.
Good luck,

The Renovate team

[mblum14]

issue appears to be in lib/modules/manager/terragrunt/modules.ts#L90. When the URL path contains double slash (e.g., //), it sets the dependency packageName to ${origin}${pathname.split('//')[0]}. In the case where the protocol isn't http or https (e.g., ssh or file), origin is 'null'.

[rarkins]

Which logic/reasoning are you using to say that git::ssh://[email protected]/hashicorp/example.git//subdir/test?ref=v1.0.4 should be converted to ssh://mygit.com/hashicorp/example.git?

I get that the git:: prefix can be removed, but why then does the git@ part get stripped from in the middle?

[mblum14]

I believe I used an older version of renovate, 37.280.0, and that was the behavior. One of our sister teams is using that version and has similarly formatted terraform source URLs. When I used that version of renovate, if my memory serves me correctly, that was the format that came out.

I don't have any strong convictions that ssh://mygit.com/hashicopr/example.git is ultimately the correct format. If the git@ should remain and doesn't break renovate from perusing the parsed repo for additional dependencies, I suppose that works for me.

[rarkins]

3/3 tests in https://github.com/renovatebot/renovate/blob/main/lib/modules/manager/terragrunt/extract.spec.ts use ssh://git@... (which also means that a similar pattern works sometimes/somehow)j

Please update the reproduction accordingly

[mblum14]

sorry, the issue isn't the ssh://git@... part, it's that plus the URL path containing a double slash. I have a branch on my fork where I add the additional examples for the specs as well as what I believe will fix the issue

[mblum14]

Additionally, the origin property, as parsed by the NodeJS URL standard library, consists of the protocol and domain. However, according to the URL standard library documentation, the origin only parses when the protocol is either http or https. This particular code path that uses the URL standard library is only token when the double slashes are present in the terraform source URL.

[avidspartan1]

Bumping this discussion - it still has the auto:reproduction tag, but mblum14 added the reproduction link to his original description. Is there another step to take here?

https://github.com/mblum14/renovate-reproduction-33370

[rarkins]

Issue creation is for repository maintainers only. I have deleted your conversion, please don't do it again.