Git commit missing Cargo.lock despite changes being present

[maxdymond]

Minimal reproduction: https://github.com/maxdymond/renovate-33630/

How are you running Renovate?

Self-hosted Renovate

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Gitlab, 38.94.3

Please tell us more about your question or problem

We're running Renovate with a postUpgrade task against a repository with a Cargo virtual workspace (top-level package points at a number of subpackages). The final action of that postUpgrade task is to show changed files:

INFO  Running: ['git', 'status', '--porcelain']
2025-01-15 10:28:59,453 INFO  Changed files:  M .gitlab-ci.yml
<snip 3 files>
 M CHANGELOG.md
 M Cargo.lock
<snip 4 files>

The postUpgrade task has bumped the version number of the package, and so that information needs to be written to the Cargo.lock file.

This correlates in the renovate logfile with trace-level enabled:

{
    "branch": "renovate/all",
    "addedOrModifiedFiles": [
        ".gitlab-ci.yml",
...
        "CHANGELOG.md",
        "Cargo.lock",
...
],
    "msg": "Added or modified files",
    "time": "2025-01-15T10:28:59.603Z",
}
...
{
    "branch": "renovate/all",
    "file": "Cargo.lock",
    "pattern": "**/*",
    "msg": "Post-upgrade file saved",
    "time": "2025-01-15T10:28:59.610Z",
}

However, after this, the commit only ends up with 9 changes:

{
    "msg": "Setting git author name: <snip: author>",
    "time": "2025-01-15T10:28:59.641Z",
}
{
    "msg": "Setting git author email: <snip: author>",
    "time": "2025-01-15T10:28:59.707Z",
}
{
    "branch": "renovate/all",
    "deletedFiles": [],
    "ignoredFiles": [],
    "result": {
        "author": null,
        "branch": "renovate/all",
        "commit": "14a766af26deb8bd851c3c9e839b63993aa1e3ab",
        "root": false,
        "summary": {
            "changes": 9,
            "insertions": 42,
            "deletions": 17
        }
    },
    "msg": "git commit",
    "time": "2025-01-15T10:29:01.740Z",
}

Cargo.lock isn't in .gitignore, so I'm at a loss to explain why Renovate hasn't added it to the Git commit.

The only potentially weird thing here is that the base Renovate code is also trying to update Cargo.lock but doesn't actually do anything:

  "stderr": "    Updating `<registry>` index
     Locking 0 packages to latest compatible versions

From one log line which traces out the config object, I notice that updatedPackageFiles contains all my updated "files", not including Cargo.lock, and updatedArtifacts contains twelve instances of Cargo.lock:

{
    "updatedArtifacts": [
        "Cargo.lock",
        "Cargo.lock",
        "Cargo.lock",
        "Cargo.lock",
        "Cargo.lock",
        "Cargo.lock",
        "Cargo.lock",
        "Cargo.lock",
        "Cargo.lock",
        "Cargo.lock",
        "Cargo.lock",
        "Cargo.lock"
    ],
    "msg": "Updated 12 lock files",
    "time": "2025-01-15T10:25:18.986Z",
}

Is it possible that Renovate is treating Cargo.lock differently and not including it in the commit properly?

Logs (if relevant)

As per question.

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this. Discussions without reproductions are less likely to be converted to Issues.

Please follow these steps:

1. Read our guide on creating a minimal reproduction.
2. Go to our minimal reproduction template repository.
3. Select the Use this template button to create a new repository based on the template.
4. Work on the minimal reproduction in your own repository.
5. Fill out the information in your repository's README.md.
6. Add the link to your reproduction to the first post of your Discussion. If you are not the original author, you can post a new comment with the link.

If you need help with running renovate on your minimal reproduction repository, please refer to our Running Renovate guide.
Good luck,

The Renovate team

[rarkins]

It's OK to have a reproduction which doesn't use the hosted bot. In that case please provide in the repo:

• minimal code/config to reproduce (per above instructions)
• additional config.js with minimal global config (e.g. allowed fields, etc)

[maxdymond]

@rarkins I think I've diagnosed the bug by adding logs to the Renovate source; is it worth me posting that information instead? I'm unsure I can recreate this repo as is.

[rarkins]

Sure, please go ahead

[maxdymond]

@rarkins Cool.

So the issue here is that the initial pass from Renovate generates a number of Cargo.lock entries in the updatedArtifacts array, as seen above. I don't think postUpgradeCommandsExecutor is written with this in mind;

renovate/lib/workers/repository/update/branch/execute-post-upgrade-commands.ts

Line 143 in 1f4790c

const existingUpdatedArtifacts = updatedArtifacts.find(

does a updatedArtifacts.find, which only returns the first instance of the artifact in the updatedArtifacts array, and updates the contents of that artifact with the post upgrade content. However, the remaining instances of the artifact in the array are not updated!

I can verify this with a loop running before and after merging the postUpgrade file contents into updatedArtifacts:

          const hash = crypto.createHash('sha256').update(artifact.contents).digest('hex');

          logger.info({
            artifactPath: artifact.path,
            hash,
          }, 'MD3: Premerge artifact');

=>

 INFO: MD3: Premerge artifact (repository=[repo]], branch=renovate/all)
       "artifactPath": "Cargo.lock",
       "hash": "a054815a908f8572a286cad461592b7250c3b32445f8114b78cd1e472d68d78e"
 INFO: MD3: Premerge artifact (repository=[repo]], branch=renovate/all)
       "artifactPath": "Cargo.lock",
       "hash": "a054815a908f8572a286cad461592b7250c3b32445f8114b78cd1e472d68d78e"
[repeated several times]

...skip...

       INFO: MD3: Postmerge artifact (repository=[repo]], branch=renovate/all)
       "artifactPath": "Cargo.lock",
       "hash": "a558f8464e8e861dcaf4c5a3c455f2b6674943e6980dc46b7a7d62f72f864781"  <<<<< note a different hash
 INFO: MD3: Postmerge artifact (repository=[repo]], branch=renovate/all)
       "artifactPath": "Cargo.lock",
       "hash": "a054815a908f8572a286cad461592b7250c3b32445f8114b78cd1e472d68d78e"  <<<<< same hash as before

These are then passed to the Git code, which prepares the commit:

renovate/lib/util/git/index.ts

Line 1053 in 1f4790c

if (typeof file.contents === 'string') {

which copies the file contents in from the passed CommitFilesConfig.

I think the upshot of this is that if any entries in the updatedArtifacts array are duplicated by the stock Renovate process, and then modified by a postUpgradeTask, that the modifications are lost entirely.

I think the desired behaviour here probably is that updatedArtifacts is deduplicated before passing it to postUpgradeTasks, but I'm guessing.

(Equally this might have already been fixed by a later version, but it's not obvious that that has happened)

[maxdymond]

@rarkins have added a minimal reproduction repository at https://github.com/maxdymond/renovate-33630/

[github-actions[bot]]

Hi there,

We appreciate the effort you put into creating a reproduction. But your reproduction is not minimal yet, or is missing fields we ask for in our template.

Please re-read the steps and documentation given in the previous message, and try again to create a minimal reproduction.

Thanks, the Renovate team

[maxdymond]

Hi,

I'm confused, sorry. From your guide:

1. Put your minimal reproduction repository on GitHub, only use GitLab or Bitbucket if needed: ✅
2. Use the fewest number of repository files and dependencies: ✅ there are no files in there that are unnecessary for triggering this issue apart from LICENSE, which is from the fork of the minimal production repository.
3. Reduce your Renovate config to a minimum: ✅ the configuration I've given you is what triggers the grouping of Cargo.lock instances before postUpgradeTasks are called.
4. Remove private or secret information: ✅ there's no private or secret information
5. Create a readme.md file (or edit the template readme.md) and: ✅
6. Explain the Current behavior and Expected behavior ✅ I've detailed the current and expected behaviour
7. Link to the Renovate Issue or Discussion ✅ there's a link to this discussion
8. Use headings to organize the information ✅ there are headings
9. Set the repository visibility to public ✅ the repository is public
10. Give us the link to the repository ✅ the link is in the top post.

Is the issue here that the first header isn't just a number, rather than being "Renovate discussion 33630"? If so, wow. I will update that.

[rarkins]

The config is complicated and not minimal at all. If it's truly required to have grouping, modified commit messages etc then this is an extremely narrow edge case not worth time investigating further because it won't affect other users

[maxdymond]

The modified commit messages are the only way I've found so far to group together lockfile maintenance with patch/minor/major updates, but I will take a look at whether I can cut this down further.

[maxdymond]

It doesn't appear as though I can cut this down further, so it looks like from the discussion above that we'll just have to patch it ourselves in our local copy.

[maxdymond]

In case anyone in future hits this, this is the patch I've gone with for this issue: 0002-deduplicate-artifacts.patch

You get the following output with DEBUG logging:

DEBUG: Updated 7 package files (repository=[repo], branch=renovate/all)
DEBUG: Adding unique artifact (repository=[repo], branch=renovate/all)
       "uniqueItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Ignoring duplicate artifact (repository=[repo], branch=renovate/all)
       "duplicateItem": {"type": "addition", "path": "Cargo.lock", "contents": "[content]"}
DEBUG: Updated 1 lock files (repository=[repo], branch=renovate/all)
       "updatedArtifacts": ["Cargo.lock"]

but you only get the one instance of updatedArtifacts which is then passed to postUpgradeTasks.