Library not added to "DB" when no valid releases are available

[per1234]

Describe the problem

Metadata for the libraries is stored persistently by the sync command in a JSON file (referred to as "DB" in the codebase).

The file consists of two arrays:

• Libraries: list of the libraries
• Releases: list of all releases of all libraries

The sync command only generates an entry in the Libraries array when there is at least one valid release for the library. Although we require at least one valid release to be present at the time the library is registered, it is still possible for this condition to occur:

• The library maintainer may have deleted the tag for the valid release in the window of time between the registration and the first run of the scheduled indexer job (this likely occurs due to them taking action on a best practices warning about the library name from Arduino Lint that is shown in the comment from the bot on the submission PR).
• The library maintainer requested a release removal for all valid release.

This is problematic because it means that maintenance commands for the library will fail with an ambiguous and confusing error message:

$ libraries-repository-engine modify --repo-url https://example.com FooLib
2025/07/04 07:55:35 Loaded 8141 libraries from DB
error: Library of name FooLib not found

$ libraries-repository-engine modify --types Contributed FooLib
2025/07/04 07:56:49 Loaded 8141 libraries from DB
error: Library of name FooLib not found

$ libraries-repository-engine remove FooLib
2025/07/04 07:53:18 Loaded 8141 libraries from DB
error: Library name FooLib not found

🐛 There is no way to know whether the command failed due to an error, or was expected to fail due to a lack of a valid release in the library repository.

To reproduce

Setup

$ DEMO_FOLDER=\"/tmp/libraries-repository-engine-demo\"

$ mkdir \"$DEMO_FOLDER\"

$ curl -fsSL https://raw.githubusercontent.com/arduino/arduino-lint/main/etc/install.sh | BINDIR=\"$DEMO_FOLDER\" sh

$ echo "
{
  \"BaseDownloadUrl\": \"http://www.example.com/\",
  \"LibrariesFolder\": \"$DEMO_FOLDER/libraries/\",
  \"LogsFolder\": \"$DEMO_FOLDER/logs/\",
  \"LibrariesDB\": \"$DEMO_FOLDER/libraries_db.json\",
  \"LibrariesIndex\": \"$DEMO_FOLDER/library_index.json\",
  \"GitClonesFolder\": \"$DEMO_FOLDER/gitclones\",
  \"DoNotRunClamav\": true,
  \"ArduinoLintPath\": \"$DEMO_FOLDER/arduino-lint\"
}
" > "$DEMO_FOLDER/config.json"

$ echo "
https://github.com/arduino-libraries/Servo.git|Arduino|Servo
https://github.com/arduino-libraries/LuckyShield.git|Arduino|Arduino_LuckyShield
" > "$DEMO_FOLDER/registry.txt"

$ touch "$DEMO_FOLDER/libraries_db.json"

Demo

$ libraries-repository-engine sync --config-file "$DEMO_FOLDER/config.json" "$DEMO_FOLDER/registry.txt"

[...]

2025/07/04 15:22:19 Scraping https://github.com/arduino-libraries/LuckyShield.git
2025/07/04 15:22:21 Checking out tag: v1.0.0
2025/07/04 15:22:21 Release Lucky Shield:1.0.0 has wrong library name, should be Arduino_LuckyShield
2025/07/04 15:22:21 Checking out tag: v1.0.2
2025/07/04 15:22:21 Release Lucky Shield:1.0.2 has wrong library name, should be Arduino_LuckyShield
2025/07/04 15:22:21 Checking out tag: v1.0.3
2025/07/04 15:22:21 Release Lucky Shield:1.0.3 has wrong library name, should be Arduino_LuckyShield

2025/07/04 08:22:21 Completed worker!
2025/07/04 15:22:19 Scraping https://github.com/arduino-libraries/Servo.git

[...]

2025/07/04 08:22:30 Completed worker!
2025/07/04 08:22:30 ...DONE

$ cat "$DEMO_FOLDER/libraries_db.json"

{
  "Libraries": [
    {
      "Name": "Servo",
      "Repository": "https://github.com/arduino-libraries/Servo.git",
      "LatestCategory": "Device Control"
    }
  ],
  "Releases": [

    [...]

  ]
}

$ libraries-repository-engine modify --config-file "$DEMO_FOLDER/config.json" --repo-url https://example.com Arduino_LuckyShield

2025/07/04 08:30:13 Loaded 1 libraries from DB
error: Library of name Arduino_LuckyShield not found

$ libraries-repository-engine modify --config-file "$DEMO_FOLDER/config.json" --types Contributed Arduino_LuckyShield

2025/07/04 08:31:12 Loaded 1 libraries from DB
error: Library of name Arduino_LuckyShield not found

$ libraries-repository-engine remove --config-file "$DEMO_FOLDER/config.json" Arduino_LuckyShield

2025/07/04 08:28:46 Loaded 1 libraries from DB
error: Library name Arduino_LuckyShield not found

Expected behavior

Correctly configured modify and remove commands are successful even when the library does not have any valid releases.

This can be achieved by changing the behavior of the sync command so that it always adds an entry to the Libraries array in the "DB", even in the case where the library doesn't have any valid releases.

libraries-repository-engine version

22d4c14

Additional context

Example of the current behavior causing confusion:

arduino/library-registry#4978 (review)

Issue checklist

• I searched for previous reports in the issue tracker
• My report contains all necessary details