Proposal: Scheduled Updater scaffolded workflow instead of Renovate

[davereid]

On another project I wrote a "Scheduled updater" workflow that ran Thursday mornings or on-demand as needed. It would do the following:

• Start up DDEV
• Create a new 1 day backup in Pantheon
• Sync the new Pantheon backup into DDEV
• Run an "update" command which ran:
  ** composer update composer/composer (if the dependency is required by anything, ideally not, because if composer is not updated first, scaffolding files fail
  ** composer update
  ** npm update
  ** drush updatedb --yes
  ** npm run build --if-available
  ** drush features:import:all --yes
  ** drush config:export --yes
• Pushes the code changes to a pull request using a GitHub app

The benefits we found were not having to manage several individual PRs for each dependency update. It's an all or nothing, which is a strong choice.

I want to start discussions with the Lullabot Support and Maintenance team about this to see if this would help solve a need by providing this into Drainpipe for all to have available if needed, and would reduce the need for Renovate entirely.

[deviantintegral]

One benefit of Renovate is that it also handles yarn, Dockerfiles, docker compose, github actions, and more.

The benefits we found were not having to manage several individual PRs for each dependency update. It's an all or nothing, which is a strong choice.

Personally, I think that individual PRs with smaller changes are a better practice. It makes it easier to test and merge, since there's fewer changes. Once you start re-batching updates together, it pushes teams towards delaying updates entirely.

[davereid]

Yeah, I think there's a benefit to smaller update PRs if you have all the automated, regression, and visual diff testing able to auto-approve things. There's also issues with multidev limits on Pantheon where it was a 10 or 20 multidev limit, where if things aren't getting merged promptly, then it can cause development to stall. Plus being able to capture the configuration exports from update hooks I think is a huge pro. I think we could make it available as an alternate update path that consumers could opt-in to using and not something that's forced on everyone.

[marcoscano]

If the updates happen fairly regularly (eg once a week), I actually think having a single PR is more manageable from a QA standpoint, since issues caught by automated/manual QA wouldn't be too hard to troubleshoot. If the updates happen once a month or less frequently, I can see how this could become an issue.

Plus being able to capture the configuration exports from update hooks I think is a huge pro

IMO this is a game changer, and it's why I would love to have this option

I think we could make it available as an alternate update path that consumers could opt-in to using and not something that's forced on everyone

Huge +1 for opt-in

[penyaskito]

At Iowa we are using Site schema that blocks any update with schema changes. LSM devs then create a ticket and a different PR for updating those and do more extensive QA.

We have #529 here to integrate part of it on drainpipe.

What if... when a renovate PR detects config schema changes, THEN it launches what you are proposing here for that single dependency, with the updates to the Site schema and a config export?

I'm pretty sure that would save LSM a ton of time, while still having the benefits you described + the benefits of smaller and traceable updates.

[deviantintegral]

Based on our various projects using Renovate, I'm going to close this issue as "won't fix". A few reasons:

• Renovate continues to work well across a wide variety of projects.
• Renovate itself is open-source, and also well maintained.
• Renovate can work with multiple VCS forges, such as GitLab and Bitbucket, and be self-hosted.
• Renovate supports a huge number of ecosystems including docker compose, go, github actions, yarn, Python, and many more. Many of our projects would need to use a home-grown updater plus Renovate.

Now, given Renovate is open-source, I think if we want to implement a more robust automation around Drupal specific configuration, then we should discuss with the maintainers upstream about how to best implement that within the context of using Renovate. That way, we're contributing to a much larger and more broadly used open-source project rather than focusing narrowly on the enterprise-focused Drupal community.

Thanks for everyone's thoughts on this!