[rush] publish -a has confusing documentation

[Sushisource]

Hello there,

Just started trying to use rush and immediately ran into some weirdness with publish -a on the first attempt to use it.

I ran rush change, all the logs were generated properly, but then upon trying to dry run publish, it actually goes and deletes the change files. Note that no previous changelogs or anything else existed before I ran rush change.

Here's some output. The deletes actually happened, which I would expect would not happen in a dry run. There's quite a bit more output afterwards but it all seems fine. Running publish -a again, which I would expect to be idempotent, now does nothing.

EDIT: I'm realizing this is probably really a documentation problem. The -h output makes it clear -a really isn't dry run, but this page sure makes it seem that way: https://rushjs.io/pages/maintainer/publishing/

The publish -h output also mentions a --commit flag which isn't a valid flag. In general I think the rush change -> rush publish flow isn't very well explained by the docs.

How, for example, would I set up my CI to auto-publish packages? How can I skip the versioning prompt?

^(*rushify) λ rush publish -a                                                                            (dollarcoin) 14:21:57


Rush Multi-Project Build Tool 5.3.0 - https://rushjs.io


Starting "rush publish"

Finding changes in: /home/spencer/dev/dollarcoin/common/changes

* DRYRUN: git checkout -b publish-1540416124174

* APPLYING: patch update for bank-node-api to 1.0.2
 - [patch] rushv1
 - [dependency] Updating dependency "dollar-client" from `^0.4.1` to `^0.4.2`

* APPLYING: patch update for chain-explorer-backend to 1.0.2
 - [patch] rushv1

* APPLYING: patch update for chain-explorer-ui to 0.0.2
 - [patch] rushv1

* APPLYING: patch update for dollar-client to 0.4.2
 - [patch] rushv1

* APPLYING: patch update for ledger-sync to 1.0.2
 - [patch] rushv1

* APPLYING: patch update for sawtooth-wrapper-api to 1.0.2
 - [patch] rushv1
 - [dependency] Updating dependency "dollar-client" from `^0.4.1` to `^0.4.2`

* APPLYING: patch update for stakeholder-backend to 1.0.2
 - [patch] rushv1

* APPLYING: patch update for stakeholder-ui to 0.0.2
 - [patch] rushv1

* APPLYING: patch update for wallet to 1.0.2
 - [patch] rushv1
 - [dependency] Updating dependency "dollar-client" from `^0.4.1` to `^0.4.2`

* APPLYING: Changelog update for "[email protected]".

* APPLYING: Changelog update for "[email protected]".

* APPLYING: Changelog update for "[email protected]".

* APPLYING: Changelog update for "[email protected]".

* APPLYING: Changelog update for "[email protected]".

* APPLYING: Changelog update for "[email protected]".

* APPLYING: Changelog update for "[email protected]".

* APPLYING: Changelog update for "[email protected]".

* APPLYING: Changelog update for "[email protected]".

* DELETING: 9 change file(s).
 - /home/spencer/dev/dollarcoin/common/changes/bank-node-api/rushify_2018-10-24-21-19.json
Deleting: /home/spencer/dev/dollarcoin/common/changes/bank-node-api/rushify_2018-10-24-21-19.json
 - /home/spencer/dev/dollarcoin/common/changes/chain-explorer-backend/rushify_2018-10-24-21-19.json
Deleting: /home/spencer/dev/dollarcoin/common/changes/chain-explorer-backend/rushify_2018-10-24-21-19.json
 - /home/spencer/dev/dollarcoin/common/changes/chain-explorer-ui/rushify_2018-10-24-21-19.json
Deleting: /home/spencer/dev/dollarcoin/common/changes/chain-explorer-ui/rushify_2018-10-24-21-19.json
 - /home/spencer/dev/dollarcoin/common/changes/dollar-client/rushify_2018-10-24-21-19.json
Deleting: /home/spencer/dev/dollarcoin/common/changes/dollar-client/rushify_2018-10-24-21-19.json
 - /home/spencer/dev/dollarcoin/common/changes/ledger-sync/rushify_2018-10-24-21-19.json
Deleting: /home/spencer/dev/dollarcoin/common/changes/ledger-sync/rushify_2018-10-24-21-19.json
 - /home/spencer/dev/dollarcoin/common/changes/sawtooth-wrapper-api/rushify_2018-10-24-21-19.json
Deleting: /home/spencer/dev/dollarcoin/common/changes/sawtooth-wrapper-api/rushify_2018-10-24-21-19.json
 - /home/spencer/dev/dollarcoin/common/changes/stakeholder-backend/rushify_2018-10-24-21-19.json
Deleting: /home/spencer/dev/dollarcoin/common/changes/stakeholder-backend/rushify_2018-10-24-21-19.json
 - /home/spencer/dev/dollarcoin/common/changes/stakeholder-ui/rushify_2018-10-24-21-19.json
Deleting: /home/spencer/dev/dollarcoin/common/changes/stakeholder-ui/rushify_2018-10-24-21-19.json
 - /home/spencer/dev/dollarcoin/common/changes/wallet/rushify_2018-10-24-21-19.json
Deleting: /home/spencer/dev/dollarcoin/common/changes/wallet/rushify_2018-10-24-21-19.json

* DRYRUN: git add .

[octogonz]

EDIT: I'm realizing this is probably really a documentation problem. The -h output makes it clear -a really isn't dry run, but this page sure makes it seem that way: https://rushjs.io/pages/maintainer/publishing/

All the docs are open source. Feel free to propose a correction. There's even an "Improve this page" button to help you create a PR. :-)

The publish -h output also mentions a --commit flag which isn't a valid flag. In general I think the rush change -> rush publish flow isn't very well explained by the docs.

Yes. This feature's design is fairly disorganized, because it was built over a couple years, and we kept discovering new requirements as it came together. Lately publishing is the area of Rush that consistently gets the most questions and confusion. But in order to really make it better, we will need to make some breaking changes, which is why serious efforts have been procrastinated.

Fortunately our team is now finishing up some commitments from this summer, so we may finally have time to address this in Rush 6. I just opened #904 to track the status.

How, for example, would I set up my CI to auto-publish packages? How can I skip the versioning prompt?

What do you mean by the "versioning prompt"?

[Sushisource]

@pgonzal Thanks for the reply. Sorry I did a bit of a drive-by "what's going on?" dump.

Making things more clear in rush 6 sounds awesome, but it'd be fantastic to have just some additional documentation in 5 before the improvements are rolled out. Even just a rundown of "here's what the flow looks like if you want to auto-publish from CI every time code is merged into master" would be super helpful. In trying to get this working I'm running into questions like:

• When is the right time to run rush publish -a? Should it be run on the feature branch, or on master?
• Should it be done by CI, or by the developer?
• How does publish decide something actually needs to be published to npm?

By the versioning prompt I meant running publish was prompting me to enter version numbers before publishing to NPM. I would expect only rush change to issue prompts. I unfortunately don't have the exact steps I took to get to that state but will post them if/when I hit that again.

Thanks!

[olivierlacan]

@octogonz Were there any changes since 2018 on this front? Is there any chance you could answer the above questions?

[octogonz]

Sure!

-When is the right time to run rush publish -a? Should it be run on the feature branch, or on master?

In a simple setup, publishing is run against the master branch and happens on an automated schedule. At my work we publish Monday-Thursday at 10am, for example.

For advanced releases, you can also create a "release branch" (not "feature branch" technically). In that case, publishing is typically manually performed when you're ready to do a release. And often in that scenario, you use a version policy which allows you to manually configure what kind of version bump you want, rather than relying on the rush change.

• Should it be done by CI, or by the developer?

The best practice is that a CI job should always perform publishing. Individual employees normally do not have access to the credentials for publishing packages; instead they would run a CI job which publishes a branch on their behalf. This way every package that gets published has an audit trail -- who published it? who approved it? what branch was used? how was it built?

Rush is designed around this assumption, but of course it's also possible to invoke the commands manually, for example for testing.

-How does publish decide something actually needs to be published to npm?

In the recommended version, rush version and rush publish are used together.

• rush version examines the change files created by rush change and then modifies package.json files to bump versions appropriately. It also generates the CHANGELOG.md file

• rush publish simply iterates through each publishable project in your configuration, inspects the version number from package.json and checks the NPM registry to see whether that version exists. If not, then it will be published. This "idempotent" design is intended for situations where for example you are publishing 30 packages and it fails halfway through, maybe due to a network timeout. The idea is that you can restart the rush publish portion and it will pick up where it left off.

As far as HOW you make Rush do things, currently our answer is basically to copy the recipe from azure-pipelines/templates/publish.yaml. The CLI options are a bit confusing. The plan is to improve that by moving most of these options into config files in a future version of Rush. I've been involved with that project and am interested in feedback/requirements/ideas that people might want to share.