Systematization of CD in our repos

[consideRatio]

I believe we have a lot to gain by making it easier and quicker to release with the help of Continuously Deployment (CD). If we make the complexity of making a release become git tag -a x.y.z && git push --tags then I'd be very happy :D

It is extra relevant if we end up with a chain of things that needs to be released for something to be fixed. From my own experience, it can become too scary to do it if it isn't quite simple.

Technical types of deployments

Publishing a pip package

To CD a pip package, do...

Publishing a Helm chart

We publish Helm charts in zero-to-jupyterhub-k8s and binderhub. This is done like this ...

???

Release guidelines

Should we have common release guidelines among repositories? If it is easy to make a release, we could encourage a practice of making beta releases for example.

I guess what I'm suggesting in this issue mainly is to support making releases using git only.

[betatim]

For repo2docker we have been using the "create a tag and travis does the rest" workflow and it is great!

The config for it is here https://github.com/jupyter/repo2docker/blob/11fc69f8d62a39f494109d2926cb763c022821c6/.travis.yml#L77-L86 and it works well. It si the same recipe I used for scikit-optimize previously. (side comment: pypi.org has been doing cool stuff wit h2FA and tokens and so on, basically better auth mechanisms, so maybe the recipe needs revamping soon/you can tell this is a snippet that I created many years ago and since have been copy&pasting).

It is really useful to have something like versioneer that figures out the version from the tag. A lot of people have a love-hate relationship with versioneer but I would recommend to "just use it" to get started. Then at a later point consider switching.

For BinderHub we don't have this, I'd say that is Ok as it is mostly a "helm package". It feels like Chartpress does everything we need there.

Release cadence: for BinderHub we release every time we merge something into master. I really like the continuous release practice and it seems to fit well with the ideas of CI, CD, small releases, continuous improvement from devops.

For repo2docker we try and cut a release every 3months. It is easy to do, keeps people who like releases happy, creates "fix points". Doing it regularly without "waiting for X" means it is easy to decide to not wait for X because it will be in the next release which is only a few months away.

[choldgraf]

The one thing that I think we miss with this "make a release for every commit" approach is that it makes it harder for us to communicate to the outside world what is being updated. Perhaps there is a way to make "checkpoints" that let us create a changelog but without making any kind of special commitment to stability, backwards-support, etc.

btw, I wrote a little CLI to quickly generate markdown that gives summaries / links of github activity in a repository. Perhaps it would be useful for us to use this in generating changelogs quickly? https://github.com/choldgraf/github-activity

For example, here was the output of running github-activity jupyterhub/team-compass 2019-09-01 (in a details tag so I don't clutter this issue)

---

2019-09-01...today

(full changelog)

Other closed PRs

• September Meeting Notes #212 (@Zsailer)
• fixing google groups link #210 (@choldgraf)
• [WIP] adding team check-in process #194 (@choldgraf)

Closed issues

• September Team Meeting 2019 #199 (@Zsailer)
• Moore foundation grant final report #183 (@choldgraf)
• Some thoughts on the SRE guidelines: Upgrading k8s #180 (@sgibson91)
• Discuss applying for a CZI "Essential OS for Science" grant #156 (@choldgraf)

Opened PRs

• September Meeting Notes #212 (@Zsailer)
• fixing google groups link #210 (@choldgraf)

Opened issues

• A round of GKE credits for mybinder.org (2019-2020) #214 (@choldgraf)
• Systematization of CD in our repos #213 (@consideRatio)
• October Team Meeting 2019 #211 (@Zsailer)
• @sgibson91 unavailability #209 (@sgibson91)
• Software Sustainability Institute 2020 Fellowship application #208 (@sgibson91)
• Adding @manics to the JupyterHub team #207 (@consideRatio)
• JupyterHub / BinderHub team responsibilities #206 (@consideRatio)
• mybinder.org subdomain: turing.mybinder.org #205 (@sgibson91)
• Adding Gesis to the federation #204 (@betatim)
• mybinder.org subdomain: gesis.mybinder.org #203 (@betatim)

[manics]

For updating version strings I've used bump2version on a couple projects. You run it with the level of the release "major", "minor" or "patch", it figures out the version, updates any required files, and creates the tag. E.g. bumpversion minor resulted in ome/omero-signup@7fe8d6b
No need to lookup the current version string, deal with numbers, or worry about making a typo in the tag.

[betatim]

I think a good way to combine "continuous releases" with "keep the world updated" is to have a blog post every N months that tells people what happened in the last month. Can call it "the quarterly X checkpoint" as well as blog posts when there is a big new feature. I see more and more companies moving towards decoupling technical releases from publicity pushes to reduce stress and coordination required. (If you pay close attention you will see new features appear and only a while later will there be an announcement about it. However most people don't pay close attention and for sure the "general public" doesn't so essentially no one notices.)

We already did this when we created the federation. We had it running for a few weeks to debug stuff and then announced it when we knew it was solid.

The markdown generator looks very cool for helping to create release notes!!

[choldgraf]

@betatim I like the idea of regular blog posts!

[manics]

I've just discovered https://github.com/toolmantim/release-drafter (via ansible/molecule#2367) for automatically drafting release notes.
Seems to operate using GitHub Actions.

[willingc]

@betatim I would prefer to pass on Versioneer. You can tell where I stand on the love-not_love debate 😉 It's a lot of overhead to a repo for minimal benefit IMHO.

[manics]

There are a couple of requests for new releases:

• Include inherited members in GitLab auth checks oauthenticator#283 (related to Update oauthenticator to include GitLab authorization fixes zero-to-jupyterhub-k8s#1455)
• Unable to install on Jupyterlab 1.0.2 jupyter-server-proxy#153

oauthenticator should be straightforward to automate using a Travis Pypi deployment, shall we aim to get that in place for the next release? Two questions:

• Which pypi user account are the secrets associated with
• Is it better to include encrypted secrets in the .travis.yml file (Systematization of CD in our repos #213 (comment)) or set a secret environment variable in the Travis repo settings e.g.: https://github.com/ome/scc/blob/v0.12.6/.travis.yml#L30-L36 (I find this easier than figuring out the travis command to create the encrypted string)

jupyter-server-proxy is a bit more complicated since it requires a pypi and npm release, though both are supported by Travis

[betatim]

I have a slight preference for storing the secrets in .travis.yml because that way we "for free" get a log of when and who changes them.

https://pypi.org/project/oauthenticator/ suggests that only @minrk has credentials for updating it. Maybe we can create a "jupyterhubteam" account that is a maintainer like we have a "mybinderteam" for https://pypi.org/project/binderhub/?

For jupyter-server-proxy I have no idea how to even get started on making a release :(

[manics]

A dedicated service account makes sense. Pypi recently added support for token authentication which can be scoped to a single package: https://pypi.org/help/#apitoken
I've tested it on:
https://github.com/manics/jupyter-notebookparams/blob/d0f663c9374bf9a7421912c86371e91e3c5c4572/.travis.yml#L10-L16

I'm happy to look into jupyter-server-proxy when everyone's agreed on this issue

[betatim]

The API token approach looks nice! I hadn't realised this had shipped yet (still dazzled by the 2FA support).

We could also add the mybinderteam account to the jupyter-server-proxy package as it is used a lot in Binder. I think I should have access to that account and can share a API token scoped to that package with you.

[minrk]

Let's make a jupyterhub-team bot account with token access for upload on PyPI. That sounds excellent.