[RFC] Making the changelog audience-oriented

[webknjaz]

So I talked to @pradyunsg privately and he thought that it might be a good idea to have a follow-up discussion regarding something that I mentioned in #12539. I'm pasting it below:

Hmm... looking back at our actual usage of the changelog, it might make sense to drop this section entirely and split the deprecations and removals into separate categories.

I also looked into a few recent releases @ https://pip.pypa.io/en/latest/news/ with Ctrl+F for "Process". The draft section has news about contributor and downstream-facing changes in this category. The 24.0 one is downstream/packaging; 23.3 is contrib; 23.2 is a feature deprecation (?); 22.2 is more of a bugfix (or forward compat / feature).

So I think that it is indeed misused, however this is probably a consequence of the category being generic and its purpose being seemingly underdefined. Changing it into specific focused and well-documented categories, like I did elsewhere, might be the thing that could improve clarity going forward. Building a shared understanding of what's the purpose of the fragment types is useful regardless of removal.

I understand where the proposal to get rid of it is coming from but it seems to me that making it clearer by splitting would be more useful to the changelog consumers compared to sticking these entries into categories that might be even less related.

Originally posted by @webknjaz in #12539 (comment)

I was recently improving the changelog structure in the @aio-libs projects, looking into what pip does, among others. I ended up putting a similar section towards the end but re-framing it as several more granular sections like contributor-facing and downstream/packaging-friendly. See https://yarl.aio-libs.org/en/latest/changes/ + https://github.com/aio-libs/yarl/blob/bd5ff24/towncrier.toml#L18-L68. I documented their purpose like so https://yarl.aio-libs.org/en/latest/contributing/guidelines/#alright-so-how-to-add-a-news-fragment.

I thought, you'll find this example useful, perhaps.

Originally posted by @webknjaz in #12539 (comment)

---

With the above in mind, I propose replacing the process change fragment type with two new types — contrib and packaging. These would target two groups of people that somewhat intersect, sometimes, but not always — contributors to pip (people who are impacted by automation and contribution process changes) and packagers (mostly the downstream maintainers), respectively.

As I see it, the old process' purpose was so vague that it resulted in it being misused as misc since its purpose was broad and not defined very clearly. By defining the suggested changelog categories granularly, the chance of accidental misuse is going to be reduced. Besides, there would be an obvious place in the news for these changelog consumers to look at. The positioning would remain towards the end since most readers are end-users (and care about features, deprecations, breaking changes).

Example of changes to go to contrib: adding a step to the pre-PR checklist, changing behavior of nox commands, recording things related to the contribution/review process in the docs, new limitations/updates on the dev/test envs (locally and in CI), changing testing prerequisites (e.g. testrunner reconfiguration).

Example of changes for packaging: bumping runtime and build deps, including/excluding files from sdists, dropping/adding support for Python runtimes, testrunner reconfiguration (since they normally run tests too, when repackaging).

cc @sbidoul

[sbidoul]

@webknjaz I agree there were misuses of the Process section and I take a mental note to be more careful when reviewing the news fragments.

Your proposed changes make a lot of sense but I'm not sure it's worth the effort?

[webknjaz]

Changing the fragment types and adding some explanatory guidance is a one time thing. Then, people would have a document to follow. It could be useful to have a flow-chart even.
Writing coherent changelogs is objectively hard. I observed this in how others struggle to write up something useful with the intended audience in mind as well as myself (although, I'm a changelog nerd, often looking for ways to improve). Ideally, people should have common understanding of how to compose entries as well as post-process it before releasing.
It's not that the proposed entries are not added — they are, but to the wrong places. This means that many people would need those mental notes, which is where the problem lies — this has extra cognitive load, meaning extended maintenance burden.
So my opinion is that having clear instructions of how to categorize changelog entries that people are submitting anyway, would contribute to reducing this burden as people wouldn't be choosing wrong categories all the time.
In my experience, most of the time I'm the only one who is trying to make the changelogs nicer in many projects. This demonstrates how much of a burden this is for many maintainers in general. And this prompted me that the effort of setting this up is quite worth it (to me, that is) and I only hope to reduce the connected burden for others.

As for the process as a category I think that in your PR @pradyunsg even suggested removing it completely. Which I think makes sense because it's not clear who it's for in the first place. As it is unclear in what cases one would want to add a note to that section. It's not interesting for the end-users to see how the maintainers changed their operation over time, but is interesting to a subset who are contributors. That's something that could be explained. Still, I'm not sure how well this fragment name corresponds its intended purpose, then.

[webknjaz]

UPD: last week I participated in the pytest development sprint and implemented the same thing there. Folks seem to be pretty happy with the new categories. And the effort to implement the new config is pretty low (mostly copying and adjusting a few things).

@sbidoul @pradyunsg so if there's interest in moving this forward, I'll be happy to submit a PR for discussion. Or you could tag whoever you think might have opinions/be affected for commenting here…

Personally, I'm convinced that this is a kind of low-effort high-impact change that would be useful in the project, which is why I'm implementing it in more of mine.

[sbidoul]

Please go ahead with a PR.