| authors | title | state |
|---|---|---|
Nicolas Joseph <[email protected]> |
Request for Discussion (RFD) Process |
published |
This is a Request for Discussion (RFD) for the RFD process itself. This RFD is intended to be a living document that evolves as the process evolves. It is heavily inspired by the process used by Oxide Computer Company. We started from the before mentioned RFD and adapted it to our needs.
Writing down ideas is important: it allows them to be rigorously formulated (even while nascent), candidly discussed and transparently shared. We capture the written expression of an idea in a Request for Discussion (RFD), a document in the original spirit of the IETF Request for Comments, as expressed by RFC 3:
The content of a note may be any thought, suggestion, etc. related to the software or other aspect of the network. Notes are encouraged to be timely rather than polished. Philosophical positions without examples or other specifics, specific suggestions or implementation techniques without introductory or background explication, and explicit questions without any attempted answers are all acceptable. The minimum length for a note is one sentence.
These standards (or lack of them) are stated explicitly for two reasons. First, there is a tendency to view a written statement as ipso facto authoritative, and we hope to promote the exchange and discussion of considerably less than authoritative ideas. Second, there is a natural hesitancy to publish something unpolished, and we hope to ease this inhibition.
Similar to RFCs, our philosophy of RFDs is to allow both timely discussion of rough ideas, while still becoming a permanent repository for more established ones. Depending on their state, RFDs may be quickly iterated on in a branch, discussed actively as part of a pull request to be merged, or commented upon after having been published.
The following are examples of when an RFD is appropriate, these are intended to be broad:
- An architectural or design decision for the software
- Change to an API or command-line tool
- Change to a process
- A design for testing
- An Engineering design for a new feature
RFDs not only apply to technical ideas but for the overall project processes as well. If you have an idea to improve the way something is being done, you have the power to make your voice heard by adding to discussion.
At the start of every RFD document, we’d like to include a brief amount of metadata. The metadata format is based on the python-markdown2 metadata format. It’d look like:
—--
authors: Lola Smith <[email protected]>, Tom Jones <[email protected]>
title: My title here
state: prediscussion
—--We keep track of three pieces of metadata:
- authors: the authors (and therefore owners) of an RFD. They should be listed with their name and e-mail address.
- title: the title of the RFD.
- state: must be one of the states discussed below.
- discussion: for RFDs that are in or beyond the discussion state, this should be a link to the PR to integrate the RFD; see below for details.
An RFD can be in one of the following six states:
prediscussionideationdiscussionpublishedcommittedabandoned
A document in the prediscussion state indicates that the work is not yet ready for discussion, but that the RFD is effectively a placeholder. The prediscussion state signifies that work iterations are being done quickly on the RFD in its branch in order to advance the RFD to the discussion state.
A document in the ideation state contains only a description of the topic that the RFD will cover, providing an indication of the scope of the eventual RFD. Unlike the prediscussion state, there is no expectation that it is undergoing active revision. Such a document can be viewed as a scratchpad for related ideas. Any member of the team is encouraged to start active development of such an RFD (moving it to the prediscussion state) with or without the participation of the original author. It is critical that RFDs in the ideation state are clear and narrowly defined.
Documents under active discussion should be in the discussion state. At this point a discussion is being had for the RFD in a Pull Request.
Once (or if) discussion has converged and the Pull Request is ready to be merged, it should be updated to the published state before merge. Note that just because something is in the published state does not mean that it cannot be updated and corrected. See the Making changes to an RFD section for more information.
The prediscussion state should be viewed as essentially a collaborative extension of an engineer’s notebook, and the discussion state should be used when an idea is being actively discussed. These states shouldn’t be used for ideas that have been committed to, organizationally or otherwise; by the time an idea represents the consensus or direction, it should be in the published state.
Once an idea has been entirely implemented, it should be in the committed state. Comments on ideas in the committed state should generally be raised as issues — but if the comment represents a call for a significant divergence from or extension to committed functionality, a new RFD may be called for; as in all things, use your best judgment.
Finally, if an idea is found to be non-viable (that is, deliberately never implemented) or if an RFD should be otherwise indicated that it should be ignored, it can be moved into the abandoned state.
We will go over this in more detail. Let’s walk through the life of a RFD.
There is a script in this repository, ./scripts/rfd/new.js, that will automate the process.
$ ./scripts/rfd/new.js "My title here"Once you have run the script, you will have a new branch with a placeholder RFD. You can then push your branch to the remote and open a draft pull request.
If you wish to create a new RFD by hand, or understand the process in greater detail, read on.
Note
Never at anytime through the process do you push directly to the main branch. Once your pull request (PR) with your RFD in your branch is merged into main, then the RFD will appear in the main branch.
You will first need to reserve the number you wish to use for your RFD. This number should be the next available RFD number from looking at the current git branch -r output.
Now you will need to create a new git branch, named after the RFD number you wish to reserve. This number should have leading zeros if less than 4 digits. Before creating the branch, verify that it does not already exist:
$ git branch -rl "*rfd/0042"If you see a branch there (but not a corresponding sub-directory in rfd in main), it is possible that the RFD is currently being created; stop and check with co-workers before proceeding! Once you have verified that the branch doesn’t exist, create it locally and switch to it:
$ git checkout -b "rfd/0042"Now create a placeholder RFD. You can do so with the following commands:
$ mkdir -p rfd/0042
$ cp scripts/rfd/template.mdx rfd/0042/README.mdxFill in the RFD number and title placeholders in the new doc and add your name as an author. The status of the RFD at this point should be prediscussion.
If your preference is to use asciidoc, that is acceptable as well, however the examples in this flow will assume markdown.
Push your changes to your RFD branch in the RFD repo.
$ git add rfd/0042/README.mdx
$ git commit -m ‘rfd(0042): Adding placeholder for RFD <Title>’
$ git push origin rfd/0042And Open a Pull Request, the pull request can be in draft mode until you are ready to open it up for discussion.
After your Pull Request is opened, a github action will automatically add a folder for your RFD in the main branch. This will allow others to see that the RFD number has been claimed and is being worked on.
Now you can gather your thoughts and get your RFD to a state where you would like to get feedback and discuss with others. It’s recommended to push your branch remotely to make sure the changes you make stay in sync with the remote in case your local gets damaged.
It is up to you as to whether you would like to squash all your commits down to one before opening up for feedback, or if you would like to keep the commit history for the sake of history.
When you are ready to get feedback on your RFD, make sure all your local changes are pushed to the remote branch. At this point you are likely at the stage where you will want to change the status of the RFD from prediscussion to discussion for a fully formed RFD or to ideation for one where only the topic is specified. Do this in your branch.
Along with your RFD content, update the RFD’s state to discussion in your branch, then:
$ git commit -am ‘rfd(0042): Add RFD for <Title>’
$ git push origin "rfd/0042"Open a pull request on GitHub to merge your branch, in this case rfd/0042 into the main branch.
After the pull request is opened, anyone subscribed to the repo will get a notification that you have opened a pull request and can read your RFD and give any feedback.
The comments you choose to accept from the discussion are up to you as the owner of the RFD, but you should remain empathetic in the way you engage in the discussion.
For those giving feedback on the pull request, be sure that all feedback is constructive. Put yourself in the other person’s shoes and if the comment you are about to make is not something you would want someone commenting on an RFD of yours, then do not make the comment.
After there has been time for folks to leave comments, the RFD can be merged into main and changed from the discussion state to the published state. The timing is left to your discretion: you decide when to open the pull request, and you decide when to merge it. As a guideline, 3-5 business days to comment on your RFD before merging seems reasonable — but circumstances (e.g., time zones, availability of particular expertise, length of RFD) may dictate a different timeline, and you should use your best judgment. In general, RFDs shouldn’t be merged if no one else has read or commented on it; if no one is reading your RFD, it’s time to explicitly ask someone to give it a read!
Discussion can continue on published RFDs! The discussion: link in the metadata should be retained, allowing discussion to continue on the original pull request. If an issue merits more attention or a larger discussion of its own, an issue may be opened, with the synopsis directing the discussion.
Any discussion on an RFD in the can still be made on the original pull request to keep the sprawl to a minimum. Or if you feel your comment post-merge requires a larger discussion, an issue may be opened on it — but be sure to reflect the focus of the discussion in the issue synopsis (e.g., “RFD 42: add consideration of Delta Lake”), and be sure to link back to the original PR in the issue description so that one may find one from the other.
After your RFD has been merged, there is always opportunity to make changes. The easiest way to make a change to an RFD is to make a pull request with the change you would like to make. If you are not the original author of the RFD name your branch after the RFD # (e.g. 0001) and be sure to @ the original authors on your pull request to make sure they see and approve of the changes.
Changes to an RFD will go through the same discussion and merge process as described above.
Once an RFD has become implemented — that is, once it is not an idea of some future state but rather an explanation of how a system works — its state should be moved to be committed. This state is essentially no different from published, but represents ideas that have been more fully developed. While discussion on committed RFDs is permitted (and changes allowed), they would be expected to be infrequent.
The best part about the RFD process is that it itself is expressed in this RFD; if you want to change the process itself, you can apply the RFD process to its own RFD: chime in on the discussion link or open an issue as dictated by its current state!
We will start with a very small set of tooling and expand it as we need.
We have a script that will automate the process of creating a new RFD. It will create the branch, create the placeholder RFD, and push the branch.
We have a very simple github actions script that will claim an RFD number when a pull request is created. It will create a folder for the RFD in the main branch and add a placeholder README.mdx file.