Remove autogenerated files from coder/coder

[mtojek]

Problem Statement

Whenever we modify the database schema or gRPC contract, it results in at least three additional files being altered and included in a pull request. These changes often clutter PRs, making it harder to review them effectively.

Do we really need to store these generated files in the repository? Reducing the number of committed files could make it easier to spot accidental schema or contract changes, while also improving PR clarity.

Proposed Solution

Since making coder/coder go-gettable is no longer a common practice, we could leverage the Makefile to generate the necessary code dynamically when building the binaries.

This approach would:
✅ Reduce unnecessary diffs in PRs
✅ Improve code review efficiency
✅ Minimize accidental changes in schema or contracts

Would love to hear thoughts on this!

[johnstcn]

✅ Minimize accidental changes in schema or contracts

Anecdotally, I've noticed accidental schema changes because of generated files being changed. The above argument holds if reviewers simply ignore all changes to generated files, which I suspect may be the case.

If we remove generated files from VCS, then we may need some additional safeguards around changes to the "sources of truth" of these generated files. Perhaps CODEOWNERS?

[mafredri]

Would it be enough to just mark any problematic files here in case they're missing?

❯ cat .gitattributes
# Generated files
coderd/apidoc/docs.go linguist-generated=true
docs/reference/api/*.md linguist-generated=true
docs/reference/cli/*.md linguist-generated=true
coderd/apidoc/swagger.json linguist-generated=true
coderd/database/dump.sql linguist-generated=true
peerbroker/proto/*.go linguist-generated=true
provisionerd/proto/*.go linguist-generated=true
provisionerd/proto/version.go linguist-generated=false
provisionersdk/proto/*.go linguist-generated=true
*.tfplan.json linguist-generated=true
*.tfstate.json linguist-generated=true
*.tfstate.dot linguist-generated=true
*.tfplan.dot linguist-generated=true
site/e2e/provisionerGenerated.ts linguist-generated=true
site/src/api/typesGenerated.ts linguist-generated=true
site/src/pages/SetupPage/countries.tsx linguist-generated=true

This collapses them in the review.

I worry about the project not being buildable without generated files. And if it requires a certain tool or version to generate the specific file that would work, that may be a problem.

[mtojek]

Would it be enough to just mark any problematic files here in case they're missing?

I can challenge this argument. If we want to hide these files, do we need them?

I worry about the project not being buildable without generated files. And if it requires a certain tool or version to generate the specific file that would work, that may be a problem.

It will require some experimentation, but it could be beneficial to eliminate the boilerplate. Perhaps some can be simply removed.

[mafredri]

I can challenge this argument. If we want to hide these files, do we need them?

My take is that yes, we need them. A simple use-case: git pull and go build ./cmd/coder, which is not possible otherwise. I do that often enough that not having the ability would be somewhat annoying. This will result in weird errors since your local generated files are out of sync.

Having them available also lets you inspect the actual code that runs. This is valuable when, but not limited to, reviewing changes to code generation.

It will require some experimentation, but it could be beneficial to eliminate the boilerplate. Perhaps some can be simply removed.

I do see some benefit and understand the wish to remove it. But I would rather invest time in making the DX better around code-gen. I don't buy into generated code being distracting for a reviewer (I find it collapsed most of the time). It's mainly the PR author that has to deal with the mess, which could probably be alleviated with improvements to code-gen speed and git hooks.

---

A different approach I might be able to get behind would be to remove generated files from the authors commits, but have a bot add them to the PR. But I'm not sure how that would work in practice but it would solve both issues (keeping generated code and removing noise).

[mtojek]

My take is that yes, we need them. A simple use-case: git pull and go build ./cmd/coder, which is not possible otherwise

This is something I might be missing - why can't we default to git pull & make build-cli instead?

A different approach I might be able to get behind would be to remove generated files from the authors commits, but have a bot add them to the PR

IMHO too much magic :)

[mafredri]

This is something I might be missing - why can't we default to git pull & make build-cli instead?

Sure, in some cases. But I'm not a fan of the extra overhead (requiring full toolchain). This also won't work in your code editor. You sync your git branch, but the generated files will be out-of-sync and your editor filled with errors.

If a new gen tool is added you also need to install it before you can continue, a slight inconvenience, especially if working from the editor.

Perhaps before entertaining this further, we would need a table of which file(s) would be removed and what pros/cons that would have. I'm especially interested in the pros since I'd like to see proof that there's a tangible benefit to removing them, and that it doesn't outweigh the cons.

[ThomasK33]

My perspective on this is that I have worked on several projects involving Kubernetes controllers, API servers, and clients. Most software related to Kubernetes relies heavily on code generation and comments on the structs. The projects also included all dependencies vendored using go mod vendor and later go work vendor, as that feature had just been added.

I’ve found it somewhat acceptable to have generated files checked in, but also quite annoying when someone forgets to update them, leading to problems when dealing with the aftermath.

A simple use-case: git pull and go build ./cmd/coder, which is not possible otherwise.

Technically, that isn’t possible today because the dependencies the coder relies on are not vendored and may not yet be cached before the first run locally.

I’m not advocating for vendoring everything; that would be a bit too extreme for my taste. However, there are many benefits to explicitly vendoring all dependencies, such as supply chain security, not relying on goproxy or sumdb, and the ability to review dependency updates and their code changes.

We are already using some tools to reduce boilerplate and minimize the amount of vendored code in our repository by utilizing the Go toolchain.
Of course, this has better integration into editors and other tools, but one should be aware that we are already using some layers of abstraction.

If a new gen tool is added you also need to install it before you can continue, a slight inconvenience, especially if working from the editor.

That also changes with Go 1.24 and the tool directive in the go.mod, but I don’t want to go beyond the scope of this conversation. (See: https://tip.golang.org/doc/go1.24#tools)

But I'm not a fan of the extra overhead (requiring full toolchain).

Out of curiosity, what does “full toolchain” mean in this context? Make is typically preinstalled on most Linux distributions.