|
| 1 | +# Strict Field Validation |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +By default, when your controller writes an object that contains fields not defined in the CRD schema, the API server: |
| 6 | + |
| 7 | +- Accepts the request |
| 8 | +- Drops the unknown fields |
| 9 | +- May only log a warning |
| 10 | + |
| 11 | +This can hide bugs and version skew between: |
| 12 | +- The controller code (Go types) and |
| 13 | +- The CRD schema installed in the cluster |
| 14 | + |
| 15 | +`controller-runtime` exposes [client.WithFieldValidation][client-field-validation-docs] to turn on strict server-side field validation for all client writes. When enabled, the API server returns a hard error instead of silently dropping unknown fields. |
| 16 | + |
| 17 | +We **do not enable this by default** in scaffolds because it can be too aggressive during upgrades. Instead, we show how to wire it as an opt-in flag. |
| 18 | + |
| 19 | +## What does it solve |
| 20 | + |
| 21 | +Strict validation prevents silent failures when your controller code and CRD schemas get out of sync. |
| 22 | + |
| 23 | +For example, you add a new field `status.newField` to your controller, but the CRD in the cluster hasn't been updated yet. When the controller calls `client.Status().Patch(...)`: |
| 24 | + |
| 25 | +**Without strict validation:** |
| 26 | +- API server drops `status.newField` silently |
| 27 | +- Controller sees no error |
| 28 | +- Field never appears on the object → confusing debugging |
| 29 | + |
| 30 | +**With strict validation:** |
| 31 | +- API server returns clear error |
| 32 | +- Controller knows CRDs need updating |
| 33 | +- Fails fast instead of silent data loss |
| 34 | + |
| 35 | +<aside class="warning"> |
| 36 | +<h1>Not included in default scaffold</h1> |
| 37 | + |
| 38 | +This feature is **not scaffolded by default** because it requires careful deployment coordination. |
| 39 | + |
| 40 | +**The problem:** Standard deployment tools (`make deploy`, `helm install`) apply CRDs and controller simultaneously with no ordering guarantees. When strict validation is enabled and the controller starts before CRDs finish updating, **all writes fail** until manual intervention. |
| 41 | + |
| 42 | +**The solution:** You need external tooling (separate Helm charts, CI/CD pipeline stages, custom scripts) to ensure CRDs are upgraded and established before the controller starts. |
| 43 | + |
| 44 | +</aside> |
| 45 | + |
| 46 | +## Upgrade scenario example |
| 47 | + |
| 48 | +Consider a common upgrade scenario: you deploy a new controller version that adds `status.newField` to your types, but the CRD in the cluster hasn't been updated yet. When the controller tries to write the new field: |
| 49 | + |
| 50 | +```go |
| 51 | +if err := r.Status().Patch(ctx, foo, patch); err != nil { |
| 52 | + // handle error |
| 53 | +} |
| 54 | +``` |
| 55 | + |
| 56 | +**Without strict validation**, the API server accepts the request but silently drops `status.newField`. The controller sees no error, but the field never appears on the object. This makes debugging difficult because there's no indication that something went wrong. |
| 57 | + |
| 58 | +**With strict validation**, the API server rejects the request with a 400 BadRequest error. The controller receives a clear error indicating a CRD–controller mismatch, making it obvious what needs to be fixed. |
| 59 | + |
| 60 | +While this catches bugs fast, it also means any write operation will fail when a new controller runs against old CRDs. The failures continue until the CRDs are updated. That's why strict validation is **off by default**. |
| 61 | + |
| 62 | +## How strict field validation works |
| 63 | + |
| 64 | +`controller-runtime` lets you wrap a client: |
| 65 | + |
| 66 | +```go |
| 67 | +strictClient := client.WithFieldValidation( |
| 68 | + baseClient, |
| 69 | + metav1.FieldValidationStrict, |
| 70 | +) |
| 71 | +``` |
| 72 | + |
| 73 | +All write operations (Create, Update, Patch) from `strictClient` send `fieldValidation=strict` to the API server. |
| 74 | + |
| 75 | +The API server: |
| 76 | +- Returns an error when the payload has unknown or invalid fields |
| 77 | +- Does not perform the write |
| 78 | + |
| 79 | +You can still override per call: |
| 80 | + |
| 81 | +```go |
| 82 | +cli.Create(ctx, obj, client.FieldValidation(metav1.FieldValidationWarn)) |
| 83 | +``` |
| 84 | + |
| 85 | + |
| 86 | +## When to use it |
| 87 | + |
| 88 | +Strict validation is a good fit when: |
| 89 | + |
| 90 | +- You own both the CRDs and the controllers |
| 91 | +- Your upgrade process applies CRDs first or ensures they update together |
| 92 | +- You want to fail fast when a controller writes fields not in the schema |
| 93 | +- You want to catch bugs in your types or conversions early |
| 94 | +- You use typed schemas, or explicitly mark dynamic data with `x-kubernetes-preserve-unknown-fields: true` |
| 95 | + |
| 96 | +This works well in development, CI, and production environments where you control the deployment order. |
| 97 | + |
| 98 | + |
| 99 | +## When NOT to use it |
| 100 | + |
| 101 | +Avoid strict validation in production when: |
| 102 | + |
| 103 | +- Controllers and CRDs upgrade independently (common in Helm, OLM deployments) |
| 104 | +- You manage third-party CRDs whose schemas evolve independently |
| 105 | +- Your CRDs use unstructured/dynamic data without `x-kubernetes-preserve-unknown-fields` |
| 106 | +- You need upgrade tolerance when controller and CRD versions are temporarily mismatched |
| 107 | + |
| 108 | +In these scenarios, strict validation causes BadRequest errors during upgrades. That's why it's: |
| 109 | +- **Off by default** in scaffolds |
| 110 | +- **Opt-in via flag** for those who need it |
| 111 | + |
| 112 | +## Wiring an opt-in flag in cmd/main.go |
| 113 | + |
| 114 | +This feature is **not scaffolded by default**. Follow these steps to add it manually. |
| 115 | + |
| 116 | +### Step 1: Add the strictManager wrapper |
| 117 | + |
| 118 | +In `cmd/main.go`, add this type definition after the `init()` function: |
| 119 | + |
| 120 | +```go |
| 121 | +// strictManager wraps the manager to reject unknown fields instead of silently dropping them. |
| 122 | +// When the controller writes a field that doesn't exist in the CRD, the write fails immediately. |
| 123 | +// This helps catch typos and version mismatches between your code and cluster CRDs. |
| 124 | +type strictManager struct { |
| 125 | + ctrl.Manager |
| 126 | + strictClient client.Client |
| 127 | +} |
| 128 | + |
| 129 | +func (m *strictManager) GetClient() client.Client { |
| 130 | + return m.strictClient |
| 131 | +} |
| 132 | +``` |
| 133 | + |
| 134 | +### Step 2: Add required imports |
| 135 | + |
| 136 | +Add these imports to `cmd/main.go`: |
| 137 | + |
| 138 | +```go |
| 139 | +import ( |
| 140 | + // ... your existing imports ... |
| 141 | + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" |
| 142 | + "sigs.k8s.io/controller-runtime/pkg/client" |
| 143 | +) |
| 144 | +``` |
| 145 | + |
| 146 | +### Step 3: Add the command-line flag |
| 147 | + |
| 148 | +In the `main()` function, where other flags are defined, add: |
| 149 | + |
| 150 | +```go |
| 151 | +func main() { |
| 152 | + var metricsAddr string |
| 153 | + var enableLeaderElection bool |
| 154 | + var probeAddr string |
| 155 | + var strictFieldValidation bool // Add this |
| 156 | + |
| 157 | + flag.StringVar(&metricsAddr, "metrics-bind-address", ":8080", "...") |
| 158 | + flag.StringVar(&probeAddr, "health-probe-bind-address", ":8081", "...") |
| 159 | + flag.BoolVar(&enableLeaderElection, "leader-elect", false, "...") |
| 160 | + |
| 161 | + // Add this flag |
| 162 | + flag.BoolVar(&strictFieldValidation, "strict-field-validation", false, |
| 163 | + "Reject unknown fields instead of dropping them. Useful for dev/CI, and production if CRDs upgrade before controllers.") |
| 164 | + |
| 165 | + // ... rest of your code ... |
| 166 | +} |
| 167 | +``` |
| 168 | + |
| 169 | +### Step 4: Wrap the manager conditionally |
| 170 | + |
| 171 | +After creating the manager with `ctrl.NewManager()`, add this wrapper logic: |
| 172 | + |
| 173 | +```go |
| 174 | +mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{ |
| 175 | + Scheme: scheme, |
| 176 | + // ... your other options ... |
| 177 | +}) |
| 178 | +if err != nil { |
| 179 | + setupLog.Error(err, "unable to start manager") |
| 180 | + os.Exit(1) |
| 181 | +} |
| 182 | + |
| 183 | +// When enabled, the controller rejects writes with unknown fields instead of silently dropping them. |
| 184 | +// This is useful for catching bugs in development, but causes problems in production when you upgrade |
| 185 | +// the controller before the CRDs - all writes will fail until CRDs are updated. |
| 186 | +// |
| 187 | +// Safe for: development, CI, and production only with external tooling to ensure CRDs upgrade first. |
| 188 | +// Not safe for: make deploy, helm install, or when you apply everything at once. The scaffolded project |
| 189 | +// has no built-in mechanism to ensure CRDs upgrade before the controller - you need external solutions. |
| 190 | +var finalMgr ctrl.Manager = mgr |
| 191 | +if strictFieldValidation { |
| 192 | + finalMgr = &strictManager{ |
| 193 | + Manager: mgr, |
| 194 | + strictClient: client.WithFieldValidation( |
| 195 | + mgr.GetClient(), |
| 196 | + metav1.FieldValidationStrict, |
| 197 | + ), |
| 198 | + } |
| 199 | +} |
| 200 | + |
| 201 | +// Use finalMgr for all subsequent setup |
| 202 | +if err := (&controller.MyReconciler{ |
| 203 | + Client: finalMgr.GetClient(), |
| 204 | + Scheme: finalMgr.GetScheme(), |
| 205 | +}).SetupWithManager(finalMgr); err != nil { |
| 206 | + setupLog.Error(err, "unable to create controller", "controller", "My") |
| 207 | + os.Exit(1) |
| 208 | +} |
| 209 | + |
| 210 | +// Continue using finalMgr for health checks, starting manager, etc. |
| 211 | +if err := finalMgr.AddHealthzCheck("healthz", healthz.Ping); err != nil { |
| 212 | + setupLog.Error(err, "unable to set up health check") |
| 213 | + os.Exit(1) |
| 214 | +} |
| 215 | + |
| 216 | +if err := finalMgr.Start(ctrl.SetupSignalHandler()); err != nil { |
| 217 | + setupLog.Error(err, "problem running manager") |
| 218 | + os.Exit(1) |
| 219 | +} |
| 220 | +``` |
| 221 | + |
| 222 | +<aside class="note"> |
| 223 | +<h1>Important: Use finalMgr everywhere</h1> |
| 224 | + |
| 225 | +After wrapping the manager, use `finalMgr` instead of `mgr` for: |
| 226 | +- Controller setup |
| 227 | +- Webhook setup |
| 228 | +- Health checks |
| 229 | +- Starting the manager |
| 230 | + |
| 231 | +This ensures all components use the wrapped client with strict validation. |
| 232 | + |
| 233 | +</aside> |
| 234 | + |
| 235 | +[client-field-validation-docs]: https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/client#WithFieldValidation |
0 commit comments