Improve Network Policy setup & doc

README.md

@@ -70,8 +70,6 @@ metadata:
   name: cluster
 spec:
   namespace: netobserv
-  networkPolicy:
-    enable: false
   consolePlugin:
     standalone: true
   processor:
@@ -94,8 +92,8 @@ EOF
 
 A few remarks:
 - You can change the Prometheus and Loki URLs depending on your installation. This example works if you use the "standalone" installation described above, with `install.loki=true` and `install.prom-stack=true`. Check more configuration options for [Prometheus](https://github.com/netobserv/network-observability-operator/blob/main/docs/FlowCollector.md#flowcollectorspecprometheus-1) and [Loki](https://github.com/netobserv/network-observability-operator/blob/main/docs/FlowCollector.md#flowcollectorspecloki-1).
-- You can enable networkPolicy, which makes the operator lock down the namespaces that it manages; however, this is highly dependent on your cluster topology, and may cause malfunctions, such as preventing NetObserv pods from communicating with the Kube API server.
-- The processor env `SERVER_NOTLS` means that the communication between eBPF agents and Flowlogs-pipeline won't be encrypted. To enable TLS, you need to supply the TLS certificates to Flowlogs-pipeline (a Secret named `flowlogs-pipeline-cert`), and the CA to the eBPF agents (a ConfigMap named `flowlogs-pipeline-ca` in the privileged namespace). [Check this issue](https://github.com/netobserv/network-observability-operator/issues/2360) if you want to help making it simpler.
+- Depending on the Kubernetes distribution and CNI, NetObserv may come secured by default with a built-in network policy. You can force installing it or not by setting `spec.networkPolicy.enable` in `FlowCollector`. If the built-in policy does not work as intended, it is recommended to turn it off and create your own instead. NetObserv runs some highly privileged workloads, thus it is important to keep it as much isolated as possible. See [NetworkPolicy.md](./docs/NetworkPolicy.md) for more details on how to create a policy.
+- The processor env `SERVER_NOTLS` means that the communication between eBPF agents and Flowlogs-pipeline won't be encrypted. To enable TLS, you need to supply the TLS certificates to Flowlogs-pipeline (a Secret named `flowlogs-pipeline-cert`), and the CA to the eBPF agents (a ConfigMap named `flowlogs-pipeline-ca` in the privileged namespace).
 
 To view the test console, you can port-forward 9001:
 
@@ -256,9 +254,7 @@ More information about multi-tenancy can be found on [this page](https://github.
 
 For a production deployment, it is highly recommended to lock down the `netobserv` namespace (or wherever NetObserv is installed) using network policies.
 
-You can set `spec.networkPolicy.enable` to `true` to make NetObserv install automatically a network policy. The policy may need to be fined-tuned for your environment (e.g. for access to kube apiserver, or Prometheus), by adding authorized namespaces.
-
-A simple example of network policy is [provided here](https://github.com/netobserv/documents/blob/main/examples/lockdown-netobserv.yaml).
+You can set `spec.networkPolicy.enable` to `true` to make NetObserv install automatically a network policy. The policy may need to be fined-tuned for your environment (e.g. for access to kube apiserver, or Prometheus), by adding authorized namespaces. More information [here](./docs/NetworkPolicy.md).
 
 #### Communications
 

api/flowcollector/v1beta2/flowcollector_types.go

@@ -96,8 +96,10 @@ type FlowCollectorSpec struct {
 type NetworkPolicy struct {
 	// Deploys network policies on the namespaces used by NetObserv (main and privileged).
 	// These network policies better isolate the NetObserv components to prevent undesired connections from and to them.
-	// This option is enabled by default when using with OVNKubernetes, and disabled otherwise (it has not been tested with other CNIs).
-	// When disabled, you can manually create the network policies for the NetObserv components.
+	// Because it cannot be tested with all CNIs, this option is only enabled by default when NetObserv runs in a known
+	// supported environment, and it is disabled by default otherwise.
+	// When disabled, it is highly recommended to create network policies manually, to prevent undesired accesses.
+	// More information: https://github.com/netobserv/network-observability-operator/blob/main/docs/NetworkPolicy.md.
 	// +optional
 	Enable *bool `json:"enable,omitempty"`
 

api/flowcollector/v1beta2/flowcollector_validation_webhook.go

@@ -21,6 +21,7 @@ type NetworkType string
 const (
 	OpenShiftSDN  NetworkType = "OpenShiftSDN"
 	OVNKubernetes NetworkType = "OVNKubernetes"
+	Kindnet       NetworkType = "Kindnet"
 )
 
 type clusterInfo interface {
@@ -112,8 +113,8 @@ func (v *validator) validateNetPol() {
 			v.warnings = append(v.warnings, fmt.Sprintf("Could not detect CNI: %s", err.Error()))
 		} else if cni == OpenShiftSDN && v.fc.NetworkPolicy.Enable != nil && *v.fc.NetworkPolicy.Enable {
 			v.warnings = append(v.warnings, "OpenShiftSDN detected with unsupported setting: spec.networkPolicy.enable; this setting will be ignored; to remove this warning set spec.networkPolicy.enable to false.")
-		} else if cni != OVNKubernetes && v.fc.DeployNetworkPolicyOtherCNI() {
-			v.warnings = append(v.warnings, "Network policy is enabled via spec.networkPolicy.enable, despite not running OVN-Kubernetes: this configuration has not been tested; to remove this warning set spec.networkPolicy.enable to false.")
+		} else if cni == "" && v.fc.DeployNetworkPolicy(false) {
+			v.warnings = append(v.warnings, "Network policy is enabled via spec.networkPolicy.enable, despite running on an unknown CNI: this configuration has not been tested; to remove this warning set spec.networkPolicy.enable to false.")
 		}
 	} else {
 		v.warnings = append(v.warnings, "Unknown environment, cannot detect the CNI in use")

api/flowcollector/v1beta2/flowcollector_validation_webhook_test.go

@@ -1163,8 +1163,8 @@ func TestValidateNetPol(t *testing.T) {
 					NetworkPolicy: NetworkPolicy{Enable: ptr.To(true)},
 				},
 			},
-			cni:              "unknown",
-			expectedWarnings: admission.Warnings{"Network policy is enabled via spec.networkPolicy.enable, despite not running OVN-Kubernetes: this configuration has not been tested; to remove this warning set spec.networkPolicy.enable to false."},
+			cni:              "",
+			expectedWarnings: admission.Warnings{"Network policy is enabled via spec.networkPolicy.enable, despite running on an unknown CNI: this configuration has not been tested; to remove this warning set spec.networkPolicy.enable to false."},
 		},
 	}
 

api/flowcollector/v1beta2/helper.go

@@ -177,11 +177,10 @@ func (spec *FlowCollectorFLP) GetMetricsPort() int32 {
 	return port
 }
 
-func (spec *FlowCollectorSpec) DeployNetworkPolicyOVN() bool {
-	return spec.NetworkPolicy.Enable == nil || *spec.NetworkPolicy.Enable
-}
-
-func (spec *FlowCollectorSpec) DeployNetworkPolicyOtherCNI() bool {
+func (spec *FlowCollectorSpec) DeployNetworkPolicy(trueByDefault bool) bool {
+	if trueByDefault {
+		return spec.NetworkPolicy.Enable == nil || *spec.NetworkPolicy.Enable
+	}
 	return spec.NetworkPolicy.Enable != nil && *spec.NetworkPolicy.Enable
 }
 

bundle/manifests/flows.netobserv.io_flowcollectors.yaml

@@ -4297,8 +4297,10 @@ spec:
                     description: |-
                       Deploys network policies on the namespaces used by NetObserv (main and privileged).
                       These network policies better isolate the NetObserv components to prevent undesired connections from and to them.
-                      This option is enabled by default when using with OVNKubernetes, and disabled otherwise (it has not been tested with other CNIs).
-                      When disabled, you can manually create the network policies for the NetObserv components.
+                      Because it cannot be tested with all CNIs, this option is only enabled by default when NetObserv runs in a known
+                      supported environment, and it is disabled by default otherwise.
+                      When disabled, it is highly recommended to create network policies manually, to prevent undesired accesses.
+                      More information: https://github.com/netobserv/network-observability-operator/blob/main/docs/NetworkPolicy.md.
                     type: boolean
                 type: object
               processor:

config/crd/bases/flows.netobserv.io_flowcollectors.yaml

@@ -3946,8 +3946,10 @@ spec:
                       description: |-
                         Deploys network policies on the namespaces used by NetObserv (main and privileged).
                         These network policies better isolate the NetObserv components to prevent undesired connections from and to them.
-                        This option is enabled by default when using with OVNKubernetes, and disabled otherwise (it has not been tested with other CNIs).
-                        When disabled, you can manually create the network policies for the NetObserv components.
+                        Because it cannot be tested with all CNIs, this option is only enabled by default when NetObserv runs in a known
+                        supported environment, and it is disabled by default otherwise.
+                        When disabled, it is highly recommended to create network policies manually, to prevent undesired accesses.
+                        More information: https://github.com/netobserv/network-observability-operator/blob/main/docs/NetworkPolicy.md.
                       type: boolean
                   type: object
                 processor:

docs/FlowCollector.md

@@ -8394,8 +8394,10 @@ configuration, you can disable it and install your own instead.<br/>
         <td>
           Deploys network policies on the namespaces used by NetObserv (main and privileged).
 These network policies better isolate the NetObserv components to prevent undesired connections from and to them.
-This option is enabled by default when using with OVNKubernetes, and disabled otherwise (it has not been tested with other CNIs).
-When disabled, you can manually create the network policies for the NetObserv components.<br/>
+Because it cannot be tested with all CNIs, this option is only enabled by default when NetObserv runs in a known
+supported environment, and it is disabled by default otherwise.
+When disabled, it is highly recommended to create network policies manually, to prevent undesired accesses.
+More information: https://github.com/netobserv/network-observability-operator/blob/main/docs/NetworkPolicy.md.<br/>
         </td>
         <td>false</td>
       </tr></tbody>

docs/NetworkPolicy.md

@@ -0,0 +1,122 @@
+# NetObserv NetworkPolicy
+
+Depending on the Kubernetes distribution and CNI, NetObserv may come secured by default with a built-in network policy. You can force installing it or not by setting `spec.networkPolicy.enable` in `FlowCollector`. If the built-in policy does not work as intended, it is recommended to turn it off and create your own instead. NetObserv runs some highly privileged workloads, hence it is important to keep it as much isolated as possible.
+
+If the built-in policy looks _almost_ good, but some allowed namespaces are missing, you can add allowed namespaces in `spec.networkPolicy.additionalNamespaces`.
+
+You can find below the communication matrix that will help you create your own policy. Be aware that some pods use the host network, which not all CNI support for network policies.
+
+## Supported environments for built-in policy
+
+The following environments have been tested with the built-in policy, and will have it enabled by default:
+
+- Kindnet / kind.
+- OVN-Kubernetes (upstream), and API server in `kube-system` namespace.
+- OpenShift with OVN-Kubernetes.
+
+Feel free to ask & contribute to increase this list.
+
+## Common labels
+
+All pods deployed by NetObserv have the label `part-of: netobserv-operator`. So they can be selected via:
+
+```yaml
+  podSelector:
+    matchLabels:
+      part-of: netobserv-operator
+```
+
+## Namespaces
+
+The main namespace is `netobserv` by default, and can be configured in `FlowCollector` via `spec.namespace`.
+All pods managed by NetObserv are deployed there, except the `netobserv-ebpf-agent` pods, which are in the "privileged" namespace: it's the main namespace + `-privileged` suffix, so `netobserv-privileged` by default.
+
+## Communication flows
+
+This section describes the flows in details to help you build your network policy; However you can simplify the rules if you choose to allow in-namespace traffic, with:
+
+```yaml
+spec:
+  egress:
+  - to:
+    - podSelector: {}
+  ingress:
+  - from:
+    - podSelector: {}
+```
+
+and deploying Loki/Prometheus/Kafka (when relevant) in the same namespace.
+
+### Operator
+
+Label: `app=netobserv-operator`, default namespace: `netobserv`.
+
+**Ingress:**
+
+- Must allow traffic from Kube API Server to Webhooks: TCP, port 9443.
+- May allow traffic from Prometheus to Metrics endpoint: TCP, port 8443 (the source depends on your Prometheus setup).
+
+**Egress:**
+
+- Must allow traffic to Kube API Server: TCP, port 6443.
+
+### eBPF agents
+
+Label: `app=netobserv-ebpf-agent`, default namespace: `netobserv-privileged`. This is host-network pods.
+
+**Ingress:**
+
+- May allow traffic from Prometheus to Metrics endpoint: TCP, port 9400 (the source depends on your Prometheus setup).
+
+**Egress:**
+
+When `spec.deploymentModel` is `Service`:
+- Must allow traffic to flowlogs-pipeline (`app=flowlogs-pipeline`), TCP, default port 2055 (port configurable in `spec.processor.advanced.port`).
+
+When `spec.deploymentModel` is `Kafka`:
+- Must allow traffic to Kafka, TCP, port depends on your Kafka setup.
+
+When `spec.deploymentModel` is `Direct`:
+- Must allow traffic to flowlogs-pipeline (`app=flowlogs-pipeline`), TCP, default port 2055 (port configurable in `spec.processor.advanced.port`). `flowlogs-pipeline` are also host-network pods (same host).
+
+### Flowlogs-pipeline
+
+Label: `app=flowlogs-pipeline`, default namespace: `netobserv`.
+
+**Ingress:**
+
+- Must allow traffic from Prometheus to Metrics endpoint: TCP, port 9401 (the source depends on your Prometheus setup).
+
+When `spec.deploymentModel` is `Service`:
+- Must allow traffic from agents (`app=netobserv-ebpf-agent`), TCP, default port 2055 (port configurable in `spec.processor.advanced.port`).
+
+When `spec.deploymentModel` is `Direct`:
+- Must allow traffic from agents (`app=netobserv-ebpf-agent`), TCP, default port 2055 (port configurable in `spec.processor.advanced.port`). `flowlogs-pipeline` are also host-network pods (same host).
+
+**Egress:**
+
+When `spec.deploymentModel` is `Kafka`:
+- Must allow traffic to Kafka, TCP, port depends on your Kafka setup.
+
+When using Loki (`spec.loki.enabled`):
+- Must allow traffic to Loki, TCP, port depends on your Loki setup (usually 3100).
+
+When exporters are configured (`spec.exporters`):
+- Must allow traffic to exporters (refer to the exporter configuration).
+
+### Web console
+
+Label: `app=netobserv-plugin`, default namespace: `netobserv`.
+
+**Ingress:**
+
+- If you set up an Ingress route/gateway to the web console, configure it accordingly to allow incoming user traffic.
+
+- May allow traffic from Prometheus to Metrics endpoint: TCP, port 9002 (the source depends on your Prometheus setup).
+
+**Egress:**
+
+- Must allow traffic to Prometheus and AlertManager, TCP, as defined in `spec.prometheus.querier`.
+
+When using Loki (`spec.loki.enabled`):
+- Must allow traffic to Loki, TCP, port depends on your Loki setup (usually 3100).

helm/README.md

@@ -66,8 +66,6 @@ metadata:
   name: cluster
 spec:
   namespace: netobserv
-  networkPolicy:
-    enable: false
   consolePlugin:
     standalone: true
   processor:
@@ -90,7 +88,7 @@ EOF
 
 A few remarks:
 - You can change the Prometheus and Loki URLs depending on your installation. This example works if you use the "standalone" installation described above, with `install.loki=true` and `install.prom-stack=true`. Check more configuration options for [Prometheus](https://github.com/netobserv/network-observability-operator/blob/main/docs/FlowCollector.md#flowcollectorspecprometheus-1) and [Loki](https://github.com/netobserv/network-observability-operator/blob/main/docs/FlowCollector.md#flowcollectorspecloki-1).
-- You can enable networkPolicy, which makes the operator lock down the namespaces that it manages; however, this is highly dependent on your cluster topology, and may cause malfunctions, such as preventing NetObserv pods from communicating with the Kube API server.
+- Depending on the Kubernetes distribution and CNI, NetObserv may come secured by default with a built-in network policy. You can force installing it or not by setting `spec.networkPolicy.enable` in `FlowCollector`. If the built-in policy does not work as intended, it is recommended to turn it off and create your own instead. NetObserv runs some highly privileged workloads, thus it is important to keep it as much isolated as possible. See [NetworkPolicy.md](https://github.com/netobserv/network-observability-operator/blob/main/docs/NetworkPolicy.md) for more details on how to create a policy.
 - The processor env `SERVER_NOTLS` means that the communication between eBPF agents and Flowlogs-pipeline won't be encrypted. To enable TLS, you need to supply the TLS certificates to Flowlogs-pipeline (a Secret named `flowlogs-pipeline-cert`), and the CA to the eBPF agents (a ConfigMap named `flowlogs-pipeline-ca` in the privileged namespace).
 
 To view the test console, you can port-forward 9001:

helm/crds/flows.netobserv.io_flowcollectors.yaml

@@ -3950,8 +3950,10 @@ spec:
                       description: |-
                         Deploys network policies on the namespaces used by NetObserv (main and privileged).
                         These network policies better isolate the NetObserv components to prevent undesired connections from and to them.
-                        This option is enabled by default when using with OVNKubernetes, and disabled otherwise (it has not been tested with other CNIs).
-                        When disabled, you can manually create the network policies for the NetObserv components.
+                        Because it cannot be tested with all CNIs, this option is only enabled by default when NetObserv runs in a known
+                        supported environment, and it is disabled by default otherwise.
+                        When disabled, it is highly recommended to create network policies manually, to prevent undesired accesses.
+                        More information: https://github.com/netobserv/network-observability-operator/blob/main/docs/NetworkPolicy.md.
                       type: boolean
                   type: object
                 processor: