From 92fb5369abebb9ad3e0a019d3e1815fa68f8eb13 Mon Sep 17 00:00:00 2001 From: DeepFlow-pengbin <1473371932@qq.com> Date: Thu, 6 Nov 2025 16:29:10 +0800 Subject: [PATCH 1/2] =?UTF-8?q?=E6=9B=B4=E6=96=B0=20agent-group-config=20?= =?UTF-8?q?=E5=86=85=E5=AE=B9?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../02-input/02-tracing/04-opentelemetry.md | 57 +++++-------------- 1 file changed, 15 insertions(+), 42 deletions(-) diff --git a/docs/zh/08-integration/02-input/02-tracing/04-opentelemetry.md b/docs/zh/08-integration/02-input/02-tracing/04-opentelemetry.md index 4fd8845d..26b79d8e 100644 --- a/docs/zh/08-integration/02-input/02-tracing/04-opentelemetry.md +++ b/docs/zh/08-integration/02-input/02-tracing/04-opentelemetry.md @@ -77,6 +77,9 @@ end kubectl apply -n open-telemetry -f https://raw.githubusercontent.com/deepflowio/deepflow-demo/main/open-telemetry/open-telemetry.yaml ``` +> 注：若使用自有环境的 OpenTelemetry，建议将对应 Kubernetes Service 的 `Service.spec.internalTrafficPolicy` 参数配置为 `Local`，这样可使业务数据直接发送至本机 Otel，避免源地址转换导致 DeepFlow 数据关联异常。 +> + 安装完毕之后，可以在环境里看到这样一个组件清单： ```bash @@ -89,9 +92,9 @@ kubectl get all -n open-telemetry | Service | otel-agent | | ConfigMap | otel-agent | -如果你需要使用其他版本或更新的 opentelemetry-collector-contrib， -请在 [otel-docker](https://hub.docker.com/r/otel/opentelemetry-collector-contrib/tags) 仓库中， -找到你想要的镜像版本，然后使用如下命令更新镜像： +如果使用其他版本 opentelemetry，可以在 [otel-docker](https://hub.docker.com/r/otel/opentelemetry-collector-contrib/tags) 官方仓库中拉取并更新镜像： + +> 注：自 0.80.0 版本起，OpenTelemetry 修改了 SkyWalking Receiver 的 Parent Span ID，[此改动](https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/21799)会影响 DeepFlow 的关联逻辑。如你使用了 SkyWalking Receiver，请务必使用 0.79.0 或更早的版本；如未使用，则任意版本均可。 ```bash LATEST_TAG="xxx" # FIXME @@ -137,50 +140,20 @@ processors: service: pipelines: traces: - processors: [k8sattributes, resource] # 确保 k8sattributes processor 先被处理 + ## 确保 k8sattributes processor 先被处理 + processors: [k8sattributes, resource] exporters: [otlphttp] ``` # 配置 DeepFlow -接下来我们需要开启 deepflow-agent 的数据接收服务。 - -首先我们确定 deepflow-agent 所在的采集器组 ID，一般为名为 default 的组的 ID： - -```bash -deepflow-ctl agent-group list -``` - -确认该采集器组是否已经有了配置： - -```bash -deepflow-ctl agent-group-config list -``` - -若已有配置，将其导出至 yaml 文件中便于进行修改： - -```bash -deepflow-ctl agent-group-config list -o yaml > your-agent-group-config.yaml -``` - -修改 yaml 文件，确认包含如下配置项： - -```bash -vtap_group_id: -external_agent_http_proxy_enabled: 1 # required -external_agent_http_proxy_port: 38086 # optional, default 38086 -``` - -更新采集器组的配置： - -``` -deepflow-ctl agent-group-config update -f your-agent-group-config.yaml -``` - -如果采集器组还没有配置，可使用如下命令基于 your-agent-group-config.yaml 文件新建配置： +接下来通过 [deepflow-ctl](../best-practice/agent-advanced-config) 添加或更新 [agent-group-config](../configuration/agent/) 后使 agent 接收 otel 数据： ```bash -deepflow-ctl agent-group-config create -f your-agent-group-config.yaml +inputs: + integration: + enabled: true + listen_port: 38086 ``` # 基于 Spring Boot Demo 体验 @@ -199,11 +172,11 @@ kubectl apply -n deepflow-otel-spring-demo -f https://raw.githubusercontent.com/ ## 查看追踪数据 -前往 Grafana，打开 `Distributed Tracing` Dashboard，选择 `namespace = deepflow-otel-spring-demo` 后，可选择一个调用进行追踪。 +在 Grafana 中查看 `DeepFlow Templates` 目录下 `Distributed Tracing` 面板，通过图中上方变量变量过滤后查看追踪效果。 DeepFlow 能够将 OpenTelemetry、eBPF、BPF 获取到的追踪数据关联展示在一个 Trace 火焰图中， 覆盖一个 Spring Boot 应用从业务代码、系统函数、网络接口的全栈调用路径，实现真正的全链路分布式追踪，效果如下： -![OTel Spring Demo](https://yunshan-guangzhou.oss-cn-beijing.aliyuncs.com/pub/pic/2022082363044b24c3b37.png) +![OTel Spring Demo](https://yunshan-guangzhou.oss-cn-beijing.aliyuncs.com/pub/pic/fda8a329505c8fc0070e416d607751c2_20251106154838.png) 你也可以访问 [DeepFlow Online Demo](https://ce-demo.deepflow.yunshan.net/d/Distributed_Tracing/distributed-tracing?var-namespace=deepflow-otel-spring-demo&from=deepflow-doc) 查看效果。 From 421af3c6172a4dfdecc2c28a4f8e143e576c5686 Mon Sep 17 00:00:00 2001 From: DeepFlow-pengbin <1473371932@qq.com> Date: Thu, 13 Nov 2025 14:52:30 +0800 Subject: [PATCH 2/2] =?UTF-8?q?=E6=9B=B4=E6=96=B0=20agent=20=E7=BB=84?= =?UTF-8?q?=E9=85=8D=E7=BD=AE=E6=A0=BC=E5=BC=8F?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 将适配 k8s macvlan cni 使用的旧版本组配置，更新为新版本格式 --- .../03-special-environment-deployment.md | 78 ++----------------- 1 file changed, 8 insertions(+), 70 deletions(-) diff --git a/docs/zh/04-best-practice/03-special-environment-deployment.md b/docs/zh/04-best-practice/03-special-environment-deployment.md index d3dc4729..bbf77166 100644 --- a/docs/zh/04-best-practice/03-special-environment-deployment.md +++ b/docs/zh/04-best-practice/03-special-environment-deployment.md @@ -41,81 +41,19 @@ permalink: /best-practice/special-environment-deployment/ ### 仅采集 RootNS 中的网卡流量 -K8s 使用 macvlan CNI 时，在 rootns 下只能看到所有 POD 共用的一个虚拟网卡，此时需要对 deepflow-agent 进行额外的配置： +当 Kubernetes 环境中使用 macvlan CNI 时，只能看到所有 Pod 在 rootns 下共享的一个虚拟网卡，因此需要通过 [deepflow-ctl](./agent-advanced-config) 更改 agent [组配置](./../configuration/agent/)适配该网络模式： -1. 创建 agent-group 和 agent-group-config： - - ```bash - deepflow-ctl agent-group create macvlan - deepflow-ctl agent-group-config create macvlan - ``` - -2. 获取 macvlan agent-group ID： - - ```bash - deepflow-ctl agent-group list | grep macvlan - ``` - -3. 新建 agent-group-config 配置文件 `macvlan-agent-group-config.yaml`: +1. agent-group-config 适配 macvlan CNI: ```yaml - vtap_group_id: g-xxxxxx - ## Regular Expression for TAP (Traffic Access Point) - ## Length: [0, 65535] - ## Default: - ## Localhost: lo - ## Common NIC: eth.*|en[osipx].* - ## QEMU VM NIC: tap.* - ## Flannel: veth.* - ## Calico: cali.* - ## Cilium: lxc.* - ## Kube-OVN: [0-9a-f]+_h$ - ## Note: Regular expression of NIC name for collecting traffic - tap_interface_regex: eth0 - ## Traffic Tap Mode - ## Default: 0, means local. - ## Options: 0, 1 (virtual mirror), 2 (physical mirror, aka. analyzer mode) - ## Note: Mirror mode is used when deepflow-agent cannot directly capture the - ## traffic from the source. For example: - ## - in the K8s macvlan environment, capture the Pod traffic through the Node NIC - ## - in the Hyper-V environment, capture the VM traffic through the Hypervisor NIC - ## - in the ESXi environment, capture traffic through VDS/VSS local SPAN - ## - in the DPDK environment, capture traffic through DPDK ring buffer - ## Use Analyzer mode when deepflow-agent captures traffic through physical switch - ## mirroring. - tap_mode: 1 - ``` - -4. 创建 agent-group-config： - - ```bash - deepflow-ctl agent-group-config create -f macvlan-agent-group-config.yaml + inputs: + cbpf: + af_packet: + interface_regex: ^(tap.*|cali.*|veth.*|eth.*|en[osipx].*|lxc.*|lo|[0-9a-f]+_h)$ + common: + capture_mode: 1 ``` -5. 修改 deepflow-agent 的 agent-group： - ```bash - kubectl edit cm -n deepflow deepflow-agent - ``` - 添加配置： - ```yaml - vtap-group-id-request: g-xxxxx - ``` - 停止 deepflow-agent： - ```bash - kubectl -n deepflow patch daemonset deepflow-agent -p '{"spec": {"template": {"spec": {"nodeSelector": {"non-existing": "true"}}}}}' - ``` - 通过 deepflow-ctl 删除 macvlan 的 agent： - ```bash - deepflow-ctl agent delete - ``` - 启动 deepflow-agent： - ```bash - kubectl -n deepflow patch daemonset deepflow-agent --type json -p='[{"op": "remove", "path": "/spec/template/spec/nodeSelector/non-existing"}]' - ``` - 查看 deepflow agent list， 确保 agent 加入了 macvlan group： - ```bash - deepflow-ctl agent list - ``` ### 同时采集 RootNS 和 PodNS 中的网卡流量