Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Local EVPN route leaking from unicast peer #17920

Open
2 tasks done
alcelafranque opened this issue Jan 24, 2025 · 7 comments
Open
2 tasks done

Local EVPN route leaking from unicast peer #17920

alcelafranque opened this issue Jan 24, 2025 · 7 comments
Labels
triage Needs further investigation

Comments

@alcelafranque
Copy link

Description

Hi, I'm not sure about the title of the issue.
I also don't know if the behavior is expected.

I'm in a case where routers with FRR do EVPN vxlan, I have several VRFs, and we do RT import on some VRFs to make them communicate.

This concept works very well, but it gets complicated in the case where we have a bgp ip-unicast session between one of its routers in a VRF to another router.

The route is re-announced in EVPN (route type 5) and the leak works well on the routers receiving this announcement. However, local leaking does not work.

Version

R2# show version
FRRouting 10.1.2_git (R2) on Linux(6.12.7).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--prefix=/usr' '--sysconfdir=/etc' '--localstatedir=/var' '--sbindir=/usr/lib/frr' '--libdir=/usr/lib' '--enable-rpki' '--enable-vtysh' '--enable-multipath=64' '--enable-vty-group=frrvty' '--enable-user=frr' '--enable-group=frr' '--enable-pcre2posix' '--enable-scripting' 'CC=gcc' 'CXX=g++'
R2#

How to reproduce

I made a lab containerlab if that helps.
https://github.com/alcelafranque/network-lab/tree/main/frr-evpn-vxlan-vrf-leak-ip-unicast

If that's not enough, here are the settings for R1:

frr defaults datacenter
log syslog informational
service integrated-vtysh-config
!
vrf vrf1
 vni 100
exit-vrf
!
vrf vrf2
 vni 200
exit-vrf
!
router bgp 64600
 bgp router-id 10.10.10.1
 neighbor BGP peer-group
 neighbor BGP remote-as internal
 neighbor 10.0.0.2 peer-group BGP
 !
 address-family ipv4 unicast
  redistribute local
  redistribute connect
  neighbor BGP activate
  neighbor BGP soft-reconfiguration inbound
  neighbor BGP allowas-in 1
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor BGP activate
  neighbor BGP soft-reconfiguration inbound
  advertise-all-vni
 exit-address-family
exit
!
router bgp 64600 vrf vrf1
 bgp router-id 10.10.10.1
 neighbor LB peer-group
 neighbor LB remote-as 64600
 bgp listen range 10.0.110.0/24 peer-group LB
 !
 address-family ipv4 unicast
  neighbor LB activate
  redistribute connected
 exit-address-family
 !
 address-family ipv6 unicast
  neighbor LB activate
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
  advertise ipv6 unicast
  rd 64600:100
  route-target import 64600:100
  route-target import 64600:200
  route-target export 64600:100
 exit-address-family
!
router bgp 64600 vrf vrf2
 bgp router-id 10.10.10.1
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family ipv6 unicast
  neighbor LB activate
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
  advertise ipv6 unicast
  rd 64600:200
  route-target import 64600:100
  route-target import 64600:200
  route-target export 64600:200
 exit-address-family
exit
!
ip prefix-list loopbacks_ips seq 10 permit 0.0.0.0/0 le 32
!
route-map MAP_VTEP_IN permit 1
exit
!
route-map MAP_VTEP_OUT permit 1
exit
!
route-map correct_src permit 1
 match ip address prefix-list loopbacks_ips
 set src 10.10.10.1
exit
!
ip nht resolve-via-default
!
ip protocol bgp route-map correct_src
!
end

For LB1

frr defaults datacenter
log syslog informational
service integrated-vtysh-config
!
router bgp 64600
  bgp router-id 10.0.10.2
  no bgp default ipv4-unicast
  neighbor R1 peer-group
  neighbor R1 remote-as internal
  neighbor 10.0.110.1 peer-group R1
  address-family ipv4 unicast
   neighbor R1 activate
   redistribute local route-map loopbacks_ips
   redistribute connect route-map loopbacks_ips
  exit-address-family
  address-family ipv6 unicast
   neighbor R1 activate
   redistribute local route-map loopbacks_ips6
   redistribute connect route-map loopbacks_ips6
  exit-address-family
exit
!
ip prefix-list loopbacks_ips seq 10 permit 203.0.113.1/32
ipv6 prefix-list loopbacks_ips6 seq 10 permit 2001:db8:1::2/128
!
route-map loopbacks_ips permit 1
 match ip address prefix-list loopbacks_ips
exit
!
route-map loopbacks_ips6 permit 1
 match ipv6 address prefix-list loopbacks_ips6
exit
!
end

And for R2

frr defaults datacenter
log syslog informational
service integrated-vtysh-config
!
vrf vrf1
 vni 100
exit-vrf
!
vrf vrf2
 vni 200
exit-vrf
!
router bgp 64600
 bgp router-id 10.10.10.2
 neighbor BGP peer-group
 neighbor BGP remote-as internal
 neighbor 10.0.0.1 peer-group BGP
 !
 address-family ipv4 unicast
  redistribute local
  redistribute connect
  neighbor BGP activate
  neighbor BGP soft-reconfiguration inbound
  neighbor BGP allowas-in 1
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor BGP activate
  neighbor BGP soft-reconfiguration inbound
  advertise-all-vni
 exit-address-family
exit
!
router bgp 64600 vrf vrf1
 bgp router-id 10.10.10.2
 neighbor BGP peer-group
 neighbor LB peer-group
 neighbor LB remote-as 64600
 bgp listen range 10.0.10.0/24 peer-group LB
 !
 address-family ipv4 unicast
  neighbor LB activate
  neighbor LB route-map MAP_LB_IN in
  neighbor LB route-map MAP_LB_OUT out
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
  rd 64600:100
  route-target import 64600:100
  route-target import 64600:200
  route-target export 64600:100
 exit-address-family
!
router bgp 64600 vrf vrf2
 bgp router-id 10.10.10.2
 address-family ipv4 unicast
  redistribute connected
 exit-address-family
 !
 address-family l2vpn evpn
  advertise ipv4 unicast
  rd 64600:200
  route-target import 64600:100
  route-target import 64600:200
  route-target export 64600:200
 exit-address-family
exit
!
ip prefix-list lb_vips seq 10 permit 10.0.0.0/24 le 32
ip prefix-list loopbacks_ips seq 10 permit 0.0.0.0/0 le 32
!
route-map MAP_LB_IN permit 1
 match ip address prefix-list lb_vips
exit
!
route-map MAP_LB_OUT deny 1
exit
!
route-map MAP_VTEP_IN permit 1
exit
!
route-map MAP_VTEP_OUT permit 1
exit
!
route-map correct_src permit 1
 match ip address prefix-list loopbacks_ips
 set src 10.10.10.2
exit
!
ip nht resolve-via-default
!
ip protocol bgp route-map correct_src
!
end

Expected behavior

I would have liked the leak to work locally too

R1# show ip route vrf vrf1
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF vrf1:
C>* 10.0.110.0/24 is directly connected, l2vni110, 00:00:14
L>* 10.0.110.1/32 is directly connected, l2vni110, 00:00:14
B>* 203.0.113.1/32 [200/0] via 10.0.110.2, l2vni110, weight 1, 00:00:09

R1# show ip route vrf vrf2
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF vrf2:
B>* 10.0.110.0/24 [200/0] via 10.10.10.2, vx0 (vrf default) onlink, label 100, weight 1, 00:00:47
B>* 203.0.113.1/32 [200/0] via 10.0.110.2, l2vni110 (vrf 1), weight 1, 00:00:09

R2# show ip route vrf vrf1
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF vrf1:
C>* 10.0.110.0/24 is directly connected, l2vni110, 00:02:04
L>* 10.0.110.1/32 is directly connected, l2vni110, 00:02:04
B>* 10.0.110.2/32 [200/0] via 10.10.10.1, l3vni100 onlink, weight 1, 00:02:01
B>* 203.0.113.1/32 [200/0] via 10.10.10.1, l3vni100 onlink, weight 1, 00:02:00
R2#

R2# show ip route vrf vrf2
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF vrf2:
B>* 10.0.110.0/24 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:02:08
B>* 10.0.110.2/32 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:02:08
B>* 203.0.113.1/32 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:02:07

Actual behavior

R1# show ip route vrf vrf1
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF vrf1:
C>* 10.0.110.0/24 is directly connected, l2vni110, 00:00:14
L>* 10.0.110.1/32 is directly connected, l2vni110, 00:00:14
B>* 203.0.113.1/32 [200/0] via 10.0.110.2, l2vni110, weight 1, 00:00:09

R1# show ip route vrf vrf2
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF vrf2:
B>* 10.0.110.0/24 [200/0] via 10.10.10.2, vx0 (vrf default) onlink, label 100, weight 1, 00:00:47

R2# show ip route vrf vrf1
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF vrf1:
C>* 10.0.110.0/24 is directly connected, l2vni110, 00:02:04
L>* 10.0.110.1/32 is directly connected, l2vni110, 00:02:04
B>* 10.0.110.2/32 [200/0] via 10.10.10.1, l3vni100 onlink, weight 1, 00:02:01
B>* 203.0.113.1/32 [200/0] via 10.10.10.1, l3vni100 onlink, weight 1, 00:02:00
R2#

R2# show ip route vrf vrf2
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF vrf2:
B>* 10.0.110.0/24 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:02:08
B>* 10.0.110.2/32 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:02:08
B>* 203.0.113.1/32 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:02:07

Additional context

No response

Checklist

  • I have searched the open issues for this bug.
  • I have not included sensitive information in this report.
@alcelafranque alcelafranque added the triage Needs further investigation label Jan 24, 2025
@lsang6WIND
Copy link

lsang6WIND commented Jan 24, 2025 •
edited
Loading

To enable route leak, you have to send: import/export vrf/vpn command.

Have you tried, on you R1 vrf2?

address-family ipv4 unicast
import vrf

@alcelafranque
Copy link
Author

Thank you for your suggestion Isang, what surprises me is that leak's behavior when receiving EVPN routes is not the same as when the routes are transformed into local EVPNs.

In my configuration, R2 successfully imports EVPN routes from VRF1 into VRF2. The import is not performed locally only.

Yes, you're right, VRF imports can be a way, but what I've noticed is that the routes are re-exported from the VRF on which we've imported them.

When i activate import vrf vrf1 on vrf2 on R1 :
view for R2

R2# do show bgp l2vpn evpn 203.0.113.1/32
BGP routing table entry for 64600:100:[5]:[0]:[32]:[203.0.113.1]
Paths: (1 available, best #1)
  Not advertised to any peer
  Route [5]:[0]:[32]:[203.0.113.1] VNI 100
  Local
    10.10.10.1(R1) from R1(10.0.0.1) (10.10.10.1)
      Origin incomplete, metric 0, localpref 100, valid, internal, bestpath-from-AS Local, best (First path received)
      Extended Community: RT:64600:100 ET:8 Rmac:aa:bb:cc:00:00:01
      Last update: Sat Jan 25 12:15:16 2025
BGP routing table entry for 64600:200:[5]:[0]:[32]:[203.0.113.1]
Paths: (1 available, best #1)
  Not advertised to any peer
  Route [5]:[0]:[32]:[203.0.113.1] VNI 200
  Local
    10.10.10.1(R1) from R1(10.0.0.1) (10.10.10.1)
      Origin incomplete, metric 0, localpref 100, valid, internal, bestpath-from-AS Local, best (First path received)
      Extended Community: RT:64600:200 ET:8 Rmac:aa:bb:cc:00:00:01
      Last update: Sat Jan 25 12:25:57 2025

R2# show ip route vrf vrf2
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF vrf2:
B>* 10.0.110.0/24 [200/0] via 10.10.10.1, l3vni200 onlink, weight 1, 00:00:03
B>* 10.0.110.2/32 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:12:22
B>* 203.0.113.1/32 [200/0] via 10.10.10.1, l3vni200 onlink, weight 1, 00:00:03

Without import vrf

R2# do show bgp l2vpn evpn 203.0.113.1/32
BGP routing table entry for 64600:100:[5]:[0]:[32]:[203.0.113.1]
Paths: (1 available, best #1)
  Not advertised to any peer
  Route [5]:[0]:[32]:[203.0.113.1] VNI 100
  Local
    10.10.10.1(R1) from R1(10.0.0.1) (10.10.10.1)
      Origin incomplete, metric 0, localpref 100, valid, internal, bestpath-from-AS Local, best (First path received)
      Extended Community: RT:64600:100 ET:8 Rmac:aa:bb:cc:00:00:01
      Last update: Sat Jan 25 12:15:16 2025

R2# show ip route vrf vrf2
Codes: K - kernel route, C - connected, L - local, S - static,
       R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
       f - OpenFabric, t - Table-Direct,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

VRF vrf2:
B>* 10.0.110.0/24 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:00:07
B>* 10.0.110.2/32 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:13:25
B>* 203.0.113.1/32 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:00:07

@ton31337
Copy link
Member

Could you enable debug bgp vpn leak-from-vrf and debug bgp vpn leak-to-vrf and show the logs? There should be a reason why the import failed/denied.

@lsang6WIND
Copy link

Without import vrf:

  • 203.0.113.1/32 [200/0] via 10.10.10.1, vx0 (vrf default) onlink, label 100, weight 1, 00:00:07

It is normal that we have a label ?

For information, there are no bgp evpn l2vpn vrf route leaking related topotests.

@alcelafranque
Copy link
Author

alcelafranque commented Jan 27, 2025 •
edited
Loading

Could you enable debug bgp vpn leak-from-vrf and debug bgp vpn leak-to-vrf and show the logs? There should be a reason why the import failed/denied.

Of course, I'll give you the return on R1/R2 by flapping the unicast session between LB1 and R1.

R1# debug bgp vpn leak-from-vrf
enabled debug bgp vpn leak-from-vrf
R1# debug bgp vpn leak-to-vrf
enabled debug bgp vpn leak-to-vrf
R1# terminal monitor
2025-01-27 10:30:00.766 [INFO] bgpd: [N9HHH-F8H1M] %ADJCHANGE: neighbor 10.0.110.2(LB1) in vrf vrf1 Up
2025-01-27 10:30:01.817 [DEBG] bgpd: [RZCQT-4CNXA] vpn_leak_from_vrf_update: from vrf VRF vrf1
2025-01-27 10:30:01.817 [DEBG] bgpd: [P9443-89910] vpn_leak_from_vrf_update: vrf1 skipping: export not set
2025-01-27 10:30:01.817 [DEBG] bgpd: [RZCQT-4CNXA] vpn_leak_from_vrf_update: from vrf VRF vrf1
2025-01-27 10:30:01.817 [DEBG] bgpd: [P9443-89910] vpn_leak_from_vrf_update: vrf1 skipping: export not set
2025-01-27 10:30:01.817 [INFO] bgpd: [M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv4 Unicast from 10.0.110.2 in vrf vrf1
2025-01-27 10:30:01.817 [INFO] bgpd: [M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv6 Unicast from 10.0.110.2 in vrf vrf1
2025-01-27 10:30:01.817 [DEBG] bgpd: [RZCQT-4CNXA] vpn_leak_from_vrf_update: from vrf VRF vrf1
2025-01-27 10:30:01.817 [DEBG] bgpd: [P9443-89910] vpn_leak_from_vrf_update: vrf1 skipping: export not set

R2#
2025-01-27 10:33:33.960 [DEBG] bgpd: [RZCQT-4CNXA] vpn_leak_from_vrf_update: from vrf VRF vrf1
2025-01-27 10:33:33.960 [DEBG] bgpd: [MGNXE-Z5FS5] vpn_leak_from_vrf_update: vrf1 path_vrf->type=10, EC{64600:100 ET:8 Rmac:aa:bb:cc:00:00:01}
2025-01-27 10:33:33.960 [DEBG] bgpd: [P9443-89910] vpn_leak_from_vrf_update: vrf1 skipping: export not set
2025-01-27 10:33:33.960 [DEBG] bgpd: [RZCQT-4CNXA] vpn_leak_from_vrf_update: from vrf VRF vrf2
2025-01-27 10:33:33.960 [DEBG] bgpd: [MGNXE-Z5FS5] vpn_leak_from_vrf_update: vrf2 path_vrf->type=10, EC{64600:100 ET:8 Rmac:aa:bb:cc:00:00:01}
2025-01-27 10:33:33.960 [DEBG] bgpd: [P9443-89910] vpn_leak_from_vrf_update: vrf2 skipping: export not set
2025-01-27 10:33:33.960 [DEBG] bgpd: [RZCQT-4CNXA] vpn_leak_from_vrf_update: from vrf VRF vrf1
2025-01-27 10:33:33.960 [DEBG] bgpd: [MGNXE-Z5FS5] vpn_leak_from_vrf_update: vrf1 path_vrf->type=10, EC{64600:100 ET:8 Rmac:aa:bb:cc:00:00:01}
2025-01-27 10:33:33.960 [DEBG] bgpd: [P9443-89910] vpn_leak_from_vrf_update: vrf1 skipping: export not set
2025-01-27 10:33:33.960 [DEBG] bgpd: [RZCQT-4CNXA] vpn_leak_from_vrf_update: from vrf VRF vrf2
2025-01-27 10:33:33.960 [DEBG] bgpd: [MGNXE-Z5FS5] vpn_leak_from_vrf_update: vrf2 path_vrf->type=10, EC{64600:100 ET:8 Rmac:aa:bb:cc:00:00:01}
2025-01-27 10:33:33.960 [DEBG] bgpd: [P9443-89910] vpn_leak_from_vrf_update: vrf2 skipping: export not set

For information, there are no bgp evpn l2vpn vrf route leaking related topotests.

If you wish in the future I can make a topotest that can look like the lab I attached on the first message.

@ton31337
Copy link
Member

Is this related skipping: export not set?

@alcelafranque
Copy link
Author

alcelafranque commented Jan 29, 2025 •
edited
Loading

hello ton31337,

I don't know if the question is for me, at least I didn't understand it.

In my research, I understood that this configuration was enough to export VRF routes into EVPN:

 address-family l2vpn evpn
  advertise ipv4 unicast
  advertise ipv6 unicast

And in this case, the EVPN export/import seems to work well for neighbors that receive the EVPN routes.

I attempted the following configuration:

On VRF1:

 address-family ipv4 unicast
  redistribute connected
  rt vpn export 64600:100
  export vpn
 exit-address-family

And on VRF2:

 address-family ipv4 unicast
  redistribute connected
  rt vpn import 64600:100
  import vpn
 exit-address-family

Without success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
triage Needs further investigation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ton31337 @alcelafranque @lsang6WIND