From 4ce467d4201233f8c6878acc040f15d96f12d467 Mon Sep 17 00:00:00 2001 From: kkayhan Date: Tue, 9 Jul 2024 22:53:34 +0200 Subject: [PATCH 1/3] l3evpn tutorial md files --- docs/tutorials/l3evpn/aboutl3evpn.md | 38 + docs/tutorials/l3evpn/fabric.md | 1053 ++++++++++++++++++++++++++ docs/tutorials/l3evpn/intro.md | 95 +++ docs/tutorials/l3evpn/ipvrf.md | 426 +++++++++++ 4 files changed, 1612 insertions(+) create mode 100644 docs/tutorials/l3evpn/aboutl3evpn.md create mode 100644 docs/tutorials/l3evpn/fabric.md create mode 100644 docs/tutorials/l3evpn/intro.md create mode 100644 docs/tutorials/l3evpn/ipvrf.md diff --git a/docs/tutorials/l3evpn/aboutl3evpn.md b/docs/tutorials/l3evpn/aboutl3evpn.md new file mode 100644 index 00000000..570fb6e7 --- /dev/null +++ b/docs/tutorials/l3evpn/aboutl3evpn.md @@ -0,0 +1,38 @@ +--- +comments: true +--- + + + +In this tutorial, the primary focus was deploying Layer 3 Ethernet VPN (EVPN) using Nokia's SR Linux. This networking solution enables the creation of a distributed routing instance spanning across multiple routers leveraging Virtual Extensible LAN (VxLAN) tunnels over an IP backbone. + +A critical part of this architecture's success lies in the use of different EVPN route types. These route types facilitate the dissemination of MAC addresses, IP prefixes, and Ethernet segments across the network, contributing to optimal forwarding decisions and redundancy mechanisms. + +In EVPN L3 Network Instances, EVPN RT-5 ( Route Type 5 ) is used to advertise IP prefixes. EVPN RT-2 may also be used for a specific purpose that we will explain in the section below. + +There are two models for implementing L3 routing with EVPN. + +## Interface-less ( IFL ) +Interface-less model only uses EVPN route type 5 to announce prefixes that populate the routing tables. If we look at the packet flow, the packet arrives from the client to the ip-vrf (L3 EVPN instance) and a route-table lookup is done, next hop resolves into a VxLAN tunnel and destination EVPN instance is identified by the VNI (Virtual Network Identifier). This model is simple and very similar to IP-VPN. SRLinux currently supports this model. + +

+ Overlay Diagram +

+ + +## Interface-ful ( IFF ) +Interface-ful model employs a specific L2 EVPN service known as "Supplementary Broadcast Domain" (SBD) to join L3 EVPN instances. Each L3 EVPN instance is linked via an IRB interface to the SBD, which serves as a central backbone connecting all L3 instances throughout the data center. The routing table entries for L3 instances are resolved into local-IRB interfaces, and through a further recursive lookup on the SBD, packets are directed to a remote-IRB interface, with the next hop being a VxLAN tunnel. + +Prefixes are announced using EVPN Route Type 5, similar to the Interface-less model. Besides that, IRB interface’s reachability information is announced using Route Type 2. Essentially RT-5 routes (client prefixes) resolve into the destinations (IRB) announced by RT-2. + +The advantage of this approach is that if a device fails, all prefixes received by RT-5 need to be invalidated. Since these prefixes resolve into an IRB interface announced by RT-2, we can invalidate all impacted prefixes by withdrawing the RT-2. Instead of sending withdrawal messages for thousands of RT-5 routes, we send a single withdrawal for the RT-2, significantly improving convergence. + +Like we said above the reachability information of IRB interfaces are announced using RT-2 ( MAC/IP route ). +The user either has to configure an IP address for each IRB interface and the IRB MAC/IP is announced to other peers via RT-2. +It is also possible to have the IRB interface without an IP address, in this way only a MAC address is announced via RT-2. This is called Interface-ful unnumbered model. + +When RT-5 announces prefixes, it must include the IRB-IP or IRB-MAC in the RT-5 communities so that the peer knows which IRB interface to use for prefix resolution. + +

+ Overlay Diagram +

\ No newline at end of file diff --git a/docs/tutorials/l3evpn/fabric.md b/docs/tutorials/l3evpn/fabric.md new file mode 100644 index 00000000..7a77581c --- /dev/null +++ b/docs/tutorials/l3evpn/fabric.md @@ -0,0 +1,1053 @@ +--- +comments: true +--- + + + +# Routing for Underlay & Overlay + +Underlay routing is used for the physical network's connectivity, while overlay routing is used to create virtual networks on top of the physical infrastructure. + +## Underlay Routing + +Before setting up an EVPN overlay, it's necessary to implement a routing protocol to ensure all leaf VxLAN Termination End Points (VTEPs) are reachable across the IP fabric. + +SR Linux supports the following routing protocols for the underlay network: + +* ISIS + +* OSPF + +* BGP + +BGP is strongly recommended for data center fabrics as described in [RFC7938](https://datatracker.ietf.org/doc/html/rfc7938), offering several advantages, here are few of them that I think are the most important: + +* **Scalability:** BGP announces only the best paths, unlike IGPs that share the entire link state database. This results in lower hardware resource consumption for BGP compared to IGP. + +* **Flexible Policy:** BGP provides numerous attributes for policy matching, offering extensive options for traffic steering. + +* **Smaller Failure Impact Radius with BGP compared to IGP:** + - In case of link failure in an ISIS/OSPF network, all devices need to run SPF on the entire link state database. The failure impact radius is the whole network. + + - In case of link failure in an eBGP network, only devices one hop away need to recalculate the best path, this is because eBGP announces all routes with next-hop self and the next hop remains unchanged. The failure impact radius is only 1 hop. + +One of the disadvantage of BGP was that BGP did not have neighbor discovery like IGP protocols have. However SR Linux can automatically establish BGP peers using the BGP Unnumbered feature. BGP unnumbered involves setting up BGP sessions without allocating a specific, unique IP address for each interface engaging in a BGP session. + +BGP IPv6 Unnumbered utilizes: + +* **IPv6 Link-Local Addresses:** Employed for communication on the same network segment, these addresses aren't routed outside their segment. In unnumbered BGP configurations, interfaces use IPv6 link-local addresses to form BGP sessions without needing a unique global IP address per interface. + +* **Router Advertisements:** As part of the Neighbor Discovery Protocol, Router Advertisements enable routers to broadcast their presence and share various information about the link and the Internet Layer on an IPv6 subnet. In BGP unnumbered, RA messages are used to announce/learn the peer’s link-local address. + +In the diagram below, a Spine is dynamically peering eBGP with each Leaf using IPv6 unnumbered. This is what we will achieve at the end of this chapter. + +

+ Fabric Diagram +

+ + +### Physical Interface Configuration + +The initial step involves setting up the physical interfaces for SRLinux to connect and initiate BGP peerings with other routers in the DC fabric. + +On each leaf and spine we will bring up the relevant [interface](../../kb/ifaces.md) and configure a routed [subinterface](../../kb/ifaces.md#subinterfaces) to achieve L3 connectivity. + +We begin with connecting to the CLI of our nodes via SSH[^1]: + +```bash +# connecting to leaf1 +ssh clab-l3evpn-leaf1 +``` + +Then on each node we enter into [candidate configuration mode](../../kb/cfgmgmt.md#configuration-modes) and proceed with the relevant interfaces' configuration. + +Let's witness the step by step process of an interface configuration on a `leaf1` switch with providing the paste-able snippets for the rest of the nodes + +1. Enter the `candidate` configuration mode to make edits to the configuration + + ```srl + Welcome to the srlinux CLI. + Type 'help' (and press ) if you need any help using this. + + + --{ running }--[ ]-- + A:leaf1# enter candidate + ``` + +2. The prompt will indicate we entered the candidate data store. In the following steps we will enter the commands to make changes to the candidate config and at the end we will commit. + + ```srl + --{ candidate shared default }--[ ]-- + A:leaf1# + ``` + +4. Create a subinterface under a physical interface and enable IPv6. We will not configure an IP address but a link-local address will be automatically configured for this interface. + + ```srl + set / interface ethernet-1/49 subinterface 1 ipv6 admin-state enable + ``` + +5. Configure the interface to send router advertisement messages to directly connected peers, informing them of the interface's IP address. + + ```srl + set / interface ethernet-1/49 subinterface 1 ipv6 router-advertisement router-role admin-state enable + ``` + +6. Attach the configured subinterfaces to the default network instance (aka GRT). + + ```srl + set / network-instance default interface ethernet-1/49.1 + ``` + + +7. Apply the configuration changes by issuing a `commit now` command. The changes will be written to the running configuration. + + ```srl + commit now + ``` + +Below you will find the relevant configuration snippets[^2] for leafs and spine of our fabric which you can paste in the terminal while being in candidate mode. + +/// tab | leaf1 + +```srl +set / interface ethernet-1/49 admin-state enable +set / interface ethernet-1/49 subinterface 1 ipv6 admin-state enable +set / interface ethernet-1/49 subinterface 1 ipv6 router-advertisement router-role admin-state enable + +set / network-instance default interface ethernet-1/49.1 +``` + +/// + +/// tab | leaf2 + +```srl +set / interface ethernet-1/49 admin-state enable +set / interface ethernet-1/49 subinterface 1 ipv6 admin-state enable +set / interface ethernet-1/49 subinterface 1 ipv6 router-advertisement router-role admin-state enable + +set / network-instance default interface ethernet-1/49.1 +``` + +/// + +/// tab | spine + +```srl +set / interface ethernet-1/1 admin-state enable +set / interface ethernet-1/1 subinterface 1 ipv6 admin-state enable +set / interface ethernet-1/1 subinterface 1 ipv6 router-advertisement router-role admin-state enable + +set / interface ethernet-1/2 admin-state enable +set / interface ethernet-1/2 subinterface 1 ipv6 admin-state enable +set / interface ethernet-1/2 subinterface 1 ipv6 router-advertisement router-role admin-state enable + +set / network-instance default interface ethernet-1/1.1 +set / network-instance default interface ethernet-1/2.1 +``` + +/// + +Once those snippets are committed to the running configuration with `commit now` command, we can ensure that the changes have been applied by showing the interface status. + +Below highlighted, you will see that an IPv6 link-layer address is auto assigned to each interface. This address is not routable and is not announced to other peers by default. + +/// tab | leaf1 + +```srl hl_lines="10" +--{ running }--[ ]-- +A:leaf1# show interface ethernet-1/49 +==================================================================== +ethernet-1/49 is up, speed 100G, type None + ethernet-1/49.1 is up + Network-instances: + * Name: default (default) + Encapsulation : null + Type : routed + IPv6 addr : fe80::1805:2ff:feff:31/64 (link-layer, preferred) +-------------------------------------------------------------------- +==================================================================== + +--{ running }--[ ]-- + +``` + +/// + +/// tab | leaf2 + +```srl hl_lines="10" +--{ running }--[ ]-- +A:leaf2# show interface ethernet-1/49 +==================================================================== +ethernet-1/49 is up, speed 100G, type None + ethernet-1/49.1 is up + Network-instances: + * Name: default (default) + Encapsulation : null + Type : routed + IPv6 addr : fe80::18d9:3ff:feff:31/64 (link-layer, preferred) +-------------------------------------------------------------------- +==================================================================== + +--{ running }--[ ]-- +``` + +/// + +/// tab | spine + +```srl hl_lines="10 18" + +--{ running }--[ ]-- +A:spine# show interface ethernet-1/{1..2} +==================================================================== +ethernet-1/1 is up, speed 100G, type None + ethernet-1/1.1 is up + Network-instances: + * Name: default (default) + Encapsulation : null + Type : routed + IPv6 addr : fe80::181d:4ff:feff:1/64 (link-layer, preferred) +-------------------------------------------------------------------- +ethernet-1/2 is up, speed 100G, type None + ethernet-1/2.1 is up + Network-instances: + * Name: default (default) + Encapsulation : null + Type : routed + IPv6 addr : fe80::181d:4ff:feff:2/64 (link-layer, preferred) +-------------------------------------------------------------------- +==================================================================== + +--{ running }--[ ]-- +``` + +/// + +Below is the ARP/ND neighbors list, constructed from the received Router Advertisement messages from the link. The critical information is the neighbor IP address. + +/// tab | leaf1 + +```srl +--{ running }--[ ]-- +A:leaf1# show arpnd neighbors interface ethernet-1/49 ++-------------------+-------------------+----------------------------------------+-------------------+--------------------------------------+-------------------+--------------------------------------+-------------------+ +| Interface | Subinterface | Neighbor | Origin | Link layer address | Current state | Next state change | Is Router | ++===================+===================+========================================+===================+======================================+===================+======================================+===================+ +| ethernet-1/49 | 1 | fe80::181d:4ff:feff:1 | dynamic | 1A:1D:04:FF:00:01 | stale | 3 hours from now | true | ++-------------------+-------------------+----------------------------------------+-------------------+--------------------------------------+-------------------+--------------------------------------+-------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + Total entries : 1 (0 static, 1 dynamic) +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + +--{ running }--[ ]-- +``` + +/// + +/// tab | leaf2 + +```srl +--{ running }--[ ]-- +A:leaf2# show arpnd neighbors interface ethernet-1/49 ++-------------------+-------------------+----------------------------------------+-------------------+--------------------------------------+-------------------+--------------------------------------+-------------------+ +| Interface | Subinterface | Neighbor | Origin | Link layer address | Current state | Next state change | Is Router | ++===================+===================+========================================+===================+======================================+===================+======================================+===================+ +| ethernet-1/49 | 1 | fe80::181d:4ff:feff:2 | dynamic | 1A:1D:04:FF:00:02 | stale | 3 hours from now | false | ++-------------------+-------------------+----------------------------------------+-------------------+--------------------------------------+-------------------+--------------------------------------+-------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + Total entries : 1 (0 static, 1 dynamic) +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + +--{ running }--[ ]-- +``` + +/// + +/// tab | spine + +```srl +--{ running }--[ ]-- +A:spine# show arpnd neighbors interface ethernet-1/{1..2} ++-------------------+-------------------+----------------------------------------+-------------------+--------------------------------------+-------------------+--------------------------------------+-------------------+ +| Interface | Subinterface | Neighbor | Origin | Link layer address | Current state | Next state change | Is Router | ++===================+===================+========================================+===================+======================================+===================+======================================+===================+ +| ethernet-1/1 | 1 | fe80::1805:2ff:feff:31 | dynamic | 1A:05:02:FF:00:31 | stale | 3 hours from now | true | +| ethernet-1/2 | 1 | fe80::18d9:3ff:feff:31 | dynamic | 1A:D9:03:FF:00:31 | stale | 3 hours from now | false | ++-------------------+-------------------+----------------------------------------+-------------------+--------------------------------------+-------------------+--------------------------------------+-------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + Total entries : 2 (0 static, 2 dynamic) +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + +--{ running }--[ ]-- +``` + +/// + + +### eBGP Unnumbered for Underlay Routing + +Now we will set up the routing protocol that will be used for exchang loopback addresses throughout the fabric. These loopbacks will be used to set up iBGP EVPN peerings, which we will cover in the following chapter. + +Here is a breakdown of the steps that are needed to configure eBGP on `leaf1` towards `spine`: + +1. **Assign Autonomous System Number** + We will use eBGP, so each router needs a unique base AS number. Typically, leaf pairs share the same unique AS number, as do spine routers. This configuration prevents routing loops, as routes announced between leaf pairs or between spines are ignored. + + ```srl + set / network-instance default protocols bgp autonomous-system 4200000001 + ``` + +1. **Assign a unique Router ID** + This is the BGP identifier reported to peers when this network-instance opens a BGP session towards another router. + + ```srl + set / network-instance default protocols bgp router-id 100.0.0.1 + ``` + +1. **Enable Address Family** + Currently only VxLAN v4 is supported and therefore we need the BGP IPv4 family to exchange the IPv4 loopback IPs that are needed for VxLAN termination. + + ```srl + set / network-instance default protocols bgp afi-safi ipv4-unicast admin-state enable + ``` + +1. **Allow Route Advertisement for eBGP** + eBGP assumes that peers are external systems and by default all incoming and outgoing routes are blocked. We will disable this behavior and permit all incoming and outgoing routes. + + ```srl + set / network-instance default protocols bgp eBGP-default-policy import-reject-all false + set / network-instance default protocols bgp eBGP-default-policy export-reject-all false + ``` + +1. **Create BGP peer-group** + A BGP peer group simplifies configuring multiple BGP peers with similar requirements by grouping them together, allowing the same policies and attributes to be applied to all peers in the group simultaneously. Here we create a group named underlay to be used for the eBGP peerings. + + ```srl + set / network-instance default protocols bgp group underlay + ``` + +1. **Configure dynamic neighbor** + We will configure the dynamic neighbor feature to establish eBGP over leaf-spine links. + + ```srl + set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 + ``` + + Then we assign this interface to the BGP group + ```srl + set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 peer-group underlay + ``` + + And we define the AS range for this router should accept dynamic peering from, in this case we defined the whole range. + ```srl + set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 allowed-peer-as [ 1..4294967295 ] + ``` + + **{Optional}** It is also possible to only allow peers that match a certain prefix. + ```srl + set / network-instance default protocols bgp dynamic-neighbors accept match fe80::/10 peer-group underlay + ``` + +2. **Commit configuration** + + Once we apply the config above (whole snippet below), we should have BGP peerings automatically established. + + ```srl + --{ +* candidate shared default }--[ network-instance default protocols bgp ]-- + A:leaf1# commit now + ``` + +Here are the config snippets per device for easy copy paste: + +/// tab | leaf1 +```srl +set / network-instance default protocols bgp autonomous-system 4200000001 +set / network-instance default protocols bgp router-id 100.0.0.1 + +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 peer-group underlay +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 allowed-peer-as [ 1..4294967295 ] + +set / network-instance default protocols bgp eBGP-default-policy import-reject-all false +set / network-instance default protocols bgp eBGP-default-policy export-reject-all false + +set / network-instance default protocols bgp afi-safi ipv4-unicast admin-state enable + +set / network-instance default protocols bgp group underlay +``` +/// + +/// tab | leaf2 +```srl +set / network-instance default protocols bgp autonomous-system 4200000002 +set / network-instance default protocols bgp router-id 100.0.0.2 + +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 peer-group underlay +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 allowed-peer-as [ 1..4294967295 ] + +set / network-instance default protocols bgp eBGP-default-policy import-reject-all false +set / network-instance default protocols bgp eBGP-default-policy export-reject-all false + +set / network-instance default protocols bgp afi-safi ipv4-unicast admin-state enable + +set / network-instance default protocols bgp group underlay +``` +/// + +/// tab | spine +```srl +set / network-instance default protocols bgp autonomous-system 65000 +set / network-instance default protocols bgp router-id 100.100.100.100 + +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/1.1 peer-group underlay +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/1.1 allowed-peer-as [ 1..4294967295 ] +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/2.1 peer-group underlay +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/2.1 allowed-peer-as [ 1..4294967295 ] + +set / network-instance default protocols bgp eBGP-default-policy import-reject-all false +set / network-instance default protocols bgp eBGP-default-policy export-reject-all false + +set / network-instance default protocols bgp afi-safi ipv4-unicast admin-state enable +set / network-instance default protocols bgp afi-safi ipv4-unicast export-policy announce_system_IP + +set / network-instance default protocols bgp group underlay +``` +/// + +### Loopback Interface for EVPN peering and VxLAN Termination + +As we will create an iBGP based EVPN control plane at a later stage, we need to configure loopback addresses for our leaf devices so that they can build an iBGP peering over those interfaces. + +In the context of the VXLAN data plane, a special kind of a loopback needs to be created - [`system0`](../../kb/ifaces.md#system) interface. + +/// note +The `system0.0` interface hosts the loopback address used to originate and typically +terminate VXLAN packets. This address is also used by default as the next-hop of all +EVPN routes. +/// + +Configuration of the `system0` interface is exactly the same as for the regular interfaces. As a best practice, the IPv4 addresses assigned to `system0` interfaces will be identical to the Router-ID. + +/// tab | leaf1 + +```srl +set / interface system0 subinterface 0 ipv4 admin-state enable +set / interface system0 subinterface 0 ipv4 address 100.0.0.1/32 + +``` + +/// + +/// tab | leaf2 + +```srl +set / interface system0 subinterface 0 ipv4 admin-state enable +set / interface system0 subinterface 0 ipv4 address 100.0.0.2/32 + +``` + +/// + +/// tab | spine + +```srl +set / interface system0 subinterface 0 ipv4 admin-state enable +set / interface system0 subinterface 0 ipv4 address 100.100.100.100/32 +``` + +/// + + +Then we will need to add the loopback/system interface to the default network instance + +```srl +set / network-instance default interface system0.0 +``` + +By default, BGP does not announce the local interface IPs, therefore we need an export policy. The policy below will announce all locally configured interfaces on the network instance but since link-local addresses are not announced and that is all we have, only the system IP (loopback) will be announced to the peers. If you wish to, you can configure a more specific export policy matching a prefix list. + +```srl +set / routing-policy policy announce_system_IP statement 1 match protocol local +set / routing-policy policy announce_system_IP statement 1 action policy-result accept + +set / network-instance default protocols bgp afi-safi ipv4-unicast export-policy announce_system_IP +``` + +After committing the configs above, a System Interface will be configured with a unique IP address (identical to the BGP Router ID as best practice) and that IP will be exported to the eBGP and announced to the neighbors. + +**Allow IPv4 packets on IPv6-only Interfaces** + +The fabric will use IPv6 interfaces to route IPv4 packets. By default, SRLinux drops IPv4 packets if the receiving interface lacks an operational IPv4 subinterface. To change this and allow IPv4 packets on IPv6-only interfaces, use the following system-wide config knob. + +```srl +set / network-instance default ip-forwarding receive-ipv4-check false +``` + + +### Verification + +As stated in the beginning of this section, the VxLAN VTEPs need to be advertised throughout the DC fabric. The `system0` interfaces we just configured are the VTEPs and they should be advertised via eBGP peering established before. The following verification commands can help ensure that. + +**BGP neighbor status** + +First, verify that the eBGP peerings are in the established state using BGP Family IPv4-Unicast. Note that all peerings are dynamic, automatically configured using the dynamic-peering feature. + +/// tab | leaf1 +```srl +A:leaf1# show network-instance default protocols bgp neighbor +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +BGP neighbor summary for network-instance "default" +Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++------------------------+-----------------------------------+------------------------+--------+-------------+-------------------+-------------------+-----------------+-----------------------------------+ +| Net-Inst | Peer | Group | Flags | Peer-AS | State | Uptime | AFI/SAFI | [Rx/Active/Tx] | ++========================+===================================+========================+========+=============+===================+===================+=================+===================================+ +| default | fe80::181d:4ff:feff:1%ethernet- | underlay | D | 65000 | established | 0d:0h:5m:3s | ipv4-unicast | [2/2/1] | +| | 1/49.1 | | | | | | | | ++------------------------+-----------------------------------+------------------------+--------+-------------+-------------------+-------------------+-----------------+-----------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Summary: +0 configured neighbors, 0 configured sessions are established,0 disabled peers +1 dynamic peers +``` +/// + +/// tab | leaf2 +```srl +A:leaf2# show network-instance default protocols bgp neighbor +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +BGP neighbor summary for network-instance "default" +Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++------------------------+-----------------------------------+------------------------+--------+-------------+-------------------+-------------------+-----------------+-----------------------------------+ +| Net-Inst | Peer | Group | Flags | Peer-AS | State | Uptime | AFI/SAFI | [Rx/Active/Tx] | ++========================+===================================+========================+========+=============+===================+===================+=================+===================================+ +| default | fe80::181d:4ff:feff:2%ethernet- | underlay | D | 65000 | established | 0d:0h:7m:59s | ipv4-unicast | [2/2/1] | +| | 1/49.1 | | | | | | | | ++------------------------+-----------------------------------+------------------------+--------+-------------+-------------------+-------------------+-----------------+-----------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Summary: +0 configured neighbors, 0 configured sessions are established,0 disabled peers +1 dynamic peers +``` +/// + +/// tab | spine +```srl +A:spine# show network-instance default protocols bgp neighbor +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +BGP neighbor summary for network-instance "default" +Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++------------------------+-----------------------------------+------------------------+--------+-------------+-------------------+-------------------+-----------------+-----------------------------------+ +| Net-Inst | Peer | Group | Flags | Peer-AS | State | Uptime | AFI/SAFI | [Rx/Active/Tx] | ++========================+===================================+========================+========+=============+===================+===================+=================+===================================+ +| default | fe80::1805:2ff:feff:31%ethernet- | underlay | D | 4200000001 | established | 0d:0h:6m:28s | ipv4-unicast | [1/1/2] | +| | 1/1.1 | | | | | | | | +| default | fe80::18d9:3ff:feff:31%ethernet- | underlay | D | 4200000002 | established | 0d:0h:8m:27s | ipv4-unicast | [1/1/2] | +| | 1/2.1 | | | | | | | | ++------------------------+-----------------------------------+------------------------+--------+-------------+-------------------+-------------------+-----------------+-----------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Summary: +0 configured neighbors, 0 configured sessions are established,0 disabled peers +2 dynamic peers +``` +/// + + +**Advertised routes** + +We configured eBGP in the fabric's underlay to advertise the VxLAN tunnel endpoints. The output below verifies that the routers are advertising the prefix of the `system0` interface to their eBGP peers: + + +/// tab | leaf1 +```srl hl_lines="12" +A:leaf1# show network-instance default protocols bgp neighbor fe80::181d:4ff:feff:1%ethernet-1/49.1 advertised-routes ipv4 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Peer : fe80::181d:4ff:feff:1%ethernet-1/49.1, remote AS: 65000, local AS: 4200000001 +Type : static +Description : None +Group : underlay +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Origin codes: i=IGP, e=EGP, ?=incomplete ++----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Network Path-id Next Hop MED LocPref AsPath Origin | ++============================================================================================================================================================================================================================+ +| 100.0.0.1/32 0 fe80::1805:2ff:feff - 100 [4200000001] i | +| :31 | ++----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +1 advertised BGP routes +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +``` +/// + +/// tab | leaf2 +```srl hl_lines="12" +A:leaf2# show network-instance default protocols bgp neighbor fe80::181d:4ff:feff:2%ethernet-1/49.1 advertised-routes ipv4 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Peer : fe80::181d:4ff:feff:2%ethernet-1/49.1, remote AS: 65000, local AS: 4200000002 +Type : static +Description : None +Group : underlay +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Origin codes: i=IGP, e=EGP, ?=incomplete ++----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Network Path-id Next Hop MED LocPref AsPath Origin | ++============================================================================================================================================================================================================================+ +| 100.0.0.2/32 0 fe80::18d9:3ff:feff - 100 [4200000002] i | +| :31 | ++----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +1 advertised BGP routes +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +``` +/// + +/// tab | spine +```srl hl_lines="12 14 33 35" +A:spine# show network-instance default protocols bgp neighbor fe80::1805:2ff:feff:31%ethernet-1/1.1 advertised-routes ipv4 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Peer : fe80::1805:2ff:feff:31%ethernet-1/1.1, remote AS: 4200000001, local AS: 65000 +Type : static +Description : None +Group : underlay +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Origin codes: i=IGP, e=EGP, ?=incomplete ++----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Network Path-id Next Hop MED LocPref AsPath Origin | ++============================================================================================================================================================================================================================+ +| 100.0.0.2/32 0 fe80::181d:4ff:feff - 100 [65000, 4200000002] i | +| :1 | +| 100.100.100.100/32 0 fe80::181d:4ff:feff - 100 [65000] i | +| :1 | ++----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +2 advertised BGP routes +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + + +A:spine# show network-instance default protocols bgp neighbor fe80::18d9:3ff:feff:31%ethernet-1/2.1 advertised-routes ipv4 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Peer : fe80::18d9:3ff:feff:31%ethernet-1/2.1, remote AS: 4200000002, local AS: 65000 +Type : static +Description : None +Group : underlay +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Origin codes: i=IGP, e=EGP, ?=incomplete ++----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Network Path-id Next Hop MED LocPref AsPath Origin | ++============================================================================================================================================================================================================================+ +| 100.0.0.1/32 0 fe80::181d:4ff:feff - 100 [65000, 4200000001] i | +| :2 | +| 100.100.100.100/32 0 fe80::181d:4ff:feff - 100 [65000] i | +| :2 | ++----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +2 advertised BGP routes +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +``` +/// + +**Route table** + +The last stop in the control plane verification would be to check if the remote loopback prefixes were installed in the `default` network-instance where we expect them to be: + + +/// tab | leaf1 +```srl hl_lines="11 13" +A:leaf1# show network-instance default route-table +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 unicast route table of network instance default +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +| Prefix | ID | Route Type | Route Owner | Active | Origin | Metric | Pref | Next-hop (Type) | Next-hop Interface | Backup Next-hop | Backup Next-hop Interface | +| | | | | | Network | | | | | (Type) | | +| | | | | | Instance | | | | | | | ++=================================+=======+============+======================+==========+==========+=========+============+====================+====================+====================+====================================+ +| 100.0.0.1/32 | 4 | host | net_inst_mgr | True | default | 0 | 0 | None (extract) | None | | | +| 100.0.0.2/32 | 0 | bgp | bgp_mgr | True | default | 0 | 170 | fe80::181d:4ff:fef | ethernet-1/49.1 | | | +| | | | | | | | | f:1 (direct) | | | | +| 100.100.100.100/32 | 0 | bgp | bgp_mgr | True | default | 0 | 170 | fe80::181d:4ff:fef | ethernet-1/49.1 | | | +| | | | | | | | | f:1 (direct) | | | | ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 routes total : 3 +IPv4 prefixes with active routes : 3 +IPv4 prefixes with active ECMP routes: 0 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +``` +/// + + +/// tab | leaf2 +```srl hl_lines="10 13" +A:leaf2# show network-instance default route-table +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 unicast route table of network instance default +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +| Prefix | ID | Route Type | Route Owner | Active | Origin | Metric | Pref | Next-hop (Type) | Next-hop Interface | Backup Next-hop | Backup Next-hop Interface | +| | | | | | Network | | | | | (Type) | | +| | | | | | Instance | | | | | | | ++=================================+=======+============+======================+==========+==========+=========+============+====================+====================+====================+====================================+ +| 100.0.0.1/32 | 0 | bgp | bgp_mgr | True | default | 0 | 170 | fe80::181d:4ff:fef | ethernet-1/49.1 | | | +| | | | | | | | | f:2 (direct) | | | | +| 100.0.0.2/32 | 4 | host | net_inst_mgr | True | default | 0 | 0 | None (extract) | None | | | +| 100.100.100.100/32 | 0 | bgp | bgp_mgr | True | default | 0 | 170 | fe80::181d:4ff:fef | ethernet-1/49.1 | | | +| | | | | | | | | f:2 (direct) | | | | ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 routes total : 3 +IPv4 prefixes with active routes : 3 +IPv4 prefixes with active ECMP routes: 0 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +``` +/// + +/// tab | spine +```srl hl_lines="10 12" +A:spine# show network-instance default route-table +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 unicast route table of network instance default +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +| Prefix | ID | Route Type | Route Owner | Active | Origin | Metric | Pref | Next-hop (Type) | Next-hop Interface | Backup Next-hop | Backup Next-hop Interface | +| | | | | | Network | | | | | (Type) | | +| | | | | | Instance | | | | | | | ++=================================+=======+============+======================+==========+==========+=========+============+====================+====================+====================+====================================+ +| 100.0.0.1/32 | 0 | bgp | bgp_mgr | True | default | 0 | 170 | fe80::1805:2ff:fef | ethernet-1/1.1 | | | +| | | | | | | | | f:31 (direct) | | | | +| 100.0.0.2/32 | 0 | bgp | bgp_mgr | True | default | 0 | 170 | fe80::18d9:3ff:fef | ethernet-1/2.1 | | | +| | | | | | | | | f:31 (direct) | | | | +| 100.100.100.100/32 | 4 | host | net_inst_mgr | True | default | 0 | 0 | None (extract) | None | | | ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 routes total : 3 +IPv4 prefixes with active routes : 3 +IPv4 prefixes with active ECMP routes: 0 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +``` +/// + +Both learned prefixes appear in the route table of the default network instance, with bgp_mgr as their owner, indicating they were added by the BGP process. The system0 interface prefix owner is the network instance manager, signifying it is a local prefix. + +**Dataplane** + +To finish the verification process let's ensure that the datapath is indeed working, and the VTEPs on both leafs can reach each other via the routed fabric underlay. + +For that we will use the `ping` command with src/dst set to loopback addresses: + +``` +A:leaf1# ping network-instance default 100.0.0.2 -I 100.0.0.1 -c 3 +Using network instance default +PING 100.0.0.2 (100.0.0.2) from 100.0.0.1 : 56(84) bytes of data. +64 bytes from 100.0.0.2: icmp_seq=1 ttl=63 time=4.72 ms +64 bytes from 100.0.0.2: icmp_seq=2 ttl=63 time=5.71 ms +64 bytes from 100.0.0.2: icmp_seq=3 ttl=63 time=5.64 ms + +--- 100.0.0.2 ping statistics --- +3 packets transmitted, 3 received, 0% packet loss, time 2004ms +rtt min/avg/max/mdev = 4.722/5.355/5.707/0.448 ms + +``` + +Perfect, the VTEPs are reachable and the fabric underlay is properly configured. We can proceed with EVPN service configuration! + +### Resulting configs + +Below you will find aggregated configuration snippets which contain the entire fabric configuration we did in the steps above. Those snippets are in the _flat_ format and were extracted with `info flat` command. + +/// note +`enter candidate` and `commit now` commands are part of the snippets, so it is possible to paste them right after you logged into the devices as well as the changes will get committed to running config. +/// + +/// tab | leaf1 + +```srl +enter candidate + +# Allow IPv4 packets on an IPv6 interface +set / network-instance default ip-forwarding receive-ipv4-check false + +# Configure the link and enable the IPv6 subinterface +set / interface ethernet-1/49 admin-state enable +set / interface ethernet-1/49 subinterface 1 ipv6 admin-state enable +set / interface ethernet-1/49 subinterface 1 ipv6 router-advertisement router-role admin-state enable + +# Configure the system interface +set / interface system0 subinterface 0 ipv4 admin-state enable +set / interface system0 subinterface 0 ipv4 address 100.0.0.1/32 + +# Add interfaces to default routing instance +set / network-instance default interface ethernet-1/49.1 +set / network-instance default interface system0.0 + +# Policy to export local routes (system) to BGP neighbors +set / routing-policy policy announce_system_IP statement 1 match protocol local +set / routing-policy policy announce_system_IP statement 1 action policy-result accept + +# BGP Configuration +set / network-instance default protocols bgp autonomous-system 4200000001 +set / network-instance default protocols bgp router-id 100.0.0.1 +set / network-instance default protocols bgp group underlay + +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 peer-group underlay +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 allowed-peer-as [ 1..4294967295 ] + +set / network-instance default protocols bgp ebgp-default-policy import-reject-all false +set / network-instance default protocols bgp ebgp-default-policy export-reject-all false + +set / network-instance default protocols bgp afi-safi ipv4-unicast admin-state enable +set / network-instance default protocols bgp afi-safi ipv4-unicast export-policy announce_system_IP + +commit now +``` + +/// + +/// tab | leaf2 + +```srl +enter candidate + +# Allow IPv4 packets on an IPv6 interface +set / network-instance default ip-forwarding receive-ipv4-check false + +# Configure the link and enable the IPv6 subinterface +set / interface ethernet-1/49 admin-state enable +set / interface ethernet-1/49 subinterface 1 ipv6 admin-state enable +set / interface ethernet-1/49 subinterface 1 ipv6 router-advertisement router-role admin-state enable + +# Configure the system interface +set / interface system0 subinterface 0 ipv4 admin-state enable +set / interface system0 subinterface 0 ipv4 address 100.0.0.2/32 + +# Add interfaces to default routing instance +set / network-instance default interface ethernet-1/49.1 +set / network-instance default interface system0.0 + +# Policy to export local routes (system) to BGP neighbors +set / routing-policy policy announce_system_IP statement 1 match protocol local +set / routing-policy policy announce_system_IP statement 1 action policy-result accept + +# BGP Configuration +set / network-instance default protocols bgp autonomous-system 4200000002 +set / network-instance default protocols bgp router-id 100.0.0.2 +set / network-instance default protocols bgp group underlay + +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 peer-group underlay +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/49.1 allowed-peer-as [ 1..4294967295 ] + +set / network-instance default protocols bgp ebgp-default-policy import-reject-all false +set / network-instance default protocols bgp ebgp-default-policy export-reject-all false + +set / network-instance default protocols bgp afi-safi ipv4-unicast admin-state enable +set / network-instance default protocols bgp afi-safi ipv4-unicast export-policy announce_system_IP + +commit now +``` + +/// + +/// tab | spine + +```srl +enter candidate + +# Allow IPv4 packets on an IPv6 interface +set / network-instance default ip-forwarding receive-ipv4-check false + +# Configure the link and enable the IPv6 subinterface +set / interface ethernet-1/1 admin-state enable +set / interface ethernet-1/1 subinterface 1 ipv6 admin-state enable +set / interface ethernet-1/1 subinterface 1 ipv6 router-advertisement router-role admin-state enable +set / interface ethernet-1/2 admin-state enable +set / interface ethernet-1/2 subinterface 1 ipv6 admin-state enable +set / interface ethernet-1/2 subinterface 1 ipv6 router-advertisement router-role admin-state enable + + +# Configure the system interface +set / interface system0 subinterface 0 ipv4 admin-state enable +set / interface system0 subinterface 0 ipv4 address 100.100.100.100/32 + +# Add interfaces to default routing instance +set / network-instance default interface ethernet-1/1.1 +set / network-instance default interface ethernet-1/2.1 +set / network-instance default interface system0.0 + +# Policy to export local routes (system) to BGP neighbors +set / routing-policy policy announce_system_IP statement 1 match protocol local +set / routing-policy policy announce_system_IP statement 1 action policy-result accept + +# BGP Configuration +set / network-instance default protocols bgp autonomous-system 65000 +set / network-instance default protocols bgp router-id 100.100.100.100 + +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/1.1 peer-group underlay +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/1.1 allowed-peer-as [ 1..4294967295 ] + +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/2.1 peer-group underlay +set / network-instance default protocols bgp dynamic-neighbors interface ethernet-1/2.1 allowed-peer-as [ 1..4294967295 ] + +set / network-instance default protocols bgp ebgp-default-policy import-reject-all false +set / network-instance default protocols bgp ebgp-default-policy export-reject-all false + +set / network-instance default protocols bgp afi-safi ipv4-unicast admin-state enable +set / network-instance default protocols bgp afi-safi ipv4-unicast export-policy announce_system_IP + +set / network-instance default protocols bgp group underlay + +commit now +``` + +/// + +## Overlay Routing + +The BGP EVPN family facilitates the exchange of overlay routes. Further details on EVPN and its mechanisms will be discussed in subsequent chapter. In this section we will focus on how BGP EVPN is configured. + +Typically, Route Reflectors (RRs) are used for iBGP peering instead of configuring a full mesh. Utilizing RRs reduces the number of BGP sessions, requiring only peering with RRs. This approach minimizes configuration efforts and allows for centralized application of routing policies. + +In our case Spine will be the EVPN RR and Leaves will be the client. + +1. **Create BGP peer-group** + A BGP peer group simplifies configuring multiple BGP peers with similar requirements by grouping them together. + + ```srl + set / network-instance default protocols bgp group overlay + ``` + +1. **Assign Autonomous System Number** + We'll use iBGP with the EVPN family, meaning all routers in this data center will share the same AS number for overlay route exchange. + + ```srl + set / network-instance default protocols bgp group overlay peer-as 55555 + set / network-instance default protocols bgp group overlay local-as as-number 55555 + ``` + +1. **Enable Address Family** + Here we are enabling the EVPN address family and disabling the IPv4 family for the overlay BGP group. + + ```srl + set / network-instance default protocols bgp group evpn afi-safi evpn admin-state enable + set / network-instance default protocols bgp group evpn afi-safi ipv4-unicast admin-state disable + ``` + +1. **Configure the neighbor** + + /// tab | leaf1 & leaf2 + Leaf devices uses Spine's System IP for BGP EVPN peering. + ```srl + set / network-instance default protocols bgp neighbor 100.100.100.100 admin-state enable + set / network-instance default protocols bgp neighbor 100.100.100.100 peer-group overlay + ``` + /// + + /// tab | spine ( RR ) + Spine is configured to establish dynamic peering with any IP address. + ```srl + set / network-instance default protocols bgp dynamic-neighbors accept match 0.0.0.0/0 peer-group overlay + ``` + /// + +1. **Configure EVPN Route Reflector (only on spine)** + + The command below will enable the route reflector functionality and only needs to be enabled on the Spine. + /// tab | spine + ```srl + set / network-instance default protocols bgp group overlay route-reflector client true + ``` + /// + + +### Verification + +The eBGP IPv4 sessions between Leaves and the spine is now active using IPv6 link-local addresses for peering. Through this eBGP peering, the IPv4 address family is distributing the loopback IPs across the DC. + +Simultaneously, the iBGP EVPN address family, set up through the loopback addresses, supports the sharing of overlay routes, which will be covered in detail in the following chapter. + + +/// tab | leaf1 +```srl +A:leaf1# show network-instance default protocols bgp neighbor +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- +BGP neighbor summary for network-instance "default" +Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++------------------+--------------------------+------------------+------+----------+---------------+---------------+-------------+--------------------------+ +| Net-Inst | Peer | Group | Flag | Peer-AS | State | Uptime | AFI/SAFI | [Rx/Active/Tx] | +| | | | s | | | | | | ++==================+==========================+==================+======+==========+===============+===============+=============+==========================+ +| default | 100.100.100.100 | overlay | S | 55555 | established | 0d:1h:13m:3s | evpn | [2/2/2] | +| default | fe80::181d:4ff:feff:1%et | underlay | D | 65000 | established | 0d:1h:13m:9s | ipv4- | [2/2/1] | +| | hernet-1/49.1 | | | | | | unicast | | ++------------------+--------------------------+------------------+------+----------+---------------+---------------+-------------+--------------------------+ +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Summary: +1 configured neighbors, 1 configured sessions are established,0 disabled peers +1 dynamic peers +``` +/// + + +/// tab | leaf2 +```srl +A:leaf2# show network-instance default protocols bgp neighbor +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- +BGP neighbor summary for network-instance "default" +Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++------------------+--------------------------+------------------+------+----------+---------------+---------------+-------------+--------------------------+ +| Net-Inst | Peer | Group | Flag | Peer-AS | State | Uptime | AFI/SAFI | [Rx/Active/Tx] | +| | | | s | | | | | | ++==================+==========================+==================+======+==========+===============+===============+=============+==========================+ +| default | 100.100.100.100 | overlay | S | 55555 | established | 0d:1h:13m:29s | evpn | [2/2/2] | +| default | fe80::181d:4ff:feff:2%et | underlay | D | 65000 | established | 0d:1h:13m:35s | ipv4- | [2/2/1] | +| | hernet-1/49.1 | | | | | | unicast | | ++------------------+--------------------------+------------------+------+----------+---------------+---------------+-------------+--------------------------+ +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Summary: +1 configured neighbors, 1 configured sessions are established,0 disabled peers +1 dynamic peers +``` +/// + + +/// tab | spine +```srl +A:spine# show network-instance default protocols bgp neighbor +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- +BGP neighbor summary for network-instance "default" +Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++------------------+--------------------------+------------------+------+----------+---------------+---------------+-------------+--------------------------+ +| Net-Inst | Peer | Group | Flag | Peer-AS | State | Uptime | AFI/SAFI | [Rx/Active/Tx] | +| | | | s | | | | | | ++==================+==========================+==================+======+==========+===============+===============+=============+==========================+ +| default | 100.0.0.1 | overlay | D | 55555 | established | 0d:1h:12m:24s | evpn | [2/0/2] | +| default | 100.0.0.2 | overlay | D | 55555 | established | 0d:1h:12m:29s | evpn | [2/0/2] | +| default | fe80::1805:2ff:feff:31%e | underlay | D | 42000000 | established | 0d:1h:12m:36s | ipv4- | [1/1/2] | +| | thernet-1/1.1 | | | 01 | | | unicast | | +| default | fe80::18d9:3ff:feff:31%e | underlay | D | 42000000 | established | 0d:1h:12m:31s | ipv4- | [1/1/2] | +| | thernet-1/2.1 | | | 02 | | | unicast | | ++------------------+--------------------------+------------------+------+----------+---------------+---------------+-------------+--------------------------+ +--------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Summary: +0 configured neighbors, 0 configured sessions are established,0 disabled peers +4 dynamic peers +``` +/// + + +[^1]: default SR Linux credentials are `admin:NokiaSrl1!`. +[^2]: the snippets were extracted with `info flat` command issued in running mode. +[^3]: you can paste those snippets right after you do `enter candidate` \ No newline at end of file diff --git a/docs/tutorials/l3evpn/intro.md b/docs/tutorials/l3evpn/intro.md new file mode 100644 index 00000000..fef7732b --- /dev/null +++ b/docs/tutorials/l3evpn/intro.md @@ -0,0 +1,95 @@ +--- +comments: true +tags: + - evpn +--- +# Introduction + + +| | | +| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| **Tutorial name** | L3 EVPN-VXLAN with SR Linux | +| **Lab components** | 3 SR Linux nodes & 2 [FRR](https://frrouting.org) | +| **Resource requirements** | :fontawesome-solid-microchip: 3vCPU
:fontawesome-solid-memory: 8 GB | +| **LAB Github Repo** | [srl-l3evpn-tutorial-lab][topofile] | +| **Lab name** | srl-l3evpn-tutorial-lab | +| **Packet captures** | [EVPN IP Prefix routes exchange][capture-evpn-rt5] | +| **Main ref documents** | [RFC 7432 - BGP MPLS-Based Ethernet VPN](https://datatracker.ietf.org/doc/html/rfc7432)
[RFC 8365 - A Network Virtualization Overlay Solution Using Ethernet VPN (EVPN)](https://datatracker.ietf.org/doc/html/rfc8365)
[RFC 9136 - IP Prefix Advertisement in Ethernet VPN (EVPN)](https://datatracker.ietf.org/doc/rfc9136/)
[Nokia 7220 SR Linux Advanced Solutions Guide][adv-sol-guide-evpn-l3]
[Nokia 7220 SR Linux EVPN-VXLAN Guide][evpn-vxlan-guide] | +| **Version information**[^1] | [`containerlab:v0.56.0`][clab-install], [`srlinux:24.3.3`][srlinux-container], [`frr:9.0.2`][frr-container] [`docker-ce:26.1.4`][docker-install] | + +EVPN serves as a control plane protocol for MAC address dissemination among routers, offering a scalable and efficient solution. +VxLAN is a datapath encapsulation, addresses the scalability issues of conventional VLANs by encapsulating Ethernet frames within UDP packets. +Together, EVPN and VxLAN facilitate the encapsulation of Layer 2 and Layer 3 traffic over an underlying IP network. + +This tutorial will lead you through configuring the DC fabric underlay. It will then cover the creation of an L3 EVPN overlay across two routers functioning as a unified virtual router. In the final chapter, it details how to peer the client with the EVPN overlay for route exchange. + +Our lab setup will resemble the following configuration: it will feature two Leaf devices linked to a Spine, collectively referred to as the DC Fabric. Attached to each Leaf, there will be a client that also functions as a CE router, capable of communicating routing protocols with the fabric. We will employ FRR for this task. For detailed information on the CE Router (Client), please check https://frrouting.org. + +

+ Fabric Diagram +

+ +## Lab deployment + +To let you follow along the configuration steps of this tutorial we created a lab that you can deploy on any Linux VM: + +The containerlab topo file and Client (FRR) startup configs can be found in the [git repo](https://github.com/srl-labs/srl-l3evpn-tutorial-lab/): + +You can deploy the lab on your Linux machine like: +``` +clab deploy -t https://github.com/srl-labs/srl-l3evpn-tutorial-lab.git +INFO[0000] Containerlab v0.56.0 started +INFO[0000] Parsing & checking topology file: l3evpn-tutorial.clab.yml +WARN[0000] Unable to load kernel module "ip_tables" automatically "load ip_tables failed: exec format error" +INFO[0000] Creating lab directory: /home/srl-l3evpn-tutorial-lab/clab-l3evpn +INFO[0000] Creating container: "leaf1" +INFO[0000] Creating container: "frr1" +INFO[0000] Creating container: "frr2" +INFO[0000] Creating container: "spine" +INFO[0000] Creating container: "leaf2" +INFO[0001] Created link: leaf1:e1-1 <--> frr1:eth1 +INFO[0001] Running postdeploy actions for Nokia SR Linux 'leaf1' node +INFO[0001] Created link: spine:e1-1 <--> leaf1:e1-49 +INFO[0001] Created link: spine:e1-2 <--> leaf2:e1-49 +INFO[0001] Running postdeploy actions for Nokia SR Linux 'spine' node +INFO[0001] Created link: leaf2:e1-1 <--> frr2:eth1 +INFO[0001] Running postdeploy actions for Nokia SR Linux 'leaf2' node +INFO[0032] Executed command "ip link set dev eth0 down" on the node "frr1". stdout: +INFO[0032] Executed command "ip link set dev eth0 down" on the node "frr2". stdout: +INFO[0032] Adding containerlab host entries to /etc/hosts file +INFO[0032] Adding ssh config for containerlab nodes ++---+-------------------+--------------+------------------------------+---------------+---------+----------------+----------------------+ +| # | Name | Container ID | Image | Kind | State | IPv4 Address | IPv6 Address | ++---+-------------------+--------------+------------------------------+---------------+---------+----------------+----------------------+ +| 1 | clab-l3evpn-frr1 | faa938ea15f3 | quay.io/frrouting/frr:9.0.2 | linux | running | 172.20.20.3/24 | 2001:172:20:20::3/64 | +| 2 | clab-l3evpn-frr2 | c719385f191d | quay.io/frrouting/frr:9.0.2 | linux | running | 172.20.20.6/24 | 2001:172:20:20::6/64 | +| 3 | clab-l3evpn-leaf1 | db43362a1c1f | ghcr.io/nokia/srlinux:24.3.3 | nokia_srlinux | running | 172.20.20.2/24 | 2001:172:20:20::2/64 | +| 4 | clab-l3evpn-leaf2 | d0ac084367fd | ghcr.io/nokia/srlinux:24.3.3 | nokia_srlinux | running | 172.20.20.4/24 | 2001:172:20:20::4/64 | +| 5 | clab-l3evpn-spine | 79bf719d2df4 | ghcr.io/nokia/srlinux:24.3.3 | nokia_srlinux | running | 172.20.20.5/24 | 2001:172:20:20::5/64 | ++---+-------------------+--------------+------------------------------+---------------+---------+----------------+----------------------+ +``` + +Containerlab will pull the git repo to your local working directory and a few seconds later containerlab finishes the deployment with providing a summary table that outlines connection details of the deployed nodes like shown above. In the "Name" column we have the names of the deployed containers and those names can be used to reach the nodes, for example to connect to the SSH of `leaf1`: + +```bash +# default credentials admin:NokiaSrl1! +ssh clab-l3evpn-leaf1 +``` + +With the lab deployed we are ready to embark on our learn-by-doing EVPN configuration journey! + +/// note +We advise the newcomers not to skip the [Configuration Basics Guide][conf-basics-guide] as it provides just enough details to survive in the configuration waters we are about to get in. +/// + +[topofile]: https://github.com/srl-labs/srl-l3evpn-tutorial-lab/ +[clab-install]: https://containerlab.srlinux.dev/install/ +[srlinux-container]: https://github.com/orgs/nokia/packages/container/package/srlinux +[frr-container]: https://quay.io/repository/frrouting/frr?tab=tags +[docker-install]: https://docs.docker.com/engine/install/ +[capture-evpn-rt5]: https://github.com/srl-labs/srl-l3evpn-tutorial-lab/blob/main/evpn_rt5.pcap +[adv-sol-guide-evpn-l3]: https://documentation.nokia.com/srlinux/24-3/books/advanced-solutions/evpn-vxlan-layer-3.html#evpn-vxlan-layer-3 +[evpn-vxlan-guide]: https://documentation.nokia.com/srlinux/24-3/books/evpn-vxlan/evpn-vxlan-tunnels-layer-3.html#evpn-vxlan-tunnels-layer-3 +[conf-basics-guide]: https://documentation.nokia.com/srlinux/24-3/title/basics.html + +[^1]: the following versions have been used to create this tutorial. The newer versions might work, but if they don't, please pin the version to the mentioned ones. diff --git a/docs/tutorials/l3evpn/ipvrf.md b/docs/tutorials/l3evpn/ipvrf.md new file mode 100644 index 00000000..af93face --- /dev/null +++ b/docs/tutorials/l3evpn/ipvrf.md @@ -0,0 +1,426 @@ +--- +comments: true +--- + + + +# L3 Overlay Instance / IP-VRF + +In the prior chapter, we set up the default network instance to facilitate the routing of VxLAN packets and exchange of the overlay routes. Now we will configure a L3 overlay domain also known as ip-vrf. + +The ip-vrf will span across two leaf switches interconnected via the spine and the transport of EVPN overlays will occur through VxLAN tunnels built upon the underlay. Spine will not be aware of the ip-vrf or any overlay domain, it will simply route VxLAN packets from one leaf to another. Subsequently, we will connect clients to this ip-vrf and clients will be able to reach each other. + + +

+ Overlay Diagram +

+ +## Physical and VxLAN Interfaces + + +1. **Create Client Facing Subinterface** + This physical interface will be facing the client device and it will serve as the default gateway for the client. + + ```srl + {default} set / interface ethernet-1/1 subinterface 1 type routed + set / interface ethernet-1/1 subinterface 1 admin-state enable + set / interface ethernet-1/1 subinterface 1 ipv4 admin-state enable + set / interface ethernet-1/1 subinterface 1 ipv4 address 192.168.1.1/24 + ``` + +1. **Create VxLAN Interface** + The tunnel interface is defined by a virtual network identifier (VNI) and an overlay network type. Type "Routed" is chosen for Layer 3 routing, while "Bridged" is used for Layer 2 switching. + + ```srl + set / tunnel-interface vxlan1 vxlan-interface 100 type routed + set / tunnel-interface vxlan1 vxlan-interface 100 ingress vni 100 + ``` + + +## L3 Network Instance ( IP-VRF ) + +1. **Create Network Instance** + + ```srl + set / network-instance ip-vrf-1 type ip-vrf + set / network-instance ip-vrf-1 admin-state enable + ``` + +1. **Attach Interfaces to the network instance** + At this step we are associating the previously configured physical subinterface and VxLAN tunnel interface with the IP-VRF. + + ```srl + set / network-instance ip-vrf-1 interface ethernet-1/1.1 + set / network-instance ip-vrf-1 vxlan-interface vxlan1.100 + ``` + +1. **Configure EVPN Parameters** + At this step we are associating the previously configured physical subinterface and VxLAN tunnel interface with the IP-VRF. + + ```srl + set / network-instance ip-vrf-1 protocols bgp-evpn bgp-instance 1 admin-state enable + set / network-instance ip-vrf-1 protocols bgp-evpn bgp-instance 1 vxlan-interface vxlan1.100 + ``` + Define an Ethernet VPN Instance number that is used as a service identifier. It can be used to to auto-derive route distinguisher and route-target. + + ```srl + set / network-instance ip-vrf-1 protocols bgp-evpn bgp-instance 1 evi 100 + ``` + + Define the route-target + + ```srl + set / network-instance ip-vrf-1 protocols bgp-vpn bgp-instance 1 route-target export-rt target:100:1 + set / network-instance ip-vrf-1 protocols bgp-vpn bgp-instance 1 route-target import-rt target:100:1 + ``` + [Optional] Configure ECMP to enable load balancing in the overlay. + + ```srl + set / network-instance ip-vrf-1 protocols bgp-evpn bgp-instance 1 ecmp 8 + ``` + +The IP-VRF is now set to route traffic. Below are the route tables for leaf1 and leaf2, showing the VRF's local interfaces and the VRF interface on the other peer, remote interface prefix on the peer is learned via bgp-evpn and highlighted for clarity. + +/// tab | leaf1 +```srl hl_lines="14" +A:leaf1# show network-instance ip-vrf-1 route-table +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 unicast route table of network instance ip-vrf-1 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +| Prefix | ID | Route Type | Route Owner | Active | Origin | Metric | Pref | Next-hop (Type) | Next-hop Interface | Backup Next-hop | Backup Next-hop Interface | +| | | | | | Network | | | | | (Type) | | +| | | | | | Instance | | | | | | | ++=================================+=======+============+======================+==========+==========+=========+============+====================+====================+====================+====================================+ +| 192.168.1.0/24 | 2 | local | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | 192.168.1.1 | ethernet-1/1.1 | | | +| | | | | | | | | (direct) | | | | +| 192.168.1.1/32 | 2 | host | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | None (extract) | None | | | +| 192.168.1.255/32 | 2 | host | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | None (broadcast) | | | | +| 192.168.2.0/24 | 0 | bgp-evpn | bgp_evpn_mgr | True | ip-vrf-1 | 0 | 170 | 100.0.0.2/32 | | | | +| | | | | | | | | (indirect/vxlan) | | | | ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 routes total : 4 +IPv4 prefixes with active routes : 4 +IPv4 prefixes with active ECMP routes: 0 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +``` +/// + +/// tab | leaf2 +```srl hl_lines="10" +A:leaf2# show network-instance ip-vrf-1 route-table +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 unicast route table of network instance ip-vrf-1 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +| Prefix | ID | Route Type | Route Owner | Active | Origin | Metric | Pref | Next-hop (Type) | Next-hop Interface | Backup Next-hop | Backup Next-hop Interface | +| | | | | | Network | | | | | (Type) | | +| | | | | | Instance | | | | | | | ++=================================+=======+============+======================+==========+==========+=========+============+====================+====================+====================+====================================+ +| 192.168.1.0/24 | 0 | bgp-evpn | bgp_evpn_mgr | True | ip-vrf-1 | 0 | 170 | 100.0.0.1/32 | | | | +| | | | | | | | | (indirect/vxlan) | | | | +| 192.168.2.0/24 | 2 | local | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | 192.168.2.1 | ethernet-1/1.1 | | | +| | | | | | | | | (direct) | | | | +| 192.168.2.1/32 | 2 | host | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | None (extract) | None | | | +| 192.168.2.255/32 | 2 | host | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | None (broadcast) | | | | ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 routes total : 4 +IPv4 prefixes with active routes : 4 +IPv4 prefixes with active ECMP routes: 0 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +``` +/// + +Below you will see client1 pinging client2 interface. +``` srl +frr1:/# ping 192.168.2.100 -I 192.168.1.100 -c3 +PING 192.168.2.100 (192.168.2.100) from 192.168.1.100: 56 data bytes +64 bytes from 192.168.2.100: seq=0 ttl=63 time=2.507 ms +64 bytes from 192.168.2.100: seq=1 ttl=63 time=2.209 ms +64 bytes from 192.168.2.100: seq=2 ttl=63 time=1.884 ms + +--- 192.168.2.100 ping statistics --- +3 packets transmitted, 3 packets received, 0% packet loss +round-trip min/avg/max = 1.884/2.200/2.507 ms +``` +/// note +We are using FRR containers as Client, you can login to the client1 with the command below. + +**docker exec -it frr1 bash** +/// + +## (Optional) BGP Peering with the CE/Client + +This step is optional and is relevant if another router, acting as our client, wants to exchange routes with the EVPN Overlay (ip-vrf). + +

+ Peering with Client +

+ +In this case, both clients have loopback IPs that need to be advertised to the L3 EVPN Network Instance (ip-vrf). This requires setting up a routing protocol between the clients (frr) and the routers they're connected to (Leaf1 & Leaf2). + +In the previous chapter, we completed the ip-vrf configuration, moving forward, we'll integrate a routing protocol within it to establish connectivity with the client. SRLinux supports OSPF, ISIS, and BGP in the overlay. We're opting for BGP because we love it for many reasons. **Please note, the FRR client BGP parameters have been pre configured.** + + +1. **AS Number and Router ID** +The initial step involves specifying the autonomous system number and router-id for this ip-vrf, which will be uniformly applied across all routers encompassed by this ip-vrf. Ultimately, these routers will function collectively as if they are a singular router distributed over multiple devices. + + ``` srl + set / network-instance ip-vrf-1 protocols bgp autonomous-system 500 + set / network-instance ip-vrf-1 protocols bgp router-id 3.3.3.3 + ``` + +1. **BGP Address Family** +Since our clients use IPv4 addresses, we activate the BGP IPv4 address family to facilitate route exchange with the client. Although the overlay supports IPv6, we have not enabled it as our clients do not have IPv6 routes to announce. + + ``` srl + set / network-instance ip-vrf-1 protocols bgp afi-safi ipv4-unicast admin-state enable + ``` + +1. **Configure the Neighbor Parameters** +We configure the BGP peer/neighbor IP and its corresponding autonomous system number, then assign the BGP neighbor to a peer group. + ``` srl + set / network-instance ip-vrf-1 protocols bgp group client + set / network-instance ip-vrf-1 protocols bgp neighbor 192.168.1.100 peer-as 1000000000 + set / network-instance ip-vrf-1 protocols bgp neighbor 192.168.1.100 peer-group client + ``` + +1. **Allow BGP to exchange routes by default** +By default, all incoming and outgoing eBGP routes are blocked. We will disable this default setting to permit all incoming and outgoing routes. + + ``` srl + set / network-instance ip-vrf-1 protocols bgp ebgp-default-policy import-reject-all false + set / network-instance ip-vrf-1 protocols bgp ebgp-default-policy export-reject-all false + ``` + + +1. **Send Default Route to the Client** +In the previous step, we disabled eBGP's default route blocking. However, eBGP doesn't automatically announce routes to the client since it treats the peer as an external system and only announces selected routes through a policy. To share overlay routes with the client, we must either configure an export route policy or enable the following feature to distribute a default route to the client. + + ``` srl + set / network-instance ip-vrf-1 protocols bgp group client send-default-route ipv4-unicast true + ``` + + +**Verification** + +Each leaf appears to have successfully established eBGP with its client. + + +/// tab | leaf1 +```srl hl_lines="10" +A:leaf1# show network-instance ip-vrf-1 protocols bgp neighbor +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +BGP neighbor summary for network-instance "ip-vrf-1" +Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++------------------------+-----------------------------------+------------------------+--------+-------------+-------------------+-------------------+-----------------+-----------------------------------+ +| Net-Inst | Peer | Group | Flags | Peer-AS | State | Uptime | AFI/SAFI | [Rx/Active/Tx] | ++========================+===================================+========================+========+=============+===================+===================+=================+===================================+ +| ip-vrf-1 | 192.168.1.100 | client | S | 1000000000 | established | 2d:1h:56m:18s | ipv4-unicast | [3/1/1] | ++------------------------+-----------------------------------+------------------------+--------+-------------+-------------------+-------------------+-----------------+-----------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Summary: +1 configured neighbors, 1 configured sessions are established,0 disabled peers +0 dynamic peers +``` +/// + +/// tab | leaf2 +```srl hl_lines="10" +A:leaf2# show network-instance ip-vrf-1 protocols bgp neighbor +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +BGP neighbor summary for network-instance "ip-vrf-1" +Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ ++----------------------+--------------------------------+----------------------+--------+------------+------------------+------------------+----------------+--------------------------------+ +| Net-Inst | Peer | Group | Flags | Peer-AS | State | Uptime | AFI/SAFI | [Rx/Active/Tx] | ++======================+================================+======================+========+============+==================+==================+================+================================+ +| ip-vrf-1 | 192.168.2.100 | client | S | 2000000000 | established | 2d:1h:57m:51s | ipv4-unicast | [3/1/1] | ++----------------------+--------------------------------+----------------------+--------+------------+------------------+------------------+----------------+--------------------------------+ +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +Summary: +1 configured neighbors, 1 configured sessions are established,0 disabled peers +0 dynamic peers +``` +/// + + +Below are the advertised and received routes from Leaf's perspective. Each leaf has announced a default route to its clients and is receiving the client's loopback IP (highlighted). + +It appears the client is re-advertising the default route back to the leaf, but the leaf is ignoring the route due to AS-Loop. + +/// tab | leaf1 +```srl hl_lines="33" +A:leaf1# show network-instance ip-vrf-1 protocols bgp neighbor 192.168.1.100 advertised-routes ipv4 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Peer : 192.168.1.100, remote AS: 1000000000, local AS: 500 +Type : static +Description : None +Group : client +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Origin codes: i=IGP, e=EGP, ?=incomplete ++----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Network Path-id Next Hop MED LocPref AsPath Origin | ++============================================================================================================================================================================================================================+ +| 0.0.0.0/0 0 192.168.1.1 - 100 [500] ? | ++----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +1 advertised BGP routes +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + + + +A:leaf1# show network-instance ip-vrf-1 protocols bgp neighbor 192.168.1.100 received-routes ipv4 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Peer : 192.168.1.100, remote AS: 1000000000, local AS: 500 +Type : static +Description : None +Group : client +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +Status codes: u=used, *=valid, >=best, x=stale +Origin codes: i=IGP, e=EGP, ?=incomplete ++---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Status Network Path-id Next Hop MED LocPref AsPath Origin | ++===========================================================================================================================================================================================================================+ +| 0.0.0.0/0 0 192.168.1.100 - 100 [1000000000, 500] ? | +| u*> 1.1.1.1/32 0 192.168.1.100 - 100 [1000000000] ? | +| * 192.168.1.0/24 0 192.168.1.100 - 100 [1000000000] ? | ++---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +3 received BGP routes : 1 used 2 valid +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +``` +/// + + + + + +/// tab | leaf2 +```srl hl_lines="33" +A:leaf2# show network-instance ip-vrf-1 protocols bgp neighbor 192.168.2.100 advertised-routes ipv4 +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +Peer : 192.168.2.100, remote AS: 2000000000, local AS: 500 +Type : static +Description : None +Group : client +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +Origin codes: i=IGP, e=EGP, ?=incomplete ++-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Network Path-id Next Hop MED LocPref AsPath Origin | ++=======================================================================================================================================================================================================+ +| 0.0.0.0/0 0 192.168.2.1 - 100 [500] ? | ++-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +1 advertised BGP routes +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ + + + +A:leaf2# show network-instance ip-vrf-1 protocols bgp neighbor 192.168.2.100 received-routes ipv4 +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +Peer : 192.168.2.100, remote AS: 2000000000, local AS: 500 +Type : static +Description : None +Group : client +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +Status codes: u=used, *=valid, >=best, x=stale +Origin codes: i=IGP, e=EGP, ?=incomplete ++---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Status Network Path-id Next Hop MED LocPref AsPath Origin | ++=========================================================================================================================================================================================================+ +| 0.0.0.0/0 0 192.168.2.100 - 100 [2000000000, 500] ? | +| u*> 2.2.2.2/32 0 192.168.2.100 - 100 [2000000000] ? | +| * 192.168.2.0/24 0 192.168.2.100 - 100 [2000000000] ? | ++---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +3 received BGP routes : 1 used 2 valid +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +``` +/// + + + +Let's examine the routing table of the VRF on each leaf. Both leaves share the same list of routes, with different next hops. Local routes resolve to a local interface, while remote routes learned from the other leaf resolve to a VxLAN tunnel. Routes resolving to a VxLAN tunnel are highlighted for clarity. + +/// tab | leaf1 +```srl hl_lines="12 18" +A:leaf1# show network-instance ip-vrf-1 route-table +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 unicast route table of network instance ip-vrf-1 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +| Prefix | ID | Route Type | Route Owner | Active | Origin | Metric | Pref | Next-hop (Type) | Next-hop Interface | Backup Next-hop | Backup Next-hop Interface | +| | | | | | Network | | | | | (Type) | | +| | | | | | Instance | | | | | | | ++=================================+=======+============+======================+==========+==========+=========+============+====================+====================+====================+====================================+ +| 1.1.1.1/32 | 0 | bgp | bgp_mgr | True | ip-vrf-1 | 0 | 170 | 192.168.1.0/24 | ethernet-1/1.1 | | | +| | | | | | | | | (indirect/local) | | | | +| 2.2.2.2/32 | 0 | bgp-evpn | bgp_evpn_mgr | True | ip-vrf-1 | 0 | 170 | 100.0.0.2/32 | | | | +| | | | | | | | | (indirect/vxlan) | | | | +| 192.168.1.0/24 | 2 | local | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | 192.168.1.1 | ethernet-1/1.1 | | | +| | | | | | | | | (direct) | | | | +| 192.168.1.1/32 | 2 | host | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | None (extract) | None | | | +| 192.168.1.255/32 | 2 | host | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | None (broadcast) | | | | +| 192.168.2.0/24 | 0 | bgp-evpn | bgp_evpn_mgr | True | ip-vrf-1 | 0 | 170 | 100.0.0.2/32 | | | | +| | | | | | | | | (indirect/vxlan) | | | | ++---------------------------------+-------+------------+----------------------+----------+----------+---------+------------+--------------------+--------------------+--------------------+------------------------------------+ +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +IPv4 routes total : 6 +IPv4 prefixes with active routes : 6 +IPv4 prefixes with active ECMP routes: 0 +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- +``` +/// + +/// tab | leaf2 +```srl hl_lines="10 14" +A:leaf2# show network-instance ip-vrf-1 route-table +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +IPv4 unicast route table of network instance ip-vrf-1 +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ ++-----------------------------+-------+------------+----------------------+----------+----------+---------+------------+------------------+------------------+------------------+--------------------------+ +| Prefix | ID | Route Type | Route Owner | Active | Origin | Metric | Pref | Next-hop (Type) | Next-hop | Backup Next-hop | Backup Next-hop | +| | | | | | Network | | | | Interface | (Type) | Interface | +| | | | | | Instance | | | | | | | ++=============================+=======+============+======================+==========+==========+=========+============+==================+==================+==================+==========================+ +| 1.1.1.1/32 | 0 | bgp-evpn | bgp_evpn_mgr | True | ip-vrf-1 | 0 | 170 | 100.0.0.1/32 | | | | +| | | | | | | | | (indirect/vxlan) | | | | +| 2.2.2.2/32 | 0 | bgp | bgp_mgr | True | ip-vrf-1 | 0 | 170 | 192.168.2.0/24 | ethernet-1/1.1 | | | +| | | | | | | | | (indirect/local) | | | | +| 192.168.1.0/24 | 0 | bgp-evpn | bgp_evpn_mgr | True | ip-vrf-1 | 0 | 170 | 100.0.0.1/32 | | | | +| | | | | | | | | (indirect/vxlan) | | | | +| 192.168.2.0/24 | 2 | local | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | 192.168.2.1 | ethernet-1/1.1 | | | +| | | | | | | | | (direct) | | | | +| 192.168.2.1/32 | 2 | host | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | None (extract) | None | | | +| 192.168.2.255/32 | 2 | host | net_inst_mgr | True | ip-vrf-1 | 0 | 0 | None (broadcast) | | | | ++-----------------------------+-------+------------+----------------------+----------+----------+---------+------------+------------------+------------------+------------------+--------------------------+ +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +IPv4 routes total : 6 +IPv4 prefixes with active routes : 6 +IPv4 prefixes with active ECMP routes: 0 +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ +``` +/// + + +Then let's send a ping between client loopbacks to conclue this chapter. + +/// tab | Ping between client loopbacks +```srl +frr1:/# ping 2.2.2.2 -I 1.1.1.1 -c3 +PING 2.2.2.2 (2.2.2.2) from 1.1.1.1: 56 data bytes +64 bytes from 2.2.2.2: seq=0 ttl=63 time=2.453 ms +64 bytes from 2.2.2.2: seq=1 ttl=63 time=1.865 ms +64 bytes from 2.2.2.2: seq=2 ttl=63 time=1.922 ms + +--- 2.2.2.2 ping statistics --- +3 packets transmitted, 3 packets received, 0% packet loss +round-trip min/avg/max = 1.865/2.080/2.453 ms +``` +/// \ No newline at end of file From 60bbfb928a9d3fc4363db13ef58801e6e4dcf0a6 Mon Sep 17 00:00:00 2001 From: kkayhan Date: Tue, 9 Jul 2024 23:21:03 +0200 Subject: [PATCH 2/3] modified mkdocs for l3evpn tutorial --- mkdocs.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mkdocs.yml b/mkdocs.yml index 5e6b54c6..7ad20b95 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -87,6 +87,11 @@ nav: - tutorials/evpn-mh/basics/index.md - Configuration: tutorials/evpn-mh/basics/conf.md - Verification: tutorials/evpn-mh/basics/verify.md + - Layer 3 EVPN Basics: + - Introduction: tutorials/l3evpn/intro.md + - Underlay and Overlay Routing: tutorials/l3evpn/fabric.md + - L3 Overlay Instance / IP-VRF: tutorials/l3evpn/ipvrf.md + - About L3 EVPN: tutorials/l3evpn/aboutl3evpn.md - Infrastructure: - KNE: - tutorials/infrastructure/kne/index.md From 69f0c8c0209daa427d01023cded564c01fae502b Mon Sep 17 00:00:00 2001 From: kkayhan Date: Wed, 10 Jul 2024 10:21:42 +0200 Subject: [PATCH 3/3] removed unusued reference --- docs/tutorials/l3evpn/fabric.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/tutorials/l3evpn/fabric.md b/docs/tutorials/l3evpn/fabric.md index 7a77581c..001a31bf 100644 --- a/docs/tutorials/l3evpn/fabric.md +++ b/docs/tutorials/l3evpn/fabric.md @@ -1049,5 +1049,4 @@ Summary: [^1]: default SR Linux credentials are `admin:NokiaSrl1!`. -[^2]: the snippets were extracted with `info flat` command issued in running mode. -[^3]: you can paste those snippets right after you do `enter candidate` \ No newline at end of file +[^2]: the snippets were extracted with `info flat` command issued in running mode. \ No newline at end of file