Arista MLAG 101

By | September 8, 2022

Article from https://networkengineering.stackexchange.com/questions/79649/arista-veos-and-eve-ng-setup

Configure Port-Channel between the switches. Usually you would want to configure at least 2 links in the Port-Channels but 4 or 8 might be required if you have considerable amounts of data between the switches or for redundancy.

Port-channel between the switches

On both MLAG-1 and MLAG-2 switches

interface Ethernet 1 - 4  
  description port-channel  
  channel-group 10 mode active 
  
  interface port-channel 10   
  description port-channel  
  switchport mode trunk  

Configure VLAN used by MLAG and disable spanning-tree on this VLAN.

On both MLAG-1 and MLAG-2 switches

vlan 4094  
   trunk group mlagpeer
  
  interface port-channel 10  
   switchport trunk group mlagpeer  

  no spanning-tree vlan 4094 

We can safely disable spanning-tree because the trunk group ensures that VLAN 4094 will only traverse port-channel 10. Trunk group names must be identical on both switches for it to work properly.

Let’s get the SVI configured on both switches so they can communicate.

 MLAG-1#  
  interface vlan 4094  
  description MLAG_VLAN  
  ip address 172.20.1.1/30  
  no shut  

 MLAG-2#  
  interface vlan 4094  
  description MLAG_VLAN  
  ip address 172.20.1.2/30  
  no shut  

Now the easy part on bringing up MLAG between the switches.

MLAG-1#  
  mlag  
  local-interface vlan 4094  
  peer-address 172.20.1.2  
  peer-link port-channel 10  
  primary-priority 1  
  domain-id mlagcore  

 MLAG-2#   
  mlag  
  local-interface vlan 4094  
  peer-address 172.20.1.1  
  peer-link port-channel 10  
  domain-id mlagcore

Note:

Primary-priority is a hidden command and it is needed for the deterministic election of MLAG Primary switch and lower priority wins. Commands related to spanning-tree comes from the MLAG primary switch. For example, when you configure spanning-tree vlan root priority on the MLAG secondary switch, the setting will not take effect.

Once the communication establishes MLAG will be active.

Verification

Notice ‘mlag-1’ switch shows that it is MLAG primary and that the status of the MLAG is active on both switches.

The command show mlag is updated to show if the MLAG configuration is consistent or inconsistent compared with the peer configuration.

For example, here is the output if the configuration is consistent.

mlag-1#show mlag
MLAG Configuration:
domain-id           :            mlagcore
local-interface     :            Vlan4094
peer-address        :          172.20.1.2
peer-link           :                Po10
peer-config         :          consistent                                         

MLAG Status:       
state               :              Active
negotiation status  :           Connected
peer-link status    :                  Up
local-int status    :                  Up
system-id           :   02:1c:73:09:79:1b
                                         
MLAG Ports:        
Disabled            :                   0
Configured          :                   0
Inactive            :                   0
Active-partial      :                   0
Active-full         :                   4

show mlag detail : Confirm MLAG is active, which peer is Primar­y/S­eco­ndary, timers, number of active­-fu­ll/­act­ive­-pa­rtial interf­aces, etc.

enter image description here

show mlag config­-sanity : Run on both peers to confirm no incons­ist­encies or issues

show mlag interfaces [detai­l|m­emb­ers­|st­ates] : Confirm inform­ation on MLAG interf­aces.

show mlag issu warnings : Displays a warning message regarding the backwa­rd-­com­pat­ibility of this feature before upgrading.

tcpdump : #bash tcpdump -nei vlan4094 port 4432 ==> to confirm control plane traffic for mlag is being sent and received.

It is also recommended that both MLAG-1 and MLAG-2 have identical LAYER 2 commands such as STP root priority etc. Because if the primary switch were to reboot, the secondary switch will takeover and STP commands would be different resulting in a sub-optimal configuration.

Now let’s configure MLAG port-channel on MLAG-1 and MLAG-2 switch and also configure regular port-channel on the access layer switch AXS1. Highly recommended to use LACP ACTIVE and not just channel-group mode on.

mlag-1#  
  interface Ethernet11  
   description MLAG:PO_ASX1  
   no shutdown  
   channel-group 11 mode active  
  interface Port-Channel11  
   description MLAG:PO_ASX1  
   switchport mode trunk  
   mlag 11  


mlag-2#  
  interface Ethernet11  
   description MLAG:PO_ASX1  
   no shutdown  
   channel-group 11 mode active  
  interface Port-Channel11  
   description MLAG:PO_ASX1  
   switchport mode trunk  
   mlag 11


ASX1#
  interface Ethernet1/9  
   description MLAG:PO_TO_MLAG-1  
   switchport mode trunk  
   spanning-tree port type edge  
   channel-group 11 mode active  
  interface Ethernet1/11  
   description MLAG:PO_TO_MLAG-2  
   switchport mode trunk  
   channel-group 11 mode active  

Let’s verify that our port-channels are up on MLAG-1 and MLAG-2 to the ASX1 switch.

 mlag-1#sh etherchannel detailed  
  Active Ports:  
     Port         Time became active    Protocol  Mode  
   -------------------- ------------------------ ------  
     Ethernet11        4:32:34          LACP    Active  
     PeerEthernet11    4:34:24          LACP    Active  


 mlag-1#show mlag interfaces detail  
                     local/remote  
  mlag     state  local  remote  oper  config  last change  changes  
 ------ ------------- ------- -------- ------- ---------- --------------   
   1  active-full  Po11  Po11  up/up  ena/ena  0:08:23 ago    24  

Notice that ‘mlag-1#’ shows ‘PeerEthernet11’ which is a good sign because it recognizes the other Ethernet11 on mlag-2 switch as a part of the port-channel.

 ASX1# show port-channel summary  
 Flags: D - Down    P - Up in port-channel (members)  
     I - Individual H - Hot-standby (LACP only)  
     s - Suspended  r - Module-removed  
     S - Switched  R - Routed  
     U - Up (port-channel)  
     M - Not in use. Min-links not met  
 --------------------------------------------------------------------------------  
 Group Port-    Type   Protocol Member Ports  
    Channel  
 --------------------------------------------------------------------------------  
 1   Po1(SU)   Eth   LACP   Eth1/9(P)  Eth1/11(P)  

Notice that from ASX1 perspective this is just a normal port-channel with no special configurations. As far as ASX1 is concerned both Eth1/9 and Eth1/11 are connected into the same switch.

Spanning-tree show commands on both mlag-1 and mlag-2 switch.

 mlag-1#sh spanning-tree vlan 101  
  Spanning tree instance for vlan 101  
  VL101  
   Spanning tree enabled protocol rapid-pvst  
   Root ID  Priority  32869  
        Address   021c.7309.791b  
        This bridge is the root  
   Bridge ID Priority  32869 (priority 32768 sys-id-ext 101)  
        Address   021c.7309.791b  
        Hello Time 2.000 sec Max Age 20 sec Forward Delay 15 sec  
  Interface    Role    State   Cost   Prio.Nbr Type  
  ---------------- ---------- ---------- --------- -------- --------------------  
  Po12       designated forwarding 1999   128.100 P2p  

 mlag-2#sh spanning-tree vlan 101  
  Spanning tree instance for vlan 101  
  VL101  
   Spanning tree enabled protocol rapid-pvst  
   Root ID  Priority  32869  
        Address   021c.7309.791b  
        This bridge is the root  
   Bridge ID Priority  32869 (priority 32768 sys-id-ext 101)  
        Address   021c.7309.791b  
        Hello Time 2.000 sec Max Age 20 sec Forward Delay 15 sec  
  Interface    Role    State   Cost   Prio.Nbr Type  
  ---------------- ---------- ---------- --------- -------- --------------------  
  Po11       designated forwarding 1999   128.100 P2p  

Notice that both switches are acting as spanning-tree root bridge. Also note that the bridge address comes from the negotiated MLAG system-id (show mlag detail).

Caveats/Thoughts/Conclusions

  • Make sure to have a deterministic MLAG primary switch.
  • Set all STP commands the same on both switches.
  • At least have 2 links between the switches for MLAG communication in a port-channel for optimal redund­ancy.
  • If you have a single homed server, traffic might need to traverse the port-channel to the other switch adding an extra hop latency.
  • Turn off STP on the MLAG peering VLAN to prevent the peer link from going into the discarding state.
  • Recomm­ended to use VLAN4094 but any VLAN can be used. Use the same VLAN throughout all MLAG domains for consis­tency.
  • Put the MLAG peering VLAN into a trunk group to ensure this VLAN isn’t used by any other ports and avoid any possible loop conditions being created.
  • Use “no autost­ate­” on the MLAG SVI to ensure this interface remains UP.
  • The Peer Link capacity should be equal to the Leaf-t­o-Spine total capacity to avoid losing capacity if the uplinks fail.
  • The MLAG Domain ID needs to be identical (case-­sen­sitive) across Peers as well as be unique in regards to other Leaf MLAG pairs.
  • The MLAG IDs need to match across Peers, but it is recomm­ended to use the same port-c­hannel ID across both Peers if possible for simplicity in operations and troubl­esh­ooting.
  • MLAG timers should be kept to the default values. Reload Delay is the interval that MLAG interfaces are disabled after an MLAG peer reboots. Non MLAG Reload Delay is the interval that non-MLAG links are disabled after an MLAG peer reboots.
  • After 4.21.1F, “­swi­tchport mode trunk native vlan tag” is no longer needed to be explicitly configured on the MLAG Peer Link.

MLAG System ID : Derived after Primary Peer is elected (lowest MAC address); persistent across reboots; LACP and STP control packets uses this to emulate one “­log­ica­l” switch.

STP : Active only on the Primary Peer; config­uration needs to be consistent across both peers for seamless failover; Secondary will forward BPDUs to Primary across the Peer Link.

TCP and UDP Port 4432 : Must be permitted in control plane ACL on both peers if non-de­fault control plane ACL is used.

MAC address table : Sync’ed between MLAG peers for active­/active member ports.

IGMP Snooping Tables : Sync’ed between MLAG peers for active­/active member ports; config­uration should be consistent across both peers.

ARP table sync? : No ARP table sync with MLAG.

L3 Sync? : MLAG is L2 active­/active techno­logy. To get an active­/active L3 Gateway use VARP, but keep in mind that though MLAG forms one logical switch between two switches, there are two separate Layer 3 control planes on each peer.

Ref: https://aristanetworks.force.com/AristaCommunity/s/article/mlag-basic-configuration

Ref: https://www.arista.com/en/support/toi/eos-4-15-2f/13733-mlag-config-check

Ref: https://aristanetworks.force.com/AristaCommunity/s/question/0D52I00007ERqClSAL/replacing-vpc-of-network

Ref: http://ithitman.blogspot.com/2014/08/configuring-arista-mlag-basic-setup.html

Ref: https://cheatography.com/sh-arista/cheat-sheets/arista-mlag/