Jumbo Frames (Nexus7k/5k)

Enabling jumbo frames in a data center can be somewhat nerve racking when you are trying to ensure you don't take the entire place down. Here are some notes that might help make the project a bit less intimidating.

General Caveats/Considerations

• Understand your topology and know the traffic flows
• Understand all hardware in your network and it's support of jumbo frames
• Enable jumbo frames on all switches in the path/environment
• Be aware of existing QoS policies implemented
  
• Overwriting an existing policy could be problematic
• Order of operations in MQC or 3CPL should be considered
• Enabling jumbo frames on a switch simply says "don't discard frames that are equal or less than this value" - nonintrusive
• Frames are not aggregated and typically are sourced from device other than the network equipment at layer 2
• Ethernet feature - use care allowing to traverse l3 interfaces
• Understand your traffic flow and connectivity
• Enable jumboframes on host interfaces with extreme care
  
• Understand the different vendor's and how they interpret the MTU command
• Does it include the header info or not?
• Enable jumbo frames on all devices in a layer 2 environment that could potentially communicate with each other
• Pairs of 5ks should be updated simultaneously using secureCRT Chat Window or Config-sync

Nexus 7k

---

System jumbomtu 9216 is default

 

N7K-AGG-1# sho run all | inc "jumbomtu" system jumbomtu 9216

 

It's a default so it doesn't show up in the configuration without the all keyword.

The interface MTU is 1500 bytes by default.

With no changes, the MTU will be 1500 on both M-series line cards and F-series Line cards.

Changes required to enable Jumbo Frames

F-Series Line Cards

• Modify the network-qos policy-map in the admin VDC
• This is a disruptive change and may cause drops on some interfaces
• Change the MTU on the individual interfaces
• This is non-intrusive
• Can only be set to either 1500 or the system jumbo MTU - any other value will display an error saying as much

Example in Admin VDC

policy-map type network-qos custom-nq-7e class type network-qos c-nq-7e-drop congestion-control tail-drop mtu 9216 class type network-qos c-nq-7e-ndrop-fcoe pause mtu 2112 system qos service-policy type network-qos custom-nq-7e

 

Example Interface Change

interface port-channel1021 switchport switchport mode fabricpath mtu 9216

 

M-Series Line Cards

• Interface level MTU change with same guideline as above

interface Ethernet3/1 mtu 9216 ip address 10.6.1.13/30 ip ospf network point-to-point ip router ospf 1 area 0.0.0.0