Configuring Nexus 5000 with Nexus 2000 Fabric Extenders

Top-of-rack switching is commonly deployed for high port-density data centers. Switches are mounted at the top of each rack that makes cabling looks neat and tidy. However, it leads to switch managability issues dealing with multiple spanning-trees when you have hundreds of layer 2 switches - not to mention firmware upgrading. To resolve these issues, we are using Nexus 5000 with multiple fabric extenders (Nexus 2000). The N2K are just like the line cards to a chassis switch e.g. Catalyst 6500, except that N5K doesn't peform L3 functions like IP routing. This setup allows you to manage the whole bunch of switches as a single switch distributed all over the data center as shown below.

Between N5K and N2K, you may connect upto 4 x fiber links . That is giving you up to 40Gbps uplink per extender. As the Nexus are running on Cisco NX-OS, most switching commands are similar to the tradition Cisco IOS. However, the setup configuration is different. You need to perform the following steps:

1) Create a virtual fex chassis
switch(config)# fex 117
! set the number of links from N5K to N2K
switch(config-fex)# pinning max-links 1 (you may set up to 4 links)
2) Associate the N2K extenders to the fex chassis
switch(config)# interface e1/17
switch(config-if)# switchport mode fex-fabric
switch(config-if)# fex associate 117
3) To verify
switch# sh int e1/17 fex-intf

You may now configure the individual switch ports on the N2K extenders like normal Cisco switch ports. Unlike other Cisco switches, most switching features are not enabled by default. You'll have to turn them on manually using the "feature" command. For example, if you wish to configure EtherChannel, you have to enable LACP using "feature lacp" command.