Cisco Nexus 5596’s with Redundant Uplinks to Catalyst 6509 Core’s Using vPC

Posted on Updated on

I recently had the opportunity to deploy a Cisco Nexus solution 5596UP switches for a healthcare customer. The Nexus switches represent Cisco’s presence in the Converged Fabric segment, which is gaining momentum recently as more IT shops seek to streamline datacenter infrastructure towards a “private cloud” approach. The Nexus 5000 series is primarily aimed at the edge, with Layer 3 capabilities available as an add-on.

The Nexus switches support vPC (Virtual Port Channels), which “allows links that are physically connected to two different Cisco Nexus 5000 Series switches or Cisco Nexus 2000 Series Fabric Extenders to appear as a single port channel by a third device.” (Cisco)

Here is a diagram of the planned configuration of vPC uplinks between the Nexus 5596UP and Catalyst 6509 (core) switches:


To setup vPC on the Nexus switches, first you need to create a vPC peer-link between the pair of Nexus switches. The peer-link must include at least 2 interfaces.

feature vpc
vpc domain 1
  role priority 4096
  system-priority 2000
  peer-keepalive destination 192.168.100.20
  auto-recovery

interface port-channel20
  switchport mode trunk
  vpc peer-link
  switchport trunk allowed vlan 100,103-104,901
  spanning-tree port type network

interface Ethernet1/23
  description Link 1 to 5596-sw2
  switchport mode trunk
  switchport trunk allowed vlan 100,103-104,901
  channel-group 20 mode active

interface Ethernet1/24
  description Link 2 to 5596-sw2
  switchport mode trunk
  switchport trunk allowed vlan 100,103-104,901
  channel-group 20 mode active

Repeat on the 2nd Nexus switch.

Next you need to create the virtual port-channels on the Nexus side. We will create one port-channel per uplink interface.

interface port-channel1
  switchport mode trunk
  vpc 1
  switchport trunk allowed vlan 100,103-104,901

interface port-channel2
  switchport mode trunk
  vpc 2
  switchport trunk allowed vlan 100,103-104,901

interface Ethernet1/1
  description uplink to Core1-7/8
  switchport mode trunk
  switchport trunk allowed vlan 100,103-104,901
  spanning-tree guard loop
  channel-group 1 mode active

interface Ethernet1/2
  description uplink to Core2-6/7
  switchport mode trunk
  switchport trunk allowed vlan 100,103-104,901
  spanning-tree guard loop
  channel-group 2 mode active

Again, repeat for the 2nd Nexus switch.

Notice that spanning-tree Loop Guard has been enabled on the uplinks to prevent STP looping issues. Also, the Allowed VLAN’s should match the VLAN ID’s allowed in the peer-link.

Finally, create the port channels on the Catalyst side. Here we will create ONE port channel per Catalyst, consisting of the uplinks from each Nexus switch, so that the Catalyst will see the Nexus pair as a single switch. Until this step is complete, the vPC status will show as down.

interface Port-channel200
 description “Connection to Nexus”
 switchport
 switchport trunk encapsulation dot1q
 switchport mode trunk
 no ip address
 spanning-tree guard root

interface TenGigabitEthernet7/6
 description NEXUS SW2 PORT1
 switchport
 switchport trunk encapsulation dot1q
 switchport mode trunk
 no ip address
 spanning-tree guard root
 channel-group 200 mode active

interface TenGigabitEthernet7/8
 description NEXUS SW1 PORT1
 switchport
 switchport trunk encapsulation dot1q
 switchport mode trunk
 no ip address
 spanning-tree guard root
 channel-group 200 mode active

Repeat this process for the 2nd Catalyst switch.

Now the vPC status should show as up-

5596-sw1(config-if)# sh vpc
Legend:
                (*) – local vPC is down, forwarding via vPC peer-link

vPC domain id                   : 1   
Peer status                     : peer adjacency formed ok      
vPC keep-alive status           : peer is alive                 
Configuration consistency status: success
Per-vlan consistency status     : success                       
Type-2 consistency status       : success
vPC role                        : primary                       
Number of vPCs configured       : 2   
Peer Gateway                    : Disabled
Dual-active excluded VLANs      : –
Graceful Consistency Check      : Enabled

vPC Peer-link status
———————————————————————
id   Port   Status Active vlans    
—   —-   —— ————————————————–
1    Po20   up     1,100,103-104                                            

vPC status
—————————————————————————-
id     Port        Status Consistency Reason           Active vlans
—— ———– —— ———– ————————– ———–
1      Po1         up     success     success                   1,100,103-104         
2      Po2         up     success     success                  1,100,103-104    

Here
is a useful Cisco reference document on vPC for the Nexus 5000 series.

Notes:

  • This configuration only applies if there is NOT VSS established between the Catalyst 6509 core’s.
  • You can rely on Spanning Tree for establishing redundant links from a Nexus pair to dual Core’s, with only one uplink marked as active by STP. This customer insisted on getting the aggregated bandwidth from both 10Gb uplinks, as they planned to converge additional applications onto the Nexus in the future.
  •  If at all possible, you should test this configuration in a lab before production deployment. At a minimum, deploy these changes during a maintenance window as there is a risk of network outage mainly due to looping behavior – spanning-tree root guard, loop guard are strongly recommended!

Happy vPC’ing!

Advertisements

19 thoughts on “Cisco Nexus 5596’s with Redundant Uplinks to Catalyst 6509 Core’s Using vPC

    Mohammed Khair said:
    September 15, 2011 at 2:24 pm

    thank you, I am having similar design but with VSS. I hope things go well 🙂

      Juan Jose said:
      March 24, 2012 at 12:42 am

      Hi, I have this escenario now, did you have any problem withh VSS and 5596 connected by VPC?,did you enable VTP in Nexus?, I mean just vtp client.

        Darow Han said:
        March 26, 2012 at 11:45 pm

        Hi Juan, I did not have VSS enabled on the upstream switches connected to the Nexus 5596’s. Also, did not enable the VTP on Nexus either.

    cisco 1941 said:
    September 20, 2011 at 10:57 pm

    Good day! Do you use Twitter? I’d like to follow you if that would be okay I’m absolutely enjoying your blog and look forward to new updates

    Brian Windle said:
    December 12, 2011 at 7:31 pm

    Anyone try this? We have a similar configuration to this and wanted to know it turned out.

    cisco 2811 said:
    July 15, 2012 at 2:28 pm

    hey, i new in the storage and servers architicture, with the same configuration that you show here and we have :
    HP ev4400 storage
    c7000 blade with flex-10
    HP SAN switch
    nexus 5000
    i saw a designe but i didn’t understand the concept,is i understand the two nexus with vPC connected to the SAN switch,and the storage with servers connected to that switch.
    i just want to understand how every component is connected to the other,thanks.

    Robert Hastings said:
    February 7, 2014 at 4:59 am

    I just configured a pair of 6509s as a VSS and instead of two separate port channels and VPCs, I created a single MEC from the VSS to a single VPC on the Nexus side. So that works out to one logical link or port channel with four members. Sweet! no spanning tree and traffic is load balanced across all four links.

      Darow Han said:
      October 28, 2014 at 4:30 pm

      Hi Robert, that’s great to hear, glad you were able to get your 6509’s into a VSS and into a single MEC. That is a more elegant solution with less moving parts, kudos!

    Jay said:
    December 9, 2014 at 7:52 pm

    I am attempting this same configuration with 7009s and 6509Es. I hope to assume all would be the same? Thanks

      Darow Han said:
      December 10, 2014 at 1:14 am

      Jay, yes if you have Nexus 7009’s instead of the 5596’s, connecting up to Catalyst 6509E without VSS, you should be able establish vPC using the listed steps.

    Graham said:
    January 30, 2015 at 11:46 am

    Hi Darow, Cisco recommend disabling loopguard globally as it’s not useful in vPC topologies (their words – not mine), as well as the uplinks to the Cats, would you recommend enabling on the vPC peer links?

    Nishant said:
    September 13, 2015 at 1:50 pm

    Hi Darow,

    Thanks for this useful post. I’m currently working on the similar setup but the upstream switch is Cisco 4500 series instead of 6500. Will the setup works similar in this case? I’ mean can 4500 series see the down stream nexus pair as one logical switch? i will be publishing only one VLAN ( say VLAN 200) over this link…

    Also planning to make one STP root on nexus pair ( peer-switch command) and HSRP on SVI VLAN 200.

    Your insight would be helpful to proceed further 🙂

    Thanks in advance,
    Nishant

    Darow Han responded:
    September 14, 2015 at 5:08 pm

    Hello Nishant,
    This setup should also be applicable if your upstream switches are Cisco 4500 Catalyst series. The 4500’s should also see the Nexus pair as a single switch using this suggested configuration.

    Hope this answers your question and keep me posted on your progress!

    Cheers,
    Darow

      Nishant said:
      October 8, 2015 at 7:27 am

      Thanks for your reply 😉

      I’m yet to start at least this Uplink setup. I have currently setup the VPCs between nexus 5ks and enabled VPC peer-switch command ( to make a single logical spanning tree root).

      So to re-confirm, still 4500 pair will see the nexus pair as a single switch with a common spanning tree root/bridge ID?? Also to ask, will all the up-link ports (4) would be in forwarding mode towards 4500 pair as per your setup?

      Regards
      Nishant

        Darow Han responded:
        October 9, 2015 at 9:05 pm

        Nishant, according to this setup each Catalyst 4500 switch will see the Nexus pair as a single switch, so both uplinks from 4500 will be active. However, note that in this scenario VSS is NOT active, therefore only one of the Cat4500 switches will be active to downstream devices due to Spanning Tree.

    Nishant said:
    October 22, 2015 at 8:55 am

    Thanks and it worked as expected 🙂

    Well , I was thinking to enable netflow to monitor uplinks from Nexus pair to 4500 catalyst.
    Which would be the best way to apply the netflow filter ? On VPC Port-channel/ SVIs or individual interfaces?

    Regards
    Nishant

    Darow Han responded:
    November 5, 2015 at 7:16 pm

    Glad to hear that you were able to get it working Nishant.
    I don’t believe that Netflow is supported on Nexus 5K series at this time though –
    https://supportforums.cisco.com/discussion/11323291/netflow-nexus-5596up

    Mohin said:
    December 11, 2015 at 4:01 pm

    I am trying to connect Cisco 4900M switches to Nexus 7010 switches with VPC config to the 4900Ms. This is not working. It looks like the 4900Ms are not sending out LACPs since the N7K portchannel is not coming up and shows no lacp PDUs in the show interface output. It looks like the interfaces come up and then disconnect right away. The portchannels at both ends are configured with mode active.
    I currently have the the links working with no cross links between the switches. So just have one link from each 4900M to just one upstream N7K switch, and no portchannel also between them.

    Any input/feedback is appreciated.

      Darow Han responded:
      December 22, 2015 at 2:25 am

      DId you create the vPC peer-link first between the Nexus 7K switches?
      Note that following my steps in the write up, you need to create the port-channel first on the Nexus switches, and then on the Catalyst switches.
      Only after all these port-channels are setup, will you see the vPC established.

      If possible, try to get config working in a lab environment first to avoid downtime.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s