Nesting NSX-T (lab) inside NSX-T (infra)

From Iwan
Jump to: navigation, search

When we build nested labs, we typically use VLANS to provide an L2 broadcast domain where we can provide networking services across. I also did this for a long time in my home lab, when I needed a new network with a new gateway I typically configured the gateway on a routed (SVI) VLAN subinterface with a new tagged VLAN. With this, I only had the gateway and then I needed to configure the other 4 switches in my home network with this new VLAN. This took too much time and effort and since I am lazy, I did not want to do this anymore, so I installed NSX-v and started to play with NSX-v. Eventually, I upgraded to NSX-T, I used the V2T migration coordinator but this went horribly wrong, so I did a reinstall. Bust this is another story for another time.

All my (nested) lab components need a network. So I decided to use NSX-T Segments for these lab VM's. This all works well, but what if I want to build a nested NSX-T lab? This means that the “GENEVE” lab network that is required to provide NSX-T "overlay" Networking services inside my nested lab is using an NSX-T "underlay" network as the transport layer.

Yes, I know … There is not a simple way to explain this better so I decided to illustrate this in a drawing:

3AF8514F-F955-496F-BA44-5CBB198735D5.png

When you look at the picture you can divide the picture into two constructs, INFRA and NESTED.

  • INFRA
    • Infra switch (physical)
      • My physical switch where the infra ESXi server is connected to
      • The interfaces of this switch connecting to the infra ESXi server are trunk interfaces and allow all VLANS across the interfaces
    • infra ESXi Host (physical)
      • One of my physical ESX servers that are connected to the physical switch/
      • This physical host also hosts the virtual machines that will behave as nested ESXi hosts
    • infra vmnic interfaces
      • The physical NICs of the ESXi host connected to the physical switch
    • infra N-VDS -> The N-VDS offered by the NSX-T Manager that is part of my lab infrastructure
      • infra NSX-T VLAN segments -> The VLAN Segments offered by my infra NSX-T Manager
        • infra vmk interfaces -> The infra vmkernel interfaces, connected to a VLAN backed segment
      • infra NSX-T GENEVE overlay segments #1 -> The GENEVE (overlay) Segments offered by my infra NSX-T Manager
        • infra Web, App and DB VM’s -> Regular VM’s connected to NSX-T VLAN backed segments
      • infra NSX-T GENEVE overlay segments #2 -> The GENEVE (overlay) Segments offered by my infra NSX-T Manager that are going to be used as UNDERLAY for the Nested parts (explained below)
  • NESTED
    • nested ESXi Host (virtual)
      • One of my virtual nested ESX servers that are connected to the infra N-VDS switch
    • nested vmnic interfaces
      • The virtual NICs of the nested ESXi host connected to the infra N-NDS switch on an infra overlay segment
    • nested N-VDS -> The N-VDS offered by the NSX-T Manager that is part of my NESTED lab infrastructure
      • infra NSX-T GENEVE overlay segments -> The GENEVE (overlay) Segments offered by my NESTED NSX-T Manager
        • nested vmk interfaces -> The nested vmkernel interfaces, connected to a (nested) GENEVE overlay backed segment
        • nested Web, App and DB vVM’s -> Double nested vVM’s connected to NSX-T GENEVE overlay backed segment

I know this is mind-blowing and pretty challenging to write about, but I did my best and hope you understand what is going on.

Retrieved from "http://www.iwanhoogendoorn.nl/index.php?title=Nesting_NSX-T_(lab)_inside_NSX-T_(infra)&oldid=4393"