1/24/12

Multiple subnets with one VMware ESXi host

As we’ve moved more and more of our critical infrastructure at The Chapel to the virtual world, I’ve struggled on occasion with the issue of setting up network cards in VM’s to work on different subnets.

This became a real issue when we migrated from our Cisco phone system to our virtualized MiTel phone system. All was good until I needed to setup the “MiTel Boarder Gateway” which acts as a firewall and SIP gateway for the phone system. Since I had to get this up and running quickly I just installed another network card then mapped it to a virtual switch in VMware and mapped the second NIC in the VM to that virtual switch.

image

This approach however is not very efficient or redundant. It also takes up valuable NIC’s and switch ports. My plan is to update this configuration with what I’ve learned when setting up our print server to work with FingerPrint which I’m going to detail below.

How to setup Vlan tagging in VMware ESXi

  1. First, you need to have a working ESXi host. The setup isn’t that hard but is more than I’m going to go into here.
  2. Setup your switch port(s) that connect to the server as a “trunk” in Cisco speak with a “Native Vlan” set to what a majority of your servers use. That way you don’t have to setup tagging on every vNIC.
  3. If your looking to have a server that needs to talk to two different subnets like a firewall or my print server running FingerPrint, add another Ethernet adapter to your VM and assign it to your default network. Mine is “VM Network”.
  4. You need to check that the Virtual Network on your primary vSwitch allows all Vlans. By default it is set to “None(0)“.  Set it to “All(4095)” or just the ones you want.
     image
    image
  5. Now, start up your VM and log in. Navigate to the Device Manager and select the network card you want to configure a different Vlan on.
    image
  6. Once you configure the tagging, make sure that you have the IP addresses setup correctly. For a firewall type VM, you will have different IP’s and gateways on different subnets. If you have a server connecting to two private networks, only set a default gateway on the “Primary” network. Windows doesn’t like it if you set different gateways to the same routed network.

That’s it. Now your servers can use different and special Vlans when needed and you don’t need to add another NIC or vSwitch each time. In my case, it allowed me to easily setup FingerPrint to communicate with our wireless network with the Bonjour protocol.

For my friends that are more versed in VMware than I, please post your comments and questions. I’m always interested in what others are doing or what the “Right” way is.

4 comments:

Justin Moore said...

Yeah, you're close - but VLAN 4095 really should only be used when you want a single vNIC to be able to deliver multiple VLANS through the vSwitch direct to your VM (like a virtual firewall for example). The problem with vlan4095 is it's passing extra traffic to every vNIC attached to the port group. In your case, vlan150 packets are hitting every vNIC and getting discarded - even the ones who don't need it, so it's just not efficient (or secure if you're in an environment that cares about that sort of thing).

Rather than configure your Port Group to 4095, you should create a port group specifically for vlan 150 and attach the vNIC to that. You can have LOTS of port groups on each vSwitch, so don't sweat that. Just make 100% sure you replicate the port group name EXACTLY on all hosts in your cluster or you'll have problems when migrating VMs from one to another using vMotion or even off-line migrations.

I'm working on a blog post today actually about my network config on our recently rebuilt vSphere clusters, but here's a teaser:
https://skitch.com/wantmoore/g52d6/gapvcenter

Jeremy Good said...

Great point Justin. I actually had the different port group for vlan 150 but then changed it to this approach. I think I'll change it back to your way for security reasons. Thanks!

AusNetIT Solutions said...

Justin,

Can you give me the Ip configuration for following

esx:

VLAN1 VM IP configuration and VLAN2 IP config?

I try to run multi subnet test network with One ESX with 1 NIC?

Thanks

Brandon said...

Very extensive guide. I'll try running Multiple subnets soon