vShield Manager 4.1 – First Impressions

My group is in the process of deploying multiple vCloud Director instances and as such I thought it would be a good idea to download and test out the vShield product suite, which is one of the required dependencies for vCloud Director. To date, my group has never had a need for vShield Manager as we use another software firewall to perform the same functions. I work more closely on the systems side of the house and as such I am not always involved with all the networking projects going on. With all of the problems experienced by the network team (no fault of there own) deploying software firewalls (all fault here) in the past, I thought it would be a good idea to be involved with vShield from the beginning.
vShield is comprised of one or more of the following components:

• vShield Manager (one required per vCenter Server, free) – vCenter Server like management tool for all vShield products
• vShield Zones (one per host, free) – firewall for traffic between VMs
• vShield Edge (requires license) – provides edge security including LB, NAT, and VPN as well as gateway services
• vShield App (one per host, requires license) – interior, vNIC level firewall for all traffic between VMs even if they are on the same host
• vShield Endpoint (one per host, requires license) – introspection-based antivirus solution

For testing purposes, I decided to deploy a vShield Manager appliance as well as a vShield Zones appliance. Deploying the vShield Manager appliance is simple and since the VM is small the task takes very little time to complete. I powered on the VM and was surprised to find that the VM booted to a login in prompt and no other information was displayed. It was only upon reading the quick start guide that I realized I needed to:

• Log into the VM with admin/default
• Type: enable
• Enter the admin password
• Type: setup
• Follow the prompts

Installation
In my opinion, this process could have been made a whole lot easier. For example, when deploying the OVF:

• Prompts could have been made available to pre-populate all the information
• The initial log in prompt could have displayed what to do / how to log in
• Instead of being put at a login prompt I simple splash screen for initial configuration could have been displayed

Once the VM was configured with that appropriate IP address the guide said to navigate to the IP address via a web browser and after accepting the SSL certificate warning, log in with same credentials as the console. While the initial login splash screen from the web browser looked very web 2.0, upon logging in I found the user interface to be very simple, which is good, but the layout was less professional than I expected.
Authentication
In the user interface, I configured access to the vCenter Server. Interestingly enough, the prompts requested the administrator user name and password. Now in the production environment I help run, the network team is not given administrator access to the vCenter Server nor would they ever be given the administrator credentials. This type of access is restricted to the systems team only. Of course, this made me wonder what permissions were actually necessary for vShield to function. The only thing stated in the documentation is that the user account must have administrator access. For testing purposes, I opted to enter the administrator credentials, but I plan to revisit this prior to production deployment.
While realizing the security implications of needing administrator permission on vCenter Server, I began to wonder how authentication worked on vShield Manager. As such, I navigated to the Users tab under Settings & Reports. Upon selecting create user, I expected to be presented with a list of options such as AD, Local, and TACACS. To my surprise, it appeared only local users were allowed. I knew the network and security teams were going to have a problem with this.
Plug-in
Next, I registered the vShield Manager plug-in with vCenter Server. Upon logging out and back into my vSphere Client session, I selected an ESXi host and then selected the vShield tab. Upon comparing the output with the web client, I quickly realized the plug-in provided the same information just within the vSphere Client. Upon selecting vShield from the Home menu, I was greeted by the same login page as seen from the web browser. I logged in and confirmed the user interface was the same as the one from the web browser. Now knowing how the plug-in worked, I wished I could have inputted the vCenter Server information from the vShield Manager console and upon doing so having the plug-in automatically register. This would save time and prevent the unnecessary step of ever logging into the vShield Manager from the web browser. I am sure some would argue that having the web browser access in addition to the vSphere Client is important, but VMware has stated multiple times that vCenter Server is to be the single point of management. To date, vCenter Server is a dependency for multiple other components to fully operate and not all additional components have a second mechanism (e.g. web browser) for accessing a management interface.
One thing I did notice with the plug-in is that if you run a task such as installing vShield Zones, the screen of vCenter Server refreshes every 5 seconds are so. If you happen to have consoles open while the task is running and try to focus on the console, you will lose focus every time the vCenter Server screen refreshes. This is extremely frustrating and a compelling reason to use the web client.
Installing vShield Zones
With everything connected up, I navigated back to my test ESXi host from the vSphere Client, selected the vShield tab and selected Install next to vShield Zones. I inputted all the required management network information and selected Install. This kicked off a variety of tasks from step 1. Installing service modules. On the install task, the process stayed at 5% for quite awhile before returning complete. Shortly thereafter, the screen refreshed back to the page where vShield Zones can be installed from and the follow error message was displayed: Previous installation of host services encountered error. I tried the operation again to confirm I entered everything correctly, but the same error occurred. Next, I decided to check the logs from the vShield Manager user interface. From the host vShield tab no logging information appeared to be visible besides the error message. Going back to the vShield user interface on the Home screen, I found a System Events tab under Settings & Reports. To my surprise, the tab returned: No events found. I selected the Audit logs tab to see if it contained any information and while it did it was only in relation to user accounts and actions. This made me wonder where all the logs were and how syslog could be configured as it was not an option from the Configuration tab of Settings & Reports. From what I can tell, syslog is not supported from vShield Manager.
Going back to the issue with deploying vShield Zones, I found the following KB article with the exact same issue: http://kb.vmware.com/kb/1028003. I hoped the resolution stated was not necessary as I was running ESXi in the default mode, which meant SSH was disabled. I performed the steps listed, however it did not fix my issue. Next, I checked the VMware communities and found: http://communities.vmware.com/message/1682569. The article pointed to a common VMware problem being the inability to resolve DNS. I confirmed DNS was working on all devices and ruled this out as an issue. While on the hosts, I ensured they could reach each other. From my own testing, I realized that the vShield Manager could not reach the ESXi hosts, however it could reach the vCenter Server instance without issue. Since I knew vShield Manager was deploying a VM and deploying VMs requires access to the ESXi hosts and not just vCenter Server, I was sure I found the issue: ACLs. I adjusted my ACLs and was successfully able to deploy vShield Zones. In this particular case, the error message was not helpful. Hopefully other error messages in the vShield product suite are more informative.
Change IP Address
In my opinion, no testing is complete until you try to break things. With everything up and operational, I decided to find out what happens when you change the IP address of vShield Manager. Here are the tests I performed:

• I logged directly in to the vShield Manager console and re-ran the setup command changing the IP address. Upon reboot, the server came up without issue, however as expected the vShield Manager plug-in no longer worked. I decided to change the IP address back to see if vShield would function as before and it did.
• I logged directly in to the vShield Manager console and re-ran the setup command changing the IP address. Upon reboot, the server came up without issue, however as expected the vShield Manager plug-in no longer worked. I logged into the vShield Manager web interface and under Configuration – vSphere Plug-in I selected Un-Register. This brought up ‘Please wait…’ AJAX overlay. One thing I noticed and had missed before is that there is no cancel button once the ‘Please wait…’ overlay is up, however you can navigate away from the page. The good news, or so I thought, was the command returned ‘Plugin is unregistered successfully’, the bad news was the command did not actually do anything. I decided to change the IP address back to see if vShield Manager would function as before and it did.
• I logged into the vShield Manager web interface and under Configuration – vSphere Plug-in I selected Un-Register. The command returned ‘Plugin is unregistered successfully’ and the plug-in was removed from vCenter Server. I logged directly in to the vShield Manager console and re-ran the setup command changing the IP address. Upon reboot, the server came up without issue. I logged into the vShield Manager web interface and under Configuration – vSphere Plug-in I selected Register. The command returned ‘Plugin is registered successfully’ and the plug-in was added to vCenter Server. I confirmed that everything that was configured on vShield Manager with the old IP address was configured the same with the new IP. Everything works as expected including the vShield Zones appliance.

Overall, it looks like changes to the IP address of vShield Manager has no negative impact. The only thing to keep is mind is that while vShield Manager is down or being reconfigured, other vShield components, such as Zones, cannot be managed.
Summary
vShield is a great idea and has a lot of potential. With that said, without even getting to configuring firewall rules several issues were experienced and the whole experience left more to be desired, which point to the lack of maturity in the product. I look forward to performing advanced tasks with vShield and plan to report back on others findings as they come up. My advice, as with any new software, is to be careful and ensure your fully test before rolling out new technology in a production environment. In this particular case, vShield is a required component to a software product that is meant to run in production: vCloud Director. While I understand the decision to merge the two products together, I only hope they will both be able to scale in size and functionality to what the cloud really needs.
UPDATE: One thing I forgot to mention was redundancy for vShield Manager. Today the only redundancy available is through VMware’s Fault Tolerance technology. Just another thing to keep in mind during deployment.

© 2011, Steve Flanders. All rights reserved.