I use the ESX Deployment Appliance (EDA) for all of my automated ESX(i) installation needs. The other day, I was attempting to configure an automated installation of ESX 4 via the EDA, but the installation kept hanging after displaying the message: No COS NICs have been added by the user. I switched to a console screen (Alt+F2) and began checking the weasel.log in /var/log. I quickly noticed that the server was attempting to PXE off of vmnic0. In ESX 3.5, I had configured the ksdevice value on the EDA to ethX. Based on this information, I updated the ksdevice field on the EDA. Upon attempting the installation a second time the same message appeared and again the installation hung. This time weasel.log was of little help as it still claimed to be booting off of vmnic0. Why was this not working?
While the default configuration of the EDA is great, I have found it necessary to make modifications to the appliance such that it could be adapted to my environmental needs. Typically, the biggest problem I have had using EDA is getting the first server to install successfully. One of the biggest reasons why an automated installation of ESX via the EDA fails is because the proper ksdevice has not been specified. To make matters worse, the ksdevice value can change from server to server depending on the number of NICs in the server.
For example, I have a server with two on-board NICs and a four-port expansion card. The NICs are laid out as follows: [ 1 ] [ 0 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ]
On ESX 3.5, the first on-board NIC (i.e. 0 above) is labeled eth4 instead of eth0 during the PXE boot process. The reason for this is because the NICs are labeled from expansion card NICs to on-board NICs and top right to bottom left. As such, in my case the top right most expansion NIC is eth0, the next is eth1, the next is eth2, the next is eth3, and since the first on-board NIC is the next right most NIC it is eth4.
To diagram this, the eth devices during the PXE process are laid out as: [ eth5 ] [ eth4 ] [ eth3 ] [ eth2 ] [ eth1 ] [ eth0 ]
This means in EDA I had to set the ksdevice value to eth4 and not eth0 in order for the installation to be successful. What makes this even more confusing is that after the installation is successful and the server boots, if I run ifconfig the first on-board NIC is eth0, the top right most expansion NIC is eth1 and the left most NIC, which in my case is on-board, is eth5. This means the layout after installation looks like: [ eth5 ] [ eth0 ] [ eth4 ] [ eth3 ] [ eth2 ] [ eth1 ]
Confused yet? I know I am.
In ESX 4.0, two things changed. First, NICs now go by vmnicX instead of ethX. Second, the NICs appear to always be in the same order during and after installation. In effect, the layout is now always: [ vmnic5 ] [ vmnic4 ] [ vmnic3 ] [ vmnic2 ] [ vmnic1 ] [ vmnic0 ]. After changing the ksdevice on EDA to vmnic4 the ESX installation displays the same warning message, however the installation continues and is successful.
I am not sure why the warning message is displayed, but the best guess I can come up with is some sort of installer issue. It appears this warning message is being seen a lot in the community. My reason for posting this is to educate people that the problem is not the warning message seen. If you are experiencing installation issues including a hung server, the warning message is a red herring. The best bet to resolving the installation issue is to check weasel.log and ensure the configuration being used for installation is valid.
© 2010, Steve Flanders. All rights reserved.
