17 March 2012

vSphere 5 Lab Setup pt 3

In part 2, the management workstation host (glados) and its components
were configured.  This post details the installation and initial setup of
the physical ESXi host (pesx0).  I was hesitant to include the following
as a separate post since it is mostly just the installation process of
ESXi and details of the various install screens.  Since it does contain
a few bits of information necessary to make the whole setup actually
function as intended, I'm including it.  It can also serve as a reference
for the virtual ESXi host installs later.  Our details for this are:
        HOST:           pesx0
        HARDWARE:       HP Pavillion p7-1254
        CPU:            1 x AMD A6-3620 Quad-Core
        MEMORY:         8 GB
        PROMPTS:        (glados [0] |pesx0> )
To start, you'll need to download the ESXi iso image from VMware.com and
write the image to bootable media.  As of this writing, the image I used
is 'VMware-VMvisor-Installer-5.0.0-469512.x86_64.iso'.  After getting
the install image, you'll want to update your BIOS settings to enable the
64 bit virtualization settings (Intel chips are VT-x (vmx) while AMD is
AMD-V (svm)).  On my host, this is found in the 'HP Setup Utility (Aptio
Setup Utility)', specifically under 'Security' -> 'System Security'.  The
option is 'Virtualization Technology':
                Virtualization Technology     > Enable
After saving the BIOS configuration, boot the system from your install
media which should bring you to a screen similar to the following:
                  ESXi-5.0.0-469512-standard Boot Menu

        ESXi-5.0.0-469512-standard Installer            <======
        Boot from local disk

                      Press [Tab] to edit options

                     Automatic boot in 8 seconds...
Selecting the first entry (ESXi Installer, noted by "<======"), will begin
the boot process.  The next several screens are mostly informational,
simply accept the defaults if prompted:
                                  Loading ESXi installer
        ===============================================================================



        <ENTER: Boot>                                      <SHIFT+O: Edit boot options>
        Automatic boot in 5 seconds...
Next screen:
                                  Loading ESXi installer
        ===============================================================================
        Loading /tboot.b00
        Loading /b.b00
        Loading /useropts.gz
        Loading /k.b00
        Loading /a.b00
        <snip...>
Initial service initialization and kernel module loads:
                        VMware ESXi 5.0.0 (VMKernel Release Build 469512)

                        VMware, Inc. VMware Virtual Platform

                        AMD A6-3620 APU with Radeon(tm) HD Graphics
                        7.5 GiB Memory


                        vmkibft loaded successfully.

                        Running cdp start


          ==================================== || || || || || || || || || || || ||
Next screen:
                                    VMware ESXi 5.0.0 Installer


                          Welcome to the VMware ESXi 5.0.0 Installation

                   VMware ESXi 5.0.0 installs on most systems but only
                   systems on VMware's Compatibility Guide are supported.

                   Consult the VMware Compatibility Guide at:
                   http://www.vmware.com/resources/compatibility

                   Select the operation to perform.

                               (Esc) Cancel      (Enter) Continue
Accept the license agreement when prompted, the installer will start
scanning your hardware:
                                    VMware ESXi 5.0.0 Installer


                                          Scanning...

                Scanning for available devices. This may take a few seconds.
Configure your storage on the ESXi host:
                                    VMware ESXi 5.0.0 Installer


                                Select a Disk to Install or Upgrade

              * Contains a VMFS partition

              Storage Device                                                Capacity
              ----------------------------------------------------------------------
              Local:
                 Local ATA Disk (t10...    (mpx.vmhba0:C0:T0:L0)          931.51 GiB
              Remote:
                 (none)


                 (Esc) Cancel    (F1) Details    (F5) Refresh    (Enter) Continue
After selecting the install disk in previous screen, select keyboard
layout, etc.  You'll then be prompted for a password for the "root" user.
Of note, the password must be at least 7 characters long:
                                    VMware ESXi 5.0.0 Installer

                            Please enter a root password (recommended)

                          Root password: ******_
                       Confirm password:

                          Passwords must be at least 7 characters long.

                          (Esc) Cancel    (F9) Back    (Enter) Continue
Following the password, secondary scanning is performed, informational
messages are given, etc.  ESXi is now ready to install:
                                    VMware ESXi 5.0.0 Installer

                                         Confirm Install

                      The installer is configured to install ESXi 5.0.0 on:
                                       mpx.vmhba0:C0:T0:L0.

                            Warning: This disk will be repartitioned.

                          (Esc) Cancel      (F9) Back      (F11) Install
After hitting 'F11', you'll see an installation progress screen:
                                    VMware ESXi 5.0.0 Installer

                                       Installing ESXi 5.0.0

                                               34 %
The last screen should be fairly quick, maybe a couple of minutes(?),
bringing us to the completed installation and prompting to reboot:
                                    VMware ESXi 5.0.0 Installer

                                      Installation Complete

                   ESXi 5.0.0 has been successfully installed.

                   ESXi 5.0.0 will operate in evaluation mode for 60 days. To
                   use ESXi 5.0.0 after the evaluation period, you must
                   register for a VMware product license. To administer your
                   server, use the vSphere Client or the Direct Control User
                   Interface.

                   Remove the installation disc before rebooting.

                   Reboot the server to start using ESXi 5.0.0.

                                          (Enter) Reboot
After the system reboots and the ESXi host is online, you'll see a screen
similar to the following:
              VMware ESXi 5.0.0 (VMKernel Release Build 469512)

              VMware, Inc. VMware Virtual Platform

              AMD A6-3620 APU with Radeon(tm) HD Graphics
              7.5 GiB Memory


              Download tools to manage this host from:
              http://0.0.0.0/

        <F2> Customize System/View Logs                         <F12> Shut Down/Restart
We'll need to make some changes so press 'F2' and supply "root" password.
This brings up the 'Customization screen'.  Navigate to 'Configure
Management Network' (noted by <=====) and hit 'Enter':
        System Customization                        Configure Management Network


        Configure Password                          Hostname:
        Configure Lockdown Mode                     localhost

        Configure Management Network   <=====       IP Address:
        Restart Management Network                  0.0.0.0
        Test Management Network
        Restore Network Settings                    To view or modify this hosts's management network settings in
        Restore Standard Switch                     detail, press <Enter>.

        Configure Keyboard
        Troubleshooting Options

        View System Logs

        View Support Information

        Reset System Configuration


                                                   <Enter> More                                     <Esc> Log Out
Since I have 2 network adapters with only one physically connected
anywhere, I'll need to update which one to use for the 'management
network':
        Configure Management Network                Network Adapters

        Network Adapters       <=============       vmnic0 (38:60:77:bf:67:49)
        VLAN (optional)                             vmnic1 (68:05:ca:01:f1:18)

        IP Configuration                            The adapters listed here provide the default network
        IPv6 Configuration                          connection to and from this host. When two or more adapters
        DNS Configuration                           are used, connections will be fault-tolerant and outgoing
        Custom DNS Suffixes                         traffic will be load-balanced.



                                                   <Enter> Change                                      <Esc> Exit
'Network Adapters' screen, select the appropriate adapter:
         Network Adapters

         Select the adapters for this host's default management network
         connection. Use two or more adapters for fault-tolerance and
         load-balancing.

             Device Name  Hardware Label (MAC Address)  Status
         [ ] vmnic0       (38:60:77:bf:67:49)           Not Connected
         [X] vmnic1       (68:05:ca:01:f1:18)           Connected

        <D> View Details  <Space> Toggle Selected         <Enter> OK  <Esc> Cancel
Hit 'Enter' to save the adapter configuration, then move down to
IP configuration.  In the next screen, I've set to use static IPv4
configuration and set the IP address and default gateway:
         IP Configuration

         This host can obtain network settings automatically if your network
         includes a DHCP server. If it does not, the following settings must be
         specified:

         ( ) Use dynamic IP address and network configuration
         (o) Set static IP address and network configuration:

         IP Address                                         [ 10.0.129.200     ]
         Subnet Mask                                        [ 255.255.255.0    ]
         Default Gateway                                    [ 10.0.129.220     ]

        <Up/Down> Select  <Space> Mark Selected           <Enter> OK  <Esc> Cancel
Of note, I'm setting the gateway even though the router VM is not yet
set up.  Additionally, since my management host, nameserver, etc.,
is on the same network segment, I don't really need to set up a default
gateway, however, I'm doing so as a formality.  Hit 'Enter' to retain the
configuration.  As an aside, IPv6 configuration is disabled by default.
Next, set up the DNS configuration to point to our lab nameserver
(lns1.lab.none (10.0.129.160)) and set the hostname of our ESXi host:
         DNS Configuration

         This host can only obtain DNS settings automatically if it also obtains
         its IP configuration automatically.

         ( ) Obtain DNS server addresses and a hostname automatically
         (o) Use the following DNS server addresses and hostname:

         Primary DNS Server     [ 10.0.129.160                                 ]
         Alternate DNS Server   [                                              ]
         Hostname               [ pesx0.lab.none                               ]

        <Up/Down> Select  <Space> Mark Selected           <Enter> OK  <Esc> Cancel
Hit 'Enter' and move onto the custom DNS suffixes:
         Custom DNS Suffixes

         DNS queries will attempt to locate hosts by appending the
         suffixes specified here to short, unqualified named.

         Use spaces or commas to separate multiple entries.

         Suffixes:              [ lab.none stor.lab.none vmo.lab.none vms.lab. ]

                                                          <Enter> OK  <Esc> Cancel
vms.lab.none is not fully shown above as it scrolled off right side of
field.  Hit 'Enter' then 'Esc' once at the "Configure Network Management"
screen to confirm the changes:
         Configure Management Network: Confirm

         You have made changes to the host's management netowrk.
         Applying these changes may result in a brief network outage,
         disconnect remote management software and affect running virtual
         machines. In case IPv6 has been enabled or disabled this will
         restart your host.

          Apply changes and restart management network?

        <Y> Yes  <N> No                                        <Esc> Cancel
After networking has been restarted, I've personally opted to enable both
the ESXi shell and SSH, which are found under "Troubleshooting options"
(we'll also need to SSH to pesx0 in just a bit):
        Troubleshooting Mode Options                ESXi Shell

        Enable ESXi Shell                           ESXi Shell is Enabled
        Enable SSH
        Modify ESXi Shell timeout                   Change current state of the ESXi Shell
        Restart Management Agents
Simply select both ESXi shell and SSH and hit 'Enter' to enable both,
then hit 'Esc'.  Both of these are disabled by default.  Also, there's no
need to restart management agents since enabling them also starts them.
After hitting 'Esc', we should be presented with a screen similar to
the following:
              VMware ESXi 5.0.0 (VMKernel Release Build 469512)

              VMware, Inc. VMware Virtual Platform

              AMD A6-3620 APU with Radeon(tm) HD Graphics
              7.5 GiB Memory


              Download tools to manage this host from:
              http://pesx0/
              http://10.0.129.200 (STATIC)

        <F2> Customize System/View Logs                         <F12> Shut Down/Restart
We're now at the end of ESXi install and initial configuration; we can
now connect to the ESXi host via vSphere Client.  For our physical ESXi
installation (pesx0), we need to set the ESXi host to allow nested 64
bit VMs.  (This is to allow our nested, virtual ESXi hosts to run 64
bit VMs.)  To do so, log into "pesx0.lab.none" (10.0.129.200) via 'ssh'.
(Since this would be the first time doing so, answer yes when prompted
about the "ssh key fingerprint":
        glados [0] ssh -l root 10.0.129.200
        Password:
        The time and date of this login have been sent to the system logs.

        VMware offers supported, powerful system administration tools.  Please
        see www.vmware.com/go/sysadmintools for details.

        The ESXi Shell can be disabled by an administrative user. See the
        vSphere Security documentation for more information.
        ~ #
Yes, I've changed the prompt in the following to allow easier
identification.  We'll need to update "/etc/vmware/config", adding
"vhv.allow = "TRUE"", which will not exist in config by default:
        pesx0> echo 'vhv.allow = "TRUE"' >> /etc/vmware/config
        pesx0> /bin/cat /etc/vmware/config
        libdir = "/usr/lib/vmware"
        authd.proxy.vim = "vmware-hostd:hostd-vmdb"
        authd.proxy.nfc = "vmware-hostd:ha-nfc"
        authd.proxy.nfcssl = "vmware-hostd:ha-nfcssl"
        authd.proxy.vpxa-nfcssl = "vmware-vpxa:vpxa-nfcssl"
        authd.proxy.vpxa-nfc = "vmware-vpxa:vpxa-nfc"
        authd.fullpath = "/sbin/authd"
        authd.soapServer = "TRUE"
        vmauthd.server.alwaysProxy = "TRUE"
        vhv.allow = "TRUE"
        pesx0> exit
        Connection to 10.0.129.200 closed.
        glados [0]
Once "vhv.allow" has been appended, the above change is immediate and no
restart is necessary.  This concludes the base ESXi host configuration.
Part 4 will detail the initial configuration on pesx0 in order to support
the rest of the lab infrastructure.

see also:
    vSphere 5 Lab Setup pt 1: The Overview
    vSphere 5 Lab Setup pt 2: The Management Hosts
    vSphere 5 Lab Setup pt 4: Network Configuration on the Physical ESXi Host
    vSphere 5 Lab Setup pt 5: Infrastructure VM Creation
    vSphere 5 Lab Setup pt 6: Infrastructure VM Configurations and Boot Images
    vSphere 5 Lab Setup pt 7: First VM Boot
    VirtuallyGhetto's writeup on nesting 64 bit VMs