User Tools

Site Tools


virtua:vmware:esx

ESX Server 3: Install & Configure

Overview

Other VMware Products

  • VDI (Virtual Desktop Infrastructure): same products as ESX server but lower license costs
  • Lab Manager: consolidate deployments of machines with groups of machines and setups

Installation

  • installation can be automated with KickStart files (RedHat Linux) for bigger ESX farms and for standardized setups (more...), KickStart files can be managed by EDA - ESX Deployment Appliance
ESX Disk Partition:: per default /var/log points to the root partition and hurt stability of the system ⇒ change it!

command line utility to show mount points:

vdf -h

Hardware requirements

  • x86 processors with virtualization capability (AMD-V, Intel VT), 64bit preferred
  • VMotion, HA require network storage (SAN/NAS) and Gb network interface for VMkernel
  • sufficient memory for ESX and VM's
  • fast CPU speed and many cores is a plus
  • sufficient physical network interfaces

Some useful commands for the VMware console

command usage
esxcfg-advcfg VMware ESX Server Advanced Configuration Option Tool
esxcfg-auth VMware ESX Server Network Management Utility
esxcfg-dumppart VMware ESX Server diagnostic partition configuration tool
esxcfg-firewall VMware ESX Server Firewall Configuration Tool
esxcfg-info VMware ESX Server Info tool
esxcfg-module VMware ESX Server Advanced Configuration Option Tool
esxcfg-mpath VMware ESX Server multipathing information
esxcfg-nas VMware ESX Server NAS configuration tool
esxcfg-nics VMware ESX Server Physical NIC information
esxcfg-resgrp VMware ESX Server resource group
esxcfg-route VMware ESX Server VMkernel IP stack default route Management Tool
esxcfg-swiscsi VMware ESX Server Software iSCSI
esxcfg-vmhbadevs show vmkernel to linux storage device mappings
esxcfg-vmknic VMware ESX Server VMkernel NIC configuration Tool
esxcfg-vswif VMware ESX Server Service Console NIC configuration Tool
esxcfg-vswitch Virtual Switch Configuration tool
esxtop display ESX Server resource utilization statistics
esxupdate VMware ESX Server software maintenance tool
findnic VMkernel network card locator
iscsid establish iSCSI connections
iscsi-ls show iSCSI target information
storageMonitor Monitors sense errors experienced by storage devices attached to VMware ESX Server
vmkdump VMkernel dumper
vmkfstools manipulate virtual disks and vmfs
vmkload_app VMkernel usermode application loader
vmkloader VMkernel loader
vmkload_mod VMkernel module loader
vm-support Script for gathering debugging information

disk thin provisioning

vmkfstools -c --createvirtualdisk #[gGmMkK]
               -d --diskformat [zeroedthick|
                                eagerzeroedthick|
                                thick|
                                thin]

Networking

Three types of network connections for virtual switches exist:

  • Service Console Port - access to ESX Server management network
  • VMkernel port - access to VMotion, iSCSI, and/or NFS/NAS
  • Virtual machine port group - access to VM networks

vmware vswitch

Traffic Shaping

  • only possible if the interface is linked to a physical nic
  • only possible in one direction: from the VM to the external network (both ways with v.4)

VLAN, Tags and Trunk

Virtual switches can take advantage of VLAN's.

Special VLAN Tags:

  • tags range from 0 - 4095 (2^12 -1)
Tag use
0 no tag
1 legacy tag (default tag)
4095 all tag (trunk)

Network Design

esx_net.jpg

Storage

  • Fiber Channel (FC) - high performance, but requires very expensive equipment (switches, cables, HBA's), protocol used is SCSI2
  • iSCSI - SCSI over IP, good perfromance but lower costs because of standard network equipment
  • HBA - Host Bus Adapter: connects i.e. the server PCI interface with block storage (iSCSI, FC)
  • NFS - very interesting for VM storage because of low overhead, easy provisioning, simple expand/decrease of volumes etc., v.3.5 only supports NFS v. 3 (no authentication)
  • VMFS - VMware File System: default file system for VMware storage (cluster capable), mapped at /vmfs/volumes on ESX per default
  • thin provisioning - create i.e. a LUN with 2TB of storage but only allocate the storage effectively used, i.e. 25GB (with ESX 3 only possible using command line tools)

FC

The server requires a FCHBA (Fiber Channel Host Bus Adapter)

provisioning :

  • Zoning - which WWN (World-Wide Name - LUN identifier) on the SAN is visible to the WWN on the client
  • LUN Masking - hide LUN's from a system which shouldn't have access

iSCSI

  • iqn - iSCSI Qualified Name: z.B. iqn.1998-01.com.vmware:train1
    • iqn - standard for all iqn's
    • 1998-1 - registration of manufacturer
    • vmware - manufacturer
    • train1 - unique name of LUN

provisioning:

  • Zoning
  • Mapping
Use the vmkping command to test a iSCSI adapter. This utility guarantees that the VMkernel network is used to connect.
If you re size a data volume on Windows you can use diskpart: select volume, extend. Under Linux use i.e. fdisk. :!: all data will be erased in both cases. To re size system volumes you need to use tools like PartitionMagic

Virtual Center

esx architecture

  • centralized management of multiple ESX instances
  • management of HA, DRS and VMotion
  • functionality to clone and taking snapshot's of VM's
  • collect system run-time statistics and store it in the VC database (statistics collection level 1- 4)
  • centralized update manager for ESX instances
  • AD interface

Modules

  • VMware Update Manager
  • VMware Converter Enterprise for VC

Requirements

  • Server: 2 GHz CPU, 2GB RAM
  • Database: Oracle or MS-SQL (Express Edition < 5 Hosts, 50 VM's)
  • OS: XP pro SP2 or Win Server 2003 SP1/R2
  • License: license server (can be installed on the same machine as service) ⇒ 14 days grace period if license server down

VM creation and management

What makes up a virtual machine? Below the most important VMware files:

File Name Description
VM_name.vmx Virtual machine configuration file
VM_name.vmdk File describing virtual disk characteristics
VM_name-flat.vmdk Preallocated virtual disk that contains the data
VM_name.nvram Virtual machine BIOS
vmware.log Virtual machine log file
vmware-#.log Files comtaining old virtual machine log entries
VM_name.vswp Virtual machine swap file
VM_name.vmsd File that describes virtual machine's snapshots

Attach special hardware devices to VM's:

Each VM has 6 PCI slots available. The default setup is the following:

  • 1x Video Card
  • 2x SCSI for storage devices (max. 4)
  • 2x Ethernet nics (max. 4)
  • 1x free
If you are going to setup WinXP clients you need to install the SCSI drives during bootup (push F6) using a Virtual Floppy drive

VMware Tools

  • optimization of VM drives (Network, SCSI)
  • improvement of remote control (mouse, seamless desktop)
  • improved memory controller (prevents swapping kernel pages)
  • time synchronization between VM and host
  • graceful shutdown
  • performance improvements up to 50%!

Cloning

VI can automatically customize cloned VM's:

  • on Windows VM's the SSID must be changed
    • you have to integrate the MS sysprep tools with VC (put the package under C:\Documents Setting\All Users\Application Data\VMware\VMware Virtual Center\sysprep\<os type>\ )

Prepare a physical machine/VM for virtualization/cloning

  1. virtualize
  2. delete unused services (i.e. using “srvinstw.exe” from MS)
  3. install VMware Tools
Cloning machines with IIS installed can cause problems because two machines will use the same IIS service account. WSUS could have problems because patches and Hotfixes with the same UID are found.

Access Control

  • if ESX is used without VC, local Linux users (VMware console) are used for user management
  • VC uses the general user FIXME vmc and manages the acces rights separately
  • VC can manage access rights per container (groups)
  • standard user roles exist, individual user roles can be created (fine grained rights management possible)
  • local or AD user groups can be assigned to a user role/group to manage rights

Resource Management

  • Limits: ⇒ CPU Cores, :!: allocating more then 1 CPU to a VM is not very efficient
  • Resource Groups
  • Memory Overcommitment (:!: Windows allocates 100% of Memory during boot up)
  • Memory Sharing (VM with same OS on a ESX host)

Resource Pools

Shares (priorities) for Memory and CPU can be allocated to resources as single VM's or Resource Pools.

Below you see the allocation of shares for standard levels (CPU):

  • Low: 500 shares per CPU
  • Normal: 1000 shares per CPU
  • High: 2000 shares per CPU

VMotion

VMotion - the capability to move a running virtual machine from one ESX host to another.

Requirements:

  • Virtual Center
  • Separated 1GB network (VMotion network)
  • Shared Storage, s

Storage VMotion

Storage VMotion - the capability to move a running virtual machine from one storage device to another

Clusters

A cluster of two ore more ESX instances is required to take advantage of HA and DRS

If a ESX instance is a memeber of a cluster it shouldn't be accessed anymore directly with a Virtual Infrastructure Client, but only over Virtual Center

DRS

DRS - Dynamic Resource Scheduler - automatic load balancing of a ESX cluster using VMotion.

There a three different modes for DRS:

  1. Manual: only gives you recommendations
  2. Partial: initial placement when booting a VM
  3. Full: move VM's automatically from ESX instance to another based on CPU and Memory utilization.

You can also set the level of sensibility DRS has (aggressive/conservative).

HA

HA - High Availability - In case of hardware failure in a cluster, the virtual severs will automatically restart on another host in the cluster.

Analyzing performance bottlenecks

As always when analyzing performance problems you should check all system and application logs. An interesting product to collect and centralize log files of several system with Syslog for ESX from Splunk.

  • Check the CPU ready time (the amount of time a VM was waiting to be serviced by a phsyical processor core)
    • allocate more shares
    • limit or shut down other VM's
    • move VM's to other ESX instances with more resources available
  • Check for memory ballooning (swapping a lot of memory to disk)
  • Check disk performance with tools like IOMeter
  • Check network perfromance with tools like NetIO

Resource Monitoring

  • VMware activity Monitor, integrated into VC

Planning VI Deployment

  • Excel Calculation Sheet provided by VMware
  • CPU and Memory peaks are relevant
  • VC contains a migration client which can collect performance data over weeks and give advice for infrastructure planing
  • Hardware
    • Licenses
    • Servers
    • Storage
    • Network
  • Migration of physical machines
/srv/wiki.niwos.com/data/pages/virtua/vmware/esx.txt · Last modified: 2011/07/07 16:34 (external edit)