I feel like this is a beat up topic, but it honestly is my one of my favorites sessions every year during VMworld (SQL best practices to be more precise, since those are usually the resource hogs) just to make sure I stay current on this topic. Technology and the ways we consume it changes rapidly so it is in everyone’s best interest to stay informed.
- Physical Host Power Management
- It is recommended to set this setting to “High performance” to make sure that the physical host is able to utilize all the resources for the VMs hosted inside it.
- How? Go to the host > Configure Tab > Power Management > Active Policy > High performance (no reboot required)
- CPU Hyper-Threading
- Hyper-threading on the physical hosts should be enabled to keep the processor pipeline busier
- This allows more opportunities to schedule CPU cycles and reduce waits/idle time in-between
- How? This is normally enabled on the BIOS for each server/blade.
- Example on Cisco UCS policy below:
- Right Sizing your Virtual Machines
- Avoid allocating more vCPUs than necessary on all VMs, specially more cores than 1 single physical CPU in your host can handle (grow as needed) unless you want to miss out on vNUMA capabilities.
- Same applies to memory, and hopefully the memory installed is evenly split per socket to begin with.
- Example: Your hosts have 2x CPUs, each with 16 cores and 512GB of memory total.
- Try to avoid VMs with more than 16 vCPUs and 256GB on them, unless you have to.
- Avoid CPU over-commitment on the host (high CPU-Ready time on VMs)
- Production cluster with general workloads, I like to recommend 3-4 vCPU to physical core ratio
- Production cluster with VDI workloads, 2-3 vCPU to physical core ratio
- Production cluster dedicated to databases, up to 2 vCPUs to physical core ratio
- Non-production clusters, 4-6 vCPUs per physical core usually works as long as same performance as production is not expected from the application owners
- Want to see if your hosts are over-committed? Pick a busy VM with at least 6+ vCPUs and check the CPU-Ready time
- How? Select VM > Monitor > Performance > Overview > Ready for VMname
- If the value is under 5 during busy times, the VM is fine and the host is not making it wait too long for clock cycles
- If the value is over 5, check a few other VMs and see if the value is consistent. Performance might be starting to regrade at that point
- Disable CPU/memory “hot-add” for VMs with 8 or more vCPUs
- Unfortunately, this setting is a double-edge sword
- Yes, it allows you to add vCPUs on the fly, however, it also disables vNUMA, which may end up creating some latency if the resources are forced to split between physical CPUs on that host
- If the VM needs more resources, simply piggy back off your patching window and add them
- How? Select VM > Edit Settings > Expand CPU > Disable CPU Hot Plug
- SCSI Controllers
- This is useful for virtual machines with high disk throughput / IOPS requirements (SQL for example)
- SCSI Controller 0 can remain as default (LSI Logic SAS), but make sure the rest are Paravirtual
- Paravirtual offers higher amount of operations and throughput with less CPU overhead on the VM
- Ideally on a SQL virtual server, add three extra controllers and separate your SQL Data, Logs, TempDB into separate disks, with each type of disk assigned to its own controller
- Standard I normally follow/recommend (all three SQL controllers added as Paravirtual type)
- SQL Data disks on Controller 1
- SQL Logs disks on Controller 2
- TempDB disk on Controller 3
- How? Select VM > Edit Settings > Add new SCSI Controller > Expand Controller > Select Paravirtual
- Virtual Machine Network Adapter
- Use VMXNET3 always, which is the default nowadays
- If you have some legacy environments, make sure to upgrade your VMs from E1000, E1000E or Flexible
- Those adapters I just mentioned are old, un-supported and have a 1GB throughput limit
- How? Remove NIC with old adapter. Add new NIC with VMXNET3 adapter. Re-configure network settings.
- Virtual Machine VMTools Software
- Try to keep your virtual machines and templates up to date
- The VMtools software enhances not only management, performance and compatibility, but they also fix certain bugs the previous version might have
- How? Select VM > Actions > Guest OD > Upgrade VMware Tools > Interactive or Automatic
- Make sure to add some the command below if you want to install silently and avoid immediate reboot
- Advanced Command:
/s /v "/qn REBOOT=ReallySuppress" - You can also enable automatic VMware tools upgrade per VM, every time the VM reboots
- How? Select VM > Edit Settings > VM Options > VMware Tools > Tools Upgrades > Check and Upgrade VMware Tools before each Power On
- Virtual Machine Hardware version
- Validate before you upgrade any VMs, that all your hosts in your cluster are able to support the VM HW version
- Do not upgrade the hardware of VMs that are out of support (Windows 2008 and below, vendor appliances)
- Update VMware tools before upgrading the VM hardware
- Create a backup or take a snapshot of the VM before you proceed, just to be safe (remove snapshot afterwards)
- Why upgrade?
- Better security (spectre and meltdown patches were introduced in version 12+)
- Compatible with newer features (virtual TPM, NVDIMM, per VM-EVC, etc)
- Higher resource maximums
- How? Shut down VM > Edit Settings > Compatibility > Upgrade VM Compatibility
- Want to piggy back off the maintenance window? Schedule VM Compatibility Upgrade on next boot instead!
Do you have any other suggestions? Please comment below!
Disclaimer: Image belongs to VMware
Thanks for Best Practices recommendation
Very informative, thanks!
This is one of the most important things to apply in managing and administering your virtual machines within your vSphere Esxi environment.
Thank you!!
100% Useful! Thank you!