vmx nic devices are the para virtualised NIC devices should always be used over the the e1000 intel devices better performance.

disable memory scrubber in Solaris guests on ESX

Time keeping in a virtual machine. And theis KB article http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1005802

esxtop resxtop remote version esxplot fling

exstop guide.

%USED time %RDY time %CSTP( fro SMP activity) %SWPWT

memory ballooning , stuff JVMs and large Oracle SGA's set memory reservation of the guest to the size of the JVM or SGA to stop the ballooning causing the guest to page or swap out to disk, add Solaris running zfs to JVM and Oracle list. There is a paper that talks about JAVA stuff specifically , go find it.

memory thin provision but don't over commit ideal performance

most common page of memory among all guest is a page of zeros :-)

If there is a memory shortage that is effecting you now the look at the %SWPWT Swap wait in esxtop, added in esx 4.0

Second Gen Virtual Hardware assist slightly reduces the esx overhead.

Sotfwate iscsi and nfs cost ~ 10% over head in CPU over Fibre, use Jumbo frames for ethernet connected storage ( nfs & iscsi)

use pvscisi for windows and linux guests. para virtualized guest was able to increase IOPs from 20,000 to 120,000 per guest, available in 4.0 and up

NFS latency stats added in VSphere 4.1.

vscsiStats /usr/lib/vmware/bin

vprobes dtrace like tool for ESX.

Look out for preconfigured vm with io tools for testing jetstream filebench orion etc.

Type of NICs can make a difference , get engaged early to select the right NIC features.