Krishna Raj Raja, VMware. This session was a super deep dive on ESXtop stats, and how to read them. In short, unless you are vSphere guru and understand how the hypervisor really works, then using ESXtop is NOT for you. Use more friendly tools like vCenter Operations manager, which takes the metrics and turns them into actionable information.
- Use vCenter Operations to manage and monitor lots of VMs and hosts. Also provides Performance, Capacity, configuration.
- For most situations vCenter Operations is the right tool, ESXTop will NOT be right tool.
- If using a Mac, set “export TERM=xterm” then launch esxtop
- Managing counters: limit the fields data captured
- In perfmon you can tune the range of data you are viewing, and can re-save that subset
- Limit data captured
- esxtop –export-entity entity.list
- vi entity.list
- esxtop –import-entity entity.list
- Memory Usage
- Granted Memory – Only grants memory the OS can access. E.g. Win7 32-bit won’t get more than 3GB of RAM. Can show less than full allocation if ballooning happens.
- MCTLSZ: How much memory was ballooned out.
- Zero counter – Zero pages, not allocated
- Active Memory – ACTV, ACTVS, ACTVF, ACTVN – counters high usually means high CPU usage. Active memory stats can go down or be low if the VM is swapping out to disk. Don’t reduce VM memory allocation just based on active memory stats. Rely on guest OS memory usage metrics, not ESXtop.
- Overhead Memory – OVHDUW (used for SVGA memory and other structures). In ESXi 5.0 and 5.1 VMs have much less overhead. OVHDMAX – includes overhead plus memory and vCPU overhead. Big gains in ESXi 5.0/5.1 over 4.x.
- Wide NUMA
- NHN, N%L, GST_ND0, GST_ND1
- Numactl –hardware
- vNUMA – VM HW v9 and VM is 8 vCPUs or more, and expose NUMA info to guest
- Useful for NUMA aware apps like SQL servcer
- World Groups
- Idle thread – one per CPU. Don’t worry about these stats, like %RUN, %WAIT
- Press “V” to show only VM world groups
- Expand a VM World Group to show all threads, SVGA is shown when 3D is shown. High %USED when lots of 3D rendering happens.
- MKS thread – Mouse, Keyboard, Screen thread. High %USED = lots of 2D activity.
- VM Provisioning
- LUN view screen. Press E to expand to view which world is doing the I/O
- CPU Usage
- Physical CPU – PCPU (physical and logical CPUs), Core = half the number of PCPUs if Hyperthreading is enabled.
- vCenter Core Utilization stats – Duplicated stats
- P0 – Full processor clock speed
- Pn – Weighted clock frequency (less than P0)
- PCPU Used – Measured against max possible CPU frequency (P0), UTIL = measured against current clock speed.
- HP BIOS – If using OS Control Mode for Power savings, you will see P0 – Pn where PSTATE MHZ shows the clock frequency for each P-state.
- Core Sharing – Can change in vCenter UI – Hyperthreaded Core Sharing. Defaults to “any” for best scheduling.
- Host Cache
- Swap to SSD – %VMWAIT means a VM is blocked from processing for some reason (e.g. swapping). Look at SWR/s and SWW/s stats for swapping metrics. LLSWR/s and LLSWW/s are metrics for swapping to SSD. %VMWAIT time should reduce.
- View Storage Accelerator – Look at MBREAD/S, look at LUN and I/O stats won’t show at much MB/s due to cache hits.
- SR-IOV
- Creates virtual functions for physical devices, such as a NIC
- Present the virtual device directly to the VM guest as a pass-through
- Does not work with vMotion.
