Achieving peak storage performance in the VMware environment involves tuning the VMware configuration beyond default values.  The following are recommended best practices for tuning VMware for use with SoftNAS.

 

VMDirectPath

 

VMDirectPath provides a means of passing a disk controller device directly through to the guest operating system (i.e., CentOS Linux).

 

To enable VMDirectPath Configuration page in the vSphere Client

 

                  1) Select the ESX host from Inventory.

                    2) Select the Configuration tab.

                   3) Select Advanced Settings under Hardware.

                   4) Edit and select your device(storage controller,physical nic)

 

Note that Intel VT-d (or equivalent) processor feature is required for support of VMDirectPath.

 

 

VM SCSI Controller - Set to Paravirtual

 

In VMware, change the SCSI controller type to "Paravirtual", which provides more efficient access to storage.

 

Physical NIC Settings

 

A host physical NIC can have settings, which can provided better utilization and performance improvement:

 

Most 1GbE or 10GbE NICs (Network Interface Cards) support a feature called interrupt moderation or interrupt  throttling, which coalesces interrupts from the NIC to the host so that the host does not get overwhelmed and spend too many CPU cycles processing interrupts.

 

To disable physical NIC interrupt moderation on the ESXi host execute the following from ESXi SSH session.

 

You can find the appropriate module parameter for your NIC by first finding the driver using the ESXi command:

 

# esxcli network nic list

 

Then find the list of module parameters for the driver used:

 

# esxcli system module parameters list -m <driver>

 

This example applies to the Intel 10GbE driver called ixgbe.

 

# esxcli system module parameters set -m ixgbe -p "InterruptThrottleRate=0"

 

 

Also, check your host for SR-IOV support, which provides additional performance and throughput in virtualized systems like VMware.

 

 

 

Adjust Network Heap Size for high network traffic

 

By default ESX server network stack allocates 64MB of buffers to handle network data.

 

   Increase  buffer allocation from 64MB to 128MB memory to handle more network data                           

 

To change Heap Size ESX Host: 

 

Configuration tab for the ESX Server host- Advanced Settings-VMkernel - Boot- VMkernel.Boot.netPktHeapMaxSize

 

 

Virtual NIC Settings

 

VM’s virtual adapter has many tuning options, which can also provide much better throughput:

 

Configure jumbo frames (MTU 9000)  in vSwitch and virtual network adapter  (be sure physical switch supports MTU 9000)

 

We recommend you choose VMXNET 3 virtual NICs

 

Disable virtual interrupt coalescing for VMXNET 3 virtual NICs as follows:

 

Go to  vSphere Client, go to VM Settings - Options tab - Advanced General - Configuration   Parameters and add an entry for ethernetX.coalescingScheme with the value of disabled.

An alternative way to disable virtual interrupt coalescing for all virtual NICs on the host which affects all VMs, not just the latency-sensitive ones, is by setting the advanced networking performance option (Configuration -Advanced Settings - Net) CoalesceDefaultOn to 0 (disabled).

 

Disable LRO

 

To do so  you need to reload the vmxnet3 driver in the SoftNAS CentOS guest operating system. Log into the SoftNAS VM using SSH (or the Desktop Console) and "su root":

 

# modprobe -r vmxnet3

 

Add the following line in /etc/modprobe.conf ( options vmxnet3 disable_lro=1)

 

Then reload the driver using:

 

# modprobe vmxnet3

 

 

Physical Host BIOS Settings

 

On most servers,  these BIOS Settings can improve the overall performance of the host:

 

·     Turn on Hyper-threading in BIOs

 

·     Confirm that the BIOS is set to enable all populated sockets for all cores

 

·     Enable “Turbo Mode” for processors that support it

 

·     Confirm that hardware-assisted virtualization features are enabled in the BIOS

 

·     Disable any other power-saving mode in the BIOS

 

·     Disable any unneeded devices from the BIOS, such as serial and USB ports

 

·     In order to allow ESXi to control CPU power-saving features, set power management in the BIOS to “OS Controlled Mode” or equivalent. Even if you don’t intend to use these power-saving features, ESXi  provides a convenient way to manage them.

 

·     C-states deeper than C1/C1E (i.e., C3, C6) allow further power savings, though with an increased chance of performance impacts. We recommend, however, that you enable all C-states in BIOS, then use ESXi host  power management to control their use.

 

 

NUMA Settings

 

NUMA systems are advanced server platforms with more than one system bus. They can harness large numbers of processors in a single system image with superior price to performance ratios. The high latency of accessing remote memory in NUMA (Non-Uniform Memory Access) architecture servers can add a non-trivial amount of latency to application performance.

 

For best performance of  latency-sensitive applications in guest OSes, all vCPUs should be scheduled on the same  NUMA node and all VM memory should fit and be allocated out of the local physical memory attached to that NUMA node.

 

Processor affinity for vCPUs to be scheduled on specific NUMA nodes, as well as memory affinity for all VM memory to be allocated from those NUMA nodes, can be set using the vSphere Client under VM Settings – Options tab – Advanced General – Configuration Parameters and adding entries for “numa.nodeAffinity=0, 1, …,” where 0, 1, etc. are the processor socket numbers.