NVMe Tiering in vSphere 8.0 Update 3 is so cool!

So if you have not seen William Lam’s post do head over there as he was the inspiration for this!

Firstly why would I want this! Well in the home lab this will allow me to use Holodeck on even more host types and I hope further down the line when ECS is at this build I can start to do some fun AI edge workloads with all this amount of memory too!

For me personally have some of VCF running on my NUC Extreme nodes would use much less power day to day

I have added a few steps to what William’s guide states just to get up and running as fast as possible, quite a few steps can also be done from the GUI and PowerCLI too.

The first action is to SSH to your host and pop the host into maintenance mode

esxcli system maintenanceMode set --enable true

Next run the following command just to list out your NVME Drives and take a note of the long string which will be needed shortly

esxcli storage core adapter device list

Next you need to make sure you have cleared all the partitions from the disk, you can use the parted tools but in all honesty use the GUI 🙂 It is way easier! Do this by selecting storage, right click the NVME and then clear partition table

From here we can now prep the drive with the tier partition with the below command replacing the /disks/…. with your GUID

esxcli system tierdevice create -d /vmfs/devices/disks/t10.NVMe____INTEL_SSDPED1K750GA_____________________000132AF4BE4D25C

Now to activate the memory tiering

esxcli system settings kernel set -s MemoryTiering -v TRUE

Finally the percentage of storage you want to use from the main system memory, this example sets it to 400%

esxcli system settings advanced set -o /Mem/TierNvmePct -i 400

If everything has gone well you should see something similar to below, your last step is to reboot the host for this to all take effect

So this is the before

And here is the after, a massive uplift to now having of 317GB RAM from the original 64!

I think I need to plan this a little better and maybe utilize a smaller drive then using my Optane device but it would be interesting to benchmark this and see how things get handled.