Are you ready to move away from SD cards into M.2 flash disks or boot from SAN? If you are one one of the many customers that is planning on leveraging SD cards for your ESXi 7.x install/upgrade (in the future whenever it is released), then this article is definitely important so you can start planning accordingly. The change is not mandatory yet, but starting with vSphere 7 the install design was modernized a bit, and the OSDATA partition which includes all the local logs (small/large core-dump, locker and scratch) was all combined as part of the OS install.
With that being said, there are now a lot more writes going directly into ESXi installation media and the SD cards are simply not durable enough to handle it. Previous versions of ESXi do not put this much stress on the cards, and most of these logs are configured to write to shared VMFS/NFS datastores, assuming you configured them to do so (otherwise, they would just reside in memory).
Quote from the official VMware knowledge base article (KB85685) that can be found here
“Some of the reasons why SD cards will be unsustainable with the next major release are:
- Reads/Writes to System Storage continue to grow, mainly to the ESX-OSDATA partition. The OSDATA partition is a critical component that is required to be always available and also be persistent across reboots. OSDATA will be used in a variety of different ways as services and applications such as vSAN and NSX grow. OSDATA is used to store some of the runtime state, and for probes, backups of items such as configuration, timestamps etc. Access to other areas on OSDATA, such as VMTools and scratch portions will also grow. With each new vSphere release, more features will start relying on the OSDATA partition.
- Performance demands and IO loads cannot be continued to be met by SD cards. For example, the demands on writes to the scratch area have been growing. Another concern is the high frequency of IO with bursts such as logs or traces.
- There is no way to reliably check and monitor for the SD card endurance, as these devices usually do not provide support for diagnostics data. Wear-out issues and remaining endurance or lifetime can go undetected. In addition, not just writes but reads are also known to cause wear issues.
- SD/USB devices are prone to wear over time and are thus subject to failures, they are not designed for enterprise class use-cases.”
If you are currently in this predicament, there are several options available for you to run or upgrade to vSphere 7.x and still leverage your current investment (SD cards) with the current hardware. Moving forward though, I would strongly recommend avoiding SD cards for vSphere compute purchases.
Some options to alleviate the amount of writes to the SD cards can be found below if you are planning to upgrade to version 7.x:
- “Reduce IOs to SD Card by reducing access to VMTools and scratch areas of OSDATA
- To store the VMTools area of OSDATA, by default, ESX will try to find a local persistent device or a SAN LUN to store VMTools. Customers can also enable options to move VMTools out of SD card into RAMDisk. See https://kb.vmware.com/s/article/83376
- For Scratch area, if there is no local HDD, SSD, NVMe device, or M.2 found, ESX will move the scratch area as well to RAMDisk automatically
- Customers can also configure and move Scratch area to SAN or NFS
- In addition, customers also have options to configure remote syslog to move logs out of OSDATA“
More information on the vSphere 6.x vs 7.x storage layout differences can be found here
