Storage Spaces is a great technology natively presented in Windows. With Storage Spaces you can create volumes with different resiliency settings, including parity (N+1 and N+2). This configuration is very efficient when storing large amounts of data.
Many users are experiencing slow performance when writing files larger than 1 GB on Parity volumes, caused by inefficient default settings.
Today I will show how to properly setup a parity volume on Windows Server 2022 Storage Spaces with good performance. First of all you need a server with a bunch of disks connected.
Some theory
In Storage Spaces your actual data is stored on NTFS formatted Volume, which is created on Virtual Disk (VD), which is created on Storage Pool, which lays on top of Physical disks.
Allocation Unit Size (AUS) is a filesystem cluster size. Any file written on filesystem will be divided in pieces of this size before writing to underlying level.
Virtual Disk is a logical device created on top of Storage Pool. This is a continuous block device underlying to filesystem. Data written on VD divided by Interleave and written to Physical Disks.
Stripe is a unit used to write data to Physical Disks. Each stripe contains data and parity Interleave-sized chunks.
Stripe Interleave is amount of data written at once to physical disk within stripe. Every write to VD is divided to Interleave-sized chunks, then parity chunks are generated, then every chunk is written to one Physical Disk.
NumberOfColumns is amount of disks used at once for stripe write operation. NumberOfColumns for VD may be lower than number of disks in Storage Pool. Each write operation may use different disks to write data, but number of chunks (data and parity) within one Stripe will always be equal to NumberOfColumns.
In following example NumberOfColumns in VD is equal to 6, and number of parity disks is 2. This way each Stripe contains 4 data and 2 parity chunks.
Disk number
The minimum amount of disks required to create a parity VD is 4 or 5 depending on parity type – single or dual. However in this case we will get a performance degradation since there is no AUS exists that will evenly divide by 3 data columns. So when choosing a minimum number of drives to create a pool, a minimum of 5 should be used for N+1 VDs and 6 for N+2 VDs.
Issue nature
By default when trying to setup Storage Spaces in Control Panel, Windows will create a volume with interleave size of 256 KB and NTFS volume with 4 KB cluster size on top of it. Do you see an issue? Here is a tip: in order to write user data to physical disks Windows will divide 4 KB clusters by 256 KB Interleave, then calculate parity on this thing and then write it to disks. Such operation will be sensitive to computing resources, and of course there would be never enough.
This problem caused by size mismatch between NTFS Allocation Unit Size, Interleave, and Number of columns.
The solution here is to allign NTFS AUS with Interleave and Number of columns. It’s relatively simple, but not implemented by default yet!
And what about 1 GB of a good speed at the beginning? By default windows will enable write-back caching on a new VD, so all of the data written to VD actually going to RAM and can be easily lost in case of power disruption. In order to avoid such situations, write caching on VD should be disabled.
Choosing right settings
Now we know where is a bottleneck located and its time to fix it.
Here is a most important part of Storage Spaces deployment procedure since most of parameters cannot be changed after VD is created. In order to change something you have to delete VD and create a new one, which can be tricky if you alredy have a data on it.
Let’s assume we have a 6 disks in Storage pool and want to create a VD with dual parity. First of all, a proper NumberOfColumns should be selected. NumberOfColumns cannot exceed number of disks in pool, so maximum available amount in this example is 6. In each stripe there would be 2 columns of parity and 4 columns of data.
In order to allign Interleave with AUS we need to calculate one from another, and I recommend first to decide what AUS will be the best for current use case. Generally, larger AUS will provide better performance on large files, while small AUS will not waste space on small files. For most NAS systems a value of 64 KB will provide a great balance.
Interleave is calculated as AUS divided by number of data columns. It means that AUS is 64 KB / 4 = 16 KB. Thats all infarmation needed and now its time to go to Powershell, as Server Manager features are very limited.
Deployment
To identify available drives, run the following command:
Get-DiskCreating a Storage Pool:
New-StoragePool -FriendlyName Pool1 -PhysicalDisks 1..6And the most interesting part – Virtual Disk:
New-VirtualDisk -StoragePoolFriendlyName Pool1 -FriendlyName VD1 -NumberOfColumns 6 -ResiliencySettingName parity -ProvisioningType Fixed -PhysicalDiskRedundancy 2 -Size 5TB -Interleave 16KB -WriteCacheSize 0This volume will be later encrypted, so there is no need in thin provisioned virtual disk. Also remember that NumberOfColumns includes both data and parity columns while only data columns used for Interleave calculation.
Now go to Disk Management, bring new disk online, and create a new volume. Format newly created volume to NTFS. Set AUS to 64 KB during formatting.
Let’s copy something on newly created Volume! Result is great, comparing to 20 MB/s on default settings.
Post-deployment tasks
It is useful to enable disk performance monitoring in Task Manager:
cmd.exe diskperf -YNow you can see load statistics on newly created volume:
Also it is a good idea to add some Hot Spare drives in pool, and in case of drive failure it will be automatically replaced:
Add-PhysicalDisk -StoragePool (Get-StoragePool Pool1) -PhysicalDisks (Get-PhysicalDisk -DeviceNumber 7) -Usage HotSpareTo verify Pool and VD configuration, run this:
Get-StoragePool Pool1 | fl
Get-VirtualDisk VD1 | flUseful links:
https://storagespaceswarstories.com
https://wasteofserver.com/storage-spaces-with-parity-very-slow-writes-solved
Leave a Reply