123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135 |
- ================================
- Device-mapper "unstriped" target
- ================================
- Introduction
- ============
- The device-mapper "unstriped" target provides a transparent mechanism to
- unstripe a device-mapper "striped" target to access the underlying disks
- without having to touch the true backing block-device. It can also be
- used to unstripe a hardware RAID-0 to access backing disks.
- Parameters:
- <number of stripes> <chunk size> <stripe #> <dev_path> <offset>
- <number of stripes>
- The number of stripes in the RAID 0.
- <chunk size>
- The amount of 512B sectors in the chunk striping.
- <dev_path>
- The block device you wish to unstripe.
- <stripe #>
- The stripe number within the device that corresponds to physical
- drive you wish to unstripe. This must be 0 indexed.
- Why use this module?
- ====================
- An example of undoing an existing dm-stripe
- -------------------------------------------
- This small bash script will setup 4 loop devices and use the existing
- striped target to combine the 4 devices into one. It then will use
- the unstriped target ontop of the striped device to access the
- individual backing loop devices. We write data to the newly exposed
- unstriped devices and verify the data written matches the correct
- underlying device on the striped array::
- #!/bin/bash
- MEMBER_SIZE=$((128 * 1024 * 1024))
- NUM=4
- SEQ_END=$((${NUM}-1))
- CHUNK=256
- BS=4096
- RAID_SIZE=$((${MEMBER_SIZE}*${NUM}/512))
- DM_PARMS="0 ${RAID_SIZE} striped ${NUM} ${CHUNK}"
- COUNT=$((${MEMBER_SIZE} / ${BS}))
- for i in $(seq 0 ${SEQ_END}); do
- dd if=/dev/zero of=member-${i} bs=${MEMBER_SIZE} count=1 oflag=direct
- losetup /dev/loop${i} member-${i}
- DM_PARMS+=" /dev/loop${i} 0"
- done
- echo $DM_PARMS | dmsetup create raid0
- for i in $(seq 0 ${SEQ_END}); do
- echo "0 1 unstriped ${NUM} ${CHUNK} ${i} /dev/mapper/raid0 0" | dmsetup create set-${i}
- done;
- for i in $(seq 0 ${SEQ_END}); do
- dd if=/dev/urandom of=/dev/mapper/set-${i} bs=${BS} count=${COUNT} oflag=direct
- diff /dev/mapper/set-${i} member-${i}
- done;
- for i in $(seq 0 ${SEQ_END}); do
- dmsetup remove set-${i}
- done
- dmsetup remove raid0
- for i in $(seq 0 ${SEQ_END}); do
- losetup -d /dev/loop${i}
- rm -f member-${i}
- done
- Another example
- ---------------
- Intel NVMe drives contain two cores on the physical device.
- Each core of the drive has segregated access to its LBA range.
- The current LBA model has a RAID 0 128k chunk on each core, resulting
- in a 256k stripe across the two cores::
- Core 0: Core 1:
- __________ __________
- | LBA 512| | LBA 768|
- | LBA 0 | | LBA 256|
- ---------- ----------
- The purpose of this unstriping is to provide better QoS in noisy
- neighbor environments. When two partitions are created on the
- aggregate drive without this unstriping, reads on one partition
- can affect writes on another partition. This is because the partitions
- are striped across the two cores. When we unstripe this hardware RAID 0
- and make partitions on each new exposed device the two partitions are now
- physically separated.
- With the dm-unstriped target we're able to segregate an fio script that
- has read and write jobs that are independent of each other. Compared to
- when we run the test on a combined drive with partitions, we were able
- to get a 92% reduction in read latency using this device mapper target.
- Example dmsetup usage
- =====================
- unstriped ontop of Intel NVMe device that has 2 cores
- -----------------------------------------------------
- ::
- dmsetup create nvmset0 --table '0 512 unstriped 2 256 0 /dev/nvme0n1 0'
- dmsetup create nvmset1 --table '0 512 unstriped 2 256 1 /dev/nvme0n1 0'
- There will now be two devices that expose Intel NVMe core 0 and 1
- respectively::
- /dev/mapper/nvmset0
- /dev/mapper/nvmset1
- unstriped ontop of striped with 4 drives using 128K chunk size
- --------------------------------------------------------------
- ::
- dmsetup create raid_disk0 --table '0 512 unstriped 4 256 0 /dev/mapper/striped 0'
- dmsetup create raid_disk1 --table '0 512 unstriped 4 256 1 /dev/mapper/striped 0'
- dmsetup create raid_disk2 --table '0 512 unstriped 4 256 2 /dev/mapper/striped 0'
- dmsetup create raid_disk3 --table '0 512 unstriped 4 256 3 /dev/mapper/striped 0'
|