Short howto for dm-cache preparation. Assuming existing VG: vg_server17 , inside PV /dev/md1 ( comprising of md RAID5: sda, sdb, sdc ).:-
1. Add SSD into server, assuming detected as /dev/sdd . Make sure OS is latest ( at least CentOS 6.6 ):-
# yum update
- Reboot.
- Prepare SSD for inclusion in LV:-
# fdisk /dev/sdd
> p (Should see no partitions)
> n
> Partition number: 1
> First cylinder: (default: start at beginning)
> Last cylinder: (default: until the end)
> p (Should see 1 partition of type 'Linux')
> t
> Hex code: 8e
> p (Should see 1 partition of type 'Linux LVM')
> w
- Create PV on SSD, and include into existing VG:-
# pvcreate /dev/sdd1
# vgextend vg_server17 /dev/sdd1
2. Assuming vm999 is existing VM inside vg_server17 ( if not exist / for testing, can create using a simple LV e.g. : lvcreate -L1T -n DataLV vg_server17 /dev/md1 )
3. Assuming SSD size is 64GB, create 58GB cache, leaving space for meta etc.:-
# lvcreate -n DataLVcache -L58G vg_server17 /dev/sdd1
4. Create meta LV ( The documentation says that the DataLVcacheMeta should be 1/1000th of the size of the DataLVcache, but a minimum of 8MB. )
# lvcreate -n DataLVcacheMeta -L58M vg_server17 /dev/sdd1
5. Activate cache pool. cachemode writeback gives max performance but sensitive to corruption if abrupt crash/shutdowns :-
# lvconvert --type cache-pool --cachemode writeback --poolmetadata vg_server17/DataLVcacheMeta vg_server17/DataLVcache
( when prompted, answer Y (yes) ).
6. Attach cache pool into existing LV ( vm199_img ):-
# lvconvert --type cache --cachepool vg_server17/DataLVcache vg_server17/vm199_img
7. IMPORTANT: Ensure your VM is still shutdown, and set I/O drivers are as follows:-
- SolusVM --> Settings (General Tab) :-
i) Disk Driver: virtio ( If Windows VM, leave as ide )
ii) Disk Cache: None
- SSH to node, go to /home/kvm/kvm199, edit kvm199.xml and append the following in 'driver' section :-
<disk type='file' device='disk'>
<source file='/dev/vg_server17/kvm199_img'/>
<target dev='hda' bus='virtio'/>
<driver cache='none' io='native' x-data-plane='on' /> <----------------- APPEND THESE SETTINGS
</disk> ^^^^^^^^^^^^^^^^^^^^^^^^^^^
- Start the VM using command line (to avoid config re-written):-
# virsh create /home/kvm/kvm199/kvm199.xml
(NOTE: These appended settings will need to be re-added every time VM is reboot using SolusVM. Use SolusVM --> VM --> Custom Config to make it permanent).
HOWTO: Remove dm-cache from existing LVM (NOTE: Must be done before removing the data LV! Otherwise will get kernel panic)
1. IMPORTANT: Make sure VM shutdown, and cached LVM unused / not mounted. (Otherwise can cause data corruption)
2. Remove cache SSD LVM, syncing any existing data in cache:-
# lvremove vg_server17/DataLVcache
3. Once data synced and cache removed, double check first to ensure no cache / meta still exist (only data LVs left)
# lvdisplay
4. Remove SSD drive partition from volume group
# vgreduce vg_server17 /dev/sdd1
5. Remove SSD drive PV
# pvremove /dev/sdd1
6. Now you should be back to normal without any SSD drives in operation:-
# lvdisplay
# vgdisplay
# pvdisplay
DONE.
NOTE: Tool to create and manage dm-cache (requires python >= 2.7):-
http://blog.oddbit.com/2014/08/16/lvcache-a-tool-for-managing-lv/
References:-
http://blog-vpodzime.rhcloud.com/?p=45
https://github.com/astersmith/esos/wiki/32_Advanced_Back_End_Storage_Setup
http://www.server-world.info/en/note?os=CentOS_6&p=lvm&f=2
http://www.ehowstuff.com/how-to-remove-volume-group-and-physical-volume-on-lvm/
http://man7.org/linux/man-pages/man7/lvmcache.7.html
