Add storage via LVM

This post is mostly notes just for myself. While I use FreeNAS locally to benefit from all its great functions such as ZFS and snapshots, I also perform a daily backup via rsync to an offsite host. That backup host was about to run out of space. Since it's my backup, it gets all the hand me downs so I barely invest in it.

Up until recently it was an AMD AM2 based system and is now AM3 based and will transition to a higher-end AM3+ when I replace another system with Ryzen. Its chokepoint is the network so I don't need anything super fast, it's just to address the fire and theft type scenarios at my main site. Just needs disk For storage I have a 64GB SSD for the OS and three spinner drives I reallocated from my main FreeNAS a while ago when it ran out of space.

These drives consist of a 4TB, 4TB, and 3TB smooshed together via LVM on an Ubuntu host. This gives me about 10TB of usable space. I just replaced two 2TB drives in the FreeNAS with larger drives so had them lying around ready to go. The case is a 3RU ATX chassis so I have plenty of space for platters.

Since it is commodity hardware-based (easy to fix!) there is no hot-swap so after cabling up the drive and starting it back up I started the add process in a few steps.

Add a Drive

Before beginning, I had to collect some items and let my heart skip a beat.

First was to locate where the drive is assigned to. Its on /dev/sdc.

 sudo fdisk -l  

Next was to get the LVM volume group name, which is vg0, the default.

 sudo vgdisplay  

Finally get the logical volume path, which is /dev/vg0/mystuff

 sudo lvdisplay  

So now I can convert it to a physical volume. I use the entire disk as I have no reason to partition them to use for other purposes.

 sudo pvcreate /dev/sdc  

Next, add the physical volume to the volume group

 sudo vgextend vg0 /dev/sdc  

Allocate the physical volume to the logical volume

 sudo lvextend -l +100%FREE /dev/vg0/mystuff  

Then finally grow the ext4 file system on the logical volume so I can use all this new space.

 sudo resize2fs /dev/vg0/mystuff  

All done, using df -h that mount went from 10TB total to 12TB.

Instead of using fdisk -l you can use cat /proc/partitions to locate the drive device. before or after you can use lsblk to get a pretty layout of your drives

 sda      8:0  0 2.7T 0 disk  
 └─vg0-stuff 253:0  0 11.8T 0 lvm /home/mystuff  
 sdb      8:16  0 59.6G 0 disk  
 ├─sdb1    8:17  0 51.7G 0 part /  
 └─sdb5    8:21  0  8G 0 part [SWAP]  
 sdc      8:32  0 1.8T 0 disk  
 └─vg0-stuff 253:0  0 11.8T 0 lvm /home/mystuff  
 sdd      8:48  0 3.7T 0 disk  
 └─vg0-stuff 253:0  0 11.8T 0 lvm /home/mystuff  
 sde      8:64  0 3.7T 0 disk  
 └─vg0-stuff 253:0  0 11.8T 0 lvm /home/mystuff  

Replace a drive

While I have space for more drives I will eventually run out so instead of getting a bigger case I can just replace drives with bigger ones. This process is easy as well.

You need to collect all the relevant information first such as the drive locations, physical, and logical volumes. Instead of lvextend you run these steps after running vgextend.

Move the data from the old drive to the new drive. This can be done live but you do have a performance hit to both drives during the process. This can take a considerable amount of time to complete so its advisable to do when the volume(s) are not too busy. Also consider power outage risk during this process. The larger the source drive the longer it will take.

 sudo pvmove /dev/sdc /dev/sde  

Finally, you remove the old drive from the volume.

 sudo vgreduce /dev/sdc  

Once done power down and physically remove. After this, you would run the resize2fs command above to grow the usable space.


-Kevin