Linux storage management

Storage management

Explore and identify block devices

Check block device "blkid", shows mount point, uuid, type, label, partition id.

IO information "blkid -i /dev/sdc1".

Show all devices"lsblk".

Create and modify partitions

Tools for working with partitions:

  • parted
  • fdisk
  • gparted
  • cfdisk

Create modify and mount filesystem

Filesystems:

  • They keep track of data on the disk
  • Linux support many natively, and support for other can be added
  • Most common are ext2(standard), ext3(journaling support), ext4(large files)
  • Other xfs (developed bySun), btrfs(b-tree file system), FAT32 (compatible with mac and window)

Create a filesystem "mkfs.ext2 /dev/sdc1".

Mounting file system "mkdir /mnt/storage" && "mount /dev/sdc1 /mnt/storage".

Reviewing and verifying "tune2fs -l /dev/sdc1".

Unmounting "umount /mnt/storage".

Create and mount encrypted partition

Encrypting a partition

  • Encrypt a partition to prevent casual access
  • dm-crypt
  • LUKS (linux unified key setup)
  • Available in the package cryptsetup

Encrypting partition "cryptsetup luksFormat /dev/sdc2".

Decrypt a partition "cryptsetup open /dev/sdc2 secret".

We need to set filesystem on this stage.

Close the device "cryptsetup close secret".

Information "cryptsetup luksDump /dev/sdc2".

Configure disk mounting

Information about mounting filesystems "/etc/fstab".

Adding filesystem to mount possible to "/etc/fstab" by providing UUID, path, type.

Mount all files systems in fstabfile "mount -a".

Possible to check results using "df -h".

Mount volumes on demand

Installing auto fs "sudo apt install autofs".

Checking status "sudo systemctl status autofs".

File for controlling mounts in autofs "/etc/auto.master".

Reconfigure swap space

Swap space:

  • when the system runs low on RAM, it can move or swap pages of memory to disk
  • The swap space can be a partition or a file
  • Allocate 2x your RAM f you have less than 2GB of RAM
  • Allocate at least 4GB if you have more than 2GB of RAM
  • Should determine what your optimal amount is
  • Know a system can have more than one available swap location
  • Keep an eye on usage with top, free, and /proc/swaps
  • Tools - swapoff, mkswap and swapon

Checking swap "cat /proc/swaps".

Before modification, we need to disable swap e.g. "swapoff /dev/sda5".

Creating new swap "dd if=/dev/zero of=/var/swapfile bs=1G count=8".

Giving the swap file correct permissions "chmod 600 /var/swapfile".

Make file as a swap file "mkswap /var/swapfile".

Enabling swap "swapon /var/swapfile".

Examine all swaps "cat /proc/swaps".

Add changes to "fstab" file to make those changes permanent.

Create redundant storage with RAID

RAID:

  • Redundant array of independent disks
  • Configure storage for reliability or speed

RAID levels:

  • RAID 0 - one volume spread or striped across many disks
  • RAID 1 - one volume mirrored on two or more disks
  • RAID 2,3,4 - various arrangement
  • RAID 5 - one volume with distributed partly on 3+ disks
  • RAID 6 - One volume with distributed partly on 4+ disks

RAID consideration:

  • Disks pr partitions in RAID 1, 5, 6 need to be the same size
  • RAID 5 space: individual disk size * (number of disks - 1)
  • RAID 6 space: individual disk size * (number of disks - 2)
  • When a disk fails, the RAID array is degraded
  • RAID works at the block level, not the file level
  • managed with mdadm
  • MD stands for multiple device (/dev/md0, /dev/md1)
  • Resync is the process that checks integrity
  • Resync can take a long time

Creating RAID:

  • Creating partitions "fdisk /dev/sdb", chose size in GB in the prompt
  • First we need to install it "apt install mdadm"
  • Create a RAID "mdadm --create --level=1 --raid-devices=2 /dev/md0 /dev/sdb1 /dev/sdb2"
  • Checking status "mdadm --detail /dev/md0"
  • Adding filesystem "mkfs.ext4 /dev/md0"
  • Adding empty folder for mounting "mkdir /mnt/myraid"
  • Mounting "mount /dev/md0 /mnt/myraid"
  • Failing a disk in RAID "mdadm --fail /dev/md0 /dev/sdb1"
  • Removing a disk from RAID "mdadm --remove /dev/md0 /dev/sdb1"
  • Adding new disk to RAID "mdadm --add /dev/md0 /dev/sdb3"
  • Stopping RAID, first we need to unmount filesystem "unmount /mnt/myraid" and then stop the RAID "mdadm --stop /dev/md0"
  • Remove the RAID "mdadm --remove /dev/md0"
  • Restore to unused partitions "mdadm --zero-superblock /dev/sdb1 /dev/sdb2 /dev/sdb3"

Unterstanding LVM

Logical volume management (LVM)

  • Builds on traditional partitions
  • uses physical volumes (PVs), volume groups (VGs), and logical volumes (LVs) to configure storage
  • Allows flexibility and adds some features
  • Installing "apt install lvm2"

Create a physical volume, volume group, and logical volume

Creating a physical volume:

  • First we need to create a partition with "fdisk/dev/sdb"
  • create a physical volume "pvcreate /dev/sdb1"
  • checking physical volumes "pvdisplay"

Creating a volume group:

  • creating a volume group out of existing physical volumes"vgcreate my_group /dev/sdb1(path to a physical volume)" possible to add more than one
  • find more details about volume group "vgdisplay" or "vgs"

Creating logical volume:

  • options for "lvcreate" -L 100G(size), -I 100%VG(use 100% of space) -I 100%FREE (use 100% of free space) -l n (use n extents)
  • get infomration about logic volumes "lvdisplay" or "lvs"

Extending a volume group and logical volumes

Extending a volume group and logical volume:

  • Checking volume group "vgdisplay"
  • Checking physical volume "pvdisplay"
  • Creating a partition "fdisk /dev/sdc"
  • Creating a physical volume "pwcreate /dev/sdc1"
  • Adding to a volume group "vgextend name_of_group /dev/sdc1"
  • Extending logical volume "lvextend -l+29615 /dev/mygroup/mydata"
  • Checking logical volume "lvdisplay"
  • Updating file system "resize2fs /dev/mygroup/mydata"

Migrating and replacing physical volume

  • Add a new PV
  • Add a new PV to a VG
  • Move extends off of an old PV "pvmove /dev/sdb1 /dev/sdd1" or "pvmove /dev/sdb1"
  • Remove an old PV from a VG "vgreduce mygroup /dev/sdb1"\

Access control lists

  • Specify access to a file for users or groups, outside of regular permissions
  • Grant access to a user who isn’t in a group that has access already

Checking permissions of a file "getacl filename".

Setting access control level for a file "setfacl -m u:usernmae:rw filename".

Removing access control level "setfacl -x user filename".

Exploring disk quotas

  • Install the "quota" package
  • Mount the filesystem with quota support
  • Create quota table on the filesystem
  • Specify quotas

Create a quota file "quotacheck -c /mnt/path".

Add quota "edquota username".

Checking if quota is turned on "quotaon -pa".

Enabling quota "quotaon /mnt/path"