Simple RAID setup with mdadm

Today I decide to add a few disks to an existing server. The server itself is very simple, has a single disk and is running Zentyal 2.0 atop Ubuntu 10.04. I’ve been experimenting with Zentyal of late. It simplifies administration for my technician who are, let’s say, not comfortable with the command line. Zentyal like similar products such as SME server, make it easier for small businesses to administer their network. I’ll speak more about Zentyal later posts perhaps.

Since Zentyal allows me to quickly backup and/or restore various configurations I’m not too concerned about redundancy of the system. This is non-critical for a small business where the 45minutes to 1hour downtime to recover the system when the hard disk crashes is acceptable. We just want the data to be cozy on a RAID 5.

You are usually best off install your RAID during the installation since most linux distros have a RAID setup in the installer but in case you forgot or simply are adding new drives to an existing server like I am. This is how to do it.

Check the drives

Obviously the first step is to plug in your drives. Ideally 3 drives of equal size to avoid waste. but this is RAID so any 3 drive will do. Check that you drives are correctly identified by the system as follow.

sebby@Hades# fdisk -l

This should show something like this for every drive with partitions /dev/sda in my case.

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0000898e

Device Boot Start End Blocks Id System
/dev/sda1 * 1 852 6835200 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 852 60802 481549313 5 Extended
/dev/sda5 852 1603 6037504 82 Linux swap / Solaris
/dev/sda6 1603 60802 475510784 83 Linux

And like this for drive with no partitions, /dev/sdb, sdc and sdd in my case.


Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000b3c15

Device Boot Start End Blocks Id System

Partitioning

*
Now we need to partition those drives before we can use them in a RAID. If you have equal size drives it’s easiest to use all available space. If you don’t you only need to ensure all partitions are exactly the same size.

There many different ways to partition a hard drive. I you have a gnome desktop installed you could use gparted which has a descent interface. On the command list I use fdisk for more complex partitioning cfidsk may be easier.

Start be evoking fdisk followed by you device

sebby@Hades# fdisk /dev/sdb
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').

Command (m for help):

If you are unfamiliar with fdisk you can type “m” here to see available option, otherwise well start by doing what the WARNING says and change a few things. At the prompt type “c” then press “enter” followed by”u” and “enter”. Those will disable DOS-compatibility mode nad change the units to sector respectively.

Command (m for help): c
DOS Compatibility flag is not set

Command (m for help): u
Changing display/entry units to sectors

Now create a new patition by pressing “n” and “enter”. Select the primary partition (p and enter) and 1 as the partition number (1 and enter)

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1

Now we need to select the partition size. Again if you are using identical drives it’s best to leave the defaults and partition the whole drive as a single partition. If you using different size drive leave the default for first sector follow by the desire partition size in K, M or G.

First sector (2048-976773167, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-976773167, default 976773167):
Using default value 976773167

Finally type “w” and “enter” to write change to the drive.

Command (m for help): w

Once your done repeat the step with the other drives.

Creating the RAID

Now we need to create the actually RAID. This is with mdadm which you can install the usual way.

# apt-get install mdadm

Once mdadm is installed create you RAID

# mdadm -C /dev/md0-v -n=3 -l=5 /dev/sd[bcd]1

Depending on the drive size this may take a while. In the meantime let’s take a quick look at what weed did here.
-C (or –create) : Indicates we want to create a multi-disk device.
/dev/md0 : The RAID device to create
-v (or –verbose) : Let us know if anything interesting is going on.
-n=3 (or –raid-devices=3) : Number of devices to be included in the RAID
-l=5 (or –level=5) : The RAID level to be used
/dev/sd[bcd]1 : The devices to include. Could also be full device path /dev/sdb1 /dev/sdc1 /dev/sdd1

As I said this may take a while. If you want to check the current status of the build just have a look at /proc/mdstat

$ cat /proc/mdstat

Or watch it

$ watch cat /proc/mdstat

Use your RAID

Now that you are a valid RAID device you need to tell your system how to use it. First we’ll need to have a file system. So let’s format the RAID. You can format it using any file system you want. I’m going with ext4.

# mke2fs -t ext4 /dev/md0

You RAId is now up and ready for use. However just like any drive we need to mount it somewhere, ideally at boot. To do this we’ll add an entry to fstab and add information about our new RAID to /etc/mdadm/mdadm.conf so mdadm know how to assemble the RAID properly before the system can mount it.

Lets copy to output of the following command to /etc/mdadm/mdadm.conf

# echo mdadm --detail --scan >> /etc/mdadm/mdadm.conf

This will append something like this to /etc/mdadm/mdadm.conf and tell mdadm to look for the 3 RAID5 devices that belong to md0

ARRAY /dev/md0 level=raid5 num-devices=3 metadata=00.90 UID=ffe1a8e1:defa717b:c3ed9300:de7bc0ab

Last we modify /etc/fstab to mount /dev/md0 at boot. Simply add the following( or similar) to fstab

/dev/md0 /home/data auto defaults,usrquota,grpquota,acl 0 2