Setup ATA over Ethernet (AoE) on Debian 8 (Initiator and Target)

Setup ATA over Ethernet (AoE) on Debian 8 (Initiator and Target)

This tutorial shows you how to setup an AoE client (initiator) and server (target) on Debian 8 /Jessie). The term AoE stands for “ATA over Ethernet” which is a storage area network (SAN) protocol that allows AoE clients to use storage devices on the (remote) AoE server over a normal ethernet network. “Remote” in this case means “inside the same LAN” because AoE is not routable outside a LAN (this is a major difference compared to iSCSI). To the AoE client (initiator), the remote storage looks like a normal, locally-attached hard drive.eval(ez_write_tag([[728,90],’howtoforge_com-box-3′,’ezslot_6′,106,’0′,’0′]));


1 Preliminary Note

I’m using two Debian 8 servers here:

  • (Initiator): IP address
  • (Target): IP address


2 Load the AoE Kernel Module on both systems


As a first step, we have to ensure that the Kernel on our servers supports ATA Over Ethernet. Run the following command as root user.

grep ATA_OVER /boot/config-`uname -r`

This should display something like this:

[email protected]:/tmp# grep ATA_OVER /boot/config-`uname -r`
[email protected]:/tmp#

This means that AoE was built in as a kernel module. Now we will check if the module is loaded:

lsmod | grep aoe

If you get nothing back, this means it’s not loaded. In this case, we can load it as follows:

modprobe aoe

Let’s check again if the module is loaded:eval(ez_write_tag([[580,400],’howtoforge_com-medrectangle-4′,’ezslot_1′,108,’0′,’0′]));

lsmod | grep aoe
[email protected]:/tmp# lsmod | grep aoe
aoe 51917 0
[email protected]:/tmp#

To have the module loaded automatically when the system boots, we add the aoe module to /etc/modules:

nano /etc/modules
# /etc/modules: kernel modules to load at boot time.
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.
# Parameters can be specified after the module name.



3 Setting up the Target (server2)


First, we set up the AoE target (server2):

apt-get install vblade

We can use unused logical volumes, image files, hard drives (e.g. /dev/sdb), hard drive partitions (e.g. /dev/sdb1) or RAID devices (e.g. /dev/md0) for the storage. In this example, I will use an image file of 20GB that is stored in the folder /storage.

mkdir /storage
dd if=/dev/zero of=/storage/storage1.img bs=1024k count=20000

This creates the image file /storage/storage1.img with a size of 20GB.

If you want to use logical volume instead, then you can create one with a size of 20GB named storage1 in the volume group vg0 like this:

lvcreate -L20G -n storage1 vg0

Now we export our storage device as follows:

vbladed 0 1 eth0 /storage/storage1.img

The first number (0) is the shelf number (major), the second (1) the slot number (minor), change these numbers to your liking. Each AoE device is identified by a couple major/minor which must be unique (if you are exporting multiple devices), with major between 0-65535 and minor between 0-255. The eth0 part tells vbladed which ethernet device to use (if you ethernet device is eth1, then use eth1 – you can find out about your ethernet devices by running



To start the export automatically whenever you boot the target, open /etc/rc.local

nano /etc/rc.local

… and add the following line to it (before the exit 0 line):

/usr/sbin/vbladed 0 1 eth0 /storage/storage1.img

4 Setting up the AoE Client / Initiator (server1)


On server1, we install the initiator:

apt-get install aoetools

Now we check what AoE storage devices are available:


Don’t worry, the command will not show any output. The command:


should now show the storage devices:

[email protected]:/tmp# aoe-stat
e0.1 20.971GB eth0 1024 up
[email protected]:/tmp#

At this point we have a new block device available on the client box named /dev/etherd/e0.1. If we have a look at the /dev tree a new node appears:

ls -la /dev/etherd/
[email protected]:/tmp# ls -la /dev/etherd/
total 0
drwxr-xr-x 2 root root 160 Mar 22 08:46 .
drwxr-xr-x 19 root root 3160 Mar 22 08:34 ..
c-w--w---- 1 root disk 152, 3 Mar 22 08:34 discover
brw-rw---- 1 root disk 152, 0 Mar 22 08:46 e0.1
cr--r----- 1 root disk 152, 2 Mar 22 08:34 err
c-w--w---- 1 root disk 152, 6 Mar 22 08:34 flush
c-w--w---- 1 root disk 152, 4 Mar 22 08:34 interfaces
c-w--w---- 1 root disk 152, 5 Mar 22 08:34 revalidate
[email protected]:/tmp#

To use that /dev/etherd/e0.1 device, we must format it:

fdisk /dev/etherd/e0.1
[email protected]:/tmp# fdisk /dev/etherd/e0.1
Welcome to fdisk (util-linux 2.25.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x2922f0be.
Command (m for help): <-- n
Partition type
p primary (0 primary, 0 extended, 4 free)
e extended (container for logical partitions)
Select (default p): <-- p
Partition number (1-4, default 1): <-- 1
First sector (2048-40959999, default 2048): <-- ENTER
Last sector, +sectors or +size{K,M,G,T,P} (2048-40959999, default 40959999): <-- ENTER
Created a new partition 1 of type 'Linux' and of size 19.5 GiB.
Command (m for help): <-- w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

Now we create a filesystem on /dev/etherd/e0.1p1

mkfs.ext4 /dev/etherd/e0.1p1
[email protected]:/tmp# mkfs.ext4 /dev/etherd/e0.1p1
mke2fs 1.42.12 (29-Aug-2014)
Creating filesystem with 5119744 4k blocks and 1281120 inodes
Filesystem UUID: 2342cd83-bd45-4975-96c0-b0f366b73778
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

… and mount it for test purposes:

mount /dev/etherd/e0.1p1 /mnt

You should now see the new device in the outputs of…

[email protected]:/tmp# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=10240k,nr_inodes=125556,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,relatime,size=204220k,mode=755)
/dev/sda1 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=23,pgrp=1,timeout=300,minproto=5,maxproto=5,direct)
mqueue on /dev/mqueue type mqueue (rw,relatime)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
rpc_pipefs on /run/rpc_pipefs type rpc_pipefs (rw,relatime)
/dev/etherd/e0.1p1 on /mnt type ext4 (rw,relatime,data=ordered)
[email protected]:/tmp#

… and

df -h
[email protected]:/tmp# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 57G 1.1G 54G 2% /
udev 10M 0 10M 0% /dev
tmpfs 200M 4.6M 195M 3% /run
tmpfs 499M 0 499M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 499M 0 499M 0% /sys/fs/cgroup
/dev/etherd/e0.1p1 20G 44M 19G 1% /mnt
[email protected]:/tmp#

You can unmount it like this:

umount /mnt

To have the device mounted automatically at boot time, e.g. in the directory /storage, we create that directory…

mkdir /storage

… and add the following line to /etc/fstab:

nano /etc/fstab
/dev/etherd/e0.1p1       /storage        ext4    defaults,auto,_netdev 0 0

This alone isn’t enough to have the device mounted at boot time because the AoE stuff gets loaded after /etc/fstab is read. Therefore we open /etc/rc.local

nano /etc/rc.local

… and add the following lines to it (before the exit 0 line):

sleep 5
mount -a

For test purposes, you can now reboot the system:


After the reboot, the device should be mounted:

[email protected]:/tmp# mount
/dev/etherd/e0.1p1 on /storage type ext4 (rw,relatime,data=ordered)
df -h
[email protected]:/tmp# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 57G 1.1G 54G 2% /
udev 10M 0 10M 0% /dev
tmpfs 200M 4.6M 195M 3% /run
tmpfs 499M 0 499M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 499M 0 499M 0% /sys/fs/cgroup
/dev/etherd/e0.1p1 20G 44M 19G 1% /storage

Now we can do a test write on the mounted partition:

touch /storage/test.txt

The check with ls command if the file has been written:

ls -la /storage
[email protected]:/tmp# ls -la /storage
total 24
drwxr-xr-x 3 root root 4096 Mar 22 09:06 .
drwxr-xr-x 23 root root 4096 Mar 22 09:05 ..
drwx------ 2 root root 16384 Mar 22 09:00 lost+found
-rw-r--r-- 1 root root 0 Mar 22 09:06 test.txt
[email protected]:/tmp#

The test.txt file has been written sucessfully on the volume that we mounted from server2.

  • AoE Protocol Definition:
  • Debian:
About the Author

Leave a Reply