NVMe over Ethernet Host and Target Drivers

NVMe over Ethernet (NVMeOE) is a deliberately simple NVMe‑over‑Fabrics transport that carries NVMe capsules directly over raw Ethernet. It skips IP, TCP and RDMA entirely, keeping only what’s needed to move data quickly and predictably when you control the link. With fewer layers on the path, it’s closer to “NVMe on the wire,” and that simplicity pays off in lower CPU overhead and easier debugging.

Compile

Current drivers are built over kernel version 6.16.11. Simply copy drivers and include into kernel source tree root directory and compile the kernel as usual.

Expose Block Devices as NVMe Drives from Target Server

1. Select block devices to expose

root@vmub2404-nvmeoe-target:~# lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
. . . . . .
. . . . . .
nvme0n1 259:0    0    20G  0 disk
nvme0n2 259:1    0    30G  0 disk

Two block devices nvme0n1 and nvme0n2 will be assigned.

2. Select Ethernet interface as destination target

root@vmub2404-nvmeoe-target:~# ip addr
. . . . . .
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:20:ce:ef brd ff:ff:ff:ff:ff:ff

Interface ens37 and its MAC address 00:0c:29:20:ce:ef will be used.

3. Run script to expose block devices as NVMe drives through NVMeOE target transport

./nvmet-eth-setup.sh nqn.test.eth ens37 /dev/nvme0n1 /dev/nvme0n2

• NQN: nqn.test.eth
• IFACE: ens37
• Block devices: /dev/nvme0n1 and /dev/nvme0n2

NVMeOE Host Connect Target Server to Access NVMe Drives

1. Select local Ethernet interface as source port

root@vmub2404-nvmeoe-host:~# ip addr
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:83:cf:01 brd ff:ff:ff:ff:ff:ff

Interface ens37 is selected as source port. Note: it has different MAC address than the destination port on target server.

2. Run script to connect NVMeOE target

./nvme-eth-connect.sh ens37 00:0c:29:20:ce:ef nqn.test.eth

• Local IFACE: ens37
• Target MAC: 00:0c:29:20:ce:ef
• Target NQN: nqn.test.eth

3. Verify devices are discovered

root@vmub2404-nvmeoe-host:~# nvme list
Node          Generic      SN                   Model   Namespace  Usage                    Format           FW Rev
------------- ------------ -------------------- ------- ---------- ------------------------ ---------------- --------
/dev/nvme1n1  /dev/ng1n1   c8b61f5dc1b8c82744ce Linux   0x2        0.00   B /  32.21  GB    512   B +  0 B   6.16.11
/dev/nvme1n2  /dev/ng1n2   c8b61f5dc1b8c82744ce Linux   0x1        0.00   B /  21.47  GB    512   B +  0 B   6.16.11