to see what Linux has to offer. While in comparison to Solaris I think
the Linux side could still mature a little, it still has potential and
a functional offering. The following is part 1 of my notes detailing
the configuration of a test Linux iSCSI target node (server side).
Configuration of an iSCSI initiator node (client side) will be detailed
in part 2. Our host details for the target notes are:
HOSTS: cobblepot (target node), humboldt (initiator node) PROMPT: host [0] OS: CentOS 5.6 Linux (RedHat Clone) IP ADDRs: cobblepot (10.0.136.17), humboldt (10.0.136.122)To start things off, Linux doesn't include a standard package for
providing iSCSI targets. There are, however, several open source
offerings available. For my usage, I've chosen iSCSI Enterprise Target
(IET) available via sourceforge (http://iscsitarget.sourceforge.net/).
The latest version at this time is iscsitarget-1.4.20.2.tar.gz, dated
2010-07-15. After downloading it, we need to build it:
cobblepot [0] pwd /opt cobblepot [0] /bin/ls iscsitarget-1.4.20.2.tar.gz cobblepot [0] /bin/gunzip -c iscsitarget-1.4.20.2.tar.gz | /bin/tar xf - cobblepot [0] cd iscsitarget-1.4.20.2 cobblepot [0] /usr/bin/make Applying Patch compat-2.6.32.patch patching file kernel/conn.c Applying Patch compat-2.6.31.patch patching file kernel/file-io.c Applying Patch compat-2.6.30.patch patching file kernel/block-io.c Hunk #1 succeeded at 280 with fuzz 2. Applying Patch compat-2.6.29.patch patching file kernel/config.c Applying Patch compat-2.6.28.patch patching file kernel/conn.c Applying Patch compat-2.6.25-2.6.27.patch patching file kernel/block-io.c <snip...> CC [M] /opt/iscsitarget-1.4.20.2/kernel/ua.o CC [M] /opt/iscsitarget-1.4.20.2/kernel/seq_list.o LD [M] /opt/iscsitarget-1.4.20.2/kernel/iscsi_trgt.o Building modules, stage 2. MODPOST CC /opt/iscsitarget-1.4.20.2/kernel/iscsi_trgt.mod.o LD [M] /opt/iscsitarget-1.4.20.2/kernel/iscsi_trgt.ko make[1]: Leaving directory `/usr/src/kernels/2.6.18-238.el5-x86_64' cobblepot [0] /usr/bin/make install `usr/ietd' -> `/usr/sbin/ietd' `usr/ietadm' -> `/usr/sbin/ietadm' `etc/initd/initd.redhat' -> `/etc/rc.d/init.d/iscsi-target' install: creating directory `/etc/iet' `etc/ietd.conf' -> `/etc/iet/ietd.conf' `etc/initiators.allow' -> `/etc/iet/initiators.allow' `etc/targets.allow' -> `/etc/iet/targets.allow' `doc/manpages/ietadm.8' -> `/usr/share/man/man8/ietadm.8' `doc/manpages/ietd.8' -> `/usr/share/man/man8/ietd.8' `doc/manpages/ietd.conf.5' -> `/usr/share/man/man5/ietd.conf.5' install: creating directory `/usr/share/doc/iscsitarget' `ChangeLog' -> `/usr/share/doc/iscsitarget/ChangeLog' `COPYING' -> `/usr/share/doc/iscsitarget/COPYING' `RELEASE_NOTES' -> `/usr/share/doc/iscsitarget/RELEASE_NOTES' `README' -> `/usr/share/doc/iscsitarget/README' `README.vmware' -> `/usr/share/doc/iscsitarget/README.vmware' `README.initiators' -> `/usr/share/doc/iscsitarget/README.initiators' install: creating directory `/lib/modules/2.6.18-238.el5/extra/iscsi' `kernel/iscsi_trgt.ko' -> `/lib/modules/2.6.18-238.el5/extra/iscsi/iscsi_trgt.ko' Running depmodWith the build out of the way, we need to load kernel module "iscsi_trgt".
Part of the install above included an init script (iscsi-target) placed
to /etc/init.d so we also need to enable it for runlevel 3:
cobblepot [0] /sbin/lsmod | /bin/grep iscsi_trgt cobblepot [1] /sbin/modprobe iscsi_trgt cobblepot [0] /sbin/lsmod | /bin/grep iscsi_trgt iscsi_trgt 119168 0 cobblepot [0] /sbin/chkconfig --add iscsi-target cobblepot [0] /sbin/chkconfig --level 3 iscsi-target on cobblepot [0] /sbin/chkconfig --list | /bin/grep iscsi-target iscsi-target 0:off 1:off 2:off 3:on 4:off 5:off 6:off cobblepot [0] /bin/ls /etc/init.d/iscsi-target /etc/rc3.d/*iscsi-target /etc/init.d/iscsi-target /etc/rc3.d/S39iscsi-target cobblepot [0] /bin/cat /proc/net/iet/session /proc/net/iet/volumeOf note, I have not yet started "iscsi-target" (/usr/sbin/ietd) because
we still need to configure it. Also, the files listed above under
/proc don't contain anything now, but will contain configured targets
and LUNs, respectively, once configured. Below, I have set up disk
"sdb" with three partitions. Partitions 1 and 2 are raw disk space (no
filesystem (FS)) and will be used as the backing stores for two LUNs.
Partition 3 has an FS, is mounted to /storage, and will be used for file
based backing stores:
cobblepot [0] /sbin/fdisk -l /dev/sdb
Disk /dev/sdb: 536 MB, 536870912 bytes
64 heads, 32 sectors/track, 512 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 96 98288 83 Linux
/dev/sdb2 97 192 98304 83 Linux
/dev/sdb3 193 512 327680 83 Linux
cobblepot [0] /bin/mkdir /storage ; /bin/mount /dev/sdb3 /storage
cobblepot [0] /bin/df -h /storage
Filesystem Size Used Avail Use% Mounted on
/dev/sdb3 310M 11M 284M 4% /storage
cobblepot [0] /bin/mkdir -p /storage/targets/humboldt0
cobblepot [0] /bin/mkdir -p /storage/targets/humboldt1
cobblepot [0] /bin/dd if=/dev/zero of=/storage/targets/humboldt0/lun.2 bs=1k count=100k
102400+0 records in
102400+0 records out
104857600 bytes (105 MB) copied, 0.380762 seconds, 275 MB/s
cobblepot [0] /bin/dd if=/dev/zero of=/storage/targets/humboldt1/lun.0 bs=1k count=100k
102400+0 records in
102400+0 records out
104857600 bytes (105 MB) copied, 0.552485 seconds, 190 MB/s
cobblepot [0] /usr/bin/find /storage/targets -type f -exec /usr/bin/du -sh {} \;
101M /storage/targets/humboldt1/lun.0
101M /storage/targets/humboldt0/lun.2
The two files created above, "humboldt0/lun.2" and "humboldt1/lun.0" areour file based backing stores for another two LUNs. With our backing
stores created, we can begin configuring our targets. Below are the
contents of the 'ietd' configuration file that I've set up. The first
"Target" section sets up a new target, assigns a local alias name, and
configures three LUNs. Two of these LUNs are our raw partitions, the
third is the path to one of our file based backing stores. The second
"Target" section creates another target and is otherwise similar to the
first section:
cobblepot [0] /bin/vi /etc/iet/ietd.conf cobblepot [0] /bin/cat /etc/iet/ietd.conf Target iqn.2011-06.com.example.cobblepot:storage-net.humboldt0 Alias humboldt0 Lun 0 Path=/dev/sdb1,Type=fileio,ScsiId=1308034281,ScsiSN=1308034287 Lun 1 Path=/dev/sdb2,Type=fileio,ScsiId=1308034812,ScsiSN=1308034818 Lun 2 Path=/storage/targets/humboldt0/lun.2,Type=fileio,ScsiId=1308034815,ScsiSN=1308034821 MaxConnections 1 MaxSessions 0 NOPInterval 1 NOPTimeout 5 DataPDUInOrder Yes DataSequenceInOrder Yes Target iqn.2011-06.com.example.cobblepot:storage-net.humboldt1 Alias humboldt1 Lun 0 Path=/storage/targets/humboldt1/lun.0,Type=fileio,ScsiId=1308035341,ScsiSN=1308035347 MaxConnections 1 MaxSessions 0 NOPInterval 1 NOPTimeout 5 DataPDUInOrder Yes DataSequenceInOrder YesNotes on the above, the entry following "Target" breaks down to:
iqn.YYYY-MM.TLD.DOMAIN.HOST:UNIQUE_IDENTIFIERLUN entries are:
Lun LUN_NUMBER Path=/PATH/TO/BACKING/STORE,Type=[fileio|blockio],ScsiId=UNIQUE_ID,ScsiSN=UNIQUE_IDPlease see manpage ietd.conf(5) for further details. Moving on,
we can set up restrictions to our targets. In "targets.allow" below,
I've allowed all targets to be discoverable from network 10.0.136.0/24.
In "initiators.allow", I've associated our targets from earlier with
only host 10.0.136.122 (humboldt, initiator node):
cobblepot [0] /bin/vi /etc/iet/targets.allow cobblepot [0] /bin/grep -v ^# /etc/iet/targets.allow ALL 10.0.136.0/24 cobblepot [0] /bin/vi /etc/iet/initiators.allow cobblepot [0] /bin/grep -v ^# /etc/iet/initiators.allow iqn.2011-06.com.example.cobblepot:storage-net.humboldt0 10.0.136.122 iqn.2011-06.com.example.cobblepot:storage-net.humboldt1 10.0.136.122(Multiple hosts for a particular target in "initiators.allow" are
specified as a comma separated list after the target.) With all
configuration complete, we can run the "iscsi-target" init script.
After doing so, we now have relevant details populated under
/proc/net/iet in "session" and "volume":
cobblepot [0] /sbin/service iscsi-target start Starting iSCSI Target: [ OK ] cobblepot [0] /bin/ps -ef | /bin/grep ietd root 6187 1 0 03:24 ? 00:00:00 /usr/sbin/ietd cobblepot [0] /bin/netstat -natp | /bin/grep 3260 tcp 0 0 0.0.0.0:3260 0.0.0.0:* LISTEN 6391/ietd cobblepot [0] /bin/cat /proc/net/iet/session tid:2 name:iqn.2011-06.com.example.cobblepot:storage-net.humboldt1 tid:1 name:iqn.2011-06.com.example.cobblepot:storage-net.humboldt0 cobblepot [0] /bin/cat /proc/net/iet/volume tid:2 name:iqn.2011-06.com.example.cobblepot:storage-net.humboldt1 lun:0 state:0 iotype:fileio iomode:wt blocks:204800 blocksize:512 path:/storage/targets/humboldt1/lun.0 tid:1 name:iqn.2011-06.com.example.cobblepot:storage-net.humboldt0 lun:0 state:0 iotype:fileio iomode:wt blocks:196576 blocksize:512 path:/dev/sdb1 lun:1 state:0 iotype:fileio iomode:wt blocks:196608 blocksize:512 path:/dev/sdb2 lun:2 state:0 iotype:fileio iomode:wt blocks:204800 blocksize:512 path:/storage/targets/humboldt0/lun.2We can also display configured target parameters of our targets via 'ietadm':
cobblepot [0] /usr/sbin/ietadm --op show --tid=1 Wthreads=8 Type=0 QueuedCommands=32 NOPInterval=1 NOPTimeout=5 cobblepot [0] /usr/sbin/ietadm --op show --tid=2 Wthreads=8 Type=0 QueuedCommands=32 NOPInterval=1 NOPTimeout=5The targets identified by "--tid" are those referenced in both the
"session" and "volume" files above. Below are the configured session
parameters relevant to target "2":
cobblepot [0] /usr/sbin/ietadm --op show --tid=2 --sid 0 InitialR2T=Yes ImmediateData=Yes MaxConnections=1 MaxRecvDataSegmentLength=8192 MaxXmitDataSegmentLength=8192 MaxBurstLength=262144 FirstBurstLength=65536 DefaultTime2Wait=2 DefaultTime2Retain=20 MaxOutstandingR2T=1 DataPDUInOrder=Yes DataSequenceInOrder=Yes ErrorRecoveryLevel=0 HeaderDigest=None DataDigest=None OFMarker=No IFMarker=No OFMarkInt=Reject IFMarkInt=RejectAt this point, our target node "cobblepot" has been configured for
client access from initiator node "humboldt". The only thing we need
to do now is configure the initiator node, which will be part 2.
see also:
ietd.conf(5)
Part 2: Configuring iSCSI Initiators in Linux
iSCSI Enterprise Target homepage
Configuring iSCSI Targets in Solaris
