(the server side). Here, in part 2, are my notes on how to set up the
initiators in Linux (the client side). Our host details for this are
the same as part 1:
HOSTS: humboldt (initiator node), cobblepot (target node) PROMPT: host [0] OS: CentOS 5.6 Linux (RedHat Clone) IP ADDRs: humboldt (10.0.136.122), cobblepot (10.0.136.17)Before proceeding, you will need to have the "iscsi-initiator-utils"
package installed and 'iscsi' and 'iscsid' enabled:
humboldt [0] /bin/rpm -q iscsi-initiator-utils iscsi-initiator-utils-6.2.0.872-6.el5 humboldt [0] /sbin/chkconfig --list | /bin/grep iscsi iscsi 0:off 1:off 2:off 3:on 4:on 5:on 6:off iscsid 0:off 1:off 2:off 3:on 4:on 5:on 6:offIf 'iscsi' and 'iscsid' are not enabled, then enable them with 'chkconfig':
humboldt [0] /sbin/chkconfig --level 3 iscsi on humboldt [0] /sbin/chkconfig --level 3 iscsid on humboldt [0] /sbin/service iscsid status iscsid (pid 1312) is running...The 'iscsi' service isn't checked above because it isn't really a service
but instead logs into known iSCSI targets when started. We can, and
will, do this manually later. If 'iscsid' isn't running, start it via
the 'service' command:
humboldt [0] /sbin/service iscsid start Starting iSCSI daemon: [ OK ]Alright, by default, there should be an "initiatorname.iscsi" file
which contains the initiator node name (INN) for our host as seen below.
Since this value must be unique, if you need to generate a new unique INN,
or this file doesn't exist, 'iscsi-iname' can be used. This command
will output to STDOUT so if you need a new INN, you'll need to update
"initiatorname.iscsi" with the output returned:
humboldt [0] /bin/cat /etc/iscsi/initiatorname.iscsi InitiatorName=iqn.1994-05.com.redhat:1d94b02baba8 humboldt [0] /sbin/iscsi-iname iqn.1994-05.com.redhat:e9281ae17241(Note, I did not update the INN on humboldt, as can be seen later.) With
that out of the way and 'iscsid' running, we will use 'iscsiadm' to
add the target node (cobblepot, 10.0.136.17) as a valid target host and
discover what targets it has available to us. The first command below
discovers the targets on cobblepot via method "sendTargets" (-t st) and
updates the local target database (discoverydb). The second 'iscsiadm'
command lists known target hosts and the third lists the method that
targets have been discovered by:
humboldt [0] /sbin/iscsiadm -m discoverydb -t st -p 10.0.136.17 -D 10.0.136.17:3260,1 iqn.2011-06.com.example.cobblepot:storage-net.humboldt1 10.0.136.17:3260,1 iqn.2011-06.com.example.cobblepot:storage-net.humboldt0 humboldt [0] /sbin/iscsiadm -m discovery -P 0 10.0.136.17:3260 via sendtargets humboldt [0] /sbin/iscsiadm -m discovery -P 1 SENDTARGETS: DiscoveryAddress: 10.0.136.17,3260 Target: iqn.2011-06.com.example.cobblepot:storage-net.humboldt0 Portal: 10.0.136.17:3260,1 Iface Name: default Target: iqn.2011-06.com.example.cobblepot:storage-net.humboldt1 Portal: 10.0.136.17:3260,1 Iface Name: default iSNS: No targets found. STATIC: No targets found. FIRMWARE: No targets found. humboldt [0] /sbin/iscsiadm -m discoverydb -t st -p 10.0.136.17 -o show # BEGIN RECORD 2.0-872 discovery.startup = manual discovery.type = sendtargets discovery.sendtargets.address = 10.0.136.17 discovery.sendtargets.port = 3260 discovery.sendtargets.auth.authmethod = None discovery.sendtargets.auth.username = <empty> discovery.sendtargets.auth.password = <empty> discovery.sendtargets.auth.username_in = <empty> discovery.sendtargets.auth.password_in = <empty> discovery.sendtargets.timeo.login_timeout = 15 discovery.sendtargets.use_discoveryd = No discovery.sendtargets.discoveryd_poll_inval = 30 discovery.sendtargets.reopen_max = 5 discovery.sendtargets.timeo.auth_timeout = 45 discovery.sendtargets.timeo.active_timeout = 30 discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768 # END RECORDThe final 'iscsiadm' command above lists configuration details governing
communication with the specified target host. With the available targets
discovered, we need to log in to them. We could run the 'iscsi' init
script but it's useful to know how to do this manually. Below we log
into both targets, first only to target "humboldt0", then to any targets
available:
humboldt [0] /sbin/iscsiadm -m node -T iqn.2011-06.com.example.cobblepot:storage-net.humboldt0 -p 10.0.136.17 -l Logging in to [iface: default, target: iqn.2011-06.com.example.cobblepot:storage-net.humboldt0, portal: 10.0.136.17,3260] Login to [iface: default, target: iqn.2011-06.com.example.cobblepot:storage-net.humboldt0, portal: 10.0.136.17,3260] successful. humboldt [0] /sbin/iscsiadm -m node -l Logging in to [iface: default, target: iqn.2011-06.com.example.cobblepot:storage-net.humboldt0, portal: 10.0.136.17,3260] Logging in to [iface: default, target: iqn.2011-06.com.example.cobblepot:storage-net.humboldt1, portal: 10.0.136.17,3260] iscsiadm: Could not login to [iface: default, target: iqn.2011-06.com.example.cobblepot:storage-net.humboldt0, portal: 10.0.136.17,3260]. iscsiadm: initiator reported error (15 - already exists) Login to [iface: default, target: iqn.2011-06.com.example.cobblepot:storage-net.humboldt1, portal: 10.0.136.17,3260] successful.The messages above tell us that we are already logged into target
"humboldt0" via error messages (these are fine) and that we are now
also logged into "humboldt1". Using 'iscsiadm' below, the first command
presents us with simple information detailing the target instances and
target names. In the second 'iscsiadm' command, we get back more useful
information including the SCSI target, channel, ID, and LUN information,
as well as the local disk attachments:
humboldt [0] /sbin/iscsiadm -m session tcp: [1] 10.0.136.17:3260,1 iqn.2011-06.com.example.cobblepot:storage-net.humboldt0 tcp: [2] 10.0.136.17:3260,1 iqn.2011-06.com.example.cobblepot:storage-net.humboldt1 humboldt [0] /sbin/iscsiadm -m session -P 3 iSCSI Transport Class version 2.0-871 version 2.0-872 Target: iqn.2011-06.com.example.cobblepot:storage-net.humboldt0 Current Portal: 10.0.136.17:3260,1 Persistent Portal: 10.0.136.17:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:1d94b02baba8 Iface IPaddress: 10.0.136.122 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 1 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ************************ Negotiated iSCSI params: ************************ HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 262144 MaxXmitDataSegmentLength: 8192 FirstBurstLength: 65536 MaxBurstLength: 262144 ImmediateData: Yes InitialR2T: Yes MaxOutstandingR2T: 1 ************************ Attached SCSI devices: ************************ Host Number: 1 State: running scsi1 Channel 00 Id 0 Lun: 0 Attached scsi disk sdb State: running scsi1 Channel 00 Id 0 Lun: 1 Attached scsi disk sdd State: running scsi1 Channel 00 Id 0 Lun: 2 Attached scsi disk sde State: running Target: iqn.2011-06.com.example.cobblepot:storage-net.humboldt1 Current Portal: 10.0.136.17:3260,1 Persistent Portal: 10.0.136.17:3260,1 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:1d94b02baba8 Iface IPaddress: 10.0.136.122 Iface HWaddress: <empty> Iface Netdev: <empty> SID: 2 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ************************ Negotiated iSCSI params: ************************ HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 262144 MaxXmitDataSegmentLength: 8192 FirstBurstLength: 65536 MaxBurstLength: 262144 ImmediateData: Yes InitialR2T: Yes MaxOutstandingR2T: 1 ************************ Attached SCSI devices: ************************ Host Number: 2 State: running scsi2 Channel 00 Id 0 Lun: 0 Attached scsi disk sdc State: runningWe can see the same LUN / local disk correlation reviewing
"/dev/disk/by-path". Reviewing "/dev/disk/by-id" provides us with the
unique SCSI ID for each related disk device. Finally, the relationship
between local disk device and SCSI target, channel, ID, and LUN can be
seen in "/proc/scsi/scsi":
humboldt [0] /bin/ls -l /dev/disk/by-path | /bin/awk '/iqn/ {print $9, $11}'
ip-10.0.136.17:3260-iscsi-iqn.2011-06.com.example.cobblepot:storage-net.humboldt0-lun-0 ../../sdb
ip-10.0.136.17:3260-iscsi-iqn.2011-06.com.example.cobblepot:storage-net.humboldt0-lun-1 ../../sdd
ip-10.0.136.17:3260-iscsi-iqn.2011-06.com.example.cobblepot:storage-net.humboldt0-lun-2 ../../sde
ip-10.0.136.17:3260-iscsi-iqn.2011-06.com.example.cobblepot:storage-net.humboldt1-lun-0 ../../sdc
humboldt [0] /bin/ls -l /dev/disk/by-id/ | /bin/awk '!/(tot|CD-R|SATA)/ {print $9, $11}'
scsi-1494554000000000031333038303334323831000000000000 ../../sdb
scsi-1494554000000000031333038303334383132000000000000 ../../sdd
scsi-1494554000000000031333038303334383135000000000000 ../../sde
scsi-1494554000000000031333038303335333431000000000000 ../../sdc
humboldt [0] /bin/cat /proc/scsi/scsi
Attached devices:
<snip...>
Host: scsi1 Channel: 00 Id: 00 Lun: 00
Vendor: IET Model: VIRTUAL-DISK Rev: 0
Type: Direct-Access ANSI SCSI revision: 04
Host: scsi1 Channel: 00 Id: 00 Lun: 01
Vendor: IET Model: VIRTUAL-DISK Rev: 0
Type: Direct-Access ANSI SCSI revision: 04
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: IET Model: VIRTUAL-DISK Rev: 0
Type: Direct-Access ANSI SCSI revision: 04
Host: scsi1 Channel: 00 Id: 00 Lun: 02
Vendor: IET Model: VIRTUAL-DISK Rev: 0
Type: Direct-Access ANSI SCSI revision: 04
Great, lots of info, let's do something now. Running 'fdisk' providesus with our disks showing that "sdb - sde" are not yet configured:
humboldt [0] /sbin/fdisk -l | /bin/grep ^Disk Disk /dev/sdb doesn't contain a valid partition table Disk /dev/sdc doesn't contain a valid partition table Disk /dev/sdd doesn't contain a valid partition table Disk /dev/sde doesn't contain a valid partition table Disk /dev/sda: 8589 MB, 8589934592 bytes Disk /dev/sdb: 100 MB, 100646912 bytes Disk /dev/sdc: 104 MB, 104857600 bytes Disk /dev/sdd: 100 MB, 100663296 bytes Disk /dev/sde: 104 MB, 104857600 bytesWe'll use "sde", which is cobblepot:/storage/targets/humboldt0/lun.2,
for a new volume. Below, I've created a new partition at "sde1",
written the partition table, and created a new ext3 filesystem (FS) on it:
humboldt [0] /sbin/fdisk /dev/sde Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): p Disk /dev/sde: 104 MB, 104857600 bytes 4 heads, 50 sectors/track, 1024 cylinders Units = cylinders of 200 * 512 = 102400 bytes Device Boot Start End Blocks Id System Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-1024, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-1024, default 1024): Using default value 1024 Command (m for help): p Disk /dev/sde: 104 MB, 104857600 bytes 4 heads, 50 sectors/track, 1024 cylinders Units = cylinders of 200 * 512 = 102400 bytes Device Boot Start End Blocks Id System /dev/sde1 1 1024 102375 83 Linux Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. humboldt [0] /sbin/fdisk -l /dev/sde Disk /dev/sde: 104 MB, 104857600 bytes 4 heads, 50 sectors/track, 1024 cylinders Units = cylinders of 200 * 512 = 102400 bytes Device Boot Start End Blocks Id System /dev/sde1 1 1024 102375 83 Linux humboldt [0] /sbin/mkfs -V -t ext3 /dev/sde1 mkfs (util-linux 2.13-pre7) mkfs.ext3 /dev/sde1 mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=1024 (log=0) Fragment size=1024 (log=0) 25688 inodes, 102372 blocks 5118 blocks (5.00%) reserved for the super user First data block=1 Maximum filesystem blocks=67371008 13 block groups 8192 blocks per group, 8192 fragments per group 1976 inodes per group Superblock backups stored on blocks: 8193, 24577, 40961, 57345, 73729 Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 33 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override.Since iSCSI targets and LUNs may not be presented in the same order
the next time we initiate them, our local device naming may change.
To mitigate this, I've set a label of "cpot_t0l2s1" to "sde1" via
'tune2fs' followed by mounting the new volume:
humboldt [0] /sbin/tune2fs -L cpot_t0l2s1 /dev/sde1 tune2fs 1.39 (29-May-2006) you have mail in /var/spool/mail/root humboldt [0] /sbin/tune2fs -l /dev/sde1 | /bin/egrep 'name|UUID' Filesystem volume name: cpot_t0l2s1 Filesystem UUID: bf1528d6-3b57-4688-b19c-283a5bbe3f78 humboldt [0] /bin/mount -L cpot_t0l2s1 /a humboldt [0] /bin/df -h /a Filesystem Size Used Avail Use% Mounted on /dev/sde1 97M 5.6M 87M 7% /a humboldt [0] /bin/vi /etc/fstab humboldt [0] /bin/grep cpot /etc/fstab LABEL=cpot_t0l2s1 /a ext3 _netdev 0 0To have the volume available the next time we boot the system, I've
set the volume to mount via it's label in "/etc/fstab" above. Of note,
the FS option is set to "_netdev", effectively preventing the system from
mounting this volume until network access is available. Our new volume is
now ready for usage and our other LUNs ready to be configured for usage.
Below, from either humboldt (initiator node) or cobblepot (target node),
we can see two established connections (sessions) to port 3260, one for
each target:
humboldt [0] /bin/netstat -na | /bin/grep 3260 tcp 0 0 10.0.136.122:40103 10.0.136.17:3260 ESTABLISHED tcp 0 0 10.0.136.122:40102 10.0.136.17:3260 ESTABLISHED cobblepot [0] /bin/netstat -na | /bin/grep 3260 tcp 0 0 0.0.0.0:3260 0.0.0.0:* LISTEN tcp 0 0 10.0.136.17:3260 10.0.136.122:40103 ESTABLISHED tcp 0 0 10.0.136.17:3260 10.0.136.122:40102 ESTABLISHEDAdditionally, on our target host, a check of "/proc/net/iet/session"
now shows sessions for our INN for both targets:
cobblepot [0] cat /proc/net/iet/session tid:2 name:iqn.2011-06.com.example.cobblepot:storage-net.humboldt1 sid:281475014263296 initiator:iqn.1994-05.com.redhat:1d94b02baba8 cid:0 ip:10.0.136.122 state:active hd:none dd:none tid:1 name:iqn.2011-06.com.example.cobblepot:storage-net.humboldt0 sid:562949974196736 initiator:iqn.1994-05.com.redhat:1d94b02baba8 cid:0 ip:10.0.136.122 state:active hd:none dd:none
see also:
Part 1: Configuring iSCSI Targets in Linux
Configuring iSCSI Initiators in Solaris
Configuring iSCSI Initiators in Windows 2008
