Noviembre 17

FCoE but i can talk about iSCSI

I cant talk about FCoE but i can talk about iSCSI.

first up, what are you talking to array wise? it shouldnt make a huge difference to the info to follow, just interested.

We use Dell Equallogic arrays at work, and the following config works remarkably well.

First up, get rid of any smell of LACP/PortChannel/LinkAgg between servers and switches and storage and switches. the only place you should be using link agg is switch to switch. iSCSI is a path aware protocol, in a nutshell you’re only getting 1Gbit/sec will be because the IO stream is being balanced down one cable because it is all part of the same session.

Couple of Terms of reference
Initiator – the host
Target Portal – the storage
Session, a logical connection between a initiator and a target

Now, there is basicly a 1:1 relationship between initiator and target, a 1:1 relationship between physical interface and initiator address and a 1:1 relationship between session and physical network port. This pretty much comes back to the FC world where you have one session between the HBA and the storage array, you dont run multiple sessions over a FC cable.

so given you have two cables plugged in, you will want to create two sessions back to your storage, the multipathing daemon (multipathd) on the linux box will sort out the MPIO stuff for you.

here are a bunch of config things we do on our servers to make the magic happen

first up, /etc/sysctl.conf

# Equallogic Configuration settings
# ARP Flux and Return Path Filtering are two behaviours the Linux
# employs for multiple NICs on the same subnet.  They need to be disabled
# to ensure proper MPIO operation with Equallogic storage.


then /etc/iscsi/iscsid.conf

node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 10
node.session.timeo.replacement_timeout = 15

we put this into /etc/rc.local to disable some of the tcp stack offloading

/sbin/ethtool --offload eth2 gro off
/sbin/ethtool --offload eth3 gro off

For sanity reasons we modify the default autogenerated iscsi IQN name to something more meaninful, its located in /etc/iscsi/initiatorname.iscsi


Then we turn on jumbo frames on the iSCSI interfaces

Modify /etc/sysconfig/network-scripts/ifcfg-ethX (where X is the interfaces talking to the iSCSI Array)

Add the following


now we get stuck into configuring the iscsi daemon itself
first up we create some logical interfaces within the iscsi daemon to direct traffic down

# iscsiadm --mode iface --op=new --interface iscsi0
# iscsiadm --mode iface --op=new --interface iscsi1

then we link the physical interfaces to the logical interfaces

# iscsiadm --mode iface --op=update --interface iscsi0 --name=iface.net_ifacename --value=eth2
# iscsiadm --mode iface --op=update --interface iscsi1 --name=iface.net_ifacename --value=eth3

Now we discover the targets we can talk to on our storage array

# iscsiadm -m discovery -t st -p <ARRAY IP> -I iscsi0 -I iscsi1

and finally we login to the targets we have discovered

# iscsiadm -m node --login

Right now, with all this done, you should now see two sd[a-z] devices for each LUN you have presented, basicly one of the devices goes via one ethernet cable, the other device via the other ethernet cable, protecting you against cable/switch failure, assuming correct connectivity and config of your switching fabric.

now here is a sample config of the multipath system in linux

all this goes into /etc/multipath.conf

## Use user friendly names, instead of using WWIDs as names.
defaults {
     user_friendly_names yes

## Blacklist local devices (no need to have them accidentally "multipathed")
blacklist {
     devnode "^sd[a]$"
     devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
     devnode "^hd[a-z]"
     devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"

## For EqualLogic PS series RAID arrays
devices {
     device {
          vendor                  "EQLOGIC"
          product                 "100E-00"
          path_grouping_policy    multibus
          getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
          features                "1 queue_if_no_path"
          path_checker            readsector0
          path_selector           "round-robin 0"
          failback                immediate
          rr_min_io               10
          rr_weight               priorities

## Assign human-readable name  (aliases) to the World-Wide ID's (names) of the devices we're wanting to manage
## Identify the WWID by issuing # scsi_id -gus /block/sdX

multipaths {
     multipath {
          wwid                    36090a01850e08bd883c4a45bfa0330ba
          alias                     devicename2
     multipath {
          wwid                    36090a01850e04bd683c4745bfa03b07c
          alias                     devicename1

there will be other examples in the config file around the device stanza, you’ll need to modify the vendor and product values to suit what you’re running, when the disk arrived on the machine via iscsi, dmesg would have recorded some details that would help here.

the multipaths stuff isn’t required, it just make life a lot easier

then start all the services needed

# service multipathd start
# chkconfig multipathd on

then use the following command to see if things have worked

# multipath -v2
# multipath -ll

if everything is good you should get something like this as the output from the second command

devicename1 (36090a01850e04bd683c4745bfa03b07c) dm-3 EQLOGIC,100E-00
[size=5.0T][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=4][active]
 \_ 13:0:0:0 sdd 8:48  [active][ready]
 \_ 14:0:0:0 sdb 8:16  [active][ready]
 \_ 12:0:0:0 sdc 8:32  [active][ready]
 \_ 11:0:0:0 sde 8:64  [active][ready]

the device to mount will be /dev/mapper/devicename1

Category: iSCSI, STORAGE | Los comentarios están deshabilitados en FCoE but i can talk about iSCSI