legacy-wiki
Multipath
Recovered from the older tannerjc.net wiki snapshot dated January 23, 2016.
Documentation
Diagram: http://christophe.varoqui.free.fr/graphics/macroflux.png
Correlation to SCSI layers: http://www.m00t.net/wiki/index.php?title=Scsi#Layers
Patches to make the SCSI midlayer perform possible retries before sending IO errors to the the mpath layer:
http://sources.redhat.com/lvm2/wiki/MultipathUsageGuide
building a boot from mpath initrd
Enumerating WWIDs
for i in `ls -d /sys/block/s*`; do echo $i; scsi_id -g -u -s `echo $i | sed 's/\/sys//g'` ; done;
Code layout
rpm2cpio device-mapper-multipath-0.4.7-30.el5.src.rpm | cpio -idmv
cciss_id
devmap_name --- gets the device mapper device name
kpartx --- kpartx which is used to build partitions for mpath devices
libcheckers --- path checking code (cciss_tur, directio, rdac, readsector0, tur)
libmultipath
|- alias.c ---
|- blacklist.c ---
|- callout.c ---
|- defaults.c ---
|- dict.c --- functions for checking each parm in the config
|- discovery.c --- defines how to find devices and what to check before adding the path
| * sector count
| * open sysfs path
| * read sysfs attributes
| * open sysfs path to read attributes
| * check gndbd value from sysfs
|-
multipath
multipath.conf.annotated
multipath.conf.defaults
multipath.conf.synthetic
multipathd
path_priority
aliases
jr rp: you cannot alias to mpath*
jr rp: you have to use a different naming scheme
jr rp: i have seen that exact problem before
jr and its because they are using mpath names in their aliases
rp jr, reason ?
jr rp: no clue
jr rp: if you want to use the mpath names then just ensure /var/lib/multipath/bindings has what you want
jr should accomplish the same thing
rp jr, whether there is a conflict between entries in (/var/lib/multipath/bindings) and aliases ?
jr rp: just use bindings and don't make an alias
no_path_retry
https://bugzilla.redhat.com/show_bug.cgi?id=181309
- Say for instanace that you have a device with no_paths_retry set to 10. All paths fail, 10 retrys happen, and so the device fails.. All future IO fails until a path comes back. With the patches proposed above, if you run multipath, that device will start queuing IO again, maybe for 10 more retrys… maybe forever, since multipathd might not realize that you changed the value out from under it, and start retry countdown again.
Debugging
multipath daemon
Log to console:
# multipathd -d
Log to syslog:
# multipathd -v4
Use strace:
# strace -f multipathd
You can also modify the init script so that that multipathd is always running with -v4
Change ...
start() {
test -x $DAEMON || exit 5
echo -n $Starting $prog daemon:
daemon $DAEMON
RETVAL=$?
[ $RETVAL -eq 0 ] touch $lockdir/$prog
echo
}
To ...
start() {
test -x $DAEMON || exit 5
echo -n $Starting $prog daemon:
daemon multipathd -v4
RETVAL=$?
[ $RETVAL -eq 0 ] touch $lockdir/$prog
echo
}
Enable logging on the SCSI midlayer
[root@rhel5box initscripts-8.45.25]# modinfo scsi_mod | fgrep logging
parm: scsi_logging_level:a bit mask of logging levels (int) #options scsi_mod scsi_logging_level=0x0000003d
or ..
[root@rhel5box class]# cat /sys/module/scsi_mod/parameters/scsi_logging_level
[root@rhel5box class]# echo 6 /sys/module/scsi_mod/parameters/scsi_logging_level
or ..
- echo 0xffffffff /sys/class/scsi_host/host0/debug_level
If you only want to see scan messages …
echo 448 /sys/module/scsi_mod/parameters/scsi_logging_level
Enable logging on lower layers [HBA]
[root@rhel5box initscripts-8.45.25]# modinfo qla2xxx | fgrep logging
parm: extended_error_logging:Option to enable extended error logging, Default is 0 - no logging. 1 - log errors. (int)
[root@rhel5box initscripts-8.45.25]# modinfo lpfc | fgrep logging
parm: lpfc_log_verbose:Verbose logging bit-mask (int) #options lpfc lpfc_log_verbose=0xffff
[0xffff] will turn everything on which is reasonable if you don’t have much of an idea where things are going wrong and want to see as much of the driver’s activity as you can. Be aware that this will generate a lot of log traffic, however.
If you’ve got more of a clue where things are going south then you can use the values in drivers/scsi/lpfc/lpfc_attr.c to select a more targeted set of information:
# lpfc_log_verbose: Only turn this flag on if you are willing to risk
# being deluged with LOTS of information.
# You can set a bit mask to record specific types of verbose messages:
#
# LOG_ELS 0x1 ELS events
# LOG_DISCOVERY 0x2 Link discovery events
# LOG_MBOX 0x4 Mailbox events
# LOG_INIT 0x8 Initialization events
# LOG_LINK_EVENT 0x10 Link events
# LOG_FCP 0x40 FCP traffic history
# LOG_NODE 0x80 Node table events
# LOG_MISC 0x400 Miscellaneous events
# LOG_SLI 0x800 SLI events
# LOG_FCP_ERROR 0x1000 Only log FCP errors
# LOG_LIBDFC 0x2000 LIBDFC events
# LOG_ALL_MSG 0xffff LOG all messages
So for e.g. if you just want to see FCP traffic logs and link events you can set a value of 0x50.
Finding WWPN
CentOS Multipath Index
http://www.centos.org/docs/5/html/5.1/DM_Multipath/ix01.html
Active/Passive Example
mpath2 (360a98000486e586f6b5a524f56476a56) dm-9 NETAPP,LUN -- Its dm-9 ( 253:9 )
[size=800G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=16][active] --- Here the paths are active
\_ 3:0:1:0 sdd 8:48 [active][ready]
\_ 3:0:3:0 sdf 8:80 [active][ready]
\_ 4:0:1:0 sdh 8:112 [active][ready]
\_ 4:0:2:0 sdi 8:128 [active][ready]
\_ round-robin 0 [prio=4][enabled] -- Enabled means, paths are passive .
\_ 3:0:0:0 sdc 8:32 [active][ready]
\_ 3:0:2:0 sde 8:64 [active][ready]
\_ 4:0:0:0 sdg 8:96 [active][ready]
\_ 4:0:3:0 sdj 8:144 [active][ready]
dm-0: blacklisted
dm-10: blacklisted
dm-11: blacklisted
dm-12: blacklisted
dm-13: blacklisted
dm-14: blacklisted
dm-15: blacklisted
dm-16: blacklisted
dm-17: blacklisted
dm-18: blacklisted
dm-19: blacklisted
dm-1: blacklisted
dm-20: blacklisted
dm-21: blacklisted
dm-22: blacklisted
dm-23: blacklisted
dm-24: blacklisted
dm-25: blacklisted
dm-26: blacklisted
dm-27: blacklisted
dm-28: blacklisted
dm-29: blacklisted
dm-2: blacklisted
dm-30: blacklisted
dm-3: blacklisted
dm-4: blacklisted
dm-5: blacklisted
dm-6: blacklisted
dm-7: blacklisted
dm-8: blacklisted
dm-9: blacklisted
md0: blacklisted
ram0: blacklisted
ram10: blacklisted
ram11: blacklisted
ram12: blacklisted
ram13: blacklisted
ram14: blacklisted
ram15: blacklisted
ram1: blacklisted
ram2: blacklisted
ram3: blacklisted
ram4: blacklisted
ram5: blacklisted
ram6: blacklisted
ram7: blacklisted
ram8: blacklisted
ram9: blacklisted
sda: not found in pathvec
sda: mask = 0x5
sda: bus = 1
sda: dev_t = 8:0
sda: size = 29296640
sda: vendor = LSILOGIC
sda: product = Logical Volume
sda: rev = 3000
sda: h:b:t:l = 0:1:0:0
sda: path checker = readsector0 (config file default)
sda: state = 2
sdb: not found in pathvec
sdb: mask = 0x5
sdb: bus = 1
sdb: dev_t = 8:16
sdb: size = 1048576000
sdb: vendor = IBM
sdb: product = 1726-4xx FAStT
sdb: rev = 0617
sdb: h:b:t:l = 3:0:0:0
sdb: tgt_node_name = 0x200400a0b84942d6
sdb: path checker = rdac (controller setting)
sdb: state = 4
sdc: not found in pathvec
sdc: mask = 0x5
sdc: bus = 1
sdc: dev_t = 8:32
sdc: size = 1048576000
sdc: vendor = IBM
sdc: product = 1726-4xx FAStT
sdc: rev = 0617
sdc: h:b:t:l = 4:0:0:0
sdc: tgt_node_name = 0x200400a0b84942d6
sdc: path checker = rdac (controller setting)
sdc: state = 4
sdd: not found in pathvec
sdd: mask = 0x5
sdd: bus = 1
sdd: dev_t = 8:48
sdd: size = 1048576000
sdd: vendor = IBM
sdd: product = 1726-4xx FAStT
sdd: rev = 0617
sdd: h:b:t:l = 3:0:0:1
sdd: tgt_node_name = 0x200400a0b84942d6
sdd: path checker = rdac (controller setting)
sdd: state = 2
sde: not found in pathvec
sde: mask = 0x5
sde: bus = 1
sde: dev_t = 8:64
sde: size = 426974208
sde: vendor = IBM
sde: product = 1726-4xx FAStT
sde: rev = 0617
sde: h:b:t:l = 3:0:0:2
sde: tgt_node_name = 0x200400a0b84942d6
sde: path checker = rdac (controller setting)
sde: state = 4
sdf: not found in pathvec
sdf: mask = 0x5
sdf: bus = 1
sdf: dev_t = 8:80
sdf: size = 1048576000
sdf: vendor = IBM
sdf: product = 1726-4xx FAStT
sdf: rev = 0617
sdf: h:b:t:l = 4:0:0:1
sdf: tgt_node_name = 0x200400a0b84942d6
sdf: path checker = rdac (controller setting)
sdf: state = 2
sdg: not found in pathvec
sdg: mask = 0x5
sdg: bus = 1
sdg: dev_t = 8:96
sdg: size = 426974208
sdg: vendor = IBM
sdg: product = 1726-4xx FAStT
sdg: rev = 0617
sdg: h:b:t:l = 4:0:0:2
sdg: tgt_node_name = 0x200400a0b84942d6
sdg: path checker = rdac (controller setting)
sdg: state = 4
sdh: not found in pathvec
sdh: mask = 0x5
sdh: bus = 1
sdh: dev_t = 8:112
sdh: size = 4194304
sdh: vendor = IBM
sdh: product = 1726-4xx FAStT
sdh: rev = 0617
sdh: h:b:t:l = 3:0:0:3
sdh: tgt_node_name = 0x200400a0b84942d6
sdh: path checker = rdac (controller setting)
sdh: state = 4
sdi: not found in pathvec
sdi: mask = 0x5
sdi: bus = 1
sdi: dev_t = 8:128
sdi: size = 4194304
sdi: vendor = IBM
sdi: product = 1726-4xx FAStT
sdi: rev = 0617
sdi: h:b:t:l = 4:0:0:3
sdi: tgt_node_name = 0x200400a0b84942d6
sdi: path checker = rdac (controller setting)
sdi: state = 4
sdj: not found in pathvec
sdj: mask = 0x5
sdj: bus = 1
sdj: dev_t = 8:144
sdj: size = 4269797376
sdj: vendor = IBM
sdj: product = 1726-4xx FAStT
sdj: rev = 0617
sdj: h:b:t:l = 3:0:0:5
sdj: tgt_node_name = 0x200400a0b84942d6
sdj: path checker = rdac (controller setting)
sdj: state = 4
sdk: not found in pathvec
sdk: mask = 0x5
sdk: bus = 1
sdk: dev_t = 8:160
sdk: size = 4269797376
sdk: vendor = IBM
sdk: product = 1726-4xx FAStT
sdk: rev = 0617
sdk: h:b:t:l = 4:0:0:5
sdk: tgt_node_name = 0x200400a0b84942d6
sdk: path checker = rdac (controller setting)
sdk: state = 4
sdl: not found in pathvec
sdl: mask = 0x5
sdl: bus = 1
sdl: dev_t = 8:176
sdl: size = 1048576000
sdl: vendor = IBM
sdl: product = 1726-4xx FAStT
sdl: rev = 0617
sdl: h:b:t:l = 3:0:1:0
sdl: tgt_node_name = 0x200400a0b84942d6
sdl: path checker = rdac (controller setting)
sdl: state = 2
sdm: not found in pathvec
sdm: mask = 0x5
sdm: bus = 1
sdm: dev_t = 8:192
sdm: size = 1048576000
sdm: vendor = IBM
sdm: product = 1726-4xx FAStT
sdm: rev = 0617
sdm: h:b:t:l = 3:0:1:1
sdm: tgt_node_name = 0x200400a0b84942d6
sdm: path checker = rdac (controller setting)
sdm: state = 4
sdn: not found in pathvec
sdn: mask = 0x5
sdn: bus = 1
sdn: dev_t = 8:208
sdn: size = 1048576000
sdn: vendor = IBM
sdn: product = 1726-4xx FAStT
sdn: rev = 0617
sdn: h:b:t:l = 4:0:1:0
sdn: tgt_node_name = 0x200400a0b84942d6
sdn: path checker = rdac (controller setting)
sdn: state = 2
sdo: not found in pathvec
sdo: mask = 0x5
sdo: bus = 1
sdo: dev_t = 8:224
sdo: size = 1048576000
sdo: vendor = IBM
sdo: product = 1726-4xx FAStT
sdo: rev = 0617
sdo: h:b:t:l = 4:0:1:1
sdo: tgt_node_name = 0x200400a0b84942d6
sdo: path checker = rdac (controller setting)
sdo: state = 4
sdp: not found in pathvec
sdp: mask = 0x5
sdp: bus = 1
sdp: dev_t = 8:240
sdp: size = 426974208
sdp: vendor = IBM
sdp: product = 1726-4xx FAStT
sdp: rev = 0617
sdp: h:b:t:l = 3:0:1:2
sdp: tgt_node_name = 0x200400a0b84942d6
sdp: path checker = rdac (controller setting)
sdp: state = 2
sdq: not found in pathvec
sdq: mask = 0x5
sdq: bus = 1
sdq: dev_t = 65:0
sdq: size = 426974208
sdq: vendor = IBM
sdq: product = 1726-4xx FAStT
sdq: rev = 0617
sdq: h:b:t:l = 4:0:1:2
sdq: tgt_node_name = 0x200400a0b84942d6
sdq: path checker = rdac (controller setting)
sdq: state = 2
sdr: not found in pathvec
sdr: mask = 0x5
sdr: bus = 1
sdr: dev_t = 65:16
sdr: size = 4194304
sdr: vendor = IBM
sdr: product = 1726-4xx FAStT
sdr: rev = 0617
sdr: h:b:t:l = 3:0:1:3
sdr: tgt_node_name = 0x200400a0b84942d6
sdr: path checker = rdac (controller setting)
sdr: state = 2
sds: not found in pathvec
sds: mask = 0x5
sds: bus = 1
sds: dev_t = 65:32
sds: size = 4194304
sds: vendor = IBM
sds: product = 1726-4xx FAStT
sds: rev = 0617
sds: h:b:t:l = 4:0:1:3
sds: tgt_node_name = 0x200400a0b84942d6
sds: path checker = rdac (controller setting)
sds: state = 2
sdt: not found in pathvec
sdt: mask = 0x5
sdt: bus = 1
sdt: dev_t = 65:48
sdt: size = 4269797376
sdt: vendor = IBM
sdt: product = 1726-4xx FAStT
sdt: rev = 0617
sdt: h:b:t:l = 3:0:1:5
sdt: tgt_node_name = 0x200400a0b84942d6
sdt: path checker = rdac (controller setting)
sdt: state = 2
sdu: not found in pathvec
sdu: mask = 0x5
sdu: bus = 1
sdu: dev_t = 65:64
sdu: size = 4269797376
sdu: vendor = IBM
sdu: product = 1726-4xx FAStT
sdu: rev = 0617
sdu: h:b:t:l = 4:0:1:5
sdu: tgt_node_name = 0x200400a0b84942d6
sdu: path checker = rdac (controller setting)
sdu: state = 2
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod/rev
0:1:0:0 sda 8:0 0 [undef][ready] LSILOGIC,Logical Volume
3:0:0:0 sdb 8:16 0 [undef][ghost] IBM,1726-4xx FAStT
4:0:0:0 sdc 8:32 0 [undef][ghost] IBM,1726-4xx FAStT
3:0:0:1 sdd 8:48 0 [undef][ready] IBM,1726-4xx FAStT
3:0:0:2 sde 8:64 0 [undef][ghost] IBM,1726-4xx FAStT
4:0:0:1 sdf 8:80 0 [undef][ready] IBM,1726-4xx FAStT
4:0:0:2 sdg 8:96 0 [undef][ghost] IBM,1726-4xx FAStT
3:0:0:3 sdh 8:112 0 [undef][ghost] IBM,1726-4xx FAStT
4:0:0:3 sdi 8:128 0 [undef][ghost] IBM,1726-4xx FAStT
3:0:0:5 sdj 8:144 0 [undef][ghost] IBM,1726-4xx FAStT
4:0:0:5 sdk 8:160 0 [undef][ghost] IBM,1726-4xx FAStT
3:0:1:0 sdl 8:176 0 [undef][ready] IBM,1726-4xx FAStT
3:0:1:1 sdm 8:192 0 [undef][ghost] IBM,1726-4xx FAStT
4:0:1:0 sdn 8:208 0 [undef][ready] IBM,1726-4xx FAStT
4:0:1:1 sdo 8:224 0 [undef][ghost] IBM,1726-4xx FAStT
3:0:1:2 sdp 8:240 0 [undef][ready] IBM,1726-4xx FAStT
4:0:1:2 sdq 65:0 0 [undef][ready] IBM,1726-4xx FAStT
3:0:1:3 sdr 65:16 0 [undef][ready] IBM,1726-4xx FAStT
4:0:1:3 sds 65:32 0 [undef][ready] IBM,1726-4xx FAStT
3:0:1:5 sdt 65:48 0 [undef][ready] IBM,1726-4xx FAStT
4:0:1:5 sdu 65:64 0 [undef][ready] IBM,1726-4xx FAStT
params = 1 queue_if_no_path 1 rdac 2 1 round-robin 0 2 1 65:16 1000 65:32 1000 round-robin 0 2 1 8:112 1000 8:128 1000
status = 2 0 1 0 2 1 A 0 2 0 65:16 A 0 65:32 A 0 E 0 2 0 8:112 A 0 8:128 A 0
*word = 1, len = 1
*word = queue_if_no_path, len = 16
*word = 1, len = 1
*word = rdac, len = 4
*word = 2, len = 1
*word = 1, len = 1
*word = round-robin, len = 11
*word = 0, len = 1
*word = 2, len = 1
*word = 1, len = 1
*word = 65:16, len = 5
*word = 1000, len = 4
*word = 65:32, len = 5
*word = 1000, len = 4
*word = 2, len = 1
*word = 1, len = 1
*word = 8:112, len = 5
*word = 1000, len = 4
*word = 8:128, len = 5
*word = 1000, len = 4
sdr: mask = 0x8
sdr: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdr: prio = 100
sds: mask = 0x8
sds: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sds: prio = 100
sdh: mask = 0x8
sdh: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdh: prio = 0
sdi: mask = 0x8
sdi: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdi: prio = 0
*word = 2, len = 1
*word = 1, len = 1
*word = 0, len = 1
*word = 2, len = 1
*word = A, len = 1
*word = 2, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = E, len = 1
*word = 2, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
xen-cfg (3600a0b80004942d60000028e488dd920) dm-3 IBM,1726-4xx FAStT
[size=2.0G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=200][active]
\_ 3:0:1:3 sdr 65:16 [active][ready]
\_ 4:0:1:3 sds 65:32 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:0:3 sdh 8:112 [active][ghost]
\_ 4:0:0:3 sdi 8:128 [active][ghost]
params = 1 queue_if_no_path 1 rdac 2 1 round-robin 0 2 1 8:240 1000 65:0 1000 round-robin 0 2 1 8:64 1000 8:96 1000
status = 2 0 1 0 2 1 A 0 2 0 8:240 A 0 65:0 A 0 E 0 2 0 8:64 A 0 8:96 A 0
*word = 1, len = 1
*word = queue_if_no_path, len = 16
*word = 1, len = 1
*word = rdac, len = 4
*word = 2, len = 1
*word = 1, len = 1
*word = round-robin, len = 11
*word = 0, len = 1
*word = 2, len = 1
*word = 1, len = 1
*word = 8:240, len = 5
*word = 1000, len = 4
*word = 65:0, len = 4
*word = 1000, len = 4
*word = 2, len = 1
*word = 1, len = 1
*word = 8:64, len = 4
*word = 1000, len = 4
*word = 8:96, len = 4
*word = 1000, len = 4
sdp: mask = 0x8
sdp: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdp: prio = 100
sdq: mask = 0x8
sdq: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdq: prio = 100
sde: mask = 0x8
sde: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sde: prio = 0
sdg: mask = 0x8
sdg: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdg: prio = 0
*word = 2, len = 1
*word = 1, len = 1
*word = 0, len = 1
*word = 2, len = 1
*word = A, len = 1
*word = 2, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = E, len = 1
*word = 2, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
mpath13 (3600a0b80004942c80000037c49a6aeaf) dm-2 IBM,1726-4xx FAStT
[size=204G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=200][active]
\_ 3:0:1:2 sdp 8:240 [active][ready]
\_ 4:0:1:2 sdq 65:0 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:0:2 sde 8:64 [active][ghost]
\_ 4:0:0:2 sdg 8:96 [active][ghost]
params = 1 queue_if_no_path 1 rdac 2 1 round-robin 0 2 1 8:48 1000 8:80 1000 round-robin 0 2 1 8:192 1000 8:224 1000
status = 2 0 1 0 2 1 A 0 2 0 8:48 A 0 8:80 A 0 E 0 2 0 8:192 A 0 8:224 A 0
*word = 1, len = 1
*word = queue_if_no_path, len = 16
*word = 1, len = 1
*word = rdac, len = 4
*word = 2, len = 1
*word = 1, len = 1
*word = round-robin, len = 11
*word = 0, len = 1
*word = 2, len = 1
*word = 1, len = 1
*word = 8:48, len = 4
*word = 1000, len = 4
*word = 8:80, len = 4
*word = 1000, len = 4
*word = 2, len = 1
*word = 1, len = 1
*word = 8:192, len = 5
*word = 1000, len = 4
*word = 8:224, len = 5
*word = 1000, len = 4
sdd: mask = 0x8
sdd: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdd: prio = 100
sdf: mask = 0x8
sdf: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdf: prio = 100
sdm: mask = 0x8
sdm: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdm: prio = 0
sdo: mask = 0x8
sdo: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdo: prio = 0
*word = 2, len = 1
*word = 1, len = 1
*word = 0, len = 1
*word = 2, len = 1
*word = A, len = 1
*word = 2, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = E, len = 1
*word = 2, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
prod-oses (3600a0b80004942c8000001b3487caf1f) dm-1 IBM,1726-4xx FAStT
[size=500G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=200][active]
\_ 3:0:0:1 sdd 8:48 [active][ready]
\_ 4:0:0:1 sdf 8:80 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:1:1 sdm 8:192 [active][ghost]
\_ 4:0:1:1 sdo 8:224 [active][ghost]
params = 1 queue_if_no_path 1 rdac 2 1 round-robin 0 2 1 8:176 1000 8:208 1000 round-robin 0 2 1 8:16 1000 8:32 1000
status = 2 0 1 0 2 1 A 0 2 0 8:176 A 0 8:208 A 0 E 0 2 0 8:16 A 0 8:32 A 0
*word = 1, len = 1
*word = queue_if_no_path, len = 16
*word = 1, len = 1
*word = rdac, len = 4
*word = 2, len = 1
*word = 1, len = 1
*word = round-robin, len = 11
*word = 0, len = 1
*word = 2, len = 1
*word = 1, len = 1
*word = 8:176, len = 5
*word = 1000, len = 4
*word = 8:208, len = 5
*word = 1000, len = 4
*word = 2, len = 1
*word = 1, len = 1
*word = 8:16, len = 4
*word = 1000, len = 4
*word = 8:32, len = 4
*word = 1000, len = 4
sdl: mask = 0x8
sdl: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdl: prio = 100
sdn: mask = 0x8
sdn: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdn: prio = 100
sdb: mask = 0x8
sdb: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdb: prio = 0
sdc: mask = 0x8
sdc: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdc: prio = 0
*word = 2, len = 1
*word = 1, len = 1
*word = 0, len = 1
*word = 2, len = 1
*word = A, len = 1
*word = 2, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = E, len = 1
*word = 2, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
prod-data (3600a0b80004942d6000001ce487caf70) dm-0 IBM,1726-4xx FAStT
[size=500G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=200][active]
\_ 3:0:1:0 sdl 8:176 [active][ready]
\_ 4:0:1:0 sdn 8:208 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:0:0 sdb 8:16 [active][ghost]
\_ 4:0:0:0 sdc 8:32 [active][ghost]
params = 1 queue_if_no_path 1 rdac 2 1 round-robin 0 2 1 65:48 1000 65:64 1000 round-robin 0 2 1 8:144 1000 8:160 1000
status = 2 0 1 0 2 1 A 0 2 0 65:48 A 0 65:64 A 0 E 0 2 0 8:144 A 0 8:160 A 0
*word = 1, len = 1
*word = queue_if_no_path, len = 16
*word = 1, len = 1
*word = rdac, len = 4
*word = 2, len = 1
*word = 1, len = 1
*word = round-robin, len = 11
*word = 0, len = 1
*word = 2, len = 1
*word = 1, len = 1
*word = 65:48, len = 5
*word = 1000, len = 4
*word = 65:64, len = 5
*word = 1000, len = 4
*word = 2, len = 1
*word = 1, len = 1
*word = 8:144, len = 5
*word = 1000, len = 4
*word = 8:160, len = 5
*word = 1000, len = 4
sdt: mask = 0x8
sdt: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdt: prio = 100
sdu: mask = 0x8
sdu: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdu: prio = 100
sdj: mask = 0x8
sdj: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdj: prio = 0
sdk: mask = 0x8
sdk: getprio = /sbin/mpath_prio_rdac /dev/%n (controller setting)
sdk: prio = 0
*word = 2, len = 1
*word = 1, len = 1
*word = 0, len = 1
*word = 2, len = 1
*word = A, len = 1
*word = 2, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = E, len = 1
*word = 2, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
*word = A, len = 1
*word = 0, len = 1
cas-storage (3600a0b80004942c80000037b49a6a20f) dm-5 IBM,1726-4xx FAStT
[size=2.0T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=200][active]
\_ 3:0:1:5 sdt 65:48 [active][ready]
\_ 4:0:1:5 sdu 65:64 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:0:5 sdj 8:144 [active][ghost]
\_ 4:0:0:5 sdk 8:160 [active][ghost]
RDAC conflicts with AVT
Normal priorities with an rdac hardware handler..
cas-storage (3600a0b80004942c80000037b49a6a20f) dm-5 IBM,1726-4xx FAStT
[size=2.0T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=200][active]
\_ 3:0:1:5 sdt 65:48 [active][ready]
\_ 4:0:1:5 sdu 65:64 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:0:5 sdj 8:144 [active][ghost]
\_ 4:0:0:5 sdk 8:160 [active][ghost
When AVT (advanced volume transfer) is enabled in addition to using rdac, the priorities can switch around and cause problems such as lvm commands hanging…
mpath0 (3600a0b800029b6000000afd946d6b0a1) dm-10 IBM,1815 FAStT
[size=339G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=200][active]
\_ 1:0:0:0 sdb 8:16 [active][ghost]
\_ 0:0:1:0 sdc 8:32 [active][ghost]
\_ round-robin 0 [prio=0][enabled]
\_ 0:0:0:0 sda 8:0 [active][ready]
\_ 1:0:1:0 sdd 8:48 [active][ready]
You’ll know AVT is on if something like this is in /var/log/messages ..
Jun 3 09:25:29 mtvnbur06 kernel: sd 1:0:0:0: rdac: AVT mode detected
Oracle notes on 0x20000
[https://metalink2.oracle.com/metalink/plsql/f?p=130:14:8403425595916020255::::p14_database_id,p14_docid,p14_show_header,p14_show_help,p14_black_frame,p14_font:NOT,434255.1,1,1,1,helvetica]
Jan 24 09:33:15 cascade kernel: SCSI error : 4 0 0 8 return code = 0x20000
Jan 24 09:33:15 cascade kernel: end_request: I/O error, dev sdai, sector 2239
Jan 24 09:33:15 cascade kernel: device-mapper: dm-multipath: Failing path 66:32.
- Solution:**
For example,The incompatibility between the way device-manager-multipath operates and the way the IBM FastT700 expects to be utilized.Basically, dm-multipath wants to round robin over all available paths to a logical drive, which causes I/O to flip between the controllers on the FastT. The fastT wants all I/O to remain on the prefferred path so long as it is available. The constant switching is apparently causing confusion and delay within the Fast T 700. Request the customer to engage their storage vendor for checking/replacing the hardware and providing a valid resolution for dm-multipath.