单路径变更多路径
某客户一套生产系统，由于施工期间不规范，数据库服务器直接直连存储，
现需要将服务器通过光交与存储连接，并配置多路径保证链路的冗余和负载均衡
具体思路如下:
1.收集系统信息
1.1 核对服务器磁盘归属信息(本地磁盘，存储lun)
1.2 收集所有磁盘设备的WWID信息，fdisk信息
1.3 收集文件系统fstab挂载信息(备份fstab)
1.4 收集磁盘卷、文件系统uuid信息
1.5 收集所有oracle数据文件信息(位置，大小，个数，状态)
2.配置多路径、核对信息
2.1 检查配置multipath服务
2.2 配置multipath.conf，绑定新设备名
2.3 重启操作系统
2.4 刷新multipath服务
2.5 检查校验多路径状态，新设备挂在信息
2.6 检查卷、文件系统UUID信息
2.7 启动数据库
#############
具体实施步骤#
#############
####在进行操作前，收集现有系统配置信息并对其进行备份，以便出现问题时候回退###
1.1 服务器磁盘归属信息
经核查sda为本地磁盘，其他设备均为直连存储的lun(sbd,sdc,sdd,sdg,sde,sdf)
1.2查看现有系统wwid信息
[root@oradb ~]# for i in `cat /proc/partitions | awk {'print $4'} | grep sd`; do echo "### $i: `scsi_id --whitelist /dev/$i`"; done
### sda: 361866da08f0a8c0020d6a2c31e4628f7
### sdb: 36006016093e24100920c1b587959b56c
### sdb2:
### sdc: 36006016093e241007b0c1b583729b956
### sdc1:
### sdd: 36006016093e24100c069285a00444b52
### sdg: 36006016042704500966bc2336892e911
### sdf: 360060160427045006e1c1baad56be711
### sdf1:
### sde: 36006016093e24100a3595b5b716a5de0
### sde1:
##############
fsisk -l信息 #
##############
[root@oradb ~]# fdisk -l
Disk /dev/sda: 299.4 GB, 299439751168 bytes
255 heads, 63 sectors/track, 36404 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x6ad0d56a
Device Boot Start End Blocks Id System
/dev/sda1 1 36405 292421631+ ee GPT
Disk /dev/sdb: 1649.3 GB, 1649267441664 bytes
255 heads, 63 sectors/track, 200512 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdb2 1 200512 1610612608+ 83 Linux
Disk /dev/sdc: 1649.3 GB, 1649267441664 bytes
255 heads, 63 sectors/track, 200512 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdc1 1 200512 1610612608+ 83 Linux
Disk /dev/sdd: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdg: 2199.0 GB, 2199023255552 bytes
255 heads, 63 sectors/track, 267349 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdf: 644.2 GB, 644245094400 bytes
255 heads, 63 sectors/track, 78325 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
/dev/sdf1 1 78325 629145531 83 Linux
Disk /dev/sde: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xcd0c0083
Device Boot Start End Blocks Id System
/dev/sde1 1 133674 1073736373+ 83 Linux
Disk /dev/mapper/backupvg-backup_lv: 3247.0 GB, 3246995275776 bytes
255 heads, 63 sectors/track, 394758 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
1.3 收集fstab信息
当前文件系统使用uuid信息挂载，这样的好处是新设备绑定多路径后，原文件系统的fstab信息不需要再修改
[root@oradb ~]# more /etc/fstab
# /etc/fstab
# Created by anaconda on Tue Sep 19 21:19:56 2017
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=94882757-4b38-4318-a08b-6f7f712ab516 / ext4 defaults 1 1
UUID=594fa55a-19d0-445f-8e27-4a40b0da0640 /boot ext4 defaults 1 2
UUID=d4a51b71-f7ec-404a-9870-8495c0accf64 swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
UUID=7f2f7a67-7b97-4d12-99c2-8fdd1d7bcdf0 /emcbakup ext3 defaults 0 0
UUID=5de2da02-0d90-484d-a4e2-2c68c42485c9 /dbfile ext3 defaults 0 0
UUID=498cfb57-c47c-4411-8ccb-406969e51fe4 /dbtempspace ext3 defaults 0 0
UUID=f818248d-3c9a-4473-9358-04967f422633 /newiscsi ext3 defaults 0 0
/dev/backupvg/backup_lv /backup ext4 defaults 0 0
1.4 收集磁盘、卷、文件系统uuid信息
当前vg是由sdd与sdg创建的物理卷组成(当设备绑定多路径后，pvname自动变成多路径设备名，无需对vg进行export与import)
[root@oradb ~]# pvdisplay
--- Physical volume ---
PV Name /dev/sdg
VG Name backupvg
PV Size 2.00 TiB / not usable 4.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 524287
Free PE 0
Allocated PE 524287
PV UUID shnHNI-jo6U-kqm2-qFgY-wPs7-25p7-e2Xq8l
--- Physical volume ---
PV Name /dev/sdd
VG Name backupvg
PV Size 1.00 TiB / not usable 4.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 262143
Free PE 12286
Allocated PE 249857
PV UUID fylY8E-FKuJ-g6yn-EpUS-wYMA-OvuW-seX2oI
[root@oradb ~]# vgdisplay
--- Volume group ---
VG Name backupvg
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 8
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 2
Act PV 2
VG Size 3.00 TiB
PE Size 4.00 MiB
Total PE 786430
Alloc PE / Size 774144 / 2.95 TiB
Free PE / Size 12286 / 47.99 GiB
VG UUID wCOfCY-fuYD-nwEL-EpAQ-wCji-XCtH-GzOY7j
[root@oradb ~]# lvdisplay
--- Logical volume ---
LV Path /dev/backupvg/backup_lv
LV Name backup_lv
VG Name backupvg
LV UUID f5LqSU-Dd6H-VEAq-MhO2-wov8-nk4G-dD38pw
LV Write Access read/write
LV Creation host, time oradb, 2019-06-30 18:26:38 +0800
LV Status available
# open 1
LV Size 2.95 TiB
Current LE 774144
Segments 2
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:0
查看当前文件系统UUID信息，以便后续变更后核对校准
[root@oradb backup]# blkid
/dev/sda3: UUID="94882757-4b38-4318-a08b-6f7f712ab516" TYPE="ext4"
/dev/sda1: UUID="594fa55a-19d0-445f-8e27-4a40b0da0640" TYPE="ext4"
/dev/sda2: UUID="d4a51b71-f7ec-404a-9870-8495c0accf64" TYPE="swap"
/dev/sde1: UUID="f818248d-3c9a-4473-9358-04967f422633" TYPE="ext3"
/dev/sdf1: UUID="498cfb57-c47c-4411-8ccb-406969e51fe4" TYPE="ext3"
/dev/sdb2: UUID="7f2f7a67-7b97-4d12-99c2-8fdd1d7bcdf0" TYPE="ext3"
/dev/sdc1: UUID="5de2da02-0d90-484d-a4e2-2c68c42485c9" TYPE="ext3"
/dev/sdd: UUID="fylY8E-FKuJ-g6yn-EpUS-wYMA-OvuW-seX2oI" TYPE="LVM2_member"
/dev/sdg: UUID="shnHNI-jo6U-kqm2-qFgY-wPs7-25p7-e2Xq8l" TYPE="LVM2_member"
/dev/mapper/backupvg-backup_lv: UUID="9c67c793-9f02-4fd2-834d-bbee0dbc3d57" TYPE="ext4"
当前文件系统挂载信息
[root@oradb ~]# mount
/dev/sda3 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
/dev/sdb2 on /emcbakup type ext3 (rw)
/dev/sdc1 on /dbfile type ext3 (rw)
/dev/sdf1 on /dbtempspace type ext3 (rw)
/dev/sde1 on /newiscsi type ext3 (rw)
/dev/mapper/backupvg-backup_lv on /backup type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
1.5 收集所有数据文件信息(位置，大小，个数，状态)
SQL > select file_name,status from dba_data_files
2.配置多路径
2.1 检查multipath服务
[root@oradb ~]# service multipathd status
multipathd (pid 8105) is running...
如果没安装multipath，可以使用yum自行安装
#yum –y install device-mapper* device-mapper-multipath*
2.2 配置multipath.conf，绑定新设备名
[root@oradb ~]# vi /etc/multipath.conf
blacklist {
devnode "^sda"
}
defaults {
user_friendly_names yes
udev_dir /dev
path_grouping_policy multibus
failback immediate
no_path_retry fail
}
multipaths {
multipath {
wwid 36006016093e24100920c1b587959b56c
alias mulsdb2
path_grouping_policy multibus
path_selector "round-robin 0"
}
multipath {
wwid 36006016093e241007b0c1b583729b956
alias mulsdc1
path_grouping_policy multibus
path_selector "round-robin 0"
}
multipath {
wwid 360060160427045006e1c1baad56be711
alias mulsdf1
path_grouping_policy multibus
path_selector "round-robin 0"
}
multipath {
wwid 36006016093e24100a3595b5b716a5de0
alias mulsde1
path_grouping_policy multibus
path_selector "round-robin 0"
}
multipath {
wwid 36006016093e24100c069285a00444b52
alias mulsdd
path_grouping_policy multibus
path_selector "round-robin 0"
}
multipath {
wwid 36006016042704500966bc2336892e911
alias mulsdg
path_grouping_policy multibus
path_selector "round-robin 0"
}
}
devices {
device {
vendor "COMPELNT"
product "Virtual Disk"
path_grouping_policy multibus
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
path_checker readsector0
path_selector "round-robin 0"
}
}
2.3 重启操作系统
配置好光交与多路径后，重启现有系统
2.4 刷新multipath服务，打印当前多路径信息
系统重启成功后，查看当前多路径信息，如下所示所有设备已经成功绑定
[root@oradb ~]# multipath -v2
[root@oradb ~]# multipath -ll
mulsdf1 (360060160427045006e1c1baad56be711) dm-0 DGC,VRAID
size=600G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw
`-+- policy='round-robin 0' prio=70 status=active
|- 9:0:0:0 sdb 8:16 active ready running
`- 10:0:1:0 sdm 8:192 active ready running
mulsdb2 (36006016093e24100920c1b587959b56c) dm-1 DGC,VRAID
size=1.5T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw
`-+- policy='round-robin 0' prio=30 status=active
|- 8:0:0:0 sdc 8:32 active ready running
`- 10:0:0:0 sdg 8:96 active ready running
mulsdg (36006016042704500966bc2336892e911) dm-4 DGC,VRAID
size=2.0T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw
`-+- policy='round-robin 0' prio=30 status=active
|- 9:0:0:1 sde 8:64 active ready running
`- 10:0:1:1 sdn 8:208 active ready running
mulsde1 (36006016093e24100a3595b5b716a5de0) dm-5 DGC,VRAID
size=1.0T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw
`-+- policy='round-robin 0' prio=70 status=active
|- 8:0:0:3 sdj 8:144 active ready running
`- 10:0:0:3 sdl 8:176 active ready running
mulsdd (36006016093e24100c069285a00444b52) dm-3 DGC,VRAID
size=1.0T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw
`-+- policy='round-robin 0' prio=70 status=active
|- 8:0:0:2 sdh 8:112 active ready running
`- 10:0:0:2 sdk 8:160 active ready running
mulsdc1 (36006016093e241007b0c1b583729b956) dm-2 DGC,VRAID
size=1.5T features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 emc' wp=rw
`-+- policy='round-robin 0' prio=30 status=active
|- 8:0:0:1 sdf 8:80 active ready running
`- 10:0:0:1 sdi 8:128 active ready running
2.5 检查校验多路径状态，多路径新设备挂载信息
当前挂载设备已经变更为多路径设备名,当前的fstab信息是以文件系统的UUID与逻辑卷设备名挂载，当存储变更多路径时，无需对fstab其更改
检查核对所有的信息，保证系统存储使用多路径设备名挂载后的信息与原来的挂载位置保持一致
[root@oradb ~]# mount
/dev/sda3 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
/dev/mapper/mulsdb2p2 on /emcbakup type ext3 (rw)
/dev/mapper/mulsdc1p1 on /dbfile type ext3 (rw)
/dev/mapper/mulsdf1p1 on /dbtempspace type ext3 (rw)
/dev/mapper/mulsde1p1 on /newiscsi type ext3 (rw)
/dev/mapper/backupvg-backup_lv on /backup type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
2.6 检查卷、文件系统UUID信息
所有的卷设备，已经变成多路径的设备名
[root@oradb ~]# pvdisplay
--- Physical volume ---
PV Name /dev/mapper/mulsdg
VG Name backupvg
PV Size 2.00 TiB / not usable 4.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 524287
Free PE 0
Allocated PE 524287
PV UUID shnHNI-jo6U-kqm2-qFgY-wPs7-25p7-e2Xq8l
--- Physical volume ---
PV Name /dev/mapper/mulsdd
VG Name backupvg
PV Size 1.00 TiB / not usable 4.00 MiB
Allocatable yes
PE Size 4.00 MiB
Total PE 262143
Free PE 12286
Allocated PE 249857
PV UUID fylY8E-FKuJ-g6yn-EpUS-wYMA-OvuW-seX2oI
[root@oradb ~]# vgdisplay
--- Volume group ---
VG Name backupvg
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 8
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 2
Act PV 2
VG Size 3.00 TiB
PE Size 4.00 MiB
Total PE 786430
Alloc PE / Size 774144 / 2.95 TiB
Free PE / Size 12286 / 47.99 GiB
VG UUID wCOfCY-fuYD-nwEL-EpAQ-wCji-XCtH-GzOY7j
[root@oradb ~]# lvdisplay
--- Logical volume ---
LV Path /dev/backupvg/backup_lv
LV Name backup_lv
VG Name backupvg
LV UUID f5LqSU-Dd6H-VEAq-MhO2-wov8-nk4G-dD38pw
LV Write Access read/write
LV Creation host, time oradb, 2019-06-30 18:26:38 +0800
LV Status available
# open 1
LV Size 2.95 TiB
Current LE 774144
Segments 2
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:10
文件系统UUID信息
[root@oradb ~]# blkid
/dev/sda3: UUID="94882757-4b38-4318-a08b-6f7f712ab516" TYPE="ext4"
/dev/sda1: UUID="594fa55a-19d0-445f-8e27-4a40b0da0640" TYPE="ext4"
/dev/sda2: UUID="d4a51b71-f7ec-404a-9870-8495c0accf64" TYPE="swap"
/dev/sdb1: UUID="498cfb57-c47c-4411-8ccb-406969e51fe4" TYPE="ext3"
/dev/sdc2: UUID="7f2f7a67-7b97-4d12-99c2-8fdd1d7bcdf0" TYPE="ext3"
/dev/sdh: UUID="fylY8E-FKuJ-g6yn-EpUS-wYMA-OvuW-seX2oI" TYPE="LVM2_member"
/dev/sde: UUID="shnHNI-jo6U-kqm2-qFgY-wPs7-25p7-e2Xq8l" TYPE="LVM2_member"
/dev/sdl1: UUID="f818248d-3c9a-4473-9358-04967f422633" TYPE="ext3"
/dev/sdj1: UUID="f818248d-3c9a-4473-9358-04967f422633" TYPE="ext3"
/dev/mapper/mulsdd: UUID="fylY8E-FKuJ-g6yn-EpUS-wYMA-OvuW-seX2oI" TYPE="LVM2_member"
/dev/mapper/mulsdg: UUID="shnHNI-jo6U-kqm2-qFgY-wPs7-25p7-e2Xq8l" TYPE="LVM2_member"
/dev/mapper/mulsdf1p1: UUID="498cfb57-c47c-4411-8ccb-406969e51fe4" TYPE="ext3"
/dev/mapper/mulsdb2p2: UUID="7f2f7a67-7b97-4d12-99c2-8fdd1d7bcdf0" TYPE="ext3"
/dev/mapper/mulsdc1p1: UUID="5de2da02-0d90-484d-a4e2-2c68c42485c9" TYPE="ext3"
/dev/mapper/mulsde1p1: UUID="f818248d-3c9a-4473-9358-04967f422633" TYPE="ext3"
/dev/mapper/backupvg-backup_lv: UUID="9c67c793-9f02-4fd2-834d-bbee0dbc3d57" TYPE="ext4"
/dev/sdg2: UUID="7f2f7a67-7b97-4d12-99c2-8fdd1d7bcdf0" TYPE="ext3"
/dev/sdi1: UUID="5de2da02-0d90-484d-a4e2-2c68c42485c9" TYPE="ext3"
/dev/sdk: UUID="fylY8E-FKuJ-g6yn-EpUS-wYMA-OvuW-seX2oI" TYPE="LVM2_member"
/dev/sdm1: UUID="498cfb57-c47c-4411-8ccb-406969e51fe4" TYPE="ext3"
/dev/sdn: UUID="shnHNI-jo6U-kqm2-qFgY-wPs7-25p7-e2Xq8l" TYPE="LVM2_member"
You have mail in /var/spool/mail/root
2.7 启动数据库
所有信息检查无误后，启动数据库，检查数据库数据文件大小，状态
本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。
