1、vplex 實施方案 vplex local 實施方案實施方案 2011-12-07 vplex 實施方案 emc 公司 2 目目 錄錄 1項目背景項目背景.3 2現有環境現有環境.4 2.1目前拓撲結構.4 2.2設備信息.4 2.2.1主機信息.4 2.2.2san交換機.4 2.2.3存儲系統.5 3vplex local 方案設計方案設計.6 3.1總體規劃.6 3.2環境準備及要求.6 3.2.1vplex物理要求.6 3.2.2生產主機.6 3.2.3存儲設備.6 3.2.4san交換機.7 3.3vplex 規劃.7 3.3.1ip 規劃.7 3.3.2vplex 端口規劃.7

2、3.4生產主機.7 3.5san 交換機.8 3.5.1port規劃.8 3.5.2zone規劃.9 3.6存儲陣列.10 3.6.1vnx5300 lun規劃.10 4實施步驟實施步驟.11 4.1配置 vplex local.11 4.2配置 vplex 后端口zone.20 4.3配置 vnx5300.21 4.4創建 metavolume.21 4.5vplex 封裝 lun.23 4.6創建 loggingvolume.24 4.7升級主機 powerpath.25 4.8配置 vplex 前端口zone.25 4.9創建 storage-view.25 4.10驗證數據的完整性.2

3、6 4.11安全回退.26 4.12恢復 vplex 架構.27 5附：附：.28 5.1vplex 管理方法.28 vplex 實施方案 emc 公司 3 5.2物理架構.30 vplex 實施方案 emc 公司 4 1 項目背景項目背景 隨著業務不斷擴大，信息系統顯得更加重要。目前有 tribon 系統、文件服務器、 sql server 數據庫等應用系統。操作系統均為 windowns2003。通過兩臺光線交換機與 cx500、vnx5300 連接。cx500 存儲原來的老數據，現應用系統數據已經全部遷移到 vnx5300。 本項目中 emc 提供存儲虛擬化 vplex local，通過

4、在現有的 san 環境加入 vplex 設備，來實現對 vnx5300 的統一管理，以便將來擴展存儲，進一步實現存儲間的 數據遷移、存儲間的 mirror。 vplex 實施方案 emc 公司 5 2 現有環境現有環境 2.1 目前拓撲結構 2.2 設備信息 經過前期信息調研，設備信息統計如下： 2.2.1主機信息主機信息 tribon serverfile server oswindows2003windows2003 ip10.1.1.9210.1.1.6 loadbalancepowerpath5.0.0.94powerpath5.0.0.94 hba1_ wwn21:00:00:24:

5、ff:04:43:f621:00:00:24:ff:04:43:b2 hba2_ wwn21:00:00:24:ff:04:43:2421:00:00:24:ff:04:94:1a clustern/an/a 2.2.2san 交換機交換機 switchnameds_200bds_300b vplex 實施方案 emc 公司 6 modelemc ds200bemc ds300b firmwarev5.2.1v6.3.1a ip10.1.1.17210.1.1.12 available ports910 speed4gb8gb 2.2.3存儲系統存儲系統 storagevnx5300 mode

6、lemc vnx5300 code31 ip10.1.1.25 available cap.2tb vplex 實施方案 emc 公司 7 3 vplex local 方案設計方案設計 3.1 總體規劃總體規劃 本項目 vplex 配置為兩個引擎，硬件版本 vs2，軟件版本 5.01。最終實現用戶原有 存儲 vnx5300 被 vplex 統一管理，但需要保證用戶可以安全退回無 vplex 架構。所以 在創建 device 時需要采用 1:1 方式，以便用戶可以安全回退到主機與存儲通過交換機的直 連架構，保證數據完整可用。 3.2 環境準備及要求環境準備及要求 3.2.1vplex 物理要求物

7、理要求 準備兩路 220-240v、32a 航空電源插頭供 vplex 機柜使用 光纖延時不得超過 2ms 準備一根網線用于 vplex 管理 準備一個 ip 地址用于 vplex 管理 準備 8 根光纖線用于 vplex engine1、engine2 前端口連接 ds200b、ds300b 交換機 準備 8 根光纖線用于 vplex engine1、engine2 后端口連接 ds200b、ds300b 交換機 3.2.2生產主機生產主機 多路徑管理軟件為 powerpath5.0 需要升級 powerpath5.3 或以上 3.2.3存儲設備存儲設備 準備一跟光纖跳線用于 vnx5300

8、 到 ds200b 連接 需要劃分 4*80gb lun 做為 meta-volume 需要劃分 1*10gb lun 做為 logging-volume 以備將來使用 vplex 實施方案 emc 公司 8 3.2.4san 交換機交換機 兩臺 san 交換機分別需要為 vplex 預留 6 個端口，其中 4 個為后端口，2 個前端口 需要增加 vplex 后端口與存儲的 zone，均為 wwn zone 需要增加 vplex 前端口與主機的 zone，均為 wwn zone 兩臺冗余交換機分別為 200b（4gb）與 300b（8gb） ，需要將 300b 交換機降至 4gb 3.3 vp

9、lex 規劃規劃 3.3.13.3.1ipip 規劃規劃 vplex hostnamebh ip10.1.1.18 netmask255.0.0.0 gateway 10.1.1.150 usernameservice passwordmidim7t 3.3.23.3.2vplexvplex 端口規劃端口規劃 enginedirectorfrontportbackport a00,21 b00,2 a00,22 b00,2 3.4 生產主機生產主機 在配置 vplex 時需要為生產主機申請停機時間，以便停止業務后通過 vplex 重新認盤 applicationtribon serverfil

10、e server hostname bhms5tribondata ip10.1.1.9210.1.1.6 loadbalancepowerpath5.3sp1powerpath5.3sp1 hba1_wwn 21:00:00:24:ff:04:43:f621:00:00:24:ff:04:43:b2 hba2_wwn 21:00:00:24:ff:04:43:2421:00:00:24:ff:04:94:1a vplex 實施方案 emc 公司 9 3.5 san 交換機交換機 3.5.1port 規劃規劃 switch nameds200b switch domain id1 switch

11、 ip10.1.1.172 switch firmwarev6.3.1a switch license connectionwwpnswitch port cx500 50:06:01:68:30:21:23:e1 1,0 1,1 bhms3 21:00:00:e0:8b:91:7a:ee 1,2 1,3 cx500 50:06:01:60:30:21:23:e1 1,4 ksprint 21:00:00:e0:8b:91:61:5c 1,5 1,6 bhms5 21:00:00:24:ff:04:43:24 1,7 tribondata 21:00:00:24:ff:04:94:1a 1,8

12、 vplex_e1bf0 50:00:14:42:70:73:26:00 1,9 vplex_e2ab3 50:00:14:42:60:77:6d:12 1,10 vplex_e1ab3 50:00:14:42:60:73:26:12 1,11 vnx5300 50:06:01:68:3e:a0:0f:39 1,12 vplex_e2bf0 50:00:14:42:70:77:6d:00 1,13 vplex_e2bb3 50:00:14:42:70:77:6d:12 1,14 vplex_e1bb3 50:00:14:42:70:73:26:12 1,15 switch nameds300b

13、 switch domain id1 switch ip10.1.1.12 switch firmwarev6.3.1a switch license connectionwwpnswitch port 1,0 帶庫 50:03:08:c3:8b:5b:20:05 1,1 i40 21:00:00:1b:32:10:90:cd 1,2 vnx5300 50:06:01:60:3e:a0:0f:39 1,3 bhms5 21:00:00:24:ff:04:43:f6 1,4 triondata 21:00:00:24:ff:04:43:b2 1,5 1,6 1,7 vplex 實施方案 emc

14、公司 10 1,8 vplex_e1af0 50:00:14:42:60:73:26:00 1,9 vplex_e2ab0 50:00:14:42:60:77:6d:10 1,10 vplex_e1ab0 50:00:14:42:60:73:26:10 1,11 1,12 vplex_e2af0 50:00:14:42:60:77:6d:00 1,13 vplex_e2bb0 50:00:14:42:70:77:6d:10 1,14 vplex_e1bb0 50:00:14:42:70:73:26:10 1,15 3.5.2zone 規劃規劃 switchnameds200b zonename

15、aliasportwwn e1ab3 1,11 50:00:14:42:60:73:26:12e1ab3_vnx2 vnx2 1,12 50:06:01:68:3e:a0:0f:39 e1bb3 1,15 50:00:14:42:70:73:26:12e1bb3_vnx2 vnx2 1,12 50:06:01:68:3e:a0:0f:39 e2ab3 1,10 50:00:14:42:60:77:6d:12e2ab3_vnx2 vnx2 1,12 50:06:01:68:3e:a0:0f:39 e2bb3 1,14 50:00:14:42:70:77:6d:12e2bb3_vnx2 vnx2

16、1,12 50:06:01:68:3e:a0:0f:39 e1bf01,9 50:00:14:42:70:73:26:00 tribondata_e1bf0 tribondata1,8 21:00:00:24:ff:04:94:1a e2bf01,13 50:00:14:42:70:77:6d:00 triibondata_e2bf0 tribondata1,8 21:00:00:24:ff:04:94:1a e1bf01,9 50:00:14:42:70:73:26:00 bhms5_e1bf0 bhms51,7 21:00:00:24:ff:04:43:24 e2bf01,13 50:00

17、:14:42:70:77:6d:00 bhms5_e2bf0 bhms51,7 21:00:00:24:ff:04:43:24 cx5001,0 50:06:01:68:30:21:23:e1 cx5001,4 50:06:01:60:30:21:23:e1 filezone tribondata1,8 21:00:00:24:ff:04:94:1a cx5001,0 50:06:01:68:30:21:23:e1 bhms51,7 21:00:00:24:ff:04:43:24 bhms5zone cx5001,4 50:06:01:60:30:21:23:e1 switchnameds30

18、0b zonenamealiasportwwn e1ab1 1,11 50:00:14:42:60:73:26:10e1ab1_vnx1 vnx1 1,0 50:06:01:60:3e:a0:0f:39 e1bb1 1,15 50:00:14:42:70:73:26:10e1bb1_vnx1 vnx1 1,0 50:06:01:60:3e:a0:0f:39 vplex 實施方案 emc 公司 11 e2ab1 1,10 50:00:14:42:60:77:6d:10e2ab1_vnx1 vnx1 1,0 50:06:01:60:3e:a0:0f:39 e2bb1 1,14 50:00:14:4

19、2:70:77:6d:10e2bb1_vnx1 vnx1 1,0 50:06:01:60:3e:a0:0f:39 1,1 50:03:08:c3:8b:5b:20:05i40_quantum 1,2 21:00:00:1b:32:10:90:cd trondata1,5 21:00:00:24:ff:04:43:b2 trondata_e1af0 e1af01,9 50:00:14:42:60:73:26:00 trondata1,5 21:00:00:24:ff:04:43:b2 trondata_e2af0 e2af01,13 50:00:14:42:60:77:6d:00 bhms51,

20、4 21:00:00:24:ff:04:43:f6 bhms5_e1af0 e1af01,9 50:00:14:42:60:73:26:00 bhms51,4 21:00:00:24:ff:04:43:f6 bhms5_e2af0 e2af01,13 50:00:14:42:60:77:6d:00 3.6 存儲陣列存儲陣列 3.6.1vnx5300 lun 規劃規劃 主機應用大小(gb)數量lun idraid meta volume8048,9,10,115 logging volume101125 bhms54818115 tribondata2300125 tribondata50013

21、5 tribondata500145 tribondata1500155 tribondata1247165 tribondata912175 vplex 實施方案 emc 公司 12 4 實施步驟實施步驟 實施前確保存儲上所有數據均做全備份。 4.1 配置配置 vplex local 1.將 vplex 兩個引擎的前后端口連入 ds200b、ds300b 交換機，需要 8 根光纖線 2.將筆記本 ip 設置成為 128.221.252.3，netmask255.255.255.248 然后 ssh 到 128.221.252.2 配置 vplex。用戶名：service 口令：midim7t

22、 3.更改管理 ip 及 hostname servicemanagementserver: sudo /opt/emc/vplex/tools/ipconfig/changehostname.py -n bh servicemanagementserver: vplexcli trying 127.0.0.1. connected to localhost. escape character is . vplex 實施方案 emc 公司 13 username: service password: creating logfile:/var/log/vplex/cli/session.log

23、_service_localhost_t29452_20111102080052 vplexcli:/ management-server set-ip -i 10.1.1.18:255.0.0.0 -g 10.1.1.150 eth3 4.使用 vplex 管理 ip，重新連接并登錄 5.檢查 vplex 健康情況 servicebh: vplexplatformhealthcheck system information - dual engine(medium config)system detected management server ip connectivity check -

24、 port plugged : ok ip interfaces : ok ip connectivity to directors check : ok local-com fc connectivity check - fc switch to director connectivity check : ok management server system check - process check : ok check partitions : ok cpu check : ok baseboard management control check : ok director (eng

25、ine-1-1 director 1a 128.221.252.35) health check - process check: ok cpu check: ok ssd check: ok partition check: ok rpm check: ok flashdir check: ok wwn seed check: ok health check: ok hardware module check: ok director (engine-1-1 director 1b 128.221.252.36) health check vplex 實施方案 emc 公司 14 - pro

26、cess check: ok cpu check: ok ssd check: ok partition check: ok rpm check: ok flashdir check: ok wwn seed check: ok health check: ok hardware module check: ok director (engine-1-2 director 2a 128.221.252.37) health check - process check: ok cpu check: ok ssd check: ok partition check: ok rpm check: o

27、k flashdir check: ok wwn seed check: ok health check: ok hardware module check: ok director (engine-1-2 director 2b 128.221.252.38) health check - process check: ok cpu check: ok ssd check: ok partition check: ok rpm check: ok flashdir check: ok wwn seed check: ok health check: ok hardware module ch

28、eck: ok 6.配置 vplex 注：過程中涉及到的口令均為：midim7t；驗證碼：strongswan vplexcli:/management-server/ports configuration system-setup verifying the pre-requisites for ez-setup. verification of the pre-requisites for ez-setup successful initializing cst. cstadmin: object pam vplex-pam-authority configured. cst initia

29、lization complete welcome to the vplex ez-setup wizard vplex 實施方案 emc 公司 15 the setup wizard collects information about your configuration and uses it to set up your vplex implementation. type exit at any prompt to stop. your answers will be saved so you can restart the process if needed. before sta

30、rting, please read the vplex installation and setup guide and the release notes. continue? (yes/no) yes: select a configuration option what would you like to configure? 1. configure a vplex local (single cluster). 2. configure a vplex metro (synchronous, 2 clusters). 3. configure a vplex geo (asynch

31、ronous, 2 clusters). select your configuration choice. (1-3): 1 you have selected to configure the only cluster of a vplex local. if you want to configure a vplex metro or geo, please select option 2 or 3. are you sure you want to configure a vplex local? (yes/no): yes configure authentication servi

32、ce provider (optional) you may select to use your existing ldap or active directory as a directory service to authenticate vplex users. to configure this, you will need the authentication service provider server information, and the security information to map the users. or, you may choose not to co

33、nfigure an authentication service provider at this time. you may configure an authentication service provider for authentication at any time, using vplex cli commands. vplex 實施方案 emc 公司 16 would you like to configure an authentication service provider to authenticate vplex users? (yes/no) no: select

34、 emc notification options by configuring emc event notifications, emc will be able to better serve you. would you like to configure this cluster to send event notifications to emc? (yes/no) yes: no by sending emc system reports, emc can proactively communicate known configuration risks to you. any n

35、ewly discovered information that can optimize or reduce risks to your system can also be communicated. would you like to configure this cluster to send system reports to emc? (yes/no) yes: no configure snmp to collect performance statistics the system can be configured to collect vplex performance s

36、tatistics via the snmp protocol. statistics such as i/o operations and latencies as well as director memory statistics can be collected by issuing snmp get, get-next or get-bulk messages to the snmp agent. additional details on the values collected can be found in the vplex installation and setup gu

37、ide. before you get started you will need the community string for snmp get,get-next and get-bulk messages. the default authorization level is read-only. would you like to configure the cluster to run the snmp agent service? (yes/no) no: specify certificate configuration vplex 實施方案 emc 公司 17 a certi

38、ficate authority is needed to create a digital host certificate for the management server. for security reasons, this certificate must expire in 5 years or less. for more information, see the vplex security configuration guide. a host certificate will be created to configure the web server. it also

39、secures communication between management servers for vplex geo configurations. for security reasons, this certificate must expire in 2 years or less. for more information, see the vplex security configuration guide. in addition to specifying certificate expirations, you will need to create certifica

40、te passphrases. these passphrases will be needed later in the setup process, so please make a note of it. how many years should the ca certificate remain valid before expiring? emc recommends 5 years (maximum is 5 years). 5: how many years should the host certificate remain valid before expiring? em

41、c recommends 2 years (maximum is 2 years). 2: review and finish review the configuration information below. if correct, enter yes (or simply accept the default and press enter) to start the setup process. if the values are not correct, enter no to go back and make changes or to exit the setup. clust

42、er information this is the only cluster in a vplex local. authentication service provider information an authentication service provider will not be configured on this cluster. performance statistics the performance statistics will not be configured. security certificate information the certificate

43、authority will expire in 5 years. vplex 實施方案 emc 公司 18 the host certificate will expire in 2 years. would you like to run the setup process now? (yes/no): yes setting up environment for certificate creation. certificate creation started connecting to directors connected to plex firmware director-1-1

44、-a. connected to plex firmware director-1-1-b. connected to plex firmware director-1-2-a. connected to plex firmware director-1-2-b. connection to directors successful please create a passphrase (at least 8 characters) for the certificate authority key. make a note of this passphrase as you will nee

45、d it to configure a second cluster of a vplex metro or geo. certificate authority passphrase: the password must be at least 8 characters long. please create a passphrase (at least 8 characters) for the certificate authority key. make a note of this passphrase as you will need it to configure a secon

46、d cluster of a vplex metro or geo. certificate authority passphrase: re-enter the passphrase for the certificate authority key: new ca certificate /etc/ipsec.d/cacerts/strongswancert.pem created new ca key /etc/ipsec.d/private/strongswankey.pem created ca certificate created successfully please crea

47、te a passphrase (at least 8 characters) for the local host certificate key. make a note of this passphrase as you will need it later. vplex 實施方案 emc 公司 19 host certificate passphrase: re-enter the passphrase for the certificate key: new host certificate request /etc/ipsec.d/reqs/webserverhostcertreq

48、.pem created new host certificate /etc/ipsec.d/certs/webserverhostcertfile.pem created and signed by the ca certificate /etc/ipsec.d/cacerts/strongswancert.pem web server host certificate created successfully new host certificate request /etc/ipsec.d/reqs/hostcertreq.pem created new host certificate

49、 /etc/ipsec.d/certs/hostcert.pem created and signed by the ca certificate /etc/ipsec.d/cacerts/strongswancert.pem vpn host certificate created successfully validating that the certificate authority and the host certificates have been created. certificate creation is completed successfully java keyst

50、ore setup started. verifying that all the pre-conditions for setting up java keystore are satisfied. verification of the pre-conditions for the run complete. creating java key store. your java key store has been created. java keystore setup is completed successfully setting up environment for ema ad

51、aptor configuration. directors are connected ema adaptor configuration started. verifying that all the pre-conditions for configuring the ema adaptor are satisfied. verifying that the ema adaptor file is present. verifying that the cluster tla is set. verification of the pre-conditions for the run c

52、omplete. configuring ema adaptor. ema adaptor configuration is completed successfully setup : disconnecting from all directors disconnected from remote systems director-1-1-a, director-1-1-b, director-1-2-a, director-1-2-b. running the basic setup task verifying that the system is configured properl

53、y. connecting to all the directors and validating them. connected to plex firmware director-1-1-a. vplex 實施方案 emc 公司 20 connected to plex firmware director-1-1-b. connected to plex firmware director-1-2-a. connected to plex firmware director-1-2-b. setting the cluster id on the directors and validat

54、ing them. successfully updated cluster-id for director: director-1-1-a. successfully updated cluster-id for director: director-1-1-b. successfully updated cluster-id for director: director-1-2-a. successfully updated cluster-id for director: director-1-2-b. setting the director count and validating

55、it. successfully updated director-count for director: director-1-1-a. successfully updated director-count for director: director-1-1-b. successfully updated director-count for director: director-1-2-a. successfully updated director-count for director: director-1-2-b. restarting geosynchrony software

56、 on all directors. this may take a few moments. for /engines/engine-1-2/directors/director-1-2-a: status description - - started. for /engines/engine-1-2/directors/director-1-2-b: status description - - started. for /engines/engine-1-1/directors/director-1-1-a: status description - - started. for /e

57、ngines/engine-1-1/directors/director-1-1-b: status description vplex 實施方案 emc 公司 21 - - started. waiting for firmware to start. commissioning all the directors. this may take a few moments. writing to /opt/vs1/backup/vs1_gwext.ini file. exiting esrs configuration. please enter the service account pa

58、ssword for the management server: please enter the service account password for the fibre channel switch at ip 128.221.252.34: executing on the first switch. this will take a few moments. please enter the service account password for the fibre channel switch at ip 128.221.253.34: executing on the fi

59、rst switch. this will take a few moments. enabling com and back-end ports. validating the health status of the directors. this may take a few moments. setting the director product revision. basic setup task is completed successfully. configuration complete! the setup wizard has completed the automat

60、ed portion of configuring your cluster. from this point, please follow the manual procedures defined in the installation and setup guide. 4.2 配置配置 vplex 后端口后端口 zone ds200b ds_200b:admin alicreate e1ab3,50:00:14:42:60:73:26:12 vplex 實施方案 emc 公司 22 ds_200b:admin alicreate e1bb3,50:00:14:42:70:73:26:12