惠普設備維護培訓日常檢查命令中國惠普 支持效勞事業部QIANYun2004.4編輯課件日常維護檢查工程系統日志

syslog.log，ccerrlog,dmesg系統運行狀態

cmviewcl、bdf、ioscan、vgdisplay、top、sar、swapinfo、netstat磁盤陣列狀態

armdsp–a va arraydsp–a autoraid amdsp–a fc60編輯課件/var/adm/syslog/syslog.log應檢查的內容：有無SCSIReset告警〔啟動前后的SCSIreset信息可忽略〕有無EMS告警。如：Aug1209:43:05bj_rz3EMS[2286]:EMSEventNotificationValue:"SERIOUS(4)"forResource:"/system/events/core_hw/core_hw"(Threshold:>="3")Executethefollowingcommandtoobtaineventdetails:/opt/resmon/bin/resdata-R149815298-r/system/events/core_hw/core_hw-n149815299-a但凡Value為Majorwarning、Serious或Critical的報警都應關注。有無“PVPowerfail〞、〞IOerror〞報錯LPMC如有重起操作，建議保存當前的syslog日志Syslog是系統重起到當前的日志，重起后將自動保存為OLDsyslog.log編輯課件ccerrlog283

PM

0

*6

08/17/200319:16:58

LogEntry283:08/17/200319:16:58

AlertLevel6:Systemcouldfail-attentionrequired;Keyword:

Bulkpowersupply(BPS)2failed;Status:15

Loggedbypowermonitor0duringmonitoringoflowvoltagepowersupply

0x0020016a4402404f0x0000000000000000

0x5820216a4402404f0x000067071113103a

執行cclogview/var/stm/logs/os/ccerrlog，可以通過telnet檢查GSP/MP里的告警日志情況。應注意檢查是否有AlertLevel大于等于2的新條目編輯課件dmesg…… $Revision:vmunix:vw:-projselectors:CUPI80_BL2000_1108-c'VwforCUPI80_BL2000_1108build'--cupi80_bl2000_1108'CUPI80_BL2000_1108'WedNov819:24:56PST2000$MemoryInformation:physicalpagesize=4096bytes,logicalpagesize=4096bytesPhysical:4177920Kbytes,lockable:3859368Kbytes,available:3859944KbytesUsing3162bufferscontaining24576Kbytesofmemory.駐留在內存中的系統最近一段時間的日志信息：常見的異常信息：SCSIResetDetectedLPMCI-CacheerrorFileSystemFull發現后應及時觀察syslog.log中的相應條目編輯課件cmviewclCLUSTERSTATUShpclusterup

NODESTATUSSTATEGMS_STATEbjscp1aup

runninghaltedNetwork_Parameters:INTERFACESTATUSPATHNAMEPRIMARYup0/5/0/0lan1PRIMARYup0/0/0/0lan0STANDBYup1/12/0/0lan2PACKAGESTATUSSTATEAUTO_RUNNODEscppkgup

running

enabledbjscp1a……NODESTATUSSTATEGMS_STATEbjscp1bup

runninghaltedNetwork_Parameters:INTERFACESTATUSPATHNAMEPRIMARYup0/5/0/0lan1STANDBYup1/12/0/0lan2PRIMARYup0/0/0/0lan0觀察雙機狀態，執行cmviewcl–v，確認STATUS和STATE為up和running，同時包自動切換〔AUTO_RUN〕屬性為enable編輯課件bdfFilesystemkbytesusedavail%usedMountedon/dev/vg00/lvol32048004816815542424%//dev/vg00/lvol12950243885622666415%/stand/dev/vg00/lvol847063041523976315759233%/var/dev/vg00/lvol7116326470830445146461%/usr/dev/vg00/lvol42048009640810756847%/tmp/dev/vg00/lvol6104857676602428036073%/opt/dev/vg00/lvol51048576445610360240%/home檢查文件系統的使用率應檢查有無使用率大于90%的文件系統編輯課件ioscan-fnClassIH/WPathDriverS/WStateH/WTypeDescription======================================================================================root0rootCLAIMEDBUS_NEXUSioa00sbaCLAIMEDBUS_NEXUSSystemBusAdapter(803)ba00/0lbaCLAIMEDBUS_NEXUSLocalPCIBusAdapter(782)lan00/0/0/0btlan3 CLAIMED INTERFACEHPPCI10/100Base-TXCore/dev/diag/lan0/dev/ether0ext_bus00/0/1/0c720CLAIMEDINTERFACESCSIC895UltraWideSingle-Endedtarget00/0/1/0.1tgtCLAIMEDDEVICEdisk00/0/1/0.1.0sdiskNO_HWDEVICEHPDVD-ROM305/dev/dsk/c0t1d0/dev/rdsk/c0t1d0檢察IO設備是否正常應檢查有無狀態為NO_HW的設備編輯課件vgdisplayVolumegroupsVGName/dev/vg00VGWriteAccessread/writeVGStatusavailable

MaxLV255……LogicalvolumesLVName/dev/vg00/lvol1LVStatusavailable/syncd

LVSize(Mbytes)100CurrentLE25AllocatedPE50UsedPV2……PhysicalvolumesPVName/dev/dsk/c4t0d0PVName/dev/dsk/c6t0d0 AlternateLinkPVStatusavailable

TotalPE12992FreePE0AutoswitchOff顯示卷組狀態，重點檢查vg00，執行vgdisplay–vvg00檢查各項status值為available/sync，不是stale編輯課件topCPULOADUSERNICESYSIDLEBLOCKSWAITINTRSSYS00.2820.2%0.0%2.6%77.2%0.0%0.0%0.0%0.0%10.1714.6%0.0%3.4%82.0%0.0%0.0%0.0%0.0%20.3318.6%0.0%3.0%78.4%0.0%0.0%0.0%0.0%30.2013.0%0.0%4.2%82.8%0.0%0.0%0.0%0.0%40.1114.4%0.0%2.0%83.6%0.0%0.0%0.0%0.0%50.4419.8%0.0%4.2%76.0%0.0%0.0%0.0%0.0%60.2813.2%0.0%11.2%75.6%0.0%0.0%0.0%0.0%70.1714.8%0.0%1.8%83.4%0.0%0.0%0.0%0.0%

avg0.250.0%0.0%0.0%100.0%0.0%0.0%0.0%0.0%Memory:1106604K(999800K)real,1527608K(1362680K)virtual,1987924KfreePage#1/6CPUTTYPIDUSERNAMEPRINISIZERESSTATETIME%WCPU%CPUCOMMAND2?18777informix156207404K5052Ksleep9233:0230.4930.43oninit6?19002tellin1542029248K22572Ksleep5256:0317.0517.02manager0?18779informix156207404K4784Ksleep1681:279.629.60oninit觀察CPU和內存使用情況重點檢查有無占用CPU過大的進程，并檢查freememory是否足夠編輯課件sar-u10:02:18cpu%usr%sys%wio%idle10:02:2103721601185175215102723942854213175523247061043837155179system195275觀察CPU使用情況:sar–u–M310重點檢查%idle是否足夠〔一般不小于25%)編輯課件sar-vHP-UXbjscp1aB.11.00U9000/80007/07/0310:02:48text-szovproc-szovinod-szovfile-szov10:02:51N/AN/A189/66402119/736001127/12021010:02:54N/AN/A188/66402102/736001121/12021010:02:57N/AN/A187/66402067/736001114/12021010:03:00N/AN/A187/66402037/736001108/12021010:03:03N/AN/A187/66402033/736001108/12021010:03:06N/AN/A187/66402036/736001108/12021010:03:09N/AN/A187/66402033/736001108/12021010:03:12N/AN/A188/66402032/736001113/12021010:03:15N/AN/A187/66402032/736001108/12021010:03:18N/AN/A187/66402032/736001108/120210觀察文件線程資源使用情況：sar–v310重點檢查有無即將到達上限的值編輯課件sar-dHP-UXbjscp1aB.11.00U9000/80007/07/0310:03:18device%busyavquer+w/sblks/savwaitavserv10:03:21c1t6d04.330.507495.975.57c2t6d03.670.506435.864.78c4t0d01.000.5010515.112.94c4t0d11.670.5011534.493.27c4t0d21.670.5010525.162.63c4t0d31.670.5016755.012.97觀察IO使用情況:sar–d310重點檢查有無%busy過大的設備編輯課件swapinfoMbMbMbPCTSTART/MbTYPEAVAILUSEDFREEUSEDLIMITRESERVEPRINAMEdev3072030720%0-1/dev/vg00/lvol2dev3000030000%0-0/dev/vg00/lv_swapreserve-2161-2161total60722161391136%-0-觀察交換區使用情況通常swap區的使用率為0，如有0以上數值，需進行進一步檢查編輯課件netstat-inNameMtuNetworkAddressIpktsOpktslan1*1500192.9.200.0192.9.200.100lan0150015.79.48.015.79.48.170745893334436lo04136127.0.0.0127.0.0.12654026540觀察網絡連接情況，檢查有無網絡連接中斷，執行netstat-in如在網卡后帶*號那么表示網絡不通編輯課件AutoRAID狀態觀察1執行：arraydsp–i確認VA的別名〔如auto1〕執行：arraydsp–aauto1>auto1.dsp執行vi檢查生成的信息文件auto1.dsp，確認ArrayState為Ready______________________________________________________________________VendorID=HPProductID=C5447AArrayserialnumber=000000261E8B

ArrayState=READYServername=mscp1Arraytype=3Mfg.ProductCode=IJMTU00004DiskspaceusageTotalphysical=121565MB*AllocatedtoLUNs=28700MB*UsedasActiveHotspare=17366MB*Usedbynon-includeddisks=0MB*UsedforRedundancy=23727MB*Unallocated(availforLUNs)=51772MB*

編輯課件AutoRAID狀態觀察2……VendorID=HPProductID=C5447AArrayserialnumber=000000261E8B

FanF1=GOODFanF2=GOODFanF3=GOODPowersupplyPS1=GOODPowersupplyPS2=GOODPowersupplyPS3=GOODControllerX:Overallstate=GOODBattery#0state=GOODBattery#1state=GOODDRAM#0state=GOODNVRAM#0state=GOODNVRAM#1state=GOODControllerY:Overallstate=GOODBattery#0state=GOODBattery#1state=GOODDRAM#0state=GOODNVRAM#0state=GOODNVRAM#1state=GOOD編輯課件FC60狀態觀察1執行：amdsp–i確認VA的別名〔如fc1〕執行：amdsp–afc1>fc1.dsp執行vi檢查生成的信息文件fc1.dsp，確認ArrayState為Ready______________________________________________________________________VendorID=HPProductID=A5277AArrayID=000400A0B80942BCArrayAlias=fc1

ArrayState=READYServerName=bjscp1aArrayType=3Mfg.ProductCode=348-0040789Date=TueApr2223:32:52EAT2003PathtoCtlrAVSB0=0/4/0/0.8.0.5.0.0.0PathtoCtlrBVSB0=1/10/0/0.8.0.4.0.0.0DiskspaceusageTotalPhysical=440.9GBAllocatedtoLUNs=220.0GBUsedasHotSpare=0.0GBUnallocated(availforLUNs)=0.0GB

編輯課件FC60狀態觀察2…VendorID=HPProductID=A5277AArrayID=000400A0B80942BCArrayAlias=fc1

LUNStatusCapacityCtrlRAIDSegmentDisks