大話Docker系列（三）：容器網絡通信實驗SDN哪家強?15-02-09-5,911人圍觀1綜述前文已經對Docker的基本環境進行構建，本文主要通過實驗驗證Docker的bridge網絡模式在宿主機內的容器間通信和利用OpenvSwitch和Docker的none網絡模式實現跨宿主機間的容器通信，從而加深對Docker容器間網絡通信的理解。2環境準備2.1操作系統root@ubuntu:~#lsb_release-aNoLSBmodulesareavailable.DistributorID:UbuntuDescription:Ubuntu14.04.1LTSRelease:14.04Codename:trusty注建議在Ubuntu14.04.1做下面的實驗在Ubuntu12.04.5T安裝使用Docker時比較麻煩。2.2主要組件用于完成下面實驗的相關組件。2.2.1Docker1.4使用最新版本Docker1.4安裝方法（以下安裝使用均在root用戶權限下完成）:Shell———Japt-getupdateapt-getinstalldocker.io2source/etc/bash_completion.d/docker.ioapt-keyadv--keyserverhkp://:80--recv-keys436A1D7869245C8950F966E92D8576A8BA88D21E95sh-c"echodeb/ubuntudockermain\6>/etc/apt/sources.list.d/docker.list”7apt-getupdate8apt-getinstalllxc-docker2.2.2Openvswitch2.3.0使用最新版本的openvswitch2.3.0安裝方法：自行獲取官網的openvswitch2.3.10.tar.gz包開始安裝Shell./configure--with-linux=/lib/modules/'uname-r'/build2>/dev/nullMakeMakeinstallmakemodules_install/sbin/modprobeopenvswitchmkdir-p/usr/local/etc/openvswitchovsdb-toolcreate/usr/local/etc/openvswitch/conf.dbvswitchd/vswitch.ovsschemaovsdb-server--remote=punix:/usr/local/var/run/openvswitch/db.sock\--remote=db:Open_vSwitch,Open_vSwitch,manager_options\--private-key=db:Open_vSwitch,SSL,private_key\--certificate=db:Open_vSwitch,SSL,certificate\--bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert\--pidfile--detachovs-vsctl--no-waitinitovs-vswitchd--pidfile--detach2.3ovs-dockerDocker公司開發的一款開源的工具腳本，用于簡化操作ovs和Docker之間關聯使用。類似工具還有pipwork。ShelldI 頊1wget/openvswitch/ovs/raw/master/utilities/ovs-dockerchmoda+xovs-docker安裝pipeworkgitclone/jpetazzo/pipeworkcp~/pipework/pipework/usr/local/bin/3實驗3.1宿主機內容器通信Docker在啟動容器時，如果不指定網絡模式選項時，會創建名docker0的虛擬網橋，通過該網橋實現與宿主機之間的通信。3.1.1實驗目的驗證在一個宿主機內部，容器1與容器2之間如何通信。3.1.2實驗拓撲h<JSt：192.1&a.5.313.1.3實驗方法假設你已經有Ubuntu14的鏡像如果沒有可以通過dockerpullubuntu:latest下載一個。1.啟動容器c1:

1dockerrun-t-i--namecl1:5000/sdnpool/ubuntu14.04:latest/bin/bash1dockerrun-t-i--namecl1:5000/sdnpool/ubuntu14.04:latest/bin/bash2.查看Docker默認的網橋分配的網卡信息root@4c7ee3ed!49e:/flifconflgethoLinkencap:EthernetHWaddr92:42:ac:ll:00:03inet Mask;255?2559inet6dddr:fe80::42:acff:fell:3/64Scope:LinkUPBROADCAST

RXpacketsi6

TXpackets:6RUNNINGMTU1UPBROADCAST

RXpacketsi6

TXpackets:6RUNNINGMTU11509errorsi3dropped:9errors:0dropped:0Metrici1overruns;0framei0overruns:0carrier:0collisions:collisions:0

RXbytes:508（508.9B）TXbytes：598（50S.OB）Lo Linkencop:LocalLoopbackinetaddril27.0.O.1Maskiinet6addr;;;1/128Scope:HostUPLOOPBACKRUNNINGMTU：65536MetricRXpackets:0errors:3dropped:0overruns:□frame:0TXpacketsi9errorsi3dropped:^overrunsi0carrter:0collisions:0txqueuelen;0RXbytes:O（DMB）TXbytesrO（0.0B）3.啟動容器。2,并關聯容器c1,link的別名為c2-c11sudodockerrun-t-i--namec2--linkc1:c2-c11:5000/sdnpool/ubuntu14.04:latest/bin/bashroot@e7528f4ftfconfigethQLinkencapiEthernetHWaddr02:42iaci11;90^95inetaddr:172.17^,5 Mesk:255x255.0.0tnet6addr:fe80::42:acff:fell:5/64Scope:LinkUPBROADCASTRXpackets:6TXpackets:UPBROADCASTRXpackets:6TXpackets:6collistons:0RXbytes:50&RUNNINGMTU：1500errors:adropped;0errors:3dropped:0txqueuelen:19^9Metric:1overruns:&frane:aoverruns:0carrter:0C598.0B)TXbytes;509(508.0B)Lo LinkencapiLocalLoopbackinetdddr:127,9.0.1廂sk:2S5M*L。inetftaddr:::1/128Scope:HostUPLOOPBACKRUNNINGMTU：65536Metric：!RXpacketsi9errorsi9dropped:?overrunsi0framei0TXpackets:derrors:adropped:^overruns:&carrier:0collisions:0txqueuelen:0RXbytes:9(0.0B)TXbytesrO(9.0B)4.驗證在容器c2中ping容器cl的ip,可以互通root@e75Z8f4f2lb3:/ffptngPING(172/17.0.3)56(84)bytesofdata.6464646464bytes

bytes

bytes

bytes

bytesroot@e75Z8f4f2lb3:/ffptngPING(172/17.0.3)56(84)bytesofdata.6464646464bytes

bytes

bytes

bytes

bytesfromfromfromfromfrom:：：172.17*6*3:172.17*6.3：icnp_seq=ltcnp_seq=2tcFip_seq=3icnp_seq=4icnp_seq=5t±l=64ttl=54ttl=64ttl=64ttl=64msmstine=0.109ttme=0*069ttme=0?067tine=0?961tlne=0.068在容器c2中，發現其/etc/hosts文件中寫入了與cl的連接信息c2-c1:root@e7528f4f21b3:/^catyetc/hosts e752Sf4f21b3127*9.0.1 localhost::1locdlhostipb-locelhostip6-loopbackfeOO::0ipS-ff90::9tp6-nca&tpreftxff02;;1ipfi-allnodesff02::2ip6-fillrouters C2-C15.Ping外網實驗在容器c2中可直接ping外網，因為docker0作為各個容器的默認網關，通過地址轉換所有的數據包都被轉發到宿主機的etho網卡出去了。

data.tine=136time=data.tine=136time=：136ttme=136nsmsmsPINGbatdu.con(220.181.57*216)56(84)bytesof64bytesfron16:icnp_seq=lttL=4964bytesfrom16:icmp_seq=2ttl=4964bytesfrom16:icmp_seq=3ttl=493.2■宿主機容器通信3.2.1實驗目的驗證利用openvswitch的橋接+gre隧道實現容器間的跨宿主機通信。3.2.2實驗拓撲3.2.3實驗方法在主機host2上3，做如下配置：Shellovs-vsctl add-brtech-brovs-vsctladd-porttech-brtep0--setinterfacetep0type=internalifconfigtep0 netmaskovs-vsctl add-brsdn-br0ovs-vsctl setbridgesdn-br0stp_enable=trueovs-vsctladd-portsdn-br0gre0--setinterfacegre0type=greoptions:remote_ip=12.啟動容器2.啟動容器c3Shell1dockerrun-t-i--namec3--net=none--privileged=trueubuntu:latest/bin/bash默認沒有分配ip信息:。口七用巴ifconfigo Linkencap:LocalLoopbacktnetaddr:Mask1inet6&ddr:::1/128ScopeiHostJPLOOPBACKRUNNINGMTU：655S6Metric：!RXpackets:?errors19dropped:?overruns10frame19TXpockets:9errors;0dropped;0overruns;0carrier;0colltsions:0txqueuelen:0RXbyt?s:O(。頂B)TXbytes:0(0.0B)分配容器c3的網絡ip，建立sdn-br0和容器的映射:1pipeworksdn-br0fb700ea73293/24--net=none--privileged=trueubuntu:latest/bin/bash3.啟動容器c4--net=none--privileged=trueubuntu:latest/bin/bash3.啟動容器c4Shell1dockerrun-t-i--namec4root@fb700ea73293:/flifconfigethlLinkencap:EthernetHWaddr56:c2:a7:61:d8:leinetMdc192? 網?3BeastMM?。部Mask:255-255*255??tnetGaddr:fe80::54c2:a7ff:fe61:d81e/64Scope:LinkUPBROADCASTRUNNINGMULTICASTMTU：1599Metric：1RXpacketserrors:&dropped:0overruns:9frame:&TXpackets:8errors:9dropped:0overruns:0carrier:□collistonEiotxqueuelenilOOORXbytes:1429(L4KB)TXbytes:64&(648,9B)Lt> LinkencapiLocalLoopbackinetaddr:Maskiinet6addr:::1/128Scope:HostUPLOOPBACKRUNNINGMrU：655S6MetrtcilRXpacketsioerrors:0dropped:0overruns:0frame:OTXpacketsIOerrorsi&dropped:0overrunscarrier13collisionstxqueuelen:9 RXbyE;0B)TKbyt頃■蹌Shellroot|gubuntu:~ffdockerps-aDETAINERID IMAGECOMMAND/bin/bashVbin/bash"CREATED7minutesagoShellroot|gubuntu:~ffdockerps-aDETAINERID IMAGECOMMAND/bin/bashVbin/bash"CREATED7minutesagoNAMES509330^73275<4b766ea73293c3ubuntu:14*04ubuntu:14.641pipeworksdn-br0509330e73275/24rt>otgubuntu:~if&vs-vsctLshow94-fiaGf-4SQ9-af￡1-52c3a3dlfiflfQBridgetech-brPort"tepcrInterfaceHtep9Htype:internalPorttech-brInterfacetech-brtype:internalBridgeNsdn-bre"Port,,sdn-br0HInterface"sdn-bra"type:InternaLPort"典thlplSSFUInterface"vethlpl5575"PortHgreO,RInterface"greO"type:gireoptions:{renate_ip='r192.16845.31"}Port"voth邛15*9”InterfaceIFvethlpl5339,F4.在主機hostl酉己置1，通主機host2方法一樣Shella!Iovs-vsctladd-brtech-br1ovs-vsctladd-porttech-brtep0--setinterfacetep0type=internal2ifconfigtep0netmask3ovs-vsctladd-brsdn-br04ovs-vsctlsetbridgesdn-br0stp_enable=true5ovs-vsctladd-portsdn-br0gre0--setinterfacegre0type=greoptions:remote_ip=36dockerrun-t-i--namec1--net=none--privileged=true1:5000/sdnpool/ubuntu14.04:latest/bin/bashdockerrun-t-i--namec2--net=none--privileged=true1:5000/sdnpool/ubuntu14.04:latest/bin/bashdocker(3yanjw-OptiPlex-docker(3yanjw-OptiPlex-390;~$dockerpsCONTAINERID IMAGEPORTS

441ce7ecf5cfd341b391Sd29NAMES192.168.5*SlJ5990/&dnpoDl/ubuntul4.04:lates.tc2192?168.5?Ml*5990/sdnpool/ubuntu14*04:latestclCOMMAND"Vbtn/bE"/bin/b;Shell1pipeworksdn-br0441ce7ecf5cf/241pipeworksdn-br0441ce7ecf5cf/242pipeworksdn-br0d341b3915d29/24docker^ysnjw-OptiPlex-399isudoovs-vsctlshow412f4225-bf17-4bf3-S79b-a$2e7f84cF2bBridgetech-brPorttech-brInterfacetech-brtype:internalPortHtep0MInterface"tepO"type:internalBridge,,sdn-br0HPori,rvethlpll7184HInterface,rvethlpll7184,rPort"vethlpL1722W'Interface,rvethlpll7228，rPort"sdn-b「ETInterface,rsdn-br9,ptypeiLnternal.Port"grefi"Interface"greO"type:greoption如(「WEgtw_tp="3"]5.驗證結果容器2:分別ping容器1，容器3，容器4均可通。

rootg441ce7ecf5cf:/#p-ing192?158.日?1PING192M68.0.1(192.168*0.1)56(84)bytesofdata.n&64bytesfrom192.168,3.1:tcnp_seq=lttl=64ttme=0.331n&64bytesfrom192.168icnp_seq=2ttl=64time=9*231AC _…「192?168.13?1pingstatistics--1000ms2packetstransntttedf2received^0%packetLoss,timerttmtn/avg/max/mdev=3.231/0.281/9*331/9.050msroot?441ce7ecf5cf:/#ping192*168-0*SPING192M68.0.S()56(84)bytesofdata.1000ms64bytesfrom192.168,9*3:tcnp_seq=lttl=64tlme=l.88ms64bytesfron192+168.9*3:icnp_seq=2ttl.=64 992n&y——pingstatistics——2packetstransnittedf2recetvedjpacketLoesftimelOOlmsrttmtn/avg/piax/ndev=49駝/L395/1.889/。?494msroot@441ce7ecf5cf:/Sptng192*168.0*4PING192.168M.4()56(84)bytesofdata.64bytesfrom192.168.GL4:icnp_seq=lttl=64 32ms64bytesfron192^.168.0.4:icnp_seq=Zttl=64tine=0.697nsy——pingstatistics——2p己匚ketstransnitted,2receivedjD*packetloss,ttmel&Olmsrttmin/dvg/mex/ridev二3^.697/1.010/11324/91315ns6.使用SDN控制器Floodlight管理上面的Docker容器集群在主機hostl上的在主機hostl上的sdn-br0設置連接控制器：1ovs-vsctlset-controllersdn-br0tcp:1:6633在容器做互ping操作，可讓floodlight拓撲發現這些容器，最終SDN控制器完全能夠發現這些容器，如下圖：FloodliDashboardTopologySwitchesHernal.SyncManager.n.f.hub.Hub.n.(.firewall.Firewall,n.f.perernal.ShutdownServi