為工業(yè)過程控制提供及時異構(gòu)SAN驅(qū)動保證Abstract When wireless sensor networks are introduced in industrial settings, they will be part of a much larger heterogeneous sensor and actuation network that includes those sub-networks together with Ethernet cabled Programmable Logic Controllers (PLCs) and control stations. Performance issues arise in such systems. Particularly, actuation latencies are considered crucial in those scenarios. In this paper, we present an approach for planning and evaluating a heterogeneous supervisory control and data acquisition(SCADA) system with wireless sensor networks. We propose closed-loop schemas and mechanisms for measuring and verifying the latency bounds in the whole system. As a case study, we deploy and evaluate our approach on an oil refinery, where several heterogeneous networks are used to monitor and control a water treatment process. Results demonstrated the effectiveness of our method in planning, debugging and providing feedback about the network (in)ability to provide timely actuation guarantees.摘要：當(dāng)無限傳感器網(wǎng)絡(luò)被引入工業(yè)環(huán)境中，他們將成為更大的異構(gòu)傳感器和驅(qū)動網(wǎng)的一部分，包括那些子網(wǎng)絡(luò)與以太網(wǎng)電纜可編程邏輯控制器（PLC）網(wǎng)絡(luò)和控制站。在這樣的系統(tǒng)中性能問題大大出現(xiàn)了。特別是在這些情況下啟動延遲是至關(guān)重要的。在本文中，我們提出了一種方法，用于評估和規(guī)劃一個異構(gòu)的數(shù)據(jù)采集與監(jiān)控系統(tǒng)（SCADA）與無線傳感器網(wǎng)絡(luò)系統(tǒng)。我們提出了閉環(huán)模式和機(jī)制，用于測量和驗證整個系統(tǒng)的延遲界限。作為一個案例研究，我們部署和評估我們的方法在一個煉油廠，有幾個異構(gòu)網(wǎng)絡(luò)是用于監(jiān)測和控制水處理工藝。結(jié)果表明，我們的方法在有效規(guī)劃，調(diào)試和提供網(wǎng)絡(luò)反饋（中）提供及時的激勵保障能力。關(guān)鍵詞：SAN，異質(zhì)性，時間界限，閉環(huán)控制1、 引言With the evolution and increased adoption of wireless sensor technology and networks, and their easier and much cheaper deployment, there is a current trend to partially replace or complement the existing infrastructure. When deployed in industrial settings, it will be integrated on a larger heterogeneous sensor and actuation network composed by cable-based networks, wireless sensor networks (WSN), Programmable Logic Controllers (PLCs) and control stations.隨著時代的演變，無線傳感器技術(shù)和網(wǎng)絡(luò)的使用也在增加以及他們更容易和便宜得多的部署，目前的趨勢是有部分取代或補(bǔ)充現(xiàn)有的基礎(chǔ)設(shè)施目前的趨勢。當(dāng)部署在工業(yè)環(huán)境中，它將被集成在一個更大的異構(gòu)傳感器和驅(qū)動網(wǎng)絡(luò)由有線網(wǎng)絡(luò)，無線傳感器網(wǎng)絡(luò)（WSN），可編程邏輯控制器（PLC）和控制站。In that context, sensor and actuator networks require a high level of reliability concerning network latency, delay and message losses, to ensure that monitoring and control loop actions can be made within required time bounds.在這樣的背景下，傳感器和執(zhí)行器網(wǎng)絡(luò)需要高水平的可靠性有關(guān)的網(wǎng)絡(luò)延遲，延遲和消息損失，確保監(jiān)測和控制回路的動作可以在所需的時間界限。Time-base guarantees can be provided by deploying WSN networks with real-time specific algorithms that include completely pre-planned synchronous time-division mechanisms. But, sensor and actuation infrastructures typically are heterogeneous systems, composed with specific software parts, some possibly offering real-time, while others do not. For instance, in a typical setting, a Time Division Multiple Access (TDMA) protocol can be applied to wireless sensor networks, to deliver some degree of in-network time predictability. However, the monitoring and control systems frequently are heterogeneous and composed by off-the-shelf networking and computerized systems without real-time operating services.時基擔(dān)保可以由部署的WSN網(wǎng)絡(luò)的實時的具體算法，包括完全預(yù)先計劃好的同步時分機(jī)制來提供。但是，傳感器和驅(qū)動的基礎(chǔ)設(shè)施通常是異構(gòu)系統(tǒng)的組成部分，與特定的軟件，有可能提供實時的，而其他的沒有。例如，在一個典型的設(shè)置中，時分多址（TDMA）協(xié)議，可用于無線傳感器網(wǎng)絡(luò)，提供一定程度的網(wǎng)絡(luò)時間的可預(yù)測性。然而，監(jiān)測和控制系統(tǒng)通常是異構(gòu)的，由現(xiàn)成的網(wǎng)絡(luò)和沒有實時操作服務(wù)計算機(jī)系統(tǒng)組成。Performance control requirements (e.g. latencies or losses) introduce some constraints into the design and deployment of overall SAN architecture. Instead of devising a single wireless sensor network with thousands of nodes in a large monitoring and control deployment, there are a multitude of small networks with tens of nodes with planned functionalities connected with several gateways and cabled networks.性能控制要求（如延遲或損失）提出一些約束條件納入總體SAN架構(gòu)的設(shè)計和部署。與制定一個單一與成千上萬個節(jié)點的大型監(jiān)控部署的無線傳感器網(wǎng)絡(luò)不同，有數(shù)十節(jié)點與多個網(wǎng)關(guān)和有線網(wǎng)絡(luò)連接功能規(guī)劃眾多的小型網(wǎng)絡(luò)。A SCADA structure includes resource-constrained sensor nodes, such as TelosB nodes, and other more powerful nodes, such as PLCs, PC or servers, interconnect with cable or wireless links, as shown in figure 1.SCADA結(jié)構(gòu)包括資源受限的傳感器節(jié)點，如TelosB節(jié)點，和其他更強(qiáng)大的節(jié)點，如PLC，PC或服務(wù)器，以有線或無線鏈路互連，如圖1所示。圖1. 一般的SCADA系統(tǒng)Due to the nature of control, the data transmitted from sensors is only valid for a short time. If the data is delivered too late it is of limited use, as in most real-time systems.由于控制的本質(zhì)，由傳感器傳輸?shù)臄?shù)據(jù)只有在短時間內(nèi)才有效。在大多數(shù)實時系統(tǒng)中，如果數(shù)據(jù)交付太遲只能是有限的使用。In industrial processes, some functions are safety-critical and thus are dependent on the system ability to operate within specific time boundaries. The prevention of accidents is extremely important. As an example, if a set point sent by the control system to a control valve cannot be delivered, the valve should fall back into a safe state (normally fully closed or fully open) after a timeout. The timeout depends on how long the process can tolerate a “malfunction” of the actuator before entering into a possibly dangerous situation, typically ranging from milliseconds to seconds. Additionally, the control system should also detect and alert control engineers when time boundaries are exceeded or communication losses occur. This is crucial to timely detect, mitigate and avoid error propagation to the rest of the control network.在工業(yè)生產(chǎn)過程中，有些功能是安全的關(guān)鍵，因此是依賴于系統(tǒng)的能力，在特定的時間范圍內(nèi)運(yùn)行。事故的預(yù)防是非常重要的。例如，如果一個被控制系統(tǒng)發(fā)送到一個控制閥集合點不能交付，閥門在超時后應(yīng)該回到安全狀態(tài)（正常全關(guān)或全開）。超時多久取決于過程可以容忍一個“失靈”的驅(qū)動器進(jìn)入一個可能有危險的情況之前，通常從毫秒到秒。此外，該控制系統(tǒng)還應(yīng)檢測和報警控制工程師的時間界限是超過或通信損失發(fā)生。這是及時發(fā)現(xiàn)減輕和避免錯誤傳播到剩下的控制網(wǎng)絡(luò)的是至關(guān)重要。In this work we propose an approach to plan closed-loop tasks with restricted time boundaries in such a heterogeneous system. We assume TDMA wireless sensor networks and consider measures and constraints from all parts of the system, including non-real-time operating components.在這項工作中，我們提出了一個計劃閉環(huán)任務(wù)在這樣的異構(gòu)系統(tǒng)的限制時間邊界的方法。我們假設(shè)TDMA的無線傳感器網(wǎng)絡(luò)和系統(tǒng)的所有部分考慮采取措施，限制，包括非實時操作系統(tǒng)組件。The rest of the paper is organized as follows: section II reviews related work; section III describes how to dimension a SCADA system with time guarantees, and discusses the time related guarantees that each component of the architecture can provide to the system. Section IV defines two closed-loop alternatives that can provide time requirements. In section V we present the experimental setup and the oil refinery requirements. Section VI shows the experimental results. Lastly, section VII concludes the paper.本文的其余部分安排如下：第二節(jié)回顧相關(guān)工作；第三節(jié)介紹了如何維與時間保證SCADA系統(tǒng)，并討論了該體系結(jié)構(gòu)的各組件可以提供給系統(tǒng)的時間相關(guān)的保證。第四節(jié)定義了兩個閉環(huán)方案，可以提供時間要求。在第五節(jié)，我們目前的實驗裝置和煉油廠的要求。第六部分：試驗結(jié)果。最后，第七章總結(jié)全文。II. RELATED WORKII. 相關(guān)的工作In this section we first review related work on schedule based planning and monitoring tools. One important key issue in WSNs that influences whether the deployed system will be able to meet time requirements is the MAC protocol and its configurations. The next related works concern implementing real-time strategies in single WSNs.在這一部分中我們首先回顧相關(guān)的以規(guī)劃和監(jiān)測工具為基礎(chǔ)的工作表上的工作。在無線傳感器中的一個重要的問題是影響是否已部署的系統(tǒng)將能夠滿足時間要求是MAC協(xié)議及其配置。下一個相關(guān)工作與在單傳感器實施實時策略相關(guān)。In RT-Link protocol, time-slot assignment is accomplished in a centralized way at the gateway node, based on the global topology in the form of neighbor lists provided by the WSN nodes.在RT鏈路協(xié)議，時隙分配完成在網(wǎng)關(guān)節(jié)點的一個集中的方式，基于鄰居列表的無線傳感器網(wǎng)絡(luò)節(jié)點提供的全局拓?fù)湫问健irelessHART is designed to support industrial process and automation applications. In addition, WirelessHART uses at its core a synchronous MAC protocol called TSMP , which combines TDMA and Frequency Division Multiple Access (FDMA).WirelessHART是專為支持工業(yè)過程自動化中的應(yīng)用。此外，WirelessHART使用為核心的同步MAC協(xié)議稱為結(jié)合TDMA和頻分多址接入（FDMA）的TSMP 。GinMAC is a TDMA protocol that incorporates topology control mechanisms to ensure timely data delivery and reliability control mechanisms to deal with inherently fluctuating wireless links. The authors show that, under high traffic load, the protocol delivers 100% of data in time using a maximum node duty cycle as little as 2.48%.proposed protocol is also an energy efficient solution for time-critical data delivery with neglected losses.GinMAC是結(jié)合拓?fù)淇刂茩C(jī)制TDMA協(xié)議，以確保提供及時的數(shù)據(jù)和可靠的控制機(jī)制，解決固有的波動無線連接。作者表明，在高流量負(fù)載，該協(xié)議提供100%使用節(jié)點的最大占空比2.48%少時間數(shù)據(jù)。該協(xié)議也是一種被忽視的損失的時間關(guān)鍵數(shù)據(jù)提供高效節(jié)能的解決方案。PEDAMACS is another TDMA scheme including topology control and routing mechanisms. The sink centrally calculates a transmission schedule for each node, taking interference patterns into account and, thus, an upper bound for the message transfer delay can be determined. PEDAMACS is restricted by the requirement of a high-power sink to reach all nodes in the field in a single hop. PEDAMACS is analyzed using simulations, but a real-world implementation and corresponding measurements are not reported.PEDAMACS是另一個TDMA方案包括拓?fù)淇刂坪吐酚蓹C(jī)制。匯集中計算每個節(jié)點的發(fā)送時間表，以干擾模式的考慮，因此，一個上限的消息傳遞延遲可以確定。PEDAMACS由一個高功率的下沉到所有節(jié)點在該領(lǐng)域的一個單跳的要求限制。進(jìn)行了模擬PEDAMACS，但實際的實現(xiàn)和相應(yīng)的測量是不報道。Our approach is also related to monitoring tools that can be used to evaluate network performance. It requires information of latencies and other simple metrics that provide enough information about the network health to avoid safety control activation or accidents.我們的方法也與可以用來評估網(wǎng)絡(luò)性能監(jiān)測工具的相關(guān)。它要求延遲和其他簡單的指標(biāo)，提供足夠的信息，以避免網(wǎng)絡(luò)健康安全控制的激活或事故信息。There already exist some tools to monitor wireless sensor networks. These include Sympathy , Sensor Network Management System (SNMS) , Sensor Network Inspection Framework (SNIF) and Distributed Node Monitoring in Wireless Sensor Networks (DiMo) .已經(jīng)有一些工具來監(jiān)測無線傳感器網(wǎng)絡(luò)。這些包括支持，傳感器網(wǎng)絡(luò)管理系統(tǒng)（SNMS），傳感器網(wǎng)絡(luò)檢測框架（SNIF）和無線傳感器網(wǎng)絡(luò)中的分布式節(jié)點監(jiān)控（DIMO）。We have reviewed existing tools to monitor network health, and designed our own simplified tool adapted to our planning objectives. The existing tools are focused on specific goals, are complex to implement and work only inside a WSN.我們回顧了現(xiàn)有的工具來監(jiān)視網(wǎng)絡(luò)的健康，和我們自己的設(shè)計簡化工具適合我們的規(guī)劃目標(biāo)。現(xiàn)有的工具都集中在特定的目標(biāo)，實現(xiàn)起來很復(fù)雜，只有在一個無線傳感器網(wǎng)絡(luò)的工作。III. CLOSED-LOOP DIMENSIONING FOR HETEROGENEOUS NETWORKIII. 為異構(gòu)網(wǎng)絡(luò)閉環(huán)的尺寸Most performance-critical applications can be found in the domain of industrial monitoring and control. In these scenarios control loops are important and can involve any node and any part of the SCADA system. For instance, the closed loop actuation value can be determined inside any WSN subnetwork, any PLC or control station.大多數(shù)的性能關(guān)鍵的應(yīng)用程序可以在工業(yè)監(jiān)控領(lǐng)域的發(fā)現(xiàn)。在這些情況下，控制回路是重要的和可涉及的任何節(jié)點和SCADA系統(tǒng)的任何部分。例如，閉環(huán)驅(qū)動值可確定在無線傳感器網(wǎng)絡(luò)的子網(wǎng)，任何PLC或控制站。The SCADA system shown in Figure 1 assumes an industrial network with multiple WSN sub-networks. Given computational, energy and performance considerations, closed loop paths may be entirely within a single sub-network, with decision logic resident in the sink node (e.g. taking few ms) or outside the WSN (e.g. for applying more computational complex supervision controller), or it may span more than one WSN, with supervision control logic residing in one of the distributed PLC outside the WSNs (middleware servers).如圖1所示的SCADA系統(tǒng)與多個無線傳感器網(wǎng)絡(luò)的子網(wǎng)絡(luò)是一個工業(yè)網(wǎng)絡(luò)。給定的計算的，能源和性能考慮，閉環(huán)路徑可能是完全一個單獨的子網(wǎng)內(nèi)，在匯聚節(jié)點的決策邏輯的點（例如以幾毫秒）或外部的無線傳感器網(wǎng)絡(luò)（例如使用更多的計算復(fù)雜的監(jiān)督控制器），或者在一個在無線傳感器網(wǎng)絡(luò)的分布式PLC（中間件服務(wù)器）在監(jiān)督控制邏輯點下它可能跨越多個無線傳感器網(wǎng)絡(luò)。One important factor in control is the closed-loop latency. The closed loop latency is the time taken from sensing node to the actuator node, passing through closed-loop manager. It will be the time taken since the value (event) happens at sensing node to the instant when the action is performed at actuator node. Since the value must cross several parts of the system, this latency can be decomposed into latencies for each part of the path. The latency can be divided into three main parts: upstream part (from sensing node to the closed-loop manager), processing latency (it is dependable of the closed-loop algorithm) and downstream part (from closed-loop manager to the actuator).在控制的一個重要因素是閉環(huán)延遲。閉環(huán)延遲是通過閉環(huán)管理從傳感節(jié)點到執(zhí)行器節(jié)點的時間。這將是自值時間（事件）發(fā)生在傳感節(jié)點的瞬間作用于執(zhí)行器節(jié)點進(jìn)行。由于值必須跨系統(tǒng)的幾個部分，這樣的延遲可以被分解成的潛伏期為每個路徑的一部分。潛伏期可劃分為三個主要部分：上游（從傳感節(jié)點到閉環(huán)管理），處理時間（這是可靠的閉環(huán)算法）和下游部分（從閉環(huán)管理器）。The position of closed-loop manager may depend on timing restrictions and data needed to compute decisions. For instance, if minimal latency was required and a single sub-network is considered, the closed-loop manager must be deployed at sink node, but the closed-loop decision computation inside embedded devices (sink node) is very limited. In this case, the upstream latency corresponds to the time for transmission between leaf node and sink node; typically, the processing time is small, because simple closed-loop techniques are used. The downstream latency is the time to transmit a command from sink node to the actuator.閉環(huán)管理點可能取決于計算所需的時間限制和數(shù)據(jù)的決定。例如，如果最小的延遲是必需的，被認(rèn)為是一個單一的子網(wǎng)絡(luò)，閉環(huán)管理必須部署在匯聚節(jié)點，但閉環(huán)決策計算在嵌入式設(shè)備（sink節(jié)點）是非常有限的。在這種情況下，上游延遲對應(yīng)于葉節(jié)點和匯聚節(jié)點之間的傳輸時間；通常情況下，處理時間是很小的，因為使用的是簡單的閉環(huán)控制技術(shù)。下游延遲時間發(fā)送命令從匯聚節(jié)點的執(zhí)行器。Figure 2 shows a scenario example of closed-loop system where the closed-loop decision is done by the sink node. The computation capability for taking closed-loop decisions inside embedded devices is very limited. It allows, for example, the selection of which nodes will participate in the sensing and decision, which threshold and conditions will be used to trigger the actuator and the actuation value.圖2顯示了一個例子的情況的閉環(huán)系統(tǒng)，閉環(huán)的決定是由匯聚節(jié)點完成。以閉環(huán)決策在嵌入式設(shè)備的計算能力是很有限的。它允許，例如，選擇的節(jié)點參與感知和決策，閾值和條件將被用來觸發(fā)執(zhí)行器和驅(qū)動值。圖2.在匯聚節(jié)點控制決策At the sink node, when a data message from sensing nodes participating in the decision arrives, or at defined time periods, the condition and thresholds are analyzed, and the actuator is triggered if one of the defined conditions is matched.在匯聚節(jié)點，當(dāng)數(shù)據(jù)消息從傳感節(jié)點到參與決策，或在規(guī)定的時間周期，和閾值的條件進(jìn)行了分析，如果一個定義的條件相匹配執(zhí)行器觸發(fā)。The other alternative for closed-loop control with more powerful resources is to deploy the supervision control logic in one of the distributed PLC outside the WSNs. This alternative is also shown in Figure 3. In this case it is possible to read data from several WSN, to compute a decision based in more complex algorithms, and to actuate over the whole industrial network. The closed-loop algorithm will receive data coming from sensors and will produce actuation commands for the actuator(s).更強(qiáng)大的資源用于閉環(huán)控制的另一種方法是在一個分布式PLC在無線傳感器網(wǎng)絡(luò)部署的監(jiān)督控制邏輯。這個方案也如圖3所示。在這種情況下，可以從幾個傳感器讀取數(shù)據(jù)，計算出一個基于更復(fù)雜的算法的決策，并驅(qū)動整個產(chǎn)業(yè)網(wǎng)絡(luò)。閉環(huán)控制算法將接收來自傳感器的數(shù)據(jù)并將為執(zhí)行器產(chǎn)生驅(qū)動的命令。In this case the control loop may traverse multiple, most probably non-real-time hardware and software systems, nevertheless the control loop will still need to be under expected time bounds.在這種情況下，控制回路可能跨越多個，最有可能的非實時的硬件和軟件系統(tǒng)的控制回路，但仍需在預(yù)期的時間界限。圖3.在整個網(wǎng)絡(luò)的控制回路In this scenario, the first part of latency (upstream latency) can be sub divided into - WSN latency (time for transmission between leaf node and sink node); latency for sink-gateway (the time taken for the message to go from the sink to the gateway, plus gateway processing time); latency for cabled network (e.g transmission between PLC-gateway and Control Station); control station latency; and end-to-end latency (leaf node to Control Station).在這種情況下，延遲的第一部分（上游延遲）可以分為：無線傳感器網(wǎng)絡(luò)的延遲（葉節(jié)點和匯聚節(jié)點之間的傳輸時間）；為Sink網(wǎng)關(guān)的延遲（時間為消息走到網(wǎng)關(guān)，網(wǎng)關(guān)處理時間加上有線網(wǎng)絡(luò)延遲（）；PLC網(wǎng)關(guān)和控制站之間傳輸）；如控制站和終端到終端的延遲（延遲；葉節(jié)點控制站）。The second part (processing time) depends of the closedloop computation to apply, and it can take several seconds. Lastly, the third part (downstream) corresponds to the path used by a command to reach an actuator. This part can be subdivided into several subparts, such as: control station latency, gateway latency, gateway to WSN interface latency and WSN downstream latency.第二部分（處理時間）取決于閉環(huán)計算應(yīng)用，它可能需要幾秒。最后，第三部分（下游）對應(yīng)的命令到執(zhí)行器的路徑。這一部分可分為幾個部分，如：控制站的無線傳感器網(wǎng)絡(luò)網(wǎng)關(guān)的延遲，延遲，延遲和延遲的無線傳感器網(wǎng)絡(luò)網(wǎng)關(guān)接口的下游。In the next sub-sections we discuss how schedule-dictated “time guarantees” are achieved in WSN sub-networks and the issues related with time guarantees limitations and solutions for the global SCADA system.在下一小節(jié)討論進(jìn)度決定“保證”是無線傳感器網(wǎng)絡(luò)的子網(wǎng)絡(luò)實現(xiàn)的問題，時間保證的局限性和解決方案相關(guān)的全球SCADA系統(tǒng)。A. Schedule based protocol for WSN sub-networksA. 基于無線傳感器網(wǎng)絡(luò)的子網(wǎng)絡(luò)協(xié)議的時間表The medium access control protocols for wireless sensor networks can be classified into two major categories: Contention based and Schedule based. Contention based protocols can easily adapt to topology changes after being deployed, as new nodes join and others leave. These protocols are based on Carrier Sense Multiple Access (CSMA) technique and have higher costs related to message collisions, overhearing and idle listening. In contrast, schedule based protocols (TDMA) avoid collisions, overhearing and idle listening by scheduling the transmission and listening periods to specific