1、MPEG for the past, present and future of televisionMPEG 用于電視的過去、當前和未來MPEG for the past of television MPEG 用于電視的過去System 系統Transport Stream 傳送流DSM-CC 數字存儲媒體命令與控制Timed metadata 有定時的宏數據Video 視頻MPEG-2 High Profile MPEG-2 高檔次Audio 音頻MPEG-1 Audio Layer II 音頻第 2 層MPEG-2 AAC MPEG-2 高級音頻編碼2021/05/022MPEG for

2、 the past, present and future of TVAn assessment 評價MPEG has provided the means for the television business on air, cable and satellite to migrate from the analogue to the digital age MPEG 對地面、有線和衛星的電視效力從模擬過渡到數字時代，提供各種方法It has been a collective efforts 它是集體努力的結果Representatives from the entire spectru

3、m of the broadcasting world provided their requirements來自廣播界的方方面面代表已提出各種需求Packages of patent licences were created (outside of MPEG) 在 MPEG 之外，已創建成套的專利答應證Today there is virtually no broadcasting system that is not based, at least partially, on MPEG standards目前沒有一個廣播系統不是基于MPEG 規范至少部分基于MPEG for the pa

4、st, present and future of TV32021/05/08MPEG for the present of television MPEG 用于電視的當前Systems 系統Carriage of MPEG-4 on MPEG-2 TS用于在 MPEG-2 TS 上承載 MPEG-4Dynamic Adaptive Streaming over (DASH) (DASH) 上的動態自順應流媒體Video 視頻MPEG-4 Advanced Video Coding (AVC)MPEG-4 高級視頻編碼3D extensions 各種 3D 擴展Audio 音頻Various

5、forms of Advanced Audio Coding (AAC)高級音頻編碼的各種方式Universal Speech and Audio (USAC)通用的語音和音頻編碼2021/05/084MPEG for the past, present and future of TVAn assessment 評價The deployment of 2nd generation digital broadcasting systems continues successfully第二代數字廣播系統的部署繼續順利進展Use of the AVC on MPEG-2 TS allows pac

6、king twice as many programs as MPEG-2 Video in an analogue channel在單個模擬電視頻道中，采用 MPEG-2 TS 承載 AVC 視頻，替代 MPEG-2 視頻，可使節目數量翻一番With IPTV the notion of broadcasting is expandingIPTV 正在拓展廣播的概念MPEG is helping the broadcasting industryMPEG 正在協助廣播工業界To enrich the broadcasted user experience豐富了廣播用戶的體驗Through s

7、tandard technologies that serve multiple fields經過規范技術效力于多個領域MPEG for the past, present and future of TV52021/05/08MPEG for the future of televisionMPEG 用于電視之未來Well, thats what I am supposed to talk about好，下面是我今天預備討論的2021/05/086MPEG for the past, present and future of TVBroadcasting is a great busine

8、ss to be in廣播業在未來依然將是一個宏大的事業Broadcasting is a proven business with a constant revenue flow廣播業已被證明具有穩定的收益流Many technologies appear that 很多技術表現出可以：Can help new competing businesses but有助于具有競爭力的新效力Can also be used to extend the broadcasting business而且還可用于擴展廣播效力MPEG is a neutral body devoted to the deve

9、lopment of standard technologies for user communities to exploitMPEG 是一個中立團體，努力于開發各類用戶群體所需的規范化技術MPEG for the past, present and future of TV72021/05/08MPEG-H: a video standard for future broadcastingMPEG-H:用于未來廣播的視頻規范ISO/IEC 23008 High Efficiency Coding and Media Delivery in Heterogeneous Environment

10、sISO/IEC 23008：高效編碼和異構環境的媒體發送A suite of standards for media coding and delivery用于媒體編碼和發送的一整套規范Currently scheduled to appear in 3 parts目前方案包括三部分：MPEG Media Transport MPEG 媒體傳送High Efficiency Video Coding 高效視頻編碼3D Audio 3D 音頻Timeline時間表CD 委員會草案DIS 國際規范草案FDIS 國際規范定稿草案MPEG for the past, present and futu

11、re of TV82021/05/08PartCDDISFDISMMT12/0713/0113/07HEVC12/0212/0713/013D Audio14/0414/1015/01Service composition in MMTMMT多媒體傳送流中的效力組裝3D EarthUD contentsBird is flyingBird is flying鳥在飛Composition information 組裝信息friendly with html5與 HTML5 友好legacy broadcasting 傳統廣播future broadcasting 未來廣播9MMT Package

12、 MMT 信息包MPEG for the past, present and future of TV2021/05/02通用發送的各種內容3D 地球CloudsMMT EncapsulatorMMT 封裝器 Service Component List 效力組件的清單 Component location 組件的位置 Composition relationship 組裝的關系 QoE requirement 封裝質量的需求 How to deliver 如何發送 How to consume 如何消費Web 互聯網Video 視頻File 文件 Configuration 配置 Adapt

13、ive delivery 自順應發送 Sync, Delay, Loss control 同步，延遲，丟包控制MMT Frame MTT 數據幀pull 拉動pullSmart delivery in Clouds 云計算中的智能發送102021/05/08Mash-up service 混效果勞Service 1 (MMT Package1)效力 1 MMT 信息包 1Service 2 (MMT Package2)效力 2MMT 信息包2MMT assetsMMT 各種資產Audio音頻Video1視頻 1Widget小工具菜單Web page網頁Video2視頻 2112021/05/0

14、8MPEG for the past, present and future of TVSecond screen management 第二屏幕的管理Additional Information with Live Content (HTML)實況內容的附加信息HTMLIssueVideo content on 2nd device在第二設備上的視頻內容Composition information 組裝信息MMT PackageMMT 信息包122021/05/08MPEG for the past, present and future of TVMPEG-H part 2 High E

15、fficiency Video CodingMPEG-H 第 2 部分 高效視頻編碼Primary goal: 50% better compression than AVC最初目的：緊縮效率比 AVC 提高 50 %For large displays(e.g. 4kx2k) 用于大屏幕 (例如 4kx2k)For mobile (e.g. WVGA) 用于挪動效力 (例如，WVGA)Developed jointly with ITU-T by Joint Collaborative Team (JCT) on Video Coding由 MPEG 與 ITU-T 結合成立的視頻協作組制定

16、Technically in the same stream of past MPEG Video Coding standards (AVC and prior standards)技術上運用與 MPEG 以前的視頻編碼規范 (AVC 及以前規范) 一樣的傳送流Currently at Committee Draft level目前處于委員會草案的階段MPEG for the past, present and future of TV132021/05/08HEVC vs AVC Subjective Performance MeasurementsHEVC 和 AVC 客觀性能測試的對比

17、Preliminary subjective tests carried out for HEVC Committee Draft對 HEVC 委員會草案已進展初步的 客觀測試Reasonably small and non-overlapping confidence intervals合理較小的和不重疊的信任度間隔Average rate savings at same (interpolated) MOS points在一樣的 (內插的) MOS (評分平均) 點，得出可節省的平均比特率 HEVC with 67% lower rate in class B (Full HD)在 B 類

18、 (全高清) 中 HEVC 的比特率要低 67 %HEVC with 49% lower rate in class C (WVGA)在 C 類 (WVGA) 中 HEVC 的比特率要低 49 %HEVC with 58% lower rate overallHEVC 整體比特率要低 58 %2021/05/08MPEG for the past, present and future of TV14A performance sample 一個性能樣本2021/05/08MPEG for the past, present and future of TV15For those techno

19、logy aware涉及的有關技術Block-based/Variable block sizes 基于塊的/可變塊尺寸Block motion compensation 塊的運動補償Fractional-pel motion vectors 分像素的運動矢量Spatial intra prediction 空間域的幀內預測Spatial transform of residual difference殘差值的空間域變換Integer-based transform designs基于整數的各種變換設計Arithmetic or VLC-based entropy coding基于算術的或變字

20、長編碼的熵編碼In-loop filtering to form final decoded picture環內的濾波處置以構成最終的解碼圖像2021/05/08MPEG for the past, present and future of TV16Where we are (May 2021)我們身處何處 (2021 年 5 月)Activity: level of 1,000 documents per meeting crossed活動：每次會議審議約 1,000 個文件Very diverse participation from industry/academia工業界和學術界的極

21、廣泛參與Significant technical advance over prior standards與過去的各種規范相比，已有顯著的技術提高Challenge: computational/implementation complexity挑戰：計算/實現的復雜度Deliverables: 可發布的：Spec技術規范Reference software參考軟件Conformance符合性Currently only one profile 目前只需一個檔次2021/05/08MPEG for the past, present and future of TV17HEVC Scalab

22、ility Extensions/1HEVC 可伸縮性擴展/1Serve the needs of heterogeneous environment of future digital TV distribution效力于未來數字電視分發的異構環境之各種需求Multiple devices: SHDTVs, set-top boxes, tablets, smart phones, PCs etc. 多類設備：超高清電視機，機頂盒，平板電腦，智能手機，個人電腦等Wide range of processing powers, display sizes and power consumpti

23、on needs 較寬范圍的處置才干、顯示器尺寸和功耗需求Multiple networks: Cable, Satellite, Terrestrial, CDN, Cable Modem, xDSL, WiFi, 4G多類網絡：有線、衛星、地面、有線數字網絡、有線調制解調器，xDSL，WiFi，4GWide range of channel bandwidths and QoS較寬范圍的各種信道帶寬和效力質量Multiple protocols (多類協議): MPEG-2 TS, DASH, HLS, RTP, UDPMultiple services: Broadcasting, On

24、 demand, Streaming, Over The Top 多類效力：廣播，點播，媒體流、三網交融2021/05/08MPEG for the past, present and future of TV18HEVC Scalability Extensions/2 HEVC 可伸縮性的擴展/2Options for backward compatible migration paths用于向后兼容的遷移途徑之各種可選項From 720p60 to 1080p60 從 720p60 到 1080p60From1080p60 to 4kx2kp60 從 1080p60 到 4kx2kp60

25、From AVC to HEVC 從 AVC 到 HEVCTime Line 時間表May 2021 Preliminary Call for Proposals (public)2021 年 5 月，初次征求提案公開July 2021 Final Call for Proposals (public)2021 年 7 月，最終征求提案公開October 2021 2021 年 10 月Evaluation of proposals & start of collaborative design phase評價各種提案，并開場所作設計階段Development of the time line

26、 associated with standardisation phases (WD, CD/PDAM, FDIS/FDAM, DIS/AMD)按照規范化程序，設定執行時間表2021/05/08MPEG for the past, present and future of TV19Service Transitions 效力的過渡1950s: Color TV 1950 年代：彩色電視Analog, backward compatible模擬的，向后兼容1990s: Digital TV 1990 年代：數字電視New infrastructure required 需求新的根底設備Tra

27、nsitions from SD to HD 從標清過渡到高清2021s: 3D 2021 年代：三維電視Mixed results from services introduction is各種效力的導入所構成的各種混合結果Not a single format across all services已不再是各種效力運用的單一格式2021/05/08MPEG for the past, present and future of TV20Current Status of 3D Video三維視頻的現狀Increasing investments in 3D production and s

28、ervices對 3D 制造和效力的投資日益增長Increasingly capable 3D displays in the market (many competing and emerging technologies) 市場上的 3D 顯示器日益增長 (大量競爭的和交融的技術)Market is primarily stereo (mix of different formats being deployed right now) 市場上主要是雙目立體的 (不同格式的混合，目前正在部署中)Focus of current MPEG work: development of a new

29、3D format and associated compression techniques that could facilitate generation of multiview output當前 MPEG 任務的重點：設定一個新的 3D 格式及相應的緊縮技術，以支持多視輸出的生成2021/05/08MPEG for the past, present and future of TV21Existing 3DV Coding Formats現有三維視頻編碼的各種格式Pack pixels from left and right views into a single frame (l

30、oss of resolution)把來自左視和右視的各像素，拼接進單個圖像幀分辨率下降Leverage existing infrastructure and equipment, only require additional signaling to de-interleave frame 權衡現有的根底設備，并只需對圖像幀添加 “去交錯 信令Embraced by broadcasters for first phase of 3D broadcast services在 3D 電視廣播的第一階段遭到廣播業者歡迎Full-resolution coding of stereo and

31、MV video as extension of AVC 雙視和多視的視頻之全分辨率編碼是 AVC 的擴展High coding efficiency achieved via inter-view prediction techniques 經過視間預測技術獲得更高的編碼效率2D compatibility supported可以兼容二維 (2D) 電視Adopted as format for 3D Blu-ray Disc已采用為 3D 藍光盤的格式Being considered for second phase of broadcast standards 正在思索用于各廣播規范的第

32、 2 階段2021/05/08MPEG for the past, present and future of TV22Left 左Right 右左視右視視間的高效預測用于高緊縮性能Targets of Future 3DV Format未來三維視頻格式的各種目的2021/05/08MPEG for the past, present and future of TV23受限的攝像機輸入數據格式受約束的比特率基于分發數據格式左右各種雙目立體顯示器 立體視基線可變 調理深度感知自動立體的N 視顯示器 寬視角 大量的各種輸出視圖3DV Reference Framework 三維視頻參考框架Lim

33、ited # Video Inputs(e.g., 2 or 3 views)DepthEstimationVideo/DepthEncoderDecoder and View SynthesisLarger #Output ViewsBinary Representation & Reconstruction Process二進制表達和重建過程+1010001010001242021/05/08MPEG for the past, present and future of TV受限數量 # 的視頻輸入 (例如 2 或 3 個視角)深度估計視頻/深度編碼器解碼器和視圖合成大于 # 的輸出視O

34、verview of Call For Proposals提案征集書的概述Test Material & Conditions 測試素材及條件8 test sequences (1920 x1080 & 1024x768) 8 個測試序列4 target bit rates per sequence 每個序列有 4 個目的比特率2-view and 3-view test scenarios 2-視角和 3-視角的測試場景AVC and HEVC compatible test categories 與 AVC 和 HEVC 兼容的各種測試類別23 responses (12 AVC + 11

35、 HEVC) 23 個呼應 提案Evaluation in Nov 2021 2021 年 11 月的評價Objective quality (PSNR of synthesized views)客觀質量各種合成視的 PSNRSubjective assessment (stereo and auto-stereo)客觀評分雙目立體和自動立體2021/05/08MPEG for the past, present and future of TV253DV Testing Scenarios三維視頻的測試場景2021/05/08MPEG for the past, present and fu

36、ture of TV262-view Configuration 2-視配置3-view Configuration 3-視配置左右深度估計三維視頻編碼器多視演奏虛擬視輸出至立體顯示器輸出至自動立體N-視顯示器中Tool Categories 工具的分類Texture coding 紋理編碼Independent of depth 不依賴于深度E.g., inter-view prediction of color view, inter-view prediction of motion parameters and residual data例如，彩色視的視間預測，運動參數和殘差數據之間的

37、視間預測Using depth data 利用深度數據E.g., view synthesis prediction, motion prediction例如，視圖合成的預測，運動預測Depth coding 深度編碼Independent of texture 不依賴于紋理E.g., depth modeling modes, weighted prediction, reduced resolution coding例如，深度建模的各種方式，加權預測，降低分辨率編碼Using texture data 利用紋理數據E.g., motion parameter inheritance, in

38、tra prediction例如，運動參數的承繼，幀內預測2021/05/08MPEG for the past, present and future of TV273DV Current Status 三維視頻的現狀Extensive subjective evaluation of 20+ proposals to the 3DV CfP 對 3DV 征集提案的 20 多個提案之廣泛客觀評價Considered both AVC and HEVC compatible思索同 AVC 和 HEVC 的兼容性Many new coding tools proposed for MV text

39、ure and depth coding已提出多視紋理和深度的多種新編碼工具Substantial rate savings compared to capabilities of existing standards與現存規范的各種能夠性相比，可顯著節省比特率AVC and HEVC extensions standardisation under wayAVC 和 HEVC 的擴展之規范化正在進展Add support for depth添加對深度的支持New AVC-based coding tools新的基于 AVC 的編碼工具MV extensions of HEVCHEVC 對多視

40、的擴展Hybrid architectures 混合構造2021/05/08MPEG for the past, present and future of TV28R3 的特例R1 建議的最正確性能Hybrid Architectures 各種混合構造From a pure compression efficiency point of view, it is always best to use the most advanced codec從純緊縮效率的觀念來看，采用最先進的編碼器總是最正確選擇However, when introducing new services, provide

41、rs must also consider capabilities of existing receivers and transition plan 然而，在引進各種新效力時，效力提供者還必需思索現存接納機和過渡方案的各種能夠性Use Case: 例如Many terrestrial broadcast systems based on MPEG-2很多地面廣播系統基于 MPEG-2US cable systems based on mix of MPEG-2 and AVC美國有線系統基于 MPEG-2 和 AVC 的混合May not be easy to simply switch

42、codecs in near term在近期簡單切換編碼器并不容易2021/05/08MPEG for the past, present and future of TV29Multiview Extensions of HEVCHEVC 的多視擴展HEVC Stereo/MV extensions being considered已思索 HEVC 雙目立體/多視的各種擴展Extensions expected to be completed approximately 12 months after the base specification, i.e., Jan 2021這些擴展預期在

43、根本規范確定后約 12 個月完成，即 2021 年 1 月Reportedly simple extensions of HEVC can achieve 30-40% bit rate reduction compared to HEVC simulcast已有報告：對 HEVC 的簡單擴展與 HEVC 同播相比，可使比特率下降 30-40 %Further gains expected if block-level changes to codec are considered假設對編碼器思索圖像塊級的各種變卦，預期可獲得更多的增益2021/05/08MPEG for the past,

44、present and future of TV30MPEG-H part 3: 3D AudioMPEG-H 第 3 部分：三維音頻Well, there is no Systems and Video without Audio好，沒有不帶音頻的系統和視頻2021/05/08MPEG for the past, present and future of TV313D Audio requirements/1對三維音頻的需求/1High quality: quality of decoded sound perceptually transparent 高質量：解碼聲的質量在感知上是透明的

45、Localization and Envelopment: Accurate sound localization and very high sense of sound envelopment within a targeted listening area (“sweet spot)定位和組裝：在預定的監聽區 (“欣賞地點) 準確的聲學定位和極高體驗的聲學組裝Flexible Loudspeaker Placement: the transmitted audio program to be mapped to a rendering setup in which loudspeaker

46、s are in alternate locations靈敏的揚聲器擺放：傳輸的音頻節目將映射到一套演奏安裝，其中的揚聲器可處在不同位置Interactivity: Interactive modification of the sound scene rendered from the coded representation交互性：從編碼表示中演奏的聲音場景，可進展交互式調理MPEG for the past, present and future of TV322021/05/083D Audio requirements/2對三維音頻的需求/2Rendering on setups w

47、ith fewer loudspeakers (including headphones): ability to derive a signal from the transmitted program material for reproduction with fewer loudspeakers對較少揚聲器 (包括耳機) 的配置也可演奏：揚聲器較少時，依然可以從傳輸的節目素材中導出一個信號來滿足重現的需求Audio/visual alignment & consistency: ability to adapt the rendered acoustic scene to be con

48、sistent with the visual音頻/視頻的對準和一致性：可以調理演奏的聲學場景，以便與視覺場景堅持一致Efficiency for decoding on different setups: ability to be rendered on all reproduction loudspeaker setups or headphone 在不同安裝上進展解碼的有效性：可以在一切重現的揚聲器安裝或耳機上進展演奏MPEG for the past, present and future of TV332021/05/083D Audio requirements/3對三維音頻的需

49、求/3Appropriate computational complexity and system latency for the target application (Broadcasting, Spatial two-(n-)way communication / telepresence )適當的計算復雜度和系統時延：以滿足目的運用如廣播、空間的 2-(n-) 路通訊 / 遙現Transcoding for low bandwidth devices: A lower bandwidth signal can be extracted from the original progra

50、m material對窄帶寬的設備之轉碼：可從源節目素材中提取窄帶寬信號Backward compatibility: (e.g. to 5.1 channel programs and decoders or transmission of HRTF encoded signals)向后兼容性：例如，兼容 5.1 聲道節目和解碼器，或 HRTF 編碼的信號MPEG for the past, present and future of TV342021/05/08Envisioned Architecture 想象的構造Inputs 輸入Audio Channels 音頻信道“normal

51、content“常規的 內容Audio Objects 音頻對象“helicopter overflight“直升機的飛過Audio Scene 音頻情景Ambisonics環境聲效果35Outputs 輸出Loudspeakers 揚聲器Headphones 耳機Spatialized sound有空間感的聲音In-frame loudspeakers機框內置的揚聲器“Sound Bar“聲吧Channel 信道500 kb/s primary channel500 kb/s 主信道64 kb/s cellular channel64 kb/s 蜂窩網信道MPEG for the past,

52、 present and future of TV2021/05/0822.2 聲道音頻節目音頻對象音頻情景三維音頻編碼器三維音頻解碼器500 kb/s比特流3D AudioDecoder22.2 的多個揚聲器家庭影院平板電視耳機上的空間感機框內置的通話器耳機上有空間感轉碼器64 kb/s比特流智能手機，電視耳機上的空間感Audio, part of an Audio-visual Experience音頻，音視頻體驗的一部分What is new in video? 視頻中有哪些是新的？Larger, high-resolution displays 更大的高分辨率顯示器Greater se

53、nse of envelopment 組裝的更好感知Possibly closer viewing distance 觀看間隔能夠更接近More efficient compression 更高效的緊縮Wireless transmission to portable devices無線傳輸至各種便攜式設備Immersive and enveloping audio experience具有浸潤性的和組裝的音頻體驗362021/05/08MPEG for the past, present and future of TVUse Case 1 Home Theatre例如 1 家庭影院Many

54、 loudspeakers 很多揚聲器10.1 10 只揚聲器加 1 只超低音揚聲器22.2“3D Experience “三維 體驗Height loudspeakers for greater envelopment多個 “音柱 用于更大的組裝Sense that audio objects are near or distant音頻對象為近的或遠的情況時之感知When near, they are consistent with video image假設是近的，它們應該和視頻圖像堅持一致性372021/05/08MPEG for the past, present and future

55、 of TV22.2 multichannel sound system (NHK)22.2 多聲道聲音系統 (日本廣播協會)2021/05/08MPEG for the past, present and future of TV38頂層9 個聲道中層10 個聲道底層3 個聲道低頻效果2 個聲道Home Theater Issues家庭影院有關問題 Maintain compatibility with legacy systems: 5.1, stereo維持同已有系統的兼容性：5.1, 立體聲Will content providers really step up to 22.2?內容

56、提供者真的會晉級到 22.2？Will consumers adopt 22.2?: 2 front high消費者會接受 22.2？：2 個前置的 “音柱How to compress 22.2 for transmission?如何緊縮 22.2 以便傳輸？MPEG AAC 64 kb/s/channel: - 1.5 Mb/sMPEG AAC 64 kb/s/聲道：增至 1.5 Mb/sHow to render for legacy setups? 對傳統安裝如何演奏?Relatively slow-growth market 增長相對較慢的市場2021/05/08MPEG for t

57、he past, present and future of TV39Use Case 2 Personal TV例如 2 個人電視機Small but high-resolution display尺寸較小的、但高分辨率的顯示器“super-tablet “超級平板Local wireless communications 本地的無線通訊To cable or fiber home hub 銜接到有線或光纖的家庭集線器Possible audio presentation 能夠的音頻呈現方式Headphones 耳機Loudspeakers around perimeter of displ

58、ay顯示器周圍的多個揚聲器Issue: To what extent can the user get an enveloping experience from only “front loudspeakers?問題：用戶僅僅從 “前置的 多個揚聲器，終究可獲得多大程度的組裝體驗?40MPEG for the past, present and future of TV2021/05/08Use Case 3 Mobile TV例如 3 挪動電視Hand-held display: Smartphone 手持顯示器：智能手機Headphone listening: 耳機收聽Stereo, p

59、erhaps with binauralisation立體聲；也許有雙耳效應化Fastest growing market: Quick rollout and adoption of standard 增長最快的市場：迅速推進和采用規范What is new? 什么是新的？USAC for stereo; MPEG Surround for binauralisationUSAC 用于立體聲；MPEG 環繞聲用于雙耳效應化What is the “Wow factor? “叫好 要素是什么？To spur adoption of new technology 鞭策采用新技術41MPEG fo

60、r the past, present and future of TV2021/05/08User Experience 用戶體驗42Home Theatre家庭影院Tablet TV平板電視Spatialized on headphones耳機上的空間感2021/05/08MPEG for the past, present and future of TVFlexible Rendering 靈敏的演奏Content providers create 22.2 program內容提供者創建 22.2 節目Consumed on stereo, 5.1, 10.1, 22.2 layout