GB/T 44115.3-2025Information technology - Virtual reality content representation - Part 3: Audio (English PDF)

Foreword

This document is in accordance with the provisions of GB/T 1.1-2020 "Guidelines for standardization work Part

1.Structure and drafting rules for standardization documents" Drafting. This document is Part 3 of GB/T 44115 "Information Technology Virtual Reality Content Expression". GB/T 44115 has been published in Lower part.

3.Audio. This document was proposed and coordinated by the National Information Technology Standardization Technical Committee (SAC/TC28). This document was drafted by: Tsinghua University, Beijing Institute of Technology, China Electronics Technology Standardization Institute, Beijing Zitiao Network Technology Co., Ltd. Co., Ltd., Saiyinxinwei (Beijing) Electronic Technology Co., Ltd., Beijing Panorama Information Technology Co., Ltd., Zhongguancun Audiovisual Industry Technology Innovation Alliance, Tsinghua University Tianjin Institute of Electronic Information, Peking University, Migu Culture Technology Co., Ltd., Tencent Technology (Shenzhen) Co., Ltd., Beijing iQiyi Yi Technology Co., Ltd., Xiaomi Communication Technology Co., Ltd., Atmos (Beijing) Intelligent Technology Co., Ltd., Huawei Technologies Co., Ltd., China Telecom Media University. The main drafters of this document are. Dou Weibei, Wang Jing, Geng Yidan, Zhu Bocheng, Li Jingxin, Huang Chuanzeng, Liu Derong, Wu Jian, Wu Qiang, Xu Shumin, Pan Xinde, Qu Tianshu, Li Yuepeng, Shang Shidong, Wang Zhihang, Liu Changtao, Huang Weiqing, Liu Mengmei, Zhao Tianbo, Han Zerui, Wang Bin, Han Jian, Li Lin, Xu Song, Hu Su, Wang Zhe, Gao Yuan, Zhu Mengyao, Jin Cong, Zhang Weimin, Gao Wen, Huang Tiejun, Zhao Haiying, Cui Xiaoran.

With the continuous emergence of virtual reality (VR) technology, virtual reality has become an emerging media form that enhances immersion. Expression and rendering reconstruction technology plays an important role in the immersive and realistic experience of virtual reality. At present, virtual reality panoramic video technology has However, due to the lack of technical specifications for VR audio content expression and rendering reconstruction, it is difficult to realize various VR audio Intercommunication and sharing between video acquisition and production and terminal playback devices or systems. GB/T 44115 aims to establish video and audio specifications applicable to virtual reality and is intended to consist of three parts.

1.System. The purpose is to establish the transmission format and signaling format of efficiently coded virtual reality video and audio data.

2.Video. The purpose is to establish the coding representation and reconstruction methods of virtual reality panoramic video and free viewpoint video.

3.Audio. The purpose is to establish the expression of immersive audio content in virtual reality devices and related systems, including Including metadata structure and renderer system architecture and interface specifications. This document is based on the premise of following ITU-RBS.2076.2019 and is aimed at adapting to my country's panoramic audio broadcasting, immersive audio communication, virtual In the fields of virtual reality audio interaction, the collaborative relationship between VR audio metadata model and rendering system is established, as well as the VR audio metadata flow The possible multiplexing modes between audio streams form the interconnection application specifications for VR audio content expression, supporting 3DoF and 6DoF, etc. In the virtual reality scene, it involves the acquisition, production and rendering reconstruction of three types of signals based on sound channels, objects and scenes, serving virtual reality (VR/ Audio applications in AR/MR/XR) devices and related systems. This document designs a decoupled architecture between the audio metadata model, the audio codec system, and the VR audio rendering system. They are both independent of each other and work closely together to form the application specification of VR audio content expression. This document will define the VR audio coordinate system, the number of It specifies the data model, renderer system, and AVS audio metadata serial stream encapsulation format. The issuing organization of this document calls attention to the fact that when declaring compliance with this document, 6.3, 7.2, 7.5, 9.1, 9.8, 10.1, Appendix D, Appendix The use of the following 52 patents related to virtual reality audio in Appendix H, Appendix I, Appendix R, and Appendix S. The patent names are as follows: PCT/CN2021/100076, Audio rendering system, method and electronic device; PCT/CN2022/098882, Audio rendering system, method and electronic device Method and electronic device; PCT/CN2021/100062, Audio signal encoding method, device and electronic device for audio rendering; PCT/ CN2022/098850, Audio rendering system, method and electronic device; PCT/CN2021/114366, Audio metadata processing method and device PCT/CN2022/114219, Audio signal processing method and device; PCT/CN2021/121135, Fading in and out of sound path energy Output method, electronic device and medium; PCT/CN2022/122204, Audio rendering method, audio rendering device and electronic device; PCT/ CN2021/121718, a method for dynamically estimating a scene approximating a rectangular room; PCT/CN2022/122635, an audio rendering system and method; PCT/CN2021/104309, reverberation time estimation method, audio signal rendering method and electronic device; PCT/CN2022/ 103312, Audio signal rendering method, device and electronic device; PCT/CN2021/115130, Signal processing method, device and electronic device for audio rendering Device and electronic device; PCT/CN2022/115194, Signal processing method, device and electronic device for audio rendering; PCT/ CN2021/121729, System, method and electronic device for spatial audio rendering; PCT/CN2022/122657, System, method and electronic device for spatial audio rendering 202110984837.4, an audio production model and generation method, an electronic device and a storage medium; 202111102045.6, Audio program metadata and generation method, electronic device and storage medium; 202111100818.7, Audio content metadata and generation method, electronic device and storage medium; 202111102038.6, Audio object metadata and generation method, electronic device and storage medium Quality; 202111205630.9, Audio track unique identification metadata and generation method, electronic device and storage medium; 202111204386.4, An audio track Channel metadata and generation method, electronic device and storage medium; 202111202898.7, An audio stream metadata and generation method, electronic device and storage medium Storage medium; 202111308422.1, Metadata and generation method, device and medium based on audio bed audio package format; 202111308430.6, Base object Audio package format metadata and generation method, device and medium; 202111306844.5, Scene-based audio package format metadata and generation method, device and medium 202111308421.7, based on binaural audio package format metadata and generation method, device and medium; 202111021068.4, based on Based on the audio channel metadata and generation method, device and storage medium of the sound bed; 202111020417.0, based on the audio channel metadata and generation method of the object Method, device and storage medium; 202111021066.5, Scene-based audio channel metadata and generation method, device and storage medium; 202111021039.8, Metadata and generation method, device and storage medium based on binaural audio channel; 202111666346.1, A broadcast audio format Method, device, equipment and storage medium for generating audio metadata blocks; 202111666362.0, Method, device, equipment and storage medium for generating audio metadata blocks Medium; 202210588174.9, Method, device, equipment and storage medium for generating internal data structure of renderer; 202210634563.0, Using metadata Method and device for rendering audio based on a sound bed; 202210762912.7, configuration method, device, equipment and shared renderer component Storage medium; 202210760302.3, a sound bed rendering item data mapping method, device, equipment and storage medium; 202210603204.9, a rendering A method, device, apparatus and storage medium for determining rendering items of a sound bed output; 202210600880.0, a method, device, apparatus and storage medium for determining rendering items of a sound bed output 202210603208.7, a method, device, apparatus and storage medium for determining object output rendering items; 202210603212.3, a A method, device, equipment and storage medium for determining scene output rendering items; 202210603184.5, a rendering item processing method, device and storage medium for an audio renderer 202210608202.9, A scene rendering item data mapping method, device, equipment and storage medium; 202210782056.1, A method, device, apparatus and storage medium for calculating gain of an audio renderer; 202210910129.0, For object renderer Metadata parsing method, device, equipment and medium for object-based audio rendering using metadata; 202210907370.8, Method and medium for object-based audio rendering using metadata Device; 202210912275.7, Method and device for rendering scene-based audio using metadata; 202211057713.2, Using metadata Method and device for rendering object-based audio using metadata; 202211063746.8, Rendering scene-based audio using metadata 201510795213.2, A spherical microphone array distribution method for a 3D recording system; 202011306975.9, Audio Playback method and device, electronic device and storage medium; 202210451743.5, An audio processing method, device, electronic device and readable storage medium Storage medium. The issuing organization of this document takes no position on the authenticity, validity and scope of this patent. The patent holder has promised to the issuing agency of this document that he is willing to cooperate with any applicant under reasonable and non-discriminatory terms and conditions. Negotiate patent licensing. The patent holder's statement has been filed with the issuing agency of this document, and relevant information can be obtained through the following Contact information. Contact. Huang Tiejun Correspondence address. Room 2641, Science Building 2, Peking University, No.

5 Yiheyuan Road, Haidian District, Beijing Postal Code.100871 Email. tjhuang@pku.edu.cn Tel. 8610-62756172 Fax. 8610-62751638 Please note that in addition to the above patents, some of the contents of this document may still involve patents. The issuing agency of this document does not assume the responsibility for identifying patents. responsibility. Information technology Virtual reality Content expression Part

1 Scope

GB/T 44115.3-2025 specifies how audio is represented in virtual reality content in the Chinese VR standards series. Sound is what makes a virtual environment believable, and it is harder to get right than the picture: a sound has a position in the scene that must stay fixed as the listener turns their head, and reproducing that over headphones means convolving it with a head-related transfer function, while reproducing it over loudspeakers means an entirely different rendering problem. This document specifies both. It sets the serial stream syntax rules and the system framework, the metadata definition that carries the audio objects, channels and scene description, the AAMSF encapsulation format, and then the headphone renderer and the loudspeaker renderer in full, with two informative annexes on the selection of an HRTF library and on a VR audio method based on linear interpolation of HRTFs. At 395 pages it is a complete codec and rendering specification rather than a framework document. For a VR platform, a headset maker, an audio middleware vendor or a content producer working to Chinese requirements, this is the governing text.

This document specifies the presentation of immersive audio content in virtual reality (VR) devices and related systems, including metadata structures and System architecture and interface specifications of the renderer. This document is applicable to audio acquisition, transmission, rendering and playback in the fields of panoramic audio recording and broadcasting, immersive audio communication, virtual reality audio interaction, etc. Release system.

2 Normative references

The contents of the following documents constitute essential clauses of this document through normative references in this document. For referenced documents without a date, only the latest version (including all amendments) applies to this document. This document.

GB/T 5271.1 Information technology vocabulary Part

3 Terms and definitions

The terms and definitions defined in GB/T 5271.1, GB/T 5271.4, GB/T 5271.9 and GB/T 33475.3 and the following terms and definitions apply to this document.

4 Organization of data

GB/T 5271.9 Information technology vocabulary Part

9 Data communications

GB/T 17191 (all parts) Information technology Digital storage media with a data transfer rate of