GB/T 45288.3-2025Artificial intelligence - Large models - Part 3: Service capability framework and maturity assessment (English PDF)

Foreword

This document is in accordance with the provisions of GB/T 1.1-2020 "Guidelines for standardization work Part

1.Structure and drafting rules for standardization documents" Drafting. This document is part 3 of GB/T 45288 "Artificial Intelligence Big Model". GB/T 45288 has published the following parts.

1.General requirements;

2.Evaluation indicators and methods;

3.Service capability maturity assessment. Please note that some of the contents of this document may involve patents. The issuing organization of this document does not assume the responsibility for identifying patents. This document was proposed and coordinated by the National Technical Committee for Information Technology Standardization (SAC/TC28). This document was drafted by: China Electronics Standardization Institute, Huawei Technologies Co., Ltd., Inspur Cloud Information Technology Co., Ltd., Tsinghua University University, Huawei Cloud Computing Technology Co., Ltd., Institute of Automation, Chinese Academy of Sciences, Beijing Baidu Netcom Technology Co., Ltd., Shenzhen Tencent Computer Computer Systems Co., Ltd., Fit (Tianjin) Testing Technology Co., Ltd., Beijing Qihoo Technology Co., Ltd., Beijing University of Aeronautics and Astronautics, Guoneng Information Technology Co., Ltd. Information Technology Co., Ltd., Qilin Hesheng Network Technology Co., Ltd., Shanghai Artificial Intelligence Industry Association, Shanghai Enflame Technology Co., Ltd. Company, Alibaba Cloud Computing Co., Ltd., Pingtou Ge (Shanghai) Semiconductor Technology Co., Ltd., Shanghai Computer Software Technology Development Center, Zhejiang University Hua Technology Co., Ltd., Qingdao Hisense Electronic Technology Service Co., Ltd., Shanghai Artificial Intelligence Research Institute Co., Ltd., China Southern Power Grid Intelligent Technology Co., Ltd., Aerospace Information Co., Ltd., Guangdong Power Grid Co., Ltd., Peking University Changsha Institute of Computing and Digital Economy Institute of Software, Chinese Academy of Sciences, Ant Group Co., Ltd., China Mobile Communications Group Co., Ltd., Mashang Consumer Finance Co., Ltd., Shenzhen Yuntianlifei Technology Co., Ltd., Shenzhen Simo Information Technology Co., Ltd., Beijing Greenland Deepin Information Technology Co., Ltd., China Southern Power Grid Co., Ltd. Ultra-high Voltage Transmission Company, Beijing Software Product Quality Inspection Center Co., Ltd., China Electric Power Research Institute Co., Ltd., Shanghai Wenyu Information Technology Co., Ltd., Inspur Software Technology Co., Ltd., Inspur Electronic Information Industry Co., Ltd., Inspur Software Group Co., Ltd., China Electronics Technology Group Corporation Big Data Research Institute Co., Ltd., Shanghai SenseTime Energy Technology Co., Ltd., China Telecom Corporation Limited, iFLYTEK Co., Ltd., China Telecom Corporation Limited Beijing Research Institute, China Mobile (Suzhou) Software Technology Co., Ltd., Xinjiang Institute of Physical and Chemical Technology, Chinese Academy of Sciences, Hangzhou Hikvision Digital Technology Co., Ltd. Shanghai Wenyu Information Technology Co., Ltd., Northwestern Polytechnical University, Unisound Intelligent Technology Co., Ltd., Beijing University of Technology, Beijing Zhi Core Microelectronics Technology Co., Ltd. The main drafters of this document are. Xu Yang, Ma Shanshan, Yu Chao, Wang Waner, Dong Jian, Tao Jianhua, Cao Xiaoqi, Bao Wei, Huang Xiancui, Ma Chenghao, Zheng Jiajia, Zheng Zimu, Zhu Guibo, Wang Jinqiao, Liu Jing, Wang Qunbo, Yang Xu, Ma Tongsen, Jin Wei, Liu Haitao, Cao Bin, Zhang Xiangzheng, Ren Haifeng, Liu Xianglong, Liu Aishan, Zhang Xu, Chen Xi, Zhao Chunhao, Jiang Yan, Mei Jingqing, Peng Juntao, Zhang Yibo, Chen Mingang, Kong Weisheng, Liu Wei, Liu Changyu, Song Haitao, Ren Zhengguo, Shao Yanning, Liu Jianing, Zhou Hao, Yang Chao, Meng Lingzhong, Sun Xi, Jin Di, Li Kuan, Wang Zhifang, Lü Jiangbo, Hu Quanyi, Wang Ning, Wang Zhigang, Kong Hao, Mo Wenhao, Zhong Kaitao, Wang Kechen, Liu Lu, Zhang Tianlin, Jiang Hui, Liu Jingqian, Liu Weichen, Gao Jianqing, Meng Jian, Shu Juelin, Shang Xingyu, Li Xudong, Yang Yating, Zhong Kailun, Zhong Kaitao, Zhang Tao, Liang Jiaen, Liu Zheng, Zheng Zhe, Wu Shanshan.

Big models have become an important technical means for the development of artificial intelligence and play an important role in leading industrial transformation. Relevant institutions have successively researched and developed more than 100 large-scale model products and evaluation lists, making it difficult for users to effectively evaluate the technical level of artificial intelligence products. GB/T 45288 aims to specify the technical requirements, evaluation indicators and service capabilities of general large models. It is planned to consist of five parts. constitute.

1.General requirements. The purpose is to establish a reference architecture for large models and specify general technical requirements.

2.Evaluation indicators and methods. The purpose is to establish the evaluation indicators of large models and describe the evaluation methods.

3.Service capability maturity assessment. The purpose is to provide the large model service capability maturity level and assessment method.

4.Computer vision big model. The purpose is to define the concept and function of the computer vision big model and specify the technical requirements and testing methods.

5.Multimodal large models. The purpose is to define the concept and function of multimodal large models, specify technical requirements and tests method. Artificial Intelligence Big Model Part

1 Scope

Part 3 of China's national standard series on large models, giving a service capability framework and maturity assessment. The shift this part addresses is that large models are bought as a service rather than as software: an organisation contracts for platform access, for customisation of a base model on its own data, and for inference operations at a service level. What it is buying is a capability, and capabilities are what maturity models describe. The questions are the ones that decide whether a deployment succeeds - can the provider actually fine-tune on proprietary data without leaking it, can it hold latency under load, can it version and roll back a model, can it explain what changed. This part provides the capability framework and its evaluation indicators, describes the maturity levels and the assessment method, and serves both sides of the contract as well as the planning, design and implementation of such services.

This document provides a large model service capability framework and evaluation indicators, and describes the maturity level classification and evaluation of large model service capabilities. method. This document is applicable to service providers and demanders to conduct a comprehensive assessment of the capabilities of large model platforms, model customization, and reasoning operation services. It is also suitable for guiding the planning, design and implementation of large model service capabilities.

2 Normative references

The contents of the following documents constitute the essential clauses of this document through normative references in this document. For referenced documents without a date, only the version corresponding to that date applies to this document; for referenced documents without a date, the latest version (including all amendments) applies to This document.

GB/T 42018-2022 Information technology artificial intelligence platform computing resources specification

3 Terms and definitions

The following terms and definitions apply to this document.

3.1 Large-scale model platform large-scale model platform A software and hardware platform that provides various resources for developing or using large models.

Note. The large model platform does not include the large model.

3.2 Large-scale model service Services to develop and apply large models and large model systems, as well as services to support the business activities of the customer in this way.

Note. The big model system is the integration of the big model and the big model platform, and is a collection of activities, processes, etc. related to the big model service.

3.3 Toolchain A collection of software to support large model development, customization, and application.

3.4 Instruct A pair of signals consisting of the input and output of the large model.

Note. For large models of natural language processing, instructions are presented in pairs of question text and answer text.

4 Abbreviations

The following abbreviations apply to this document.