YY/T 1833.5-2024Artificial intelligence medical device - Quality requirements and evaluation - Part 5: Pre-trained models (English PDF)

Foreword

This document is in accordance with the provisions of GB/T 1.1-2020 "Guidelines for standardization work Part

1.Structure and drafting rules for standardization documents" Drafting. This document is part 5 of YY/T 1833 "Quality Requirements and Evaluation of Artificial Intelligence Medical Devices". YY/T 1833 has been published The following parts.

2.General requirements for data sets;

3.General requirements for data annotation;

5.Pre-trained models. Please note that some of the contents of this document may involve patents. The issuing organization of this document does not assume the responsibility for identifying patents. This document is proposed by the State Food and Drug Administration. This document is under the jurisdiction of the national artificial intelligence medical device standardization technical authority. This document was drafted by: Zhejiang Baishi Medical Technology Co., Ltd., China Food and Drug Inspection Institute, State Drug Administration Medical Device Technology Evaluation Center, National Health Commission Health Development Research Center, Institute of Medical Information, Chinese Academy of Medical Sciences, Suzhou Institute of Advanced Studies, University of Science and Technology, Zhejiang University, Harbin Institute of Technology, Beijing University of Posts and Telecommunications, Beijing Institute of Technology, Tsinghua University, Beijing Tianjin-Hebei National Technology Innovation Center, Shanghai Institute of Medical Device Inspection, PLA General Hospital, Philips (China) Investment Co., Ltd. Company, Huawei Technologies Co., Ltd., and Beijing Yuanying Technology Co., Ltd. The main drafters of this document are. Yan Ziye, Li Jingli, Mao Shufan, You Mao, Liu Xiaoyin, Gao Dongping, Zhou Shaohua, Wu Jian, Huang Jianhua, Zhou Xiuzhuang, Li Jianwu, Wang Xuexia, Luo Lin, Lu Yao, Wang Hao, Meng Xiangfeng, Liu Chongsheng, He Kunlun, Ge Xin, Fu Haifang, Tian Mengqiu, Zhen Hao, Li Shu.

Pre-trained models have become an important concept in the field of artificial intelligence and a common resource for the development of artificial intelligence medical devices. Deep learning algorithms based on models and transfer learning have been widely adopted in the field of artificial intelligence medical devices. The characteristics and quality vary greatly. Artificial intelligence medical devices developed based on pre-trained models have unpredictable risks that affect product safety. The industry urgently needs to propose quality requirements and evaluation methods for pre-trained models to facilitate product quality control of artificial intelligence medical devices. Provide basic protection. According to the AI medical device standard system that has been initially established in my country, YY/T 1833 "Quality Requirements and Standards for AI Medical Devices" The series of basic general standards for "Evaluation" is planned to consist of eight parts.

1.Terminology. The purpose is to provide terminology for quality evaluation activities of artificial intelligence medical devices.

2.General requirements for datasets. The purpose is to propose general quality requirements and evaluation methods for datasets.

3.General requirements for data annotation. The purpose is to propose quality requirements and evaluation methods for data annotation.

4.Traceability. The purpose is to clarify the general requirements and evaluation methods for the traceability of artificial intelligence medical devices.

5.Pre-trained models. The purpose is to standardize the quality of pre-trained models used in artificial intelligence medical devices.

6.Environmental requirements. The purpose is to standardize the operating environment requirements and evaluation methods of artificial intelligence medical devices.

7.Privacy protection requirements. The purpose is to enhance the ability of artificial intelligence medical devices to protect the privacy of subjects.

8.Ethical requirements. The purpose is to achieve the ethical requirements of artificial intelligence from a technical level and protect human rights. This document provides ideas for the quality evaluation of pre-trained models used in artificial intelligence medical devices, and also provides guidance for the subsequent development of detailed It provides a basis for the dedicated quality requirements of algorithmic models. The pre-trained models used in AI medical devices come from a wide range of sources, including medical device manufacturers, third-party suppliers, and third-party service platforms. Due to technical, commercial and policy factors, the technical details, R&D process and quality control of pre-trained models are not fully understood. In order to effectively control and trace the quality of the final products of artificial intelligence medical devices, this document The pre-trained models used in the devices and the corresponding documentation put forward quality requirements and evaluation methods to guide AI medical device manufacturers. Strengthen quality control from within. Since the pre-trained model itself is not a medical device and the technology route is in a rapid development stage, this document does not To constrain the R&D process and avoid limiting innovation. For version changes of pre-trained models, use dynamically updated third-party services, For situations such as self-learning ability, this document is evaluated for quality based on specific time and specific version. Version changes and updates refer to medical device changes. Regulations and standards are implemented. For R&D companies, this document provides a basis for the selection and quality control of pre-trained models. It provides a basis for testing activities of pre-trained models. Quality requirements and evaluation of artificial intelligence medical devices Part

1 Scope

YY/T 1833.5 is the part of the Chinese AI medical device framework that addresses the pre-trained model. It specifies the general quality requirements for pre-trained models used in artificial intelligence medical devices and the corresponding evaluation methods. A pre-trained model is a component the manufacturer usually did not train and often cannot fully characterise, which breaks the assumption behind the rest of the framework: that the data, the training and the resulting behaviour are all documented by the same party. Part 5 exists to say what has to be known about such a model before it can carry a clinical function. It is the newest part of the YY/T 1833 series that YY/T 1990 and YY/T 1991 build on. For a developer using foundation or transfer-learned models in a Chinese submission, this is the governing document.

This document specifies the general quality requirements for pre-trained models used in artificial intelligence medical devices and describes the corresponding evaluation methods. This document applies to pre-trained models used in artificial intelligence medical devices. This document is not intended for evaluation of the development process of pre-trained models.

2 Normative references

The contents of the following documents constitute essential clauses of this document through normative references in this document. For referenced documents without a date, only the version corresponding to that date applies to this document; for referenced documents without a date, the latest version (including all amendments) applies to This document.

YY/T 1833.1 Quality requirements and evaluation of artificial intelligence medical devices Part

3 Terms and definitions

The terms and definitions defined in YY/T 1833.1 and the following apply to this document.

3.1 Pre-trained model pre-trainedmodel A computational model that has been trained on a dataset and can be used as the basis for new tasks.

Note. Pre-trained models are usually used for transfer learning in a narrow sense, where the model is pre-trained on a source task and then fine-tuned on a downstream task; In a broad sense, a pre-trained model may be used as the initial value of a machine learning model for fine-tuning of downstream tasks. The source and category are described in an expanded manner.

3.2 Documentation describing various properties of the pretrained model.

3.3 taskdomaintaskdomain The domain of the specific problem or task that the machine learning model needs to solve.

Note. In the application of artificial intelligence medical devices, examples of task domains include. image lesion detection, imaging sign classification, image ROI (region of interest) classification, segmentation, ultrasound video segmentation, image report generation, ECG signal detection, image generation, process optimization, etc.

3.4 Source Task sourcetask Initial tasks for training machine learning models.

Note. A.2 explains the relationship between source tasks, downstream tasks, and pre-trained models.