YY/T 1990-2025Artificial intelligence medical device - Computer assisted analysis software for cytopathologic images - Algorithm performance test methods (English PDF)

Foreword

This document complies with the provisions of GB/T 1.1-2020 "Standardization Work Guidelines Part

1.Structure and Drafting Rules of Standardization Documents". Drafting. Please note that some content in this document may involve patents. The issuing organization of this document assumes no responsibility for identifying patents. This document was proposed by the National Medical Products Administration. This document is under the jurisdiction of the unit responsible for the standardization technology of artificial intelligence medical devices. This document was drafted by: Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences; Shandong Cancer Hospital; National Institutes for Food and Drug Control; and other relevant organizations. The Greater Bay Area Branch of the National Medical Products Administration's Medical Device Technical Review and Inspection Center, Union Hospital affiliated with Tongji Medical College of Huazhong University of Science and Technology, and the Navy The First Affiliated Hospital of Military Medical University (Shanghai Changhai Hospital), Harbin Institute of Hematology and Oncology, and Shenzhen National Research Institute of High-Performance Medical Devices have Limited Liability Company, Shenzhen Jiankang Intelligent Technology Co., Ltd., 91360 Medical Technology Nanjing Co., Ltd., Shanghai Xingmai Information Technology Co., Ltd. Shanghai United Imaging Healthcare Technology Co., Ltd., Beijing Thorough Future Technology Co., Ltd., Beijing Institute for Medical Device Testing (Beijing Medical Biotechnology Co., Ltd.) (Including the Physical Protective Equipment Inspection and Research Center), Shanghai Medical Device Inspection and Research Institute, Beijing-Tianjin-Hebei National Technology Innovation Center, and Xi'an Jiaotong University. The main drafters of this document are. Qin Wenjian, Ru Kun, Liu Dongge, Zhao Miaoqing, Li Jingli, Nie Xiu, He Miaoxia, Liu Yu, Wei Wanxu, Lin Yani, and Wang Hui. Wang Hao, He Jiaye, Meng Xiangfeng, Di Feng, He Chuan, Zhan Yiqiang, Wang Meiying, Wang Ruixia, Liu Chongsheng, Luo Lin, Li Chen.

This document serves as a methodological standard, primarily used for algorithm performance testing of AI-assisted cytopathology analysis software, focusing on cytopathology images. Applications include segmentation and cell type recognition. Given that the scope of applications and technologies in this field are still evolving, applicability and risks have been carefully considered. New quality requirements and evaluation methods proposed later are not limited by this document. Artificial intelligence medical devices for cell pathology image assistance Analysis Software Algorithm Performance Testing Methods

1 Scope

YY/T 1990 sets out how to test an AI cytopathology tool before it can be trusted with a slide. It describes the algorithm performance testing methods for software that applies artificial intelligence to the auxiliary analysis of cytopathological images, covering tasks such as segmentation and cell type recognition, and it applies to software that post-processes those images. Software involved in image acquisition, pre-processing or process optimisation is excluded. It builds directly on the YY/T 1833 series for terminology, datasets and data labelling and on YY/T 1858 for lung imaging, so it reads as the cytology member of an established family rather than a standalone text. An informative annex works through an example of test data collection and labelling. For anyone registering digital pathology or cervical screening software in China, this is the method a reviewer will expect the validation report to follow.

This document describes the algorithm performance testing method for cell pathology image-assisted analysis software using artificial intelligence technology. This document applies to auxiliary analysis software that uses artificial intelligence technology to post-process cytopathological images. This document is not applicable to software related to cytopathology image acquisition, preprocessing, and process optimization.

2 Normative references

The contents of the following documents, through normative references within the text, constitute essential provisions of this document. Dated citations are listed below. For references to documents, only the version corresponding to that date applies to this document; for undated references, the latest version (including all amendments) applies. document.

YY/T 1833.1 Quality Requirements and Evaluation of Artificial Intelligence Medical Devices Part

3 Terms and Definitions

The terms and definitions in YY/T 1833.1, YY/T 1833.2-2022, YY/T 1833.3, and YY/T 1858-2022, as well as the following terms and definitions, shall apply. The meaning applies to this document.

3.1 Digital pathology image High-resolution images acquired by combining digital sensing technology with optical magnification devices and scanning in a fully automated microscope or optical magnification system. The digital images can be viewed on a computer monitor.

3.2 stress sample Within the calibration range of a certain algorithm model, samples with extremely large or extremely small feature capacity are used to determine the generalization performance and reliability of the algorithm model. stability.

Note. Examples of stress samples, such as complex cytopathic lesions, heterogeneous lesions, rare data, or pathological images that introduce noise not included in the cytopathology slide. [Source: YY/T 1858-2022, 3.8, with modifications]

4 Algorithm Performance Testing Requirements

4.1 General Rules The algorithm performance testing process follows the requirements of section

4.1 of YY/T 1858-2022, establishing test documentation and providing a clear and standardized test plan; such as... The testing process requires retesting, but the number of retests should be limited (e.g., not exceeding the number of cell classifications performed by the algorithm) to prevent the algorithm from altering the reference standard.