GB/Z 177.2-2026Intelligence grading of artificial intelligence terminal - Part 2: General requirements (English PDF)

Foreword

This document was drafted in accordance with the rules given in GB/T 1.1-2020, Directives for standardization - Part 1: Rules for the structure and drafting of standardizing documents.

This document is Part 2 of GB/Z 177, Grading of intelligence for artificial intelligence terminals.

Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. The issuing body of this document is not to be held responsible for identifying any or all such patent rights.

This document was proposed by and is under the jurisdiction of the National Information Technology Standardization Technical Committee (SAC/TC 28).

1 Scope

This document specifies the system for grading the intelligence of artificial intelligence terminals, referred to below as the terminal, covering the division into intelligence levels, the capability elements and the key capabilities of each level, and describes the test methods.

This document applies to the grading of intelligence of terminals of every kind, and provides a reference for the design, development, application, selection and testing of terminals.

2 Normative references

The following document contains provisions which, through normative reference in this text, constitute indispensable provisions of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document, including any amendments, applies.

GB/Z 177.1, Grading of intelligence for artificial intelligence terminals - Part 1: Reference framework

3 Terms and definitions

For the purposes of this document, the terms and definitions given in GB/Z 177.1 and the following apply.

3.1 testing scenario - a composite test situation set up, within a particular application scenario, to test the key capabilities of a terminal.

3.2 testing task - the basic unit of execution in the testing of terminal intelligence. Note: a testing task can be a specific step within a testing scenario, or can be completed on its own as an independent, atomic test item.

3.3 scenario context - the dynamic set of information about the user, the device and the environment that a terminal builds in real time while carrying out a particular task, so as to understand and respond to the intent of the user accurately.

3.4 end-to-end closed-loop - the complete flow from the moment the user issues an instruction or an intent, through the perception, cognition and execution the terminal must perform, to the successful delivery of the result of the task.

3.5 personal knowledge base - the set of personalized information about a user - particular facts, relationships, preferences and habits - that a terminal learns and stores, with the user's authorization, over long-term use.

3.6 session - one process of interaction between the user and the terminal to complete a task or a group of related tasks.

3.7 session context - the set of dynamic information recorded and maintained within one session - the interaction history and the current state - so as to keep the interaction coherent and the task accurately carried out.

3.8 ability element - the basic categories that make up the top-level framework of terminal intelligence capability.

3.9 task decomposition - the capability of a terminal to break a complex intent or a multi-step instruction from the user automatically into a series of smaller, more specific and executable subtasks.

3.10 task orchestration - the capability, on the basis of task decomposition, to plan and generate an optimal or reasonable order of execution according to the logical relations, the dependencies and the priorities among the subtasks.

3.11 intent clarification - the capability, when the terminal recognizes that the intent of the user is unclear, to make that intent clearer by asking of its own accord, offering options or requesting further information.

4 Abbreviated terms

For the purposes of this document, the following abbreviated term applies. TTS: text-to-speech.

5 Overview

According to the complexity of the capability with which a terminal carries out tasks of various kinds within its functional range, and the degree of automation, the level of terminal intelligence is divided into four levels, L1 to L4. The higher the level, the higher the level of intelligence of the terminal. A terminal of a higher level covers all the capabilities of the terminals of lower levels: an L3 terminal, for instance, has at the same time all the key capabilities of L1 and of L2.

6 Division into levels

6.1 L1 response level. The terminal understands a single simple instruction, calls a determined tool according to that instruction and completes a single-step task.

6.2 L2 tool level. The terminal understands the instruction of the user and simple intents and has a simple reasoning capability; it can call preset tools and complete single-step or clearly defined multi-step execution tasks; it can generate content in at least one modality, text, audio or image; and it has short-term memory within a single session.

6.3 L3 assistance level. The terminal can understand the instruction and the intent of the user fully, can carry out intent clarification of its own accord, has a stronger reasoning capability and can complete task decomposition and task orchestration automatically; it can select and call tools dynamically and automatically; it can generate content in at least one modality, text, audio or image; and it has both short-term and long-term memory.

6.4 L4 collaboration level. To be determined. Note: according to the present state of technical development, this document gives the key intelligence capabilities for levels L1 to L3; the division of L4 and higher levels will be made clear and completed in a later revision of this document.

7 Capability elements

7.1 First-level capability elements. 7.1.1 Perception: the capability of the terminal to obtain data from sensors, system services and application services through internal and external sensors, data acquisition modules and recognition modules, and to build the scenario context information the task needs; it comprises perception of user information, of device information and of environmental information. 7.1.2 Cognition: the capability to understand the intent of the user and the related information, to reason about and analyse that information and to plan the task dynamically, so that its execution matches the goal the user expects; it comprises understanding, reasoning and planning. 7.1.3 Execution: the capability to generate a response from the output of the cognitive process and to call internal and external tools and services to achieve the goal of the task; it comprises tool calling, content generation, interconnected collaboration and expressive output. 7.1.4 Memory: the capability, with the user's authorization, to extract, store, retrieve and dynamically update the content of the interaction and the related information; it comprises short-term and long-term memory. 7.1.5 Learning: the capability to improve performance in following instructions, answering knowledge questions and carrying out particular tasks through user feedback, external knowledge input and self-reflection, so as to optimize the output and raise the level of intelligence; it comprises context-adaptive learning and continuous evolutionary learning.

7.2 Second-level capability elements. 7.2.1 Perception. 7.2.1.1 Perception of user information: the capability of the terminal to perceive information of several kinds relating to the user and to collect and recognize the user's biometric features, input content, behaviour and physiological state. Biometric features include voiceprint, face and fingerprint; input content includes text, images, audio and video; behaviour includes gesture, eye movement, remaining still and walking; physiological state includes heart rate and body temperature.

7.2.3.3 Interconnected collaboration: the capability of the terminal to exchange information and operate jointly with other devices, including remote control of external devices, migration of content and tasks across devices, and distributed collaborative processing of complex tasks over several devices. 7.2.3.4 Expressive output: the capability of the terminal to process or optimize the output content and to convert it, through the output module, into a form a person can perceive - sound, image or text; it includes feeding back to the user the state and the result of the task, adapting the manner of output to the scenario, and enhancing the effect of multimedia content.

7.2.4 Memory. 7.2.4.1 Short-term memory: the capability of the terminal to remember the content of the context of a single session. 7.2.4.2 Long-term memory: the capability of the terminal to remember the session history, the preferences of the user, the scenario context or the personal knowledge base over the long term.

7.2.5 Learning. 7.2.5.1 Context-adaptive learning: the capability of the terminal, within a single session, to optimize its behaviour and its content output by analysing the current session context and the examples the user provides. 7.2.5.2 Continuous evolutionary learning: the capability of the terminal to adjust, optimize and extend its internal models and strategies - by analysing positive and negative feedback from the user continuously, reflecting on the causes of failed tasks or deviations in the result, and learning knowledge from external documents and data - so as to raise its intelligence continuously.

8 Key capabilities

8.1 Overview. 8.1.1 Composition of the intelligence capability. The intelligence capability of a terminal is made up of two parts: a) device-side capability, meaning the capability of the terminal to achieve intelligent application functions on its own, relying on its local hardware and software resources; b) device-cloud collaborative capability, meaning the composite capability of the terminal, when connected to the internet, to support intelligent applications by scheduling the computing, storage and service resources of the terminal and of the cloud together. According to differences in computing resources and in typical application scenarios, this document divides terminals into those of strong and those of weak computing power, with different device-side capability requirements, the division following GB/Z 177.1. Because the local computing and storage resources of a terminal of weak computing power are limited, this document gives the device-side key capabilities only for terminals of strong computing power.

8.1.2 Explanation of the key capabilities. The product parts of GB/Z 177 follow the capability elements of this document and may enhance or refine the key capabilities according to the technical characteristics and the application scenario of the terminal, including but not limited to: a) adding key capabilities not covered by this document.

8.3.3.5 Learning. 8.3.3.5.1 Context-adaptive learning: the terminal should have the capability, within a single session, to adjust and optimize its content output dynamically according to the context or the examples of the user. 8.3.3.5.2 Continuous evolutionary learning: no requirement.

9 Test methods

9.1 Principles of testing. Testing of the quantitative parameters of a terminal is generally carried out by executing testing tasks, and testing of the key capabilities by executing testing scenarios. The testing scenarios follow these principles: a) representativeness, meaning that a single testing scenario covers several key capabilities, and that in designing it both how typical it is and how usable it is are fully taken into account; b) equivalence, meaning that where the testing scenario defined in the relevant product part of GB/Z 177 does not apply, a custom testing scenario may be built from the functional characteristics of the terminal, provided it matches the scenario defined in that part in the range of key capabilities covered and in the complexity of the tasks, and provided the testing party has assessed and confirmed it; c) sufficiency of coverage, meaning that the testing scenarios executed together cover all the key capabilities of that level, so that the verification of capability is complete and systematic; d) end-to-end closed loop, meaning that the execution of a single testing scenario generally has to be completed as an end-to-end closed loop, and that only when the terminal has successfully completed all the tasks of the scenario can the key capabilities it covers be judged to have passed.

9.2 Test framework. 9.2.1 Device-side capability. For the device-side key capabilities described in the relevant product part of GB/Z 177, it has to be verified that the capability is achieved by the terminal on its own. The device-side realization of a key capability may be tested in either of these ways: a) the terminal executes the testing task in an environment with no internet connection, confirming that the capability still operates normally without relying on cloud-side services; a terminal of strong computing power may use this way by preference; b) the terminal executes the testing task in a connected environment and outputs material proving that the capability is realized on the device side, such as local model call logs, local service component information and execution traces. 9.2.2 Device-cloud collaborative capability. With the terminal connected to the internet, the key capabilities of the terminal are tested according to the testing scenarios described in the relevant product part of GB/Z 177.

9.3 Determination of the intelligence level. 9.3.1 Conformity method. For classes of terminal that are relatively simple in form and whose formal characteristics, such as the modalities of perception and of expression, do not overlap, the conformity method is used: for the target intelligence level applied for, the device-side capability and the device-cloud collaborative capability of the terminal are tested against the key capabilities corresponding to that level, and only when the terminal conforms in both respects to all the key capabilities of the level applied for can it be judged to have reached that level.