Valid

GB/T 41813.2-2022Information technology—Intelligent speech interaction testing method—Part 2: Semantic understanding (English PDF)

信息技术 智能语音交互测试方法 第2部分:语义理解

Preview PDF

This is a limited preview

Buy now to download the full PDF (15 pages)

Need quotation for your management?

Get a formal quotation for GB/T 41813.2-2022 in 30 minutes.

Issued by

SAMR; SAC

Level / Type

National · Recommended

Issue date

October 14, 2022

Implementation date

May 1, 2023

Scope

GB/T 41813.2-2022 is the English-translated version of 信息技术 智能语音交互测试方法 第2部分:语义理解.

GB/T 41813.2-2022 is the second part of GB/T 41813 and describes how to test the semantic understanding side of an intelligent speech interaction system.

The functional test items it covers are intent understanding, named entity recognition, identification of sensitive information, semantic rejection, information retrieval, text similarity calculation, text modification, semantic correction, natural language generation, logical reasoning, dialogue guidance and context-sensitive multi-round conversation; the performance items are the effect and the efficiency of semantic understanding and system stability.

It also fixes the test dataset, tools, environment, execution and result determination, with a normative Appendix A for subjective experience testing. Speech service providers, their users and third-party test bodies use it to design and run such tests.

It quotes GB/T 36464 (all parts). Issued on 14 October 2022, it has applied since 1 May 2023.

Found what you need?

Document preview — GB/T 41813.2-2022

National Standard of the People's Republic of China

ICS
35.240.01
Classification
L 77

Issued by: State Administration for Market Regulation; Standardization Administration of the PRC

Contents

  • Preface
  • Introduction
  • 1 Scope
  • 2 Normative references
  • 3 Terms and Definitions
  • 4 Overview
  • 5 Test Preparation and Execution
  • 5.1 Test dataset
  • 5.2 Test Tools
  • 5.3 Test Environment
  • 5.4 Test execution
  • 5.5 Result determination
  • 6 Functional Test Method
  • 6.1 Intent Understanding
  • 6.2 Named Entity Recognition
  • 6.3 Identification of sensitive information
  • 6.4 Semantic Rejection
  • 6.5 Information Retrieval
  • 6.6 Text similarity calculation
  • 6.7 Text Modification
  • 6.8 Semantic Correction
  • 6.9 Natural Language Generation
  • 6.10 Logical Reasoning
  • 6.11 Dialogue Guide
  • 6.12 Context-sensitive multi-round conversations
  • 7 Performance Test Method
  • 7.1 The effect of semantic understanding
  • 7.2 Efficiency of Semantic Understanding
  • 7.3 System Stability
  • Appendix A (normative) Subjective Experience Test
  • A.1 Overview
  • A.2 Test item
  • A.3 Test method
  • Reference

Foreword

This document is in accordance with the provisions of GB/T 1.1-2020 "Guidelines for Standardization Work Part 1: Structure and Drafting Rules of Standardization Documents" drafted.

This document is the second part of GB/T 41813 "Information Technology Intelligent Voice Interaction Test Method". GB/T 41813 has been released the following parts.

--- Part 1: Speech recognition;

--- Part 2: Semantic understanding.

Introduction

Intelligent voice interaction is used in smart home, smart customer service, mobile terminals, vehicle terminals, as well as smart education, smart medical care, smart office, and services.

It is widely used in many fields such as robotics, and has become one of the important ways of human-computer interaction. With the deepening of intelligent voice interaction, it is necessary to The system reference framework, basic technical requirements, and Internet interface requirements for intelligent voice interaction are unified and standardized. In this regard, the state has formulated It has established the basic national standard supporting intelligent voice interaction system. On this basis, it is also necessary to use a unified test method and evaluation standard to The ability of intelligent voice interaction system is evaluated, and the basic method and basis for evaluation of products and services related to intelligent voice interaction are provided.

GB/T 41813 "Information Technology Intelligent Voice Interaction Test Method" is GB/T 36464 (all parts) "Information Technology Intelligent Language Audio Interaction System" provides basic and general testing methods. Intelligent voice interaction includes three basic loops. speech recognition, semantic understanding and speech synthesis.

The test objects, test items, test environments and test methods involved in each link are different. GB/T 41813 "Information Technology Intelligent Voice Interaction Test Methods" aims to establish and describe general test items and general test methods applicable to all aspects of intelligent voice interaction. It consists of three parts.

--- Part 1: Speech recognition. The purpose is to provide general test items and common test items for speech recognition in intelligent voice interaction applications. using the test method.

--- Part 2: Semantic understanding. The purpose is to provide general test items and common test items for semantic understanding in intelligent voice interaction applications. using the test method.

--- Part 3: Speech synthesis. The purpose is to provide general test items and common test items for speech synthesis in intelligent voice interaction applications. using the test method.

1 Scope

This document describes common test items and common test methods for semantic understanding systems in intelligent speech interaction testing.

This document applies to the semantic understanding system of intelligent voice interaction applications by intelligent voice service providers, users and third-party testing agencies Design and implementation of tests.

2 Normative references

The contents of the following documents constitute essential provisions of this document through normative references in the text. Among them, dated citations documents, only the version corresponding to that date applies to this document; for undated references, the latest edition (including all amendments) applies to this document.

GB/T 36464 (all parts) Information technology intelligent voice interactive system

3 Terms and Definitions

The terms and definitions defined in GB/T 36464 (all parts) and the following terms and definitions apply to this document.

3.1 Enables functional units to understand the intent of a person's speech

[Source: GB/T 36464.1-2020, 3.11]

3.2 Named entity

An entity that refers to a name with a specific or unique meaning.

3.3 Intent

The tasks or goals that the system needs to perform in the process of voice interaction.

4 Overview

The semantic comprehension test in the intelligent voice interaction test mainly includes functional test and performance test, as follows.

---Functional testing is used to check whether the system under test provides various functions related to semantic understanding, including. intent understanding, named entities Identification, sensitive information identification, semantic rejection, information retrieval, text similarity calculation, text modification, semantic modification, natural language generation Multi-turn dialogues that are successful, logical reasoning, dialogue guidance, and contextual relevance;

---Performance test is used to check various performances related to semantic understanding in the system under test, including. semantic understanding effect, semantic understanding efficiency and system stability.

During the testing process, online/offline functional and performance tests should be performed according to the technical specifications of the system under test. Methods that can use subjective experience To carry out the test, the subjective experience test shall be performed according to the test items and test methods described in Appendix A. The test items listed in this document can be used for […]

......
This preview omits tables, figures, formulas and parts of the technical clauses. The complete document — 15 pages — is available in the English PDF.

Referenced standards

Normative references

GB/T 36464

Similar standards

How to Download GB/T 41813.2-2022

  1. 1

    Add to cart

    Click "Download PDF" on this page and choose "Buy this standard". You can add more standards before checkout.

  2. 2

    Checkout

    Enter your email and billing details. Payment is processed securely by Stripe (cards or bank transfer). Prefer bank details? Request a quotation (min. order $300).

  3. 3

    Fast delivery

    After payment you'll receive an email confirmation. The English PDF is sent to the same address, usually within minutes and at the latest within 1 business day.

  4. 4

    Invoice included

    A tax invoice is attached to the confirmation email. Need a custom invoice? Contact us.

Related Standards

English PDF
15 pages
Fast delivery
Invoice included
View Cart

Secure payment via Stripe

Payments accepted

VisaMastercardAmerican ExpressApple PayGoogle PayStripe

GB/T 41813.2-2022

$240.00