GB/T 42382.1-2023Information technology - Neural network representation and model compression - Part 1: Convolutional neural network (English PDF)

1 Scope

The document lays down the representation and the compression process of the offline model of a convolutional neural network.

It applies to the development, the design, the testing and the evaluation of convolutional neural network models of every kind, and to their efficient application in the device and cloud domains.

A note adds that the representation and model compression methods laid down in the document do not require native support from a machine learning framework, and may be supported by means of conversion, of a toolkit or of similar forms.

2 Normative references

One document is cited: GB/T 5271.34-2006, Information technology, Vocabulary, Part 34: Artificial intelligence, neural networks. For dated references only the edition cited applies; for undated references the latest edition, including any amendments, applies.

3 Terms and definitions

3.1 codec representation: use of compression technology to reduce the size of a model. A note refers to Clause 10 for the detailed definition.

3.2 layer: hierarchical structure within a neural network. A note adds that every network layer contains several operators, for instance the input layer, the convolution layer and the fully connected layer.

3.3 reference random vector: basic symbol vector shared by the whole network.

3.4 multiple INT4 quantization: form of quantization in which one tensor is quantized into a combination of several INT4 tensors.

3.5 encapsulation representation: as printed, the definition reads that it puts out interfaces such as security information and identity verification. A note refers to model protection in Clause 11 for the detailed definition.

3.6 block structured matrix: matrix that can be divided into several blocks, each block being arranged according to some rule.

3.7 block circulant matrix: matrix each of whose blocks is a circulant matrix.

3.8 shared weight, shared weight value, shared weight tensor: where one neural network handles several different tasks, the portion of the weights that those tasks share.

3.9 shared bias, shared bias vector: where one neural network handles several different tasks, the portion of the bias that those tasks share.

3.10 convolutional neural network: feedforward neural network that uses convolution in at least one of its layers. A note adds that it is a representative deep learning algorithm in artificial intelligence applications, and refers to ISO/IEC DIS 22989:2022 for the definition.

3.11 compact representation: serialized format output after compression by any of several neural network compact representation algorithms. The example given lists structured matrices, weight sharing between multiple models, the sparse matrix of sparse coding, the quantization table of weight quantization, the decomposed matrices of low-rank decomposition and the bit representation of a binary neural network. A note refers to the compression process, the decompression process and the data generation methods in Clause 7 to Clause 9.

3.12 basic symbol vector: comprises the basic symbol vector shared by the whole network and the basic symbol vector corresponding to each layer, the latter being obtained from the basic symbol vector of the network according to an agreed rule.

3.13 correct input vector: vector obtained by multiplying the elements of the input vector of a network layer by the elements at the corresponding positions in the disturbance vector.

3.14 structured matrix: particular class of matrix that can be built into a complete matrix from a smaller quantity of data together with a definite arrangement rule.

3.15 convolution kernel granularity quantization: form of quantization in which the weights of a convolution layer are divided by output channel and each 3D convolution kernel is quantized one by one.

3.16 interoperable representation: definition of the syntax of a neural network, of the arithmetic operations it supports and of the weight formats, which are integer, floating point and fixed point. A note refers to the syntax and semantics of the neural network model in Clause 6.

3.17 quantitative scale factor: scaling ratio between the original tensor and the quantized tensor.

3.18 model compression: method that reduces the size of a neural network model and raises the efficiency with which the model runs and is transmitted.

3.19 model: neural network corresponding to the completion of a single task or of several tasks.

3.20 bias, offset, bias vector, offset vector: systematic deviation with respect to a reference value. A note refers to ISO/IEC 2382:2015 for the definition.

3.21 weight, weight tensor, connection weight, connection strength, synaptic weight: coefficient by which the input value of an artificial neuron is multiplied before it is combined with the other input values. A note refers to GB/T 5271.34-2006.

3.22 weight aggregation dimension: dimension along which the shared weight tensor and the unique weight vector are concatenated.

3.23 task: some function realized by means of a neural network. The examples given are image super-resolution and image denoising.

3.24 disturbance vector, disturbance symbol vector: symbol vector obtained by expanding the random vector so that its number of dimensions equals that of the input vector of the network layer.

3.25 neural network representation: method of defining the basic syntax, the semantics, the operations, the hyper-parameters and the bitstream of a neural network.

3.26 random vector, random symbol vector: basic symbol vector corresponding to a single network layer.

3.27 unique weight, unique weight value, unique weight tensor: where one neural network handles several different tasks, the portion of the weights that is not shared, for any one of those tasks.

3.28 unique weight list: where one neural network handles several different tasks, the portion other than the shared weight values, that is, the combination of the weight portions not shared among the several tasks.

3.29 specific offset vector: where one neural network handles several different tasks, the portion of the bias that is not shared, for any one of those tasks.

3.30 list of unique offset vectors: as printed, where one neural network handles several different tasks, the portion other than the shared weight values, that is, the combination of the weight portions not shared among the several tasks.

3.31 circulant matrix: a circulant matrix is one kind of structured matrix, its arrangement rule being that the current row or column is the circular shift of the preceding row or column.

4 Abbreviations

CNN is the convolutional neural network; DNN is the deep neural network; PQ is product quantization.

5 Conventions

5.1 Rules. The mathematical operators used in the document and their precedence are similar to those used in the C language, but integer division and the arithmetic shift operations are given a specific definition. Unless otherwise stated, numbering and counting begin at 0.

5.2 Arithmetic operators. Table 1 defines the arithmetic operators. Plus denotes addition. Minus denotes subtraction when used as a binary operator and negation when used as a unary prefix operator. The multiplication sign denotes multiplication. The superscript form denotes exponentiation, that is, a raised to the power b, and may also denote a superscript. The solidus denotes integer division with truncation towards 0: the examples given are that 7 divided by 4 and minus 7 divided by minus 4 are truncated to 1, while minus 7 divided by 4 and 7 divided by minus 4 are truncated to minus 1. The division sign denotes division with neither truncation nor rounding, as does the fraction form of a over b. The summation sign denotes the cumulative sum of the function f of i taken over all the integer values of the variable i from a to b inclusive. The percent sign denotes the modulo operation, that is, the remainder of a divided by b, where a and b are both positive integers.

6 Structure of GB/T 42382

The foreword records that the document is Part 1 of GB/T 42382, Information technology, Neural network representation and model compression, and that Part 1, Convolutional neural network, has been issued.

The introduction states that GB/T 42382 is intended to establish the rules for neural network representation and model compression applicable to convolutional neural networks, to large-scale pre-trained networks and to graph neural networks, and is planned to consist of three parts. Part 1, Convolutional neural network, is intended to establish the representation and model compression standard for convolutional neural networks. Part 2, Large-scale pre-trained model, is intended to establish the model representation, model compression and model transmission standard for large-scale pre-trained networks. Part 3, Graph neural network, is intended to establish the representation of graph data and of graph neural networks and to fix the encoding format standard for graph neural network models.

The introduction also states that neural network representation and model compression are a component of the artificial intelligence technology system and a precondition for applying artificial intelligence across the branches of the national economy, and that the inability of multi-source algorithm platforms to work together, models not being mutually convertible, has held back the spread and application of the technology; the document sets out to regulate representation and model compression in order to secure cross-platform interoperability and to raise the reuse of models.

A patent declaration in the introduction lists the clauses concerned and the patents that may be involved when conformity with the document is claimed, together with the patent holders and the contact address; the patent holders named are Peking University, Huawei Technologies, Beijing Baidu Netcom Science Technology, Xiamen University, Zhejiang University, Zhao Hengrui and the Institute of Automation of the Chinese Academy of Sciences. Annex A carries the patent list.