GB/T 38637.2-2020Internet of things—Access of sensing and controlling device—Part 2:Data management requirements (English PDF)

3.1 Perceptual data

The raw data acquired through data collection or the data representing object information processed on this basis are collectively referred to as.

[GB/T 33474-2016, definition 3.11]

3.2 Control data

The data that acts on the object to perform control operations.

[GB/T 33474-2016, definition 3.12]

3.3 Structured data

A data representation form in which the structure of each record assembled by data elements is consistent and can be used

The relationship model can be effectively described.

[GB/T 35295-2017, definition 2.2.13]

3.4 Unstructured data

Data that does not have a predefined model or is not organized in a predefined way.

[GB/T 35295-2017, definition 2.1.25]

3.5 Semi-structured data

Data model structure that does not conform to the form of a relational database or other data tables, but contains relevant tags to separate words

A form of data structure that hierarchizes records and fields.

3.6 Dirty data

Not within the given range, meaningless or illegal data format.

3.7 data analysis

The process of detailed research and generalization of data in order to extract useful information and form conclusions.

[GB/T 33745-2017, definition 2.5.4]

4.1 Data management function framework

Data management when the perception control device is connected to the gateway or IoT platform includes data collection, data processing, data exchange and data security

The data management function framework is shown in Figure 1, and functions can be selected according to requirements when specific applications are deployed.

4.2 Perception control device access architecture

According to the conceptual model of the Internet of Things in GB/T 33474-2016, the resource exchange domain, service provision domain, and operation and maintenance control domain in the IoT system

The software and hardware equipment of the company is usually deployed in the cloud or central server to form an IoT platform. Perception control device access to the framework of the Internet of Things platform

As shown in Figure 2, the perception control access device can access the IoT platform in the following two ways.

a) Indirect access. When the communication capability of the perception control device is limited, the perception control device connects to the IoT platform through the gateway.

b) Direct access. Perceptual control equipment with the function of directly accessing the IoT platform can directly connect to the IoT without passing through other logical network elements.

The interaction of perception data and control data between network platforms.

The perception control device can upload perception data when it is connected to the gateway or IoT platform, and the gateway or IoT platform can also send the perception control device

Send control data. This section only regulates the data management function when the perception control device is connected to the gateway or the Internet of Things platform.

The management functions mainly include data storage, data analysis, data preprocessing, and data security. The data management functions of the IoT platform mainly include data

Storage, data analysis, data preprocessing, data presentation, data distribution rule making, data distribution, data security.

5.1 General requirements

The requirements for the sensing data collected and the control data executed in the sensing control equipment are as follows.

a) It should support the collection of structured data, unstructured data, semi-structured data and other types of data;

b) Should support the access of multiple heterogeneous data sources and realize the conversion of data formats.

5.2 Perception data requirements

Perception data requirements may include.

a) The sensing unit generates and outputs sensing data, the content of which is the characterization data obtained by the sensing unit from the sensing object; such as temperature and humidity transmission

The sensor generates and outputs air temperature and humidity data, and the electronic sphygmomanometer generates and outputs blood pressure and pulse data.

b) For the sensing unit of intelligent sensing control equipment, the output sensing data is in the form of binary digital signal encapsulated by the communication protocol,

Can use different communication protocol encapsulation.

c) For the perception unit of non-intelligent perception control equipment, the output perception data is in the form of analog quantity, which can be received and processed by the gateway.

d) The sensing data output by the gateway is the data processed by the gateway to the original sensing data, and it communicates with various sensing units

The protocol is analyzed, encapsulated with a unified communication protocol, and the processed perception data is output in the form of a binary digital signal.

5.3 Control data requirements

Control data requirements may include.

a) The control data is transmitted to the control unit to control the controlled object; such as controlling the start or stop of the generator, adjust the temperature setting of the air conditioner

Set value, adjust the speed of the motor, etc.

b) The control data is generated by the IoT platform and sent to the control unit directly or via the gateway.

c) For the intelligent perception control device control unit, the received control data is in the form of a binary digital signal encapsulated by a communication protocol.

d) For the control unit of non-intelligent perception control equipment, the received control data is in the form of analog quantity.

6.1 Data preprocessing

6.1.1 Data heterogeneous conversion

Data heterogeneous conversion refers to the structural conversion of data carried in different protocol message formats, and conversion to perception control equipment, gateways or platforms

The data structure that can be identified and processed, the requirements are as follows.

a) Should support the communication protocol of the perception control device or platform;

b) Should support the analysis of specified sensing data packets and control data packets;

c) It should support data structure conversion through the protocol conversion module to realize data intercommunication between the perception control device and the gateway or platform.

6.1.2 Data extraction

Data extraction refers to the process of extracting data from data sources. The requirements are as follows.

a) Extraction modes such as full extraction, incremental extraction, and log-based extraction should be supported, and the extraction of geospatial information data can be supported;

b) Should support custom configuration of data extraction format and process;

c) The feature extraction of text, image, audio, video and other data should meet the requirements of 6.4 in GB/T 32630-2016.

6.1.3 Data cleaning

Data cleaning refers to the process of using existing data mining methods and methods to convert dirty data to meet data quality requirements or application requirements.

Requirements are as follows.

a) It should support the handling of abnormal data such as incorrect data, invalid data, and duplicate data;

b) Should support the filling of missing data;

c) The monitoring of data inconsistencies should be supported.

6.1.4 Data quality assessment

Data quality assessment is a process of optimizing the value of data by testing and improving the comprehensive characteristics of data in response to existing data quality problems.

Refer to GB/T 36344-2018 for the evaluation indicators and methods of data quality.

6.2 Data storage

6.2.1 General requirements

The general requirements for data storage are as follows.

a) Time series databases should be supported to store real-time data, which can be used to monitor and inspect real-time data collected by equipment, etc.;

b) It should support relational databases to store historical data, which can be used to analyze and optimize production management processes, etc.;

c) Data should be pre-processed before data storage, including data cleaning and data quality evaluation;

d) Provides a visual management interface for storage scheduling, storage monitoring, and storage management.

6.2.2 Storage scheduling

The storage scheduling requirements are as follows.

a) Support the development of storage scheduling plans, especially the planning and execution of storage node migration, expansion, replication, modification, and deletion operations;

b) It should support the dynamic scheduling of storage services to meet the needs of different users;

c) The division of storage permissions should be supported, and storage services should be optimized through mechanisms such as resource queuing and time division.

6.2.3 Storage monitoring

Storage monitoring requirements are as follows.

a) The storage monitoring plan should be customized according to requirements;

b) Storage status monitoring should be supported;

c) It should support the monitoring and tracking of all operations performed on storage locations by storage service users;

d) The acquisition of data such as storage resource information, file information and historical operations should be supported.

6.2.4 Storage Management

The storage management requirements are as follows.

a) Storage grading strategy should be supported, especially the classification of stand-alone level and cross-platform level classification;

b) A data isolation mechanism should be supported to make the data of different data users independent and invisible;

c) Should support various data model fusion functions;

d) Should support local and remote data import and export functions.

6.2.5 Storage backup

The storage backup requirements are as follows.

a) The parallel fault tolerance mechanism of the backup process should be supported, and the downtime of the backup process does not affect the availability of data;

b) Centralized control of data backup should be supported, and data backup strategies should be formulated for different usage requirements;

c) Mirror backup, redundant backup, etc. should be supported to improve data storage reliability;

d) Should support the separation of backup programs and application programs;

e) It should support compressed storage of backup data.

6.3 Data analysis

Data analysis refers to the analysis of massive sensing/control data and mining the use value of the data. The requirements are as follows.

a) Distributed mining analysis of basic distributed parallel computing processing framework should be supported;

b) Should support memory computing to realize real-time analysis;

c) Should support the definition and implementation of data analysis models;

d) Should support the definition and implementation of data mining models;

e) It should support multiple forms of analysis result output;

f) Should support the algorithm library required for data mining, and provide a variety of data mining algorithms.

6.4 Data presentation

The data presentation requirements are as follows.

a) Different types of data presentation such as structured data, semi-structured data, and unstructured data should be supported.

1) Structured data should support visualization methods such as histograms, graphs, line charts, pie charts, and radar charts;

2) Semi-structured data should support the display and statistics of commonly used data types;

3) Unstructured data should support classification and browsing.

b) It should be possible to ensure a good response speed of data presentation by means of caching, memory calculation, and compression transmission.

c) Should support the data presentation template function, and have template integration capabilities.

d) It should be extensible, especially to support the secondary development interface, which can quickly realize the required data presentation method.

7.1 Formulation of data distribution rules

Data distribution forwards the data of the IoT platform to other platforms according to preset rules. The requirements are as follows.

a) The rule script language should support the customization of business rules;

b) The reasoning mechanism of rules and the matching efficiency of rules should be ensured, and the efficiency of rule execution should not affect the main business logic;

c) Should support the resolution of rule conflicts in the rule chain;

d) Should support the query operation of the rule execution log.

7.2 Data distribution

The data distribution requirements are as follows.

a) It should support the sending of different types of data such as structured data, semi-structured data, and unstructured data;

b) It should support a data filtering mechanism that meets the requirements of the data receiver to reduce the transmission and reception of useless data;

c) The real-time and reliability of the data distribution process should be maintained.

8.1 Data integrity

The data integrity requirements are as follows.

a) The integrity of the data transmission process should meet the requirements of 6.1 and 7.1 of GB/T 37025-2018;

b) The data integrity should be tested through mechanisms such as package signature, test word verification, and reference constraints to ensure the integrity of the data storage process.

Integrity, providing solutions for incomplete data.

8.2 Data encryption

The data encryption requirements are as follows.

a) For specific sensitive fields or business data, encrypted storage should be used;

b) Should support the use of cryptographic algorithms approved by the national password management authority, and use those certified and approved by the national password management authority

Password products, follow the relevant national and industry standards for passwords;

c) Appropriate encryption algorithms, key lengths and key management mechanisms should be adopted in accordance with the requirements of relevant national security authorities;

d) The encryption process during data transmission should meet the requirements of 6.3 and 7.3 of GB/T 37025-2018.

8.3 Data access authority control

Data access permission control requirements are as follows.

a) Should support the formulation of security policies, and enable control of user access to data according to the security policies;

b) Separate management mechanisms for different levels of user permissions such as business data, system data, and database systems should be implemented, and user access

Ask to assign minimum access rights;

c) When data is provided to a third party for access, the access rights of the default account should be strictly restricted, and the password of the account should be modified regularly.

8.4 Data auditability

The data auditability requirements are as follows.

a) The scope of the audit should cover user behavior of business data, and set different levels of behavior records based on the importance of data;

b) The audit record should include the date, time, type, subject identification, object identification and results of the event;

c) It should be able to analyze based on recorded data and generate audit reports;

d) Audit records should be protected to avoid unexpected deletion, modification or overwriting.