GB/T 38643-2020Information technology—Big data—Functional testing requirements for analytic system (English PDF)
信息技术 大数据 分析系统功能测试要求
This is a limited preview
Buy now to download the full PDF (24 pages)
Need quotation for your management?
Get a formal quotation for GB/T 38643-2020 in 30 minutes.
Issued by
SAMR; SAC
Level / Type
National · Recommended
Issue date
April 28, 2020
Implementation date
November 1, 2020
Scope
GB/T 38643-2020 is the English-translated version of 信息技术 大数据 分析系统功能测试要求.
GB/T 38643-2020 is the Chinese standard "Information technology—Big data—Functional testing requirements for analytic system". Its scope clause reads: This standard specifies the functions of the data preparation module, analysis support module, data analysis module, and process planning module of the big data analysis system.
Can test requirements. This standard is applicable to guide the design, development and delivery of big data analysis systems. Its clauses include terms and definitions; data preparation module function test; analysis support module function test; data analysis module function test; functional test of process orchestration module.
Found what you need?
Document preview — GB/T 38643-2020
National Standard of the People's Republic of China
- ICS
- 35.240
- Classification
- L 67
Issued by: State Administration for Market Regulation; Standardization Administration of the PRC
Contents
- Foreword
- 1 Scope
- 2 Normative references
- 3 Terms and definitions
- 4 Abbreviations
- 5 Overview
- 6 Data preparation module function test
- 6.1 Data extraction function test
- 6.2 Data cleaning function test
- 6.3 Data conversion function test
- 6.4 Data loading function test
- 7 Analysis support module function test
- 7.1 Query function test
- 7.2 Machine learning function test
- 7.3 Statistical analysis function test
- 7.4 Visual function test
- 8 Data analysis module function test
- 8.1 Analysis mode test
- 8.2 Analysis type test
- 9 Function Test of Process Orchestration Module
- 9.1 Workflow Management Test
- 9.2 Alarm and log test
- Appendix A (informative appendix) Test example
1 Scope
This standard specifies the functions of the data preparation module, analysis support module, data analysis module, and process planning module of the big data analysis system.
Can test requirements.
This standard is applicable to guide the design, development and delivery of big data analysis systems.
2 Normative references
The following documents are indispensable for the application of this document. For dated reference documents, only the dated version applies to this article
Pieces. For undated references, the latest version (including all amendments) applies to this document.
GB/T 37721-2019 Information technology big data analysis system functional requirements
3 Terms and definitions
The following terms and definitions defined in GB/T 37721-2019 apply to this document.
3.1 Big data analysis system
On the basis of the original data and calculation framework provided by the big data storage and processing system, a system that integrates a series of tools used in the data analysis life cycle process.
5 overview
This standard provides corresponding test requirements based on the functional requirements of the big data analysis system specified in GB/T 37721-2019.Test shows
See Appendix A for examples.
6 Data preparation module function test
6.1 Data extraction function test
The data extraction function test requirements are as follows.
a) It should be tested whether the data preparation module of the big data analysis system supports the extraction of data stored in the storage system as required;
b) It should be tested whether the data preparation module of the big data analysis system provides different extraction methods for structured data and unstructured data;
c) It should be tested whether the data preparation module of the big data analysis system provides full extraction and incremental extraction modes;
d) It should be tested whether the data preparation module of the big data analysis system supports active extraction and passive addition;
e) It should be tested whether the data preparation module of the big data analysis system supports regular batch extraction;
f) Should test whether the data preparation module of the big data analysis system supports distributed data extraction, and test whether load balancing is achieved in the data extraction process.
6.2 Data cleaning function test
The data cleaning function test requirements are as follows.
a) Whether the data preparation module of the big data analysis system supports data consistency should be tested;
b) It is necessary to test whether the data preparation module of the big data analysis system supports the processing of invalid values by deleting and correcting invalid data values;
c) It is necessary to test whether the data preparation module of the big data analysis system supports the processing of missing values through operations such as filling in missing values or deleting data entries corresponding to missing values;
d) It is necessary to test whether the data preparation module of the big data analysis system supports the processing of duplicate data through operations such as merging or deleting duplicate data;
e) It should be tested whether the data preparation module of the big data analysis system provides the data comparison function before and after cleaning;
f) It should be tested whether the data preparation module of the big data analysis system supports logical contradictions, correlation verification, and cleaning of unreasonable data.
6.3 Data conversion function test
The data conversion function test requirements are as follows.
a) The column conversion operation of structured data should be used to test whether the data preparation module of the big data analysis system supports the conversion of structured data columns;
b) It is necessary to test whether the data preparation module of the big data analysis system supports the conversion of structured data through the conversion operation of the structured data;
c) Test whether the data preparation module of the big data analysis system supports structured data table conversion by performing table conversion operations on structured data;
d) It should be tested whether the data preparation module of the big data analysis system supports the structured processing of unstructured data;
e) It should be tested whether the data preparation module of the big data analysis system supports the standardized processing of text and web data, and whether it supports the conversion of document data into a single standardized form;
f) Voice and audio input should be used to test the accuracy of input recognition results and test whether the data preparation module of the big data analysis system
Support the recognition and processing of voice/audio data;
g) Image input should be used to test the accuracy of input recognition results and test whether the data preparation module of the big data analysis system supports the extraction of image information.
6.4 Data loading function test
The data loading function test requirements are as follows.
a) It should be tested whether the data preparation module of the big data analysis system supports loading the cleaned and converted data into the big data analysis system;
b) Input the converted data into the target structure according to the loaded target structure, and test whether the data preparation module of the big data analysis system supports full loading;
c) When data already exists in the target structure, new data should be added on the basis of saving the existing data to test the big data analysis
Whether the system data preparation module supports incremental loading;
d) It should be tested whether the data preparation module of the big data analysis system supports real-time loading or batch loading.
7 Analysis support module function test
7.1 Query function test
7.1.1 Query interface test
The query interface test requirements are as follows.
a) It should be tested whether the analysis support module of the big data analysis system supports query through the standard database connection interface;
b) It should be tested whether the analysis support module of the big data analysis system supports the RESTAPI query interface for query.
7.1.2 Query optimization test
The query optimization test requirements are as follows.
a) The data index should be established to test whether the analysis support module of the big data analysis system achieves the effect of query acceleration;
b) It should be tested whether the analysis support module of the big data analysis system supports accurate query and fuzzy query;
c) It should be tested whether the analysis support module of the big data analysis system supports rule-based or cost-based query optimization;
d) It should be tested whether the analysis support module of the big data analysis system supports data fragmentation and multiple copy technology;
e) It should be tested whether the analysis support module of the big data analysis system supports complex conditions and high concurrent queries through SQL;
f) It should be tested whether the analysis support module of the big data analysis system supports secondary index.
7.2 Machine learning function test
7.2.1 Data set management function test
The data set management function test requirements are as follows.
a) It should be tested whether the analysis support module of the big data analysis system can divide the input data into training set, verification set and test set;
b) Import the trained and verified model into the big data analysis system, and import the model trained in the big data system.
To test whether the analysis support module of the big data analysis system provides the function of importing and exporting machine learning models.
7.2.2 Support algorithm test
The requirements for supporting algorithm testing are as follows.
a) It should be tested whether the analysis support module of the big data analysis system supports regression and classification algorithms;
b) Test whether the analysis support module of the big data analysis system supports clustering algorithms;
c) It should be tested whether the analysis support module of the big data analysis system supports the collaborative filtering algorithm;
d) Whether the analysis support module of the big data analysis system supports dimensionality reduction algorithms should be tested;
e) It should be tested whether the analysis support module of the big data analysis system supports frequent pattern mining algorithms;
f) Whether the analysis support module of the big data analysis system supports neural network algorithms should be tested;
g) It should be checked whether it has feature extraction, feature conversion, feature selection, model selection, cross-validation, model tuning component testing
The data analysis system analyzes whether the support module provides other components of the machine learning process;
h) It should be tested whether the analysis support module of the big data analysis system supports one or more languages such as Java, Scala, Python, R, etc., and is
Whether to support the secondary development to add new operators.
7.2.3 Model evaluation function test
The big data analysis system analysis support should be tested by checking the core evaluation components such as cross-validation and model selection in the machine learning module
Whether the module can support the evaluation module of the algorithm model.
7.3 Statistical analysis function test
The statistical analysis function test requirements are as follows.
a) The analysis support module of the big data analysis system should be tested by calculating the maximum, minimum, sum, total and other statistics.
Numerical statistics of this book;
b) The analysis support module of the big data analysis system should be tested by calculating the average, median, mode and other statistics to test whether the analysis support module supports data analysis
Statistics of central tendency;
c) The analysis support module of the big data analysis system should be tested by calculating the range, variance, standard deviation and other statistics.
Statistics of the degree of dispersion;
d) It is necessary to test whether the analysis support module of the big data analysis system supports the analysis of multiple random variables by calculating covariance, correlation coefficient and other statistics.
The relationship between machine variables;
e) Test whether the analysis support module of the big data analysis system supports the customization of statistical analysis by saving commonly used statistical analysis schemes
Template capabilities.
7.4 Visual function test
The visual function test requirements are as follows.
a) The analysis support module of the big data analysis system should be tested by inputting in Excel, relational database, JSON, and XML format.
Support common data source data format as input;
b) Test whether the analysis support module of the big data analysis system supports the visual display of high-dimensional data;
c) It should be checked whether the analysis support module of the big data analysis system can be displayed in the form of histogram, pie chart, line chart, etc.
Support visual analysis tool library;
d) It should be tested whether the analysis support module of the big data analysis system supports visualization tools related to the evaluation of algorithm models.
8 Data analysis module function test
8.1 Analysis mode test
8.1.1 Offline data analysis function test
The offline data analysis function test requirements are as follows.
a) It should be tested whether the data analysis module of the big data analysis system supports structured query language;
b) It should be tested whether the data analysis module of the big data analysis system supports distributed analysis of offline data;
c) It should be tested whether the data analysis module of the big data analysis system has the ability to support third-party applications through standard interfaces;
d) It should be tested whether the data analysis module of the big data analysis system supports a computing framework such as distributed computing or parallel computing;
e) It should be tested whether the data analysis module of the big data analysis system supports the segmentation and distributed scheduling of massive work tasks;
f) It should be tested whether the data analysis module of the big data analysis system supports the integration of third-party machine learning algorithm libraries;
g) Test whether the data analysis module of the big data analysis system supports the use of memory or SSD storage as a cache;
h) It should be tested whether the data analysis module of the big data analysis system supports optimization at the distributed execution plan level;
i) It should be tested whether the data analysis module of the big data analysis system supports the analysis of text, audio and video, and image data;
j) It should be tested whether the data analysis module of the big data analysis system supports data sources in relational databases and big data storage systems.
The ability to perform cross-query, aggregation, and related operations;
k) It should be tested whether the data analysis module of the big data analysis system supports the use of GPU to accelerate analysis of specific algorithms.
8.1.2 Stream data analysis function test
The streaming data analysis function test requirements are as follows.
a) It should be tested whether the data analysis module of the big data analysis system supports batch processing after time slice;
b) It should be tested whether the data analysis module of the big data analysis system supports stream processing based on event triggering or sampling;
c) It should be tested whether the data analysis module of the big data analysis system supports data statistics on real-time streams;
d) It should be tested whether the data analysis module of the big data analysis system supports the sorting of streaming data;
e) It should be tested whether the data analysis module of the big data analysis system supports the association with the static table;
f) It should be tested whether the data analysis module of the big data analysis system supports the associated processing of multiple data streams;
g) Test whether the data analysis module of the big data analysis system supports real-time analysis tasks using sliding windows, and test the time
Whether the window size is adjustable;
h) It should be tested whether the data analysis module of the big data analysis system supports real-time data grouping and priority scheduling;
i) It should be tested whether the data analysis module of the big data analysis system supports the analysis of text, audio and video, and image data.
8.1.3 Interactive online analysis function test
The interactive online analysis function test requirements are as follows.
a) It should be tested whether the data analysis module of the big data analysis system supports distributed online data through structured query language analysis;
b) It should be tested whether the data analysis module of the big data analysis system supports ad hoc query of data through structured query language;
c) It should be tested whether the data analysis module of the big data analysis system supports the use of visualization middleware to display the data analysis results;
d) It should be tested whether the data analysis module of the big data analysis system supports the definition of calculation formulas and parameter configurations in the interactive analysis process;
e) It should be tested whether the data analysis module of the big data analysis system supports automatic saving and rollback of the interactive analysis process;
f) It should be tested whether the data analysis module of the big data analysis system supports the preservation and release of analysis results during the interactive analysis process;
g) It should be tested whether the data analysis module of the big data analysis system supports interactive data analysis based on online online analysis;
h) It should be tested whether the data analysis module of the big data analysis system supports the analysis of unstructured data.
8.2 Analysis type test
8.2.1 Predictive analysis function test
The requirements for predictive analysis function testing are as follows.
a) It should be tested whether the data analysis module of the big data analysis system supports multiple forecasting analysis methods such as trend forecasting and regression analysis;
b) It should be tested whether the data analysis module of the big data analysis system supports the presentation of the accuracy rate in the form of percentage digitization, and whether it is accurate
At least 1 digit after the decimal point;
c) It should be tested whether the data analysis module of the big data analysis system supports the use of visualization to display the analysis results;
d) It should be tested whether the data analysis module of the big data analysis system supports the release and application of the trained model.
8.2.2 Descriptive analysis function test
Descriptive analysis function test requirements are as follows.
a) It should be tested whether the data analysis module of the big data analysis system supports the use of correlation analysis methods for descriptive analysis;
b) It should be tested whether the data analysis module of the big data analysis system supports visual display of the analysis results of sample data, and whether it supports display
The model training effect, whether it supports the storage and release of the trained model;
c) It should be tested whether the data analysis module of the big data analysis system supports a good visual presentation of the analysis results.
9 Functional Test of Process Orchestration Module
9.1 Workflow management test
The workflow management test requirements are as follows.
a) Operations such as process planning and revision should be carried out by drag and drop to test whether the process planning module of the big data analysis system supports visualization
The process orchestration operation interface;
b) It is necessary to test whether the process orchestration module of the big data analysis system supports the trigger time and execution cycle of the configuration workflow.
Support the scheduling trigger mechanism of the workflow, and whether it supports configuring the trigger time or trigger event;
c) It should be tested whether the process orchestration module of the big data analysis system supports the start and stop operations of the workflow through the management interface;
d) Whether the process orchestration module of the big data analysis system supports parallel execution of multi-process tasks should be tested;
e) It should be tested whether the process orchestration module of the big data analysis system supports the serialization of workflows through data pipelines;
f) It should be tested whether the process orchestration module of the big data analysis system supports multi-person collaboration functions;
g) It should be tested whether the process orchestration module of the big data analysis system supports the persistent storage of the process orchestration results.
9.2 Alarm and log test
The alarm and log test requirements are as follows.
a) It should be tested whether the process orchestration module of the big data analysis system supports tracking of the execution status of calculations or tasks, and whether the
The task gives an alert;
b) It should be tested whether the process orchestration module of the big data analysis system supports the output of the details of the task execution status to the log.
......
This preview omits tables, figures, formulas and parts of the technical clauses. The complete document — 24 pages — is available in the English PDF.
Similar standards
How to Download GB/T 38643-2020
- 1
Add to cart
Click "Download PDF" on this page and choose "Buy this standard". You can add more standards before checkout.
- 2
Checkout
Enter your email and billing details. Payment is processed securely by Stripe (cards or bank transfer). Prefer bank details? Request a quotation (min. order $300).
- 3
Fast delivery
After payment you'll receive an email confirmation. The English PDF is sent to the same address, usually within minutes and at the latest within 1 business day.
- 4
Invoice included
A tax invoice is attached to the confirmation email. Need a custom invoice? Contact us.
Related Standards
GB/T 10003-2026 — Biaxially oriented polypropylene (BOPP) film
GB/T 10004-2026 — Plastic/Plastic laminated films and pouches for packaging
GB/T 10007-2026 — Rigid cellular plastics - Determination of shear properties
GB/T 38643-2020
$420.00