JR/T 0175-2019Specification for securities and futures industry software test (English PDF)

4.1 Test Purpose

Verify that the software meets software development contracts, system/subsystem design documents, system requirements specifications, system design specifications, product specifications Software specifications such as product specifications, operation and maintenance manuals, and operation manuals.

Software defects are found through testing to provide a basis for the quality evaluation of software products.

4.2 Test process

4.2.1 Work requirements during the test phase

The test of each test level and test type should follow this test process. This test process is divided into test estimation, test plan, test design There are five stages of planning, test execution and test report. The activities, inputs and outputs of each stage are shown in Appendix A.

See Table 1 for the working requirements to be observed in each test phase.

Table 1 Work requirements

Test phase job requirements Test estimate

a) The test range of functional test and non-functional test is reasonable, comprehensive and non-redundant;

b) When estimating the workload, the test strategy is selected reasonably and effectively;

c) Both functional test and non-functional test have completed test workload estimation, and have sufficient estimation basis;

d) If there are other tasks such as test tool development, acceptance test support, test management, etc.

Estimation of points.

Test Plan

a) Define functional and non-functional test scopes, test strategies, and access/exit requirements for test execution;

b) When there is a significant difference between the test strategy and the test strategy in the test workload estimate, a reasonable explanation should be provided;

c) The detailed test schedule is reasonable and the milestones are clear. Avoid insufficient or excessive testing.

d) The test environment deployment strategy meets the system planning requirements, and the test tool name, version, and purpose description used are clear;

e) clarify the risks that may occur during testing activities and their impact and emergency measures;

f) Clarify the problem management process, change management process, progress management process, and defect management process during the test.

Table 1 Work requirements (continued)

Test phase job requirements Test design

a) Test requirements. 100% coverage of business requirements and system requirements specifications, and appropriate reference to relevant development and design documents;

b) Test cases. 100% coverage of test requirements or 100% coverage of related design and requirements documents according to different test levels, such as single Meta-tests, integration tests cover relevant development and design documents, and system tests cover system requirements specifications.

c) The design of non-functional test scenarios fully considers the amount of business data that may have an impact, and is effective in real production environments.

Testable in a test environment.

Test execution

a) Clearly record the execution results of each test case, and the unexecuted case should indicate the reason for not executing;

b) For the core, important systems and functions, the test execution process should be left in the form of screenshots or log retention;

c) Defects and problems found in the implementation process shall be recorded in the defect management tool or defect list in a timely manner and processed in a timely manner After the test, there is no defect other than the final state.

testing report

a) Clarify the functional and non-functional test scope. If there is a difference from the test plan, explain it;

b) Defining test conclusions, describing specific passing standards and actual test results from functional and non-functional perspectives, and giving whether they pass Past conclusions

c) State the deployment of the test environment, and analyze the differences between the test environment and the production environment;

d) clear test version, test implementation, summary and analysis of defects;

e) Make suggestions for improvement, summarize lessons learned, and provide references for follow-up work.

4.2.2 Test Document Specification

The output documents of each test stage, the basic contents that should be included are shown in Table 2, and can be tailored according to actual needs. For specific templates, see the Appendix B.

Table 2 Basic contents of the document

The basic content of the stage output document Test estimate Test design complexity, test execution complexity, estimated number of test cases, test settings for each system module Count workload, test execution workload, non-functional test workload, etc.

Test plan test plan

Test target, test scope, test strategy, test schedule, test environment, test method, test Management, test reports, etc.

Test design

Test requirements require functions, verification points, verification methods, etc.

Test case

Test case ID, priority, scenario description, method/function, normal system/exception system, prerequisites, test Test purpose, test steps, test data, expected results, etc.

Test execution

Test defect list

Defect ID, defect description, reproduction steps, expected results, actual results, test rounds, test types, Found version, found date, defect owner, defect submitter, severity, priority, defect status, Root cause, solution, shutdown date, etc.

Test execution results Test case actual execution results, pass or fail, corresponding defect ID, tester, actual measurement Trial dates, screenshots of test steps, etc.

Test report test summary report Test scope, test conclusions, difference analysis between test environment and production environment, functional test/non-functional test Implementation, summary of defects and analysis of remaining problems, analysis of test results, lessons learned, improvement suggestions, etc.

Test management

Managers report progress, workload deviations, achievement of quality goals, risks and issues.

Milestone summary reports on milestone progress, cost and quality, risks and issues, lessons learned, suggestions for improvement, etc.

4.3 Test Type

4.3.1 Functional test

The main purpose, test content and test technology of functional test are as follows.

a) Purpose

Based on the system requirements specification, verify the system functional characteristics such as the required function implementation and business operations.

The purpose is to ensure that the system meets the customer's functional and ease-of-use requirements under specified conditions.

b) Test content/focus the test content/concerns include correct implementation of functions, no useless functions, and omissions of non-existent functions, as follows.

1) The implemented function is correct. it is consistent with the function required in the system requirements specification;

2) There are no useless functions. the range of functions implemented does not exceed the scope described in the system requirements specification

3) There is no omission of functions. the range of functions implemented can completely cover the scope described in the system requirements specification.

c) Testing technology Black box test technology is the main, gray box test technology is supplemented.

The black box test technology, white box test technology and gray box test technology mentioned in this standard are listed in Appendix C.

4.3.2 Performance test

The main purpose, test content and test technology of performance test are as follows.

a) Purpose

By simulating the business scenarios of various load conditions such as real and high pressure, the various performance indicators of the system are evaluated.

Including load test, stress test, capacity test, business response time test, etc.

b) Test content/focus Test content/focus includes load test, stress test, capacity test, and business response time test, as follows.

1) Load test. continuously increase the pressure on the system or keep the maximum safe load until one or more of the system Performance indicators reach safety thresholds (for example, certain resources have reached saturation, etc.) Performance indicators under conditions;

2) Stress test. When the safety load is exceeded, the system is continuously stressed, and by determining the bottleneck of a system or can not receive the performance points requested by the user to obtain the test of the maximum service level that the system can provide;

3) Capacity test. use limit values (such as the maximum number of concurrent users, database records, etc.) that reflect the capacity characteristics of the system the system exerts pressure by observing that the system has not experienced any software failure or can maintain the main the function runs normally to check the capacity index of the system;

4) Business response time test. business operation is performed during the continuous operation of stable pressure or after a certain capacity is reached in order to detect that the system response time meets the performance requirements, it records the processing time of the system.

c) Testing technology details as follows.

1) The test methods for load test, stress test and business response time test are by simulating single user and multi-user concurrent Scenario, to obtain the delay, throughput, and processing time of software system function processing. Counting with automated test tools According to the structure and load pressure structure, usually select the representative or core business module, the business with a higher number of visits Construct various performance test scenarios to perform stand-alone performance testing or concurrent performance testing. Test indicators include order peaks Value throughput rate, transaction peak throughput rate, order continuous throughput rate, transaction continuous throughput rate, order processing delay Hourly and daily settlement processing capacity, etc .;

2) The test method of capacity test is to construct corresponding capacity data under different system configurations and different business scenarios.

Assess the system's extreme capacity and hardware resource usage. Among them, the limit capacity includes the maximum number of users, the maximum processing capacity, Maximum number of transactions (execution units), maximum throughput, maximum file capacity, etc .; hardware resource usage includes memory, CPU, hard disk, network bandwidth, equipment, etc. Test indicators include daily order processing capacity and daily transaction processing capacity.

4.3.3 Reliability test

The main purpose, test content and test technology of reliability test are as follows.

a) Purpose

Reliability testing is to simulate various abnormalities or failure scenarios that may occur in the system to verify that the system is handling Fault tolerance, data recovery, and disaster recovery capabilities, and evaluate the reliability of the system, with the aim of ensuring that the system is Meet high availability requirements under certain environmental deployment conditions, which typically include maturity testing, fault tolerance testing, stability testing, and Recoverability test.

b) Test content/focus Test content/focus includes maturity test, fault tolerance test, stability test and recoverability test, as follows.

1) Maturity test. When a code judgment error occurs in a software system, an interface error during integration, a communication message error, or when business logic errors in the system and other errors caused in the design, development and other links, the verification system can eliminate the errors the impact and still work properly;

2) Fault tolerance test. When external errors such as illegal operations, data deletion, and forced termination are introduced, the system (including (Including hardware, software and ancillary programs) when an abnormality occurs, the verification system should have protective measures (handling the abnormal methods include. automatic system processing and manual intervention processing;

3) Stability test. verify the stability of the system for a long time under a certain pressure through continuous testing;

4) Recoverability test. The ability of the system to recover to a normal state in a failed state. Failure status should be system crash or the user stops and stops all functions, not the faulty running state; the normal state should at least guarantee the main function of the system all can run. Get the system recovery time (RTO) and data recovery time (RPO). For the definition of RTO and RPO, see JR/T 0145-2016.

c) Testing technology details as follows.

1) The test method for maturity test and fault tolerance test is to manually simulate failure scenarios, such as process crash and hang, network Network disconnection and delay, server downtime, or abnormal operations or data input through tools or manual manufacturing, Verify the ability of the system or component to effectively deal with failures;

2) The test method for the stability test is a 7 * 24-hour stress test, that is, imitating the production relationship of the business, for production needs, keep the order pressure at different multiples, and run the system under test 24 hours a day for 7 days. During the test period You can verify system operation by viewing the status of system processes, performing functional operations, and viewing run log information.

Correctness of state

3) The test method of the recoverability test is to simulate the analysis of the system log when a failure or disaster causes the system to fail.

Or flow calculation can get the system recovery time (RTO) and data recovery time (RPO) indicators.

4.3.4 Security test

The main purpose, test content and test technology of the security test are as follows.

a) Purpose

Through security testing methods and methods, verify that the security features of the software are consistent with expectations, and test software products against various attacks the ability to prevent situations, thereby ensuring the safety and quality of products.

b) Test content/focus details as follows.

1) Configuration management. According to the information security level protection requirements, the operating system, applications, databases, network equipment, etc.

Line security configuration;

2) Resource utilization. application memory usage and resource contention issues, such as uninitialized variables, out-of-bounds arrays, memory overflow, Resource leaks, process thread exceptions, etc .;

3) Identity authentication. The users who log in to the operating system, database system and application system are identified and authenticated, such as Use strong password policies, limit the number of consecutive login attempts, use valid verification codes, etc .;

4) Access control. Users should access system resources (such as servers, directories, files, etc.) according to their own permissions.

No unauthorized access

5) Data protection. When the application stores sensitive information, the stored sensitive information should be encrypted, including but not limited to Passwords, keys and other sensitive information should be encrypted and written into caches, file systems, databases, etc .;

6) Communication security. When the application uses a secure communication protocol (such as SSL, TLS), it should respond to the transmission of sensitive information.

Line encryption, etc.

7) Session management. Local and server settings for ...