GB/T 42584-2023Comprehensive performance evaluation specification for information project (English PDF)

1 Scope

The document lays down the evaluation system and the evaluation process for the comprehensive performance of an information project.

It applies to the various organizations that build and use information projects and to third party evaluation bodies, guiding them in setting the performance targets of the construction of an information project and giving them the criteria for evaluating its comprehensive performance.

2 Normative references

The documents referred to are GB/Z 20986-2007, Information security technology - Guidelines for the category and classification of information security incidents, and GB/T 22239-2019, Information security technology - Baseline for classified protection of cybersecurity.

Dated references apply in the dated version only; undated references apply in their latest version, amendments included.

3 Terms and definitions

3.1 Information project: a construction project based on computer technology and communication technology as its key elements, made up of one entity, or of a group of interconnected and cooperating entities, that carries out the collection, processing, storage, analysis and application of information.

3.2 Comprehensive performance: the output of an information project and the results of that output.

3.3 Information system: the man and machine system built by an information project, made up of computer hardware, network and communication equipment, computer software, information resources, information users and rules, whose purpose is to handle information flows.

3.4 Construction quality: the level of management and of technology of an information project during its construction period.

3.5 Operation and maintenance level: the level of capability in managing the IT software and hardware operating environment and the IT business system as a whole, by means of the relevant methods, means, techniques, rules, processes and documents.

3.6 Application effect: the degree to which the information system meets the needs of the users during use, and the results it produces for the stakeholders of the project.

3.7 Economic benefit: the financial profitability, the debt servicing capability and the survival capability of an information project once it has been put into use, together with its non-financial benefits.

3.8 Social benefit: the effects or the influence that an information project brings to the stakeholders of the project, once it has been put into use, in matters of environment, society, governance, health, education and safety.

3.9 Function point: a unit for measuring the functional size of software. The source given is GB/T 36964-2018, 3.12.

3.10 Incident: any event that is not part of the standard operation of a service and that causes, or may cause, an interruption of the service or a fall in the quality of the service. The source given is GB/T 24405.1-2009, 2.7.

4.1 Basic composition

The evaluation system for the comprehensive performance of an information project has two levels of indicators. There are four first-level indicators: the construction quality indicator, the operation and maintenance level indicator, the application effect indicator, and the economic and social benefit indicator. The framework of the system is shown in Figure 1.

The second-level indicators of the system are of two kinds, basic indicators and auxiliary indicators. The basic indicators are the indicators that an evaluation carried out with this document has to include and that may not be dropped during the evaluation. The auxiliary indicators are one indicator, a group of indicators or a class of indicators produced by a defined procedure for the evaluation of the comprehensive performance of a particular project, according to the aim of the evaluation, the features of the project or the requirements of the evaluation, and they are optional. The basic and auxiliary indicators laid down in the document shall meet the requirements of Annex A.

The evaluation system reaches its final score by a weighted sum of the indicators at each level, as shown in formula (1). The formula gives the score i-CORE as the sum of four products: the weight of the first-level construction quality indicator multiplied by the sum over its eight second-level indicators of the weight of each multiplied by its score; the weight of the first-level operation and maintenance level indicator multiplied by the corresponding sum over its eight second-level indicators; the weight of the first-level application effect indicator multiplied by the corresponding sum over its four second-level indicators; and the weight of the first-level economic and social benefit indicator multiplied by the corresponding sum over its two second-level indicators. A note states that the rate of change of the project investment, the rate of change of the investment per unit of production capacity, the rate of change of the net present value and the debt service coverage ratio are not counted in the overall result of the evaluation.

Figure 1 shows the four first-level indicators under the heading of the system and, under each of them, its second-level indicators, drawn in solid boxes for the basic indicators and in dashed boxes for the auxiliary indicators.

4.2 Construction quality

4.2.1 The standardization of project management mainly evaluates how far the management of the whole construction cycle of the information project follows the rules, initiation, planning, execution, monitoring and closing included.

4.2.2 The rate of technical innovation mainly evaluates the proportion of technical innovation in the basic software and hardware and in the application software during the construction of the information project, that is the proportion of technical innovation in the basic hardware, in the basic software and in the application software.

4.2.3 The degree of business support mainly evaluates how far the information project supports the business and the management, which includes the coverage of the business, the depth of application of the system and the degree of digitalization. Here the depth of application of the system means how far the information system is applied to the subdivisions of the business, and the degree of digitalization means how far the information system makes digital use of the information resources.

4.2.4 The rate of resource utilization mainly evaluates how economically the equipment resources are used during the construction of the information project, which includes the CPU load, the rate of use of dedicated storage equipment, the updating of function points and the sharing of cloud resources.

4.2.5 Data sharing mainly evaluates the interconnection and the sharing of data during the construction of the information project, which includes the integration of the systems and the state of data sharing.

4.2.6 Technical advancement mainly evaluates the new methods and new techniques obtained during or after the construction of the information project, which includes granted patents, published standards and prizes won in science and technology.

4.2.7 The rate of energy utilization mainly evaluates how efficiently energy is used during the construction of the information project, which includes the dynamic monitoring of the operating state of the equipment and the power usage effectiveness of the machine room or data centre.

4.2.8 Security protection mainly evaluates the classified protection of the cybersecurity of the information system, which includes the security of the physical environment, of the communication network, of the area boundaries and of the computing environment, the security management centre, and the security management rules, bodies, personnel, construction management and operation and maintenance management.

4.3 Operation and maintenance level

4.3.1 Availability mainly evaluates the ability of the information system to complete the required functions at a given moment or during a given period.

4.3.2 Reliability mainly evaluates the ability of the information system to provide normal service continuously while it is running.

4.3.3 Maintainability mainly evaluates how easy or how difficult the information system is to repair, to restore and to improve.

4.3.4 The timeliness of the service response mainly evaluates the ability of the information system to respond in good time to abnormal events while it is running.

4.3.5 The timeliness of incident resolution mainly evaluates the ability to resolve an incident, once it has occurred, within the agreed service time.

4.3.6 First time incident resolution mainly evaluates the ability to resolve an incident, once it has occurred, at the first attempt.

4.3.7 Security risk mainly evaluates the information security incidents that occur in the information system and the high risk vulnerabilities that are present in it.

4.3.8 Adaptability mainly evaluates whether the information system lends itself to iterative updating and to extension and upgrading.

4.4 Application effect

4.4.1 The rate of function application mainly evaluates how far the function points of the information system are applied in the existing business and management processes.

4.4.2 The stability rate of the key business mainly evaluates how stable the handling of the core business processes with the information system is.

4.4.3 User satisfaction mainly evaluates how satisfied the users are with the use of the information system, which includes ease of use, stability and the quality of the technical support.

4.4.4 User activity mainly evaluates how often the users make use of the information system.

4.5 Economic and social benefit

4.5.1 The input output ratio mainly evaluates the ratio of the return of a profit making information project to the total investment in the project.

4.5.2 The rate of achievement of the expected targets mainly evaluates the ratio of the targets already reached by the information project to the targets expected of it.

4.5.3 The rate of change of the project investment mainly evaluates the extent of the change between the budgeted investment in the information project and the investment in the final account at acceptance.

4.5.4 The rate of change of the investment per unit of production capacity mainly evaluates the difference between the actual investment per unit of production capacity after the information project has been put into use and the designed investment per unit of production capacity.

4.5.5 The rate of change of the net present value mainly evaluates the difference between the net present value of a profit making information project after it has been put into use and the forecast net present value.

4.5.6 The debt service coverage ratio mainly analyses the ability of a profit making information project to repay the funds it owes, and assesses whether the sources of funds available for repaying principal and interest after the information project has been put into use are enough to service the debt of the current period.

5 Evaluation process

5.1 On the basis of the social responsibility and the operating and management targets of the organization, and with reference to the project contract or the project task statement, the aim of the evaluation of the comprehensive performance of the information project is made clear in terms of conformity to the rules, of effectiveness and of advancement, so as to give the requirements and the basis for designing the indicator system.

5.2 On the basis of the aim of the evaluation, a complete evaluation plan is drawn up, which includes the organizational structure of the evaluation, the content of the work, the planned schedule and the interfaces of the work.

5.3 According to the design, the construction, the operation and the output of the information project and according to the aim of the evaluation, the evaluation indicators are designed, selected and fixed, and they make up the indicator system for the comprehensive performance of the information project. The design of the indicator system follows these principles: the design of the indicator system conforms to the relevant national, local, sectoral or enterprise standards or specifications; the indicator system contains both qualitative and quantitative indicators, both of which are measurable, the quantitative indicators being calculable; custom indicators may be added according to the features of the project and to the requirements of the sector or region where it lies, provided that the normal operation of the project under evaluation is not affected and no added cost is caused; the scores of all the indicators are kept on a percentage scale; and the data items of the indicators are kept as objective and as obtainable as possible. The methods of calculating the indicators laid down in the document shall meet the requirements of Annex B.

5.4 The weights of the indicators of the comprehensive performance of the information project are set level by level, by classifying the indicators and then fixing the weights; the method of setting them shall meet the requirements of Annex C.

5.5 The physical boundary, the logical boundary and the other boundaries crossed by the data input and output, by the input of resources and the output of products and services of the information project under evaluation are fixed. The division of the project boundary includes all the resources, processes and infrastructure involved in the construction, the extension or the running of the information project that have a visible or a hidden influence on the evaluation indicators.

5.6 On the basis of the indicator system and of the project boundary already fixed, a detailed survey outline is drawn up and the sources of the data to be collected, the methods of collecting them and the supporting material are fixed. The data mainly collected at this stage are: the material on the decision to build the project, on its initiation and on its construction, which includes but is not limited to the feasibility study report, the project appraisal report, the official approval of the project, the project budget and its changes, the acceptance report, the final account and the relevant contracts and agreements; the material on the running of the project, which includes but is not limited to the production and operating data and the financial data after the project has been put into use; the material on the sector in which the project lies, which includes but is not limited to the technical level, the scale, the operating situation and the labour productivity of information projects in the same sector at home and abroad; the material on the relevant national laws, policies and rules, which includes but is not limited to the laws promulgated by the state, the information policies published and the relevant rules and standards; and other relevant material, which includes but is not limited to the rules of the enterprise or of the department and the material that meets the particular features of the project under evaluation and the requirements of the evaluation.

5.7 On the basis of the data and the supporting material obtained in full, and according to the management rules of the state, of the sector or of the department, a complete quantitative and qualitative analysis and argumentation of the technical and economic situation of the construction and of the production process of the information project is made following the indicator system.

5.8 The results of the analysis and argumentation are compared with the aim of the evaluation, the project plan and the technical and economic indicators of the sector or of similar projects, the score for the comprehensive performance of the information project is obtained and the evaluation report is prepared.