Anda di halaman 1dari 41

2013/9/12

Security Level:

Introduction to Wireless Product Acceptance Optimization Solution

www.huawei.com

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Contents
Overview

Solution Introduction
FAQs Negotiation Cases

Difficulties in Project Acceptance (Test)


The Acceptance Manual is a collection of test cases. For specific projects, however, many cases may be redundant. On-site engineers may not know how to identify and delete redundant cases or choose test cases. Therefore, risks of uncertainty exist. Acceptance supporting documents (such as factory inspection reports and R&D internal test reports or documents) are missing or hard to obtain. Existing test case selection tools cannot select suitable cases correctly like on-site engineers. Acceptance contract clauses are unclear. Disputes with customers in later delivery phase puts us in an unfavorable position. Acceptance standards are determined through negotiations with customers. However, we lack negotiation strategies or proper measures when customers raise unreasonable requirements.

Five shortcomings in ability development

Materials: Hard to select proper test cases among massive test cases Documents: Lack supporting documents or hard to obtain Tools: Inapplicable to guide on-site engineers to select proper test items

Clauses: Unclear clauses causing disputes with customers


Strategies: No guidance strategy for negotiations

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 3

How Does This Solution Help

Focusing on product acceptance (provisional acceptance), this solution solves problems related to materials, supporting documents, tools, clauses, and negotiation guidance strategies, minimizes the on-site acceptance, shortens the acceptance period, and fastens payment collection.
Tools: Develop auxiliary tools, associate feature sales and acceptance, and enhance test case risk management

Documents:
Prepare supporting documents and ensure that the documents can be obtained by on-site engineers

Clauses: Draft acceptance clauses to restrict customer test requirements

Materials:
Optimize acceptance information and further classify test packages in acceptance scenarios

Strategies: Define acceptance negotiation guidance strategies to support on-site negotiations

Minimize the acceptance, shorten the acceptance period, and fasten payment collection

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 4

Contents
Overview

Solution Introduction
FAQs Negotiation Cases

Solution Introduction

Test Packages Acceptance Supporting Documents Test Case Selection Tools Clauses Negotiation Guidance Strategies

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 6

Classifying the Test Cases in the Acceptance Manual into Test Packages
Divide the original Acceptance Manual into four test packages and one strategy package according to test case attributes and test procedures.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 7

Structure of a Test Package

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 8

Associating Test Packages with Acceptance Process


Acceptance Process
Supply Acceptance Provisional Acceptance Demonstration Acceptance Sites Acceptance Network Optimizatio n Acceptance Final Accepta nce

Accepta nce Activity

Factory inspection report (optional) Industry certification report (optional) Packing list List of materials <Specialized certification package>

Product function/feature test

Network deployment service acceptance Single-site verification Equipment installation report Self-check report <Site acceptance package>

Network KPI test System KPI test

Outstanding problem settlement Trial run

Test package

<Feature verification package>

<KPI test package>

<Strategy package> The provisional acceptance is classified into demonstration acceptance and sites acceptance. The demonstration acceptance is used to test functions and features of products. The sites acceptance is used to test engineering service quality and basic services. Demonstration acceptance and sites acceptance can be performed in any order or at the same time.

Note: Engineers are not prompted to perform demonstration acceptance unless customers require testing product functions at the acceptance stage.
HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 9

Associating Test Packages with Acceptance ProcessDemonstration Acceptance

Demonstration acceptance is a new concept aiming to complete product function/feature tests within a small range and avoid large-scale tests site by site. The small range can be test beds, trial sites, and validation networks.
Trial Sites Within an Overall Network Trial sites refer to Type A certain sites to be NodeB constructed on the RNC network. The hardware and software Type B configurations of these sites are the same as those on the actual Overall Network network and can represent the functions of the actual network. Validation Network NodeB

Test Bed
RNC NodeB

Type C

BSC

BTS

Trial Sites

Laboratory

The test bed is a laboratory simulated network. The hardware and software configurations of this network are the same as those on the actual network and can represent the functions of the actual network.

RNC NodeB

RNC

Overall Network

Validation Network

The validation network refers to a subnet of the network to be constructed. The hardware and software configurations of this subnet are the same as those on the actual network and can represent the functions of the actual network.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 10

Test Package Delivery Owner and Precautions


Test Package
Specialized certification package

Delivery Owner
Supply chain/R&D

Precaution
1. Optional. If customers have test requirements, provide the factory inspection report first. 2. If customers require tests, guide customers to allow tests at Huawei headquarters. In this case, quotation is required. 1. Optional. If customers have the requirement, provide internal test reports in priority to avoid repeated tests. 2. If tests must be performed on site, control the tests within a small range, for example, in test beds, trial sites, and validation networks, to avoid large-scale tests site by site. 3. Some test cases may fail to be verified due to immature industry chain, incomplete product implementation, unobservable effect, or theoretical ultimate capacity. For such test cases, only a commitment to provide clarification documents (emulation and theory analysis) can be made. 1. Guide customers to perform tests based on a certain percentage, for example, 10%. 2. If tests involve expensive devices, quotation is required or customers provide the devices. This solution focuses on device acceptance (provisional acceptance) without involving network acceptance. Only KPI test cases are provided. Customers may have test requirements beyond the acceptance range, for example, IOTs, RF tests, new solution/feature tests, and network acceptance. A strategy package contains relevant guidance strategies to avoid impacts of the preceding tests on the acceptance.

Feature verification package

GTS/R&D

Site acceptance package

GTS

KPI test package

GTS

Strategy package

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 11

Solution Introduction

Test Packages Acceptance Supporting Documents Test Case Selection Tools Clauses Negotiation Guidance Strategies

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 12

Acceptance Supporting Documents


The acceptance supporting documents include two test packages and four types of reports.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 13

End-to-End Feature Test Report


R&D engineers shall provide test reports on the optional and special (test cases of features hard to be executed) features in the feature verification package in priority. The test reports of all features will be finally available when the features are released. These reports can be shown to customers during negotiations, reflecting the strict product quality test procedures of Huawei and persuading customers to avoid repeated tests during product acceptance.

End-to-end process of a feature

Step 1: R&D engineers design test cases based on features and provide internal test reports or documents.

Release
Feature

Sales

Step 2: Marketing personnel sell features.

Service delivery
Step 3: On-site engineers negotiate with customers to avoid tests based on the internal test reports and documents provided by R&D engineers.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 14

How to Obtain Acceptance Supporting Documents

Factory inspection report


Wireless product contact person: Zeng Jun (ID: 00106221), Wireless Test & ATE Product Dept Download address: http://10.72.128.225/texpert/Default.aspx See the External Document Archiving Database Query Guide V2.0.ppt. Download address: http://support.huawei.com/support/pages/kbcenter/view/product.do?actionFlag=detailProductSimple&we b_doc_id=SC0000526336&doc_type=123-2 These reports are released together with test cases (test packages). Download address: Chinese edition (including strategy guidance): http://support.huawei.com/support/pages/kbcenter/view/product.do?actionFlag=detailProductSimple&we b_doc_id=SC0000526336&doc_type=123-2 English edition: http://support.huawei.com/support/pages/kbcenter/view/product.do?actionFlag=detailProductSimple&we b_doc_id=SE0000414297&doc_type=123-2

Industry certification report


Version test report and feature test report

1.

2.

Note: Industry certificates are applicable to all kinds of products. Different release test reports and feature test reports are available for different kinds of products. Only the download address of the UMTS test package is given. See Appendix for download addresses of test reports of other products.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 15

Solution Introduction

Test Packages Acceptance Supporting Documents Test Case Selection Tools Clauses Negotiation Guidance Strategies

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 16

Key Improvements to Test Case Selection Tools

1. Risk management of feature test cases Feature test cases are packaged according to the difficulty of test execution, and test risk degrees are marked by "basic", "optional", and "special".
HUAWEI TECHNOLOGIES CO., LTD.

2. Associate front end (sales) and back end (acceptance and delivery) and provide automatic links to test cases from Feature ID or Model in BOQ.

Huawei Confidential

Page 17

Case Management in Excel FormatEasy to View Test Case Details

View the details of a test case by clicking Test Name Able to know details of a test case at any time Easy to execute tests and select test cases

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 18

Case Management in Excel FormatSelect Test Cases to Generate Word Files

Signature Form Acceptance Manual

Use macro to generate a test case and a signature form in Word format
1. A test cases is selected except that the Select flag is null or set to No or Cancel. 2. The editing operations are similar to those of an ordinary Excel sheet, for example, copy, paste, add, and delete. Run macro to generate a test case description file and a signature form in Word format. HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 19

Case Management in Excel FormatExecution Result Identifier and Statistics

Automatically generate test case execution results Test case execution results are marked in different colors. Execution result statistics are performed automatically on a daily or weekly basis.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 20

Case Management in Excel FormatTest Case Selection Tools

Select test cases automatically: 1. In the Selecting Test Case dialog box, determine the baseline document (Excel format) and test case package as prompted. 2. Set conditions. As shown in the figure above, select the test cases whose Model in BOQ is included in the model list and Priority is High. After the test cases meeting preset conditions are selected, marks are automatically filled in Selection Mark of the selected test cases. You can start a new search and combine the search results or start another search in the search results. 3. The selected test cases can be used to generate test case description files and signature forms in Word format. HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 21

Solution Introduction

Test Packages Acceptance Supporting Documents Test Case Selection Tools Clauses Negotiation Guidance Strategies

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 22

Technical Clauses Regarding Acceptance

No on-site acceptance is required for items that are clearly specified in standards or protocols. (For example, for sensibility and standard signaling, third-party certification reports and compatibility analysis reports can be provided to replace tests.)
Tests that require special test instruments (such as power meters, special spectrum testers, and voice/image/stream quality evaluation tools) are not performed on site. No acceptance is required for ultimate functions (for example, xx subscribers in a single cell). (Internal test reports can be provided for clarification, or witness tests can be performed with R&D engineers for important items). No commitments shall be made to verify all sold features on site. (Some customers even request us to test the features of other vendors. Internal test results and clarification documents can be provided to replace the feature verification.) No commitments shall be made to test items that require third-party cooperation (for example, terminal interworking and service platform compatibility; MVI test services can be provided to meet carrier requirements). Standard test cases can be provided. Test services can be sold separately to meet extra test requirements of customers.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 23

Acceptance Responsibility Division Clauses

Clearly specify that the carrier is the responsible person for equipment interconnection and the equipment supplier is

responsible for solving equipment problems. (In principle, no IOT is


performed during acceptance.)

Clearly specify that the carrier should prepare all third-party tools and devices required for tests (mainly trace tools). (In principle, no other test tools are recommended. Guide customers to prepare the tools.) The acceptance certificate is still acquirable if the acceptance is blocked or delayed by non-supplier reasons.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 24

Notes:

Application scenarios of clauses:

At the bidding stage, if acceptance standards are not clearly determined with customers, clauses restricting the scope of tests shall be added in contracts to prevent some risks and avoid being passive when confirming acceptance standards with customers later.

How to use the clauses:

The technical clauses that are important and widely applicable should be included in main contracts, usually in Condition part.

The technical clauses that are important and specific should be included as an acceptance attachment of contracts, also in Condition part.

Responsibility division clauses are included as an acceptance attachment in Roles and Responsibilities part of contracts.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 25

Solution Introduction

Test Packages Acceptance Supporting Documents Test Case Selection Tools Clauses Negotiation Guidance Strategies

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 26

General Negotiation Principles


1

Use the fewest test packages to meet customer requirements.

Make clear that the acceptance aims to prove the normal service provisioning of equipment, instead of verifying features.

Make clear the delivery modes of acceptance: certificates, test reports, clarification documents, tests at Huawei HQ, and on-site tests.

Huawei shall provide paid test services which are claimed to be independent of acceptance and undertaken as a separate project operation.

All test packages are concepts used internally and should be known as "acceptance test packages" for customers.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 27

Negotiation Scenario 1: All Feature Tests


Requirement Description Risk Analysis Recommended Guidance Strategy 1. Guide relevant engineers to provide release test reports. The test reports should contain statistics data that proves the verifications performed internally. 2. Guide relevant engineers to provide feature verification reports, including test cases (logs should be available in reports for tracking). 3. Test contents are within the range of baselines. Clarification documents are provided if on-site verification conditions are not met. On-site tests are allowed. The demonstration acceptance mode is recommended, and the test environment can be test beds, trial sites, or validation network. No repeated tests at each site are permitted. The acceptance at actual sites is allowed only in sampling test manner, for example, based on a percentage of 5% to 10%.

Carriers require verifying all features before the equipment is put into commercial use, which is the most common requirement. Some carriers even require verifying all features at each site.

1. Features are classified into very small categories and the on-site test workload is too heavy. 2. Features of Huawei are not clearly classified, and some features are hard or even impossible to be tested on site.

Strategy 1: Using documents to replace tests

Strategy 2: Performing tests within a smaller range Strategy 3: Performing acceptance based on a certain percentage (Note)

Note:

1. In principle, not all features need to be tested. Only the features that are purchased by customers and to be put into commercial use need to be tested.2. Feature tests should not be performed at actual sites. Strategy 3 is a compromise made with customers only when no other approach is available.
HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 28

Negotiation Scenario 2: IOTs


Requirement Description Risk Analysis Recommended Guidance Strategy
If Huawei equipment is interacted with the equipment of important carriers or mainstream suppliers, make NVIOT clarification with carriers and guide carriers to raise the IOT report requirement. For interconnection requirements in other scenarios, refer to MVI reports (without NDA) and commercial application cases. Equipment interconnection must be completed for network commissioning, but not all IOTs need to be performed. Interconnection is required to meet only commercial application requirements through close cooperation among three parties. The PAC is still acquirable if the interconnection is delayed by nonHuawei reasons.

Generally, carriers require that suppliers provide the equipment supporting interconnection with the existing network equipment and the previous interconnection reports. Some carriers even fail to obtain support from original suppliers.

The protocol compliance of inter-equipment interconnection cannot be settled through only efforts of Huawei. With NDAs signed, IOT and MVI reports shall not be provided for a third party without permission. Due to competition purposes, interconnection equipment suppliers may provide no cooperation or even block the acceptance.

Strategy 1: Using reports to replace tests

Strategy 2: Restricting the test range and specifying responsibilities

Strategy 3: Providing IOTs as a test service independent of acceptance

If carriers require all IOTs, promote the selling of MVI test services and obtain cooperation from interconnected equipment suppliers.

Note: The test service is a professional service that is operated independent of project acceptance and used to meet various test requirements of customers. In terms of acceptance negotiation strategies, the test service is a way to meet requirements and reach a balance with customers by means of quotation.
HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 29

Negotiation Scenario 3: Capacity Specification Tests


Requirement Description Risk Analysis
Capacity specifications are usually the equipment processing capabilities obtained under theoretical baseline conditions, which are hard to be tested on site. At present, the equipment capacity is very large. Ordinary test tools cannot be used to obtain the specifications of the main equipment, special test tools cannot be used for on-site tests due to restrictions of shipment, customs clearance, costs, and personnel skills.

Recommended Guidance Strategy


Strategy 1: Obtaining test-free permission Strategy 2: Using reports to replace tests Provide sufficient evidences showing the difficulty in ultimate capacity specification tests and guide customers to cancel such test requirements. Promise to provide internal test reports to reduce on-site test workload.

Some carriers pay a great concern to equipment capacity and require relevant tests. Some carriers even require verifying the ultimate specifications in real scenarios.

Strategy 3: Returning to Huawei HQ for test

Guide carriers to perform witness tests with R&D engineers at Huawei HQ.

Note: 1. The changes of capacity specification tests are complex. No commitments shall be made to meet special traffic model and networking requirements unless carriers perform the tests. 2. Any commitment related to tests involved in strategy 2 and strategy 3 shall not be made before being confirmed and agreed by R&D personnel. Tests involved in strategy 3 shall be quoted.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 30

Negotiation Scenario 4: RF Index Tests


Requirement Description

Risk Analysis

Recommended Guidance Strategy


Standard tests should comply with third-party organization certification reports, and no on-site tests are performed. Guide customers to provide certificates or reports.

Some carriers require performing RF index tests on site, even including electrical performance and environment requirements.

Index tests require complex test tools and high costs and are hard to be performed on site.

Strategy 1: Providing international organization certification reports

Replace local standards with corresponding international standards.


For test items that vary with modules, such as RF indexes, factory inspection reports can be provided as the index satisfaction evidence. Guide carriers to perform witness tests with R&D engineers at Huawei HQ.

Some tests are required to comply with local standards.

Test personnel are not familiar with local standards, standards are inconsistent, and certification costs are high.

Strategy 2: Providing factory inspection reports Strategy 3: Returning to Huawei HQ for test

Note: 1. For on-site tests, clearly specify that carriers are responsible for preparing test tools and performing tests on only one or two modules. 2. Confirm strategy 2 with the production contact person; confirm strategy 3 with the R&D contact person at Huawei HQ.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 31

Negotiation Scenario 5: Deep Test Requirements


Requirement Description Risk Analysis
Strategy 1: Performing technical clarification

Recommended Guidance Strategy

During the acceptance workshop, carriers perform finitist and theoretical analyses, increasing test requirements abruptly.

The purpose of these tests are to find commercial application schemes, which is already beyond the acceptance scope.

Tests are not necessary so long as commercial application schemes pass the technical clarification during workshop.

Some carriers also have networking, configuration, and commercial strategies, and even require FOA tests.

The scope of FOA must be strictly controlled; otherwise, it may be mixed with pre-commercial application and the commercial application may be conducted before PAC is obtained, making us passive.

Strategy 2: Conducting training

If test items are added for personnel training, the tests can be differentiated from acceptance according to training operations.

Strategy 3: Providing "test services"

If on-site tests are required, a balance can be reached by selling "test services".

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 32

Overall Strategy Model


Network acceptance

The overall strategy model is a procedure of filtering customer requirements, with increasingly higher test difficulties and risks from bottom to top. Tests at upper layers require more strict control. The overall strategy model aims to finally simplify the site acceptance through the filtering at the three lower layers.

Site acceptance package


(service acceptance)

Feature verification package


(wide product feature tests)

Specialized certification package


(professional certification reports/tests)

Strategy package
(IOT and ultimate equipment capability tests)

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 33

Contents
Overview

Solution Introduction
FAQs Negotiation Cases

Q1: Application Stage and Personnel of the Solution

Stage

Role

Activity
1. Includes the acceptance standards and test cases completed by the technical owner based on this solution in the contract. 2. Negotiates on the acceptance standards with customers under assistance of the technical owner. 3. Tries to include the suggested clauses of the solution (clauses can be selected according to project situations) in the contract if acceptance standards cannot be clearly determined with customers at this stage. 1. Specifies strategies according to acceptance items in the solution, and uses test case selection tools to customize test cases for acceptance as an attachment of the contract according to the product function range specified in the contract. 2. Helps the engineering bidding project manager to negotiate with customers on test standards and includes the test cases as an attachment of the contract. 1. If acceptance standards are not clearly specified in the contract, determines the acceptance standards with customers at the delivery preparation stage according to this solution. 2. Determines acceptance standards with customers under assistance of the technical owner. 1. If acceptance standards are not clearly specified in the contract, specifies relevant strategies according to the acceptance standards in this solution, and customizes test cases for acceptance as an attachment of the contract according to the product function range specified in the contract. 2. Helps the engineering project manager to negotiate with customers.

Engineering bidding project manager Contract bidding stage

Technical owner

Engineering delivery project manager

Contract delivery stage


Technical owner

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 35

Q2: How Is This Solution Included in the Contract at the Pre-Sales Stage?
Project management proposal
Engineering bidding document Engineering service and project management proposals Responsibility division interface Construction period schedule Acceptance proposal Acceptance flow Acceptance milestone Acceptance scope Acceptance clauses Acceptance certificate Acceptance standard/test case

After bidding document negotiation, the acceptance standards or test cases agreed by both Huawei and the customer are included in the contract as a part of the acceptance solution.
HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 36

Q3: How to Use this Solution at the After-Sales Stage?

If test cases are not specified or not clearly specified in the contract, test standards must be determined with the customer and both parties must sign for confirmation at the delivery preparation stage before the acceptance is started. No acceptance should be performed before acceptance standards

are clearly determined.

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 37

Contents
Overview

Solution Introduction
FAQs Negotiation Cases

Case 1:

Acceptance steps determined by the customer of project R in country I:


1. 2. 3. 4. 5. 6. LAB Acceptance FOA Acceptance Installations, Integration and Site Acceptance RF Cluster Acceptance Market Acceptance SDCA Level Acceptance

1667 basic test items

for LAB/FOA

The success of the tests in a previous step is the precondition for the tests in the next step.

Results after communication with the customer:

Nearly half of the test items are delivered in other modes, greatly reducing the on-site test workload and risks.
HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 39

Case 2

Test Items Required by the Customer of Project V in Country V:

The customer requires testing all features including the OSS and RAN and completing the acceptance within 30 days after installation. The workload to test all features is heavy, and some test conditions are not met on site. The time schedule is tight, and failure to complete the tasks on time may incur penalty.

Risk Analysis

Results after Communication with the Customer:

Perform feature tests in batches and test the features to be put into commercial use on the existing network before the PAC is obtained. Test other features in three months after the PAC is obtained. If test conditions are not met on site, Huawei can provide internal test reports.

Three months
Launch acceptance PAC RAN: 160 features OSS: 133 features

RAN: Internal test reports can be provided for 154 features

HUAWEI TECHNOLOGIES CO., LTD.

Huawei Confidential

Page 40

Thank you
www.huawei.com