SYSTEM TESTING AND DEPLOYMENT

Lecture Six

Chapter 7: System Testing and Deployment

System Testing & Deployment after Knowledge Codification

Capture Tools Programs, books, articles, experts KNOWLEDGE CAPTURE (Creation)

Intelligence gathering

Shells, tables, tools, frames maps, rules

KNOWLEDGE CODIFICATION

DATABASES

Explicit Knowledge

TESTING AND DEPLOYMENT

KNOWLEDGE SHARING KNOWLEDGE TRANSFER

KNOWLEDGE BASE

7-2

Chapter 7: System Testing and Deployment

Key Definitions
Logical testing answers the question, Are we building the system right? User acceptance testing checks the systems behavior in a realistic environment. Answers the question, Have we built the right system? Deployment refers to the physical transfer of the technology to the organizations operating unit

7-3

Chapter 7: System Testing and Deployment

Stages of KMSLC
Evaluate Existing Infrastructure Form the KM Team Knowledge Capture

Iterative Rapid Prototyping

Design KM Blueprint Verify and validate the KM System Implement the KM System Manage Change and Rewards Structure Post-system evaluation
7-4

Chapter 7: System Testing and Deployment

Quality Assurance

KM system should meet expectations of members represented in the KM team. Performance depend highly on quality of explicit/tacit knowledge stored in KB. For expert, quality relates to reasoning process that produces reliable and accurate solutions. For user, quality relates to systems ease of use and efficiency. For knowledge developer, quality relates to how valid the knowledge sources are and how well they are codified into KB.

7-5

Chapter 7: System Testing and Deployment

KMS Testing
To

control quality, performance and efficiency of both KB and KMS. Types of Testing
Logical

Testing, make sure that the system produces correct results. User Acceptance Testing, follows logical testing and check systems behaviour in a realistic environment.

7-6

Chapter 7: System Testing and Deployment

Hurdles in KMS Testing

Subjective Lack

nature of tacit knowledge. of reliable specifications to guide testing

How

to decide whether knowledge is complete

errors of time interface un-friendly
7-7

Human Lack User

Chapter 7: System Testing and Deployment

Logical Testing Approaches

Knowledge
Structure

base formation

of knowledge as it relates to circular or redundant errors Consistency, correctness, and completeness of knowledge base rules
Knowledge
Reliability

base functionality
of knowledge base
7-8

Confidence

Chapter 7: System Testing and Deployment

Logical Errors

Circular errors - contradictory in meaning or logic Redundancy errors - offer different approaches to the same problem Unusable knowledge - comes up no matter conditions succeed or fail Subsumption errors - if one rule is true, one knows the second rule is always true Inconsistent knowledge - the same inputs yield different results
7-9

Chapter 7: System Testing and Deployment

User Acceptance Testing Approach

Select

a person or a team for testing

on user acceptance test criteria

Decide

Develop

a set of test cases unique to the system

a log on various versions of the tests and test results the system
7-10

Maintain

Field-test

Chapter 7: System Testing and Deployment

Requirements for Test Team

Independent of the design or codification of the system

Understand the knowledge problem and computer technology Well versed in the organizations goals
Know how the system being tested will fit into the designated area of operation
7-11

Chapter 7: System Testing and Deployment

Content of Test Plan

Items

of KM system that require testing of user acceptance testing

Schedule

Description

of test methods that are approved by user and company for documenting
7-12

Guidelines

test results

Chapter 7: System Testing and Deployment

User Acceptance Criteria

Accuracy

and correctness of outcome for test items Adequacy of the solutions Adaptability to novel input Ease of use Face validity or credibility Robustness Technical/operational test
7-13

Chapter 7: System Testing and Deployment

User Acceptance Test Techniques

Face Test

validation Cases

Sub-systems

validation
7-14

Chapter 7: System Testing and Deployment

Face Validation

Developer, users and experts to evaluate the performance of the KM system Can be used to test chunks of knowledge at any phase of the knowledge-based system development life cycle. It is mainly a group effort of the knowledge developer, the domain expert, and the end user. This ad hoc tool is quite useful testing usersystem interfaces, user friendliness, and explanation facilities.
7-15

Chapter 7: System Testing and Deployment

Test Cases
Developer,

users and experts provide

cases It is more of a black-box approach, in which only system inputs and outputs are significant.

7-16

Chapter 7: System Testing and Deployment

Issues Related to Deployment

Selection of the knowledge base problem Ease of understanding the KM System Knowledge transfer Integration alternatives The issue of maintenance Organizational factors

7-17

Chapter 7: System Testing and Deployment

Successful KM System Deployment

User has prior experience with computer applications User has been involved in the building of the KM system Benefits of the KM system is immediate and measurable First impression Management support the system from start Adequate user training

7-18

Chapter 7: System Testing and Deployment

Ease of understanding the KM System

Level

of motivation of the user Computer literacy and technical background of the user Communication skills of the trainer Time availability and funding for training Place of training

7-19

Chapter 7: System Testing and Deployment

Knowledge Transfer

Transfer approach

The KM system is actually transferred from the developer directly to the working unit (end user) in the organization. The KM is transferred of knowledge-based system technology skills.

Transfer ways
One time transfer Gradually

Implementation
Stand alone implementation Integrate with other implementation

7-20

Chapter 7: System Testing and Deployment

Integration Alternative
Technical integration through the companys LAN or existing information system infrastructure Knowledge-sharing integration when the KM system is usable company-wide Decision-making flow integration when the system matches the users style of thinking Workflow reengineering when the KM system triggers changes in the workplace

7-21

Chapter 7: System Testing and Deployment

Organizational Factors

Top management support Support of the work of the champion Ensure a clean and supportive organizational climate De-emphasize role of politics Knowledge developer should remain neutral within the political arena Return on investment
7-22

Chapter 7: System Testing and Deployment

Steps in KMS Deployment

Transfer

of KM System from knowledge developer to the organizations operating unit. of KMs skills from knowledge developer to the organizations operators.
7-23

Transfer

Chapter 7: System Testing and Deployment

Factors affecting User Training

Primary
The

users knowledge of knowledge-based system

Complexity

of KM system and user interfaces ease of use

technical experience/ communication skills of training venue
7-24

Secondary
Trainers

Environment

Chapter 7: System Testing and Deployment

Pre User Training Session

Define how the KM system agrees with organizational mission

Demonstrate how the system can support organizational goals

Allocate adequate resources for the training project Advocate positive effects of the system

Explain cost-benefit analysis of the KM system

7-25

Chapter 7: System Testing and Deployment

Overcome Resistances to Change

User

attitude survey

Communication

sessions
Role

negotiation
7-26

Chapter 7: System Testing and Deployment

Post-Implementation Review

How KM System has changed accuracy/timeliness of decision making How KM System has affected attitude of end users Whether the system has caused constructive organizational changes? Whether the system has benefited the operation cost?
7-27

Chapter 7: System Testing and Deployment

Security Issues
The

new system should provide password / protocol protection. procedures should be consistently observed. should be restricted regarding update of the KB.
7-28

Security

Access

Chapter 7: System Testing and Deployment

Circular Errors
Rule
IF

1:

John and Rob are co-owners of company A

THEN John and Rob own the same company

Rule
IF

2:

John and Rob own the same company THEN John and Rob are co-owners
7-29

Chapter 7: System Testing and Deployment

Redundancy Errors
Rule
IF

1:

(Salary > 50,000 OR HOME > 100,000)

THEN Loan = yes

Rule
IF

2:

(Salary > 60,000)

THEN Loan = yes

7-30

Chapter 7: System Testing and Deployment

Unusable Knowledge
IF

humidity is 5% AND heavy_rain = yes

THEN some actions

7-31

Chapter 7: System Testing and Deployment

Subsumption Errors
Rule
IF

1:
2:

A AND B, THEN D

Rule
IF

A AND B AND C, THEN D

7-32