Knowledge acquisition

(Ch. 17

Durkin)

knowledge engineering: building expert systems

knowledge acquisition: process of extracting knowledge from an
expert, organizing it, and encoding it into a knowledge base
knowledge elicitation: extracting knowledge from an expert
knowledge acquisition is the principle bottleneck in expert system
development
many techniques and theories about how to best do this
more tools are appearing to help in this
early example: inductive inference tables
active research area
psychologists are especially interested in elicitation issues, as
it is a fundamental problem of human psychology

B. Ross Cosc 4f79

Knowledge
acquisition
Expert
data, problems,
questions
knowledge
concepts
solutions

Knowledge
engineer

Formalized
structured
knowledge

KNOWLEDGE
BASE

Needs,
usability,
feedback
Prototypes,
needs queries

End user
Also: other experts, literature
B. Ross Cosc 4f79

Some problematic
phenomena

1. Paradox of expertise: The more competent a domain expert is, the less
able she is to describe the knowledge they use to solve problems.
- studies & experience shows that experts are experts because they
compile their vast knowledge into compact, efficiently retrievable form
- as a result, they ignore lots of details about how they derive conclusions
--> intuition is prevalent; structured principles are ignored
- for example, experts use lots of generalization and pattern matching to
solve standard and new problems
2. Experts make bad knowledge engineers
- domain experts are the worst people for formalizing their own knowledge
- non-objective, unfamiliar with AI technology, ...
- need an objective view of knowledge, which isnt possible from expert
- eg. try to formalize how you go about creating a computer
program to solve some problem

B. Ross Cosc 4f79

Some problematic
phenomena
3. Don't believe everything experts say.
experts rely on intuition, compiled knowledge
unaware of the deep reasoning; use shallow reasoning
ie. often short-term memory isnt used;rather, long-term memory as obtained
via past experiences is relied upon
---> huge gaps in knowledge
because experts don't know the formal structure of their knowledge,
their descriptions will likely be wrong
- they arent used to verbalizing their expertise!
therefore, knowledge engineer must watch for knowledge that is...
- irrelevant, incomplete, incorrect, inconsistent
- knowledge engineer will formalize an expert's knowledge, and then
test it to see whether it is logically consistent

B. Ross Cosc 4f79

Steps in knowledge
acquisition
1. Collect: (elicitation)
- getting the knowledge out of the expert
- most difficult step
- lots of strategies
2. Interpret:
- review collected knowledge, organize, filter
3. Analyze:
- determining types of knowledge, conceptual relationships
- determining appropriate knowledge represention & inference
structure
4. Design:
- extracting more knowledge after using above principles
Lets look at these in more detail...
B. Ross Cosc 4f79

Tasks of main
players

Durkin 17.4

B. Ross Cosc 4f79

Preliminary
steps

Durkin 17.7

B. Ross Cosc 4f79

Interviews and
questions

Interacting with the expert is the primary means of eliciting

knowledge

17.9, 17.10

B. Ross Cosc 4f79

Interview
strategies

there are different interview techniques; some are suited to

different phases of the elicitation process
Funnel sequencing technique: interview progresses from general,
exploratory questions, to detailed questions
Prompts

Indirect

Beginning of topic ; General

Probes

Direct

End of topic ; Concrete

SUMMARIZE INTERVIEW

B. Ross Cosc 4f79

1. Unstructured
interview

a spontaneous, natural means to let expert talk freely on anything in

domain
expert verbalizes responses to general questions asked by KE
stream of consciousness sometimes used
KE keeps a minimal level of focus on topics discussed
goal: not to let KE unduly influence early explorations of knowledge

17.14, 17.15

B. Ross Cosc 4f79

10

2. Structured
interview

much more focussed and disciplined than unstructured interview

KEs task is to discover concrete information about specific
questions
topic to be explored has been established at earlier sessions
not as exploratory as unstructured --> better for advanced phases

17.18, 17.19

B. Ross Cosc 4f79

11

To interview or not to
interview

Interviewing is primary means of knowledge elicitation.

However, there are weaknesses:
procedural knowledge difficult to verbalize
easier to do than to describe
plus some knowledge (physical, artistic) not easily verablized

ineffective long-term memory

expert just doesnt remember details of problems
compiled knowledge is difficult to reconstruct

Case studies: another strategy useful in concert with interviews

B. Ross Cosc 4f79

12

3. Retrospective case
study

ask expert to review and explain a solved case

expert goes over all the steps, explaining as she or he goes
KE will record the protocol: the sequence of problem-solving
steps or strategies used by expert
types of case studies:
a) familiar case: a typical vanilla case
general info is obtained
best for early phases when foundations are sought

b) unusual case: a new problem hereforeto unseen by expert

good way to get deeper, detailed, more introspective expert
feedback
best for intermediate, later stages

17.22, 17.23

B. Ross Cosc 4f79

13

4. Observational case
study

rather than giving expert the whole case, just supply the problem
description
then watch & record the expert as he or she solves the problem
stream of consciousness useful
both familiar and unfamiliar problems can be used
familiar: more general knowledge obtained
unfamiliar: detailed, deeper insight into problem solving
obtained

17.26, 17.27, 17.30, 17.31

B. Ross Cosc 4f79

14

Summary: strategy
effectiveness

17.32, 17.33, 17.34

B. Ross Cosc 4f79

15

Analyzing the
knowledge
1. data from expert interview & observation is then transcribed into
text form
important to document all data: date, who, what,...
2. the text is interpreted
identifying chunks: labelling key parts of the knowledge
what portions of knowledge? what are they?

3. Analyzing (sorting) the knowledge:

interrelating the knowledge with previous sessions
determining its representation in domain-friendly notation
converting it to KB language
this is done iteratively and incrementally
must pass it by expert for confirmation and corrections

knowledge dictionary: akin to data dictionary in DB systems

a system document that indexes all terms, rules, etc

B. Ross Cosc 4f79

16

Example transcript
(step 1)

17.11

B. Ross Cosc 4f79

17

Interpreted Transcript
(step 2)

17.12

B. Ross Cosc 4f79

18

Interpreting
transcript

17.36, 17.37

B. Ross Cosc 4f79

19

Knowledge
analysis

Graphical representation of knowledge is an effective means of

organizing it
both KE and expert can understand
idea is that graphical notations close the semantic gap
between expert knowledge and formalized form
Some techniques
cognitive maps: hierarchical, frame-like graphs
inference networks/trees: AND-OR tree
flowcharts: great for procedural knowledge
decision tree
example table (from which decision tree, neural net derivable)
contemporary knowlege engineering tools incorporate graphical
denotations of KB

B. Ross Cosc 4f79

20

Graphical
representations

17.7, 17.8, 17.9, 17.10

B. Ross Cosc 4f79

21

Conclusi
on
research in AI, psychology is forming models of how people & experts
organize knowledge, learn, and do problem solving
- these models will give means for determining the best way to
extract knowledge from experts, and encode it into a KB
in the meantime, knowledge engineers (experts themselves) rely on
experience for acquiring knowledge and constructing expert systems
- what about: an expert system for creating expert systems?
KE is quite an interesting and challenging
- lucrative profession
- active research area

B. Ross Cosc 4f79

22