CONCEPT LEARNING AND

THE GENERAL-TO-SPECIFIC
ORDERING

WHAT IS A CONCEPT?
A Concept is a subset of subjects or events defined over
a larger set. For example, We refer to the set of
everything (all objects) as the set of things. Animals are a
subset of things,Anim
and birds are a subset of animals.
Thin
gs

Cars

als

Birds

WHAT IS A CONCEPT?
In more technical terms, a concept is a boolean-valued
function defined over this larger set.
For example, a function defined over all animals whose
value is true for birds and false for every other animal.

WHAT IS A CONCEPT LEARNING?

Given a set of examples labeled as members or nonmembers of a concept, concept-learning consists of
automatically inferring the general definition of this
concept.
In other words, concept-learning consists of
approximating a boolean-valued function from training
examples of its input and output

EXAMPLE OF A CONCEPT LEARNING

TASK
Concept : Good Days for Watersports (values: Yes, No)
Attributes/Features:

Sky (values: Sunny, Cloudy, Rainy)

AirTemp (values: Warm, Cold)
Humidity (values: Normal, High)
Wind (values: Strong, Weak)
Water (values: Warm, Cool)
Forecast (values: Same, Change)

Example of a Training Point:

<Sunny, Warm, High, Strong, Warm, Same, Yes>

EXAMPLE OF A CONCEPT LEARNING

TASK
Day

Sky

Airtemp

Humidit
y

Wind

Water

Forecast WaterSp
ort

Sunny

Warm

Normal

Strong

Warm

Same

Yes

Sunny

Warm

High

Strong

Warm

Same

Yes

Rainy

Cold

High

Strong

Warm

Change

No

Sunny

Warm

High

Strong

Cool

Change

Yes

Chosen Hypothesis Representation:

Conjunction of constraints on each attribute where:
? means any value is acceptable
0 means no value is acceptable
Example of a hypothesis <?, cold, high, ?, ?, ?>

EXAMPLE OF A CONCEPT LEARNING

TASK
Goal : To infer the best concept-description from the set
of all possible hypotheses (best means which best
generalizes to all (known or unknown) elements of the
instance space).
Most General Hypothesis: Everyday is a good day for
water sports
<?, ?, ?, ?, ?, ?>

Most Specific Hypothesis: No day is a good day for

water sports
<0, 0, 0, 0, 0, 0>

TERMINOLOGY AND NOTATION

CONCEPT LEARNING AS SEARCH

Concept Learning can be viewed as the task of
searching through a large space of hypotheses implicitly
defined by the hypothesis representation.

GENERAL & SPECIFIC ORDERING

In each specific concept learning tasks, there exists a hypothesis
space(H) with a set of all possible hypotheses
Each hypothesis is described by a tuple of constraints on the
attributes of the concept. The constraints may be

- ?, which is a general value

- 0, which is a specific value
Convert the collection of hypothesis into a form where they indicate
the result of the task from a very specific task to a general task
represented as:
<0, 0, 0, . . . , 0> to <?, ?, ?, . ?>

GENERAL & SPECIFIC ORDERING

<0, 0, 0, . . . , 0> indicates a very specific order where
the given attributes to learn from are all very restricted
and dont pass given criteria.
<?, ?, ?, . . . , ?> indicates a very general order which
means the given attribute can be substitute with any of
its set of values to pass the given criteria.

ORDERING WITH AN EXAMPLE

Lets consider the example as seen in the previous
concept learning. Where the target concept to be learnt
is the Days on which the person enjoys WaterSport

APPLYING FIND-S ALGORITHM

The Find-S algorithm is used to find the maximally
specific hypothesis, example, the form in which it
satisfies all the given hypothesis for the required result.
Note: Find-S Algorithm only considers positive results /
results in favour and ignores negative results.
In the given example, the 4 given hypothesis h1, h2, h3,
h4 are denoted by the results on Day 1, 2, 3, and 4
respectively. For the FIND-S Algorithm only h1, h2, and h4
will be considered because c(x)={Yes} for these

Inductive Bias
Given the following examples:
Sunny,Warm,Normal,Strong,Cool,Change, yes
Cloudy,Warm,Normal,Light,Warm,Same, yes
Rainy,Warm,Normal,Light,Warm,Same, no
There is no hypothesis in our space that can represent
this concept. The closest we can come
is?,Warm,Normal,Light,Warm,Same
So, it is a biased learner

Unbiased Learner
We could define an unbiased hypothesis space by allowing arbitrary
conjunctions, disjunctions, and negations.
The number of possible hypothesis (ignoring semantic repetition)
is2|X|.
The Candidate-Elimination algorithm, if applied to this hypothesis
space, will be unable to generalize beyond the observed examples!
So, in order to learn the concept we would need to present every
single example.
If we use a partially-learned concept, thenexactly halfof the
hypotheses will vote positive and half negative for every unseen
example.

Three Learners with Different

Biases
Rote Learner - no inductive bias
Candidate Elimination - the target concept can be
represented in its hypothesis space - can classify some
instances that the Rote Learner will not.
Find-S - in addition that all instances are negative
instances until the opposite is entailed by its other
knowledge or most specific hypothesis over others.

Formal Definition
Consider
1. concept learning algorithm L
2. instances X, target concept c
3. training examples Dc={x,c(x)}
4. let L(xi,Dc)denote the classification assigned to the instance x i by L
after training on data Dc.
Definition: The inductive bias of L is any minimal set of assertions B such
that for any target concept c and corresponding training examples of D
xiX(BDcxi)L(xi,Dc)
where A B means A logically entails B

 For example, the inductive bias of the CandidateElimination algorithm (with voting) is the assumption
that the target concept is contained in the hypothesis
space.
That is, if we make that assumption then the
classificationfollows logically (by deduction).

CANDIDATE-ELIMINATION algorithm
diagram