Concepts of Inductive Logic

Programming
Vasile Alaiba <alaiba@info.uaic.ro>

BPM Research Group

Department of Computer Science, “Al.I.Cuza” Univ. Iasi
http://profs.info.uaic.ro/~alaiba/cercbpm
 Summary
• Fundamentals
• Induction Logic Programming
• Attractive features and practical
difficulties
• Relevance to BPM
• References
 What is Inductive Logic
Programming?
• Intersection of:
– Logic Programming (Prolog)
– Inductive Machine Learning
• Informally:
– induction of Prolog programs
 Logic Programming
• Algorithm = Logic + Control
• Logic Programming = Declarative (logic)
• Program = facts + rules = knowledge base
• Execution = resolving a goal (predicate)
starting with a knowledge base
 Prolog Example
male(‘Bob’).
female(‘Sharon’).
father(‘Bob’, ‘Sharon’).
parent(U, V) :- father(U, V).
%%%%% end of Prolog program (knowledge base)

?- parent(‘Bob’, ‘Sharon’).
Yes
?-parent(P, ‘Sharon’).
P = ‘Bob’
 Inductive Machine Learning
• Develop tools and techniques to:
– induce hypotheses from observations
(examples)
– synthesise new knowledge from experience
• Main limitations of classical IML
– limited knowledge representation formalism
– difficulties in using substantial background
knowledge in the learning process
Induction as Inverted Deduction
• Induction is just the inverse of deduction!
• Given some data D and some partial
background knowledge B, learning can be
described as generating a hypothesis h
that, together with B, explains D.
• Training data D of the form (xi, f(xi))
• Where xi denotes the ith training instance
and f(xi) denotes its target value
Inverse Entailment Operators
• An inverse entailment operator O takes
the training data D = {(xi, f(xi))} and
background knowledge B as input and
produces as output a hypothesis h:
– O(B, D) = h such as
(IE)
 An example (1)
• Target concept to be learned is:
– “pairs of people (u, v) such that the child of u is v”
– represented by predicate Child(u, v)
• A single positive example:
– f(xi): Child(Bob, Sharon)
• Instances described by:
– xi: Male(Bob), Female(Sharon), Father(Sharon, Bob)
• General background knowledge:
– B: Parent(u, v) ← Father(u, v)
 An example (2)
• In this case, two hypotheses that satisfy
the constraint (IE) are:
– h1: Child(u, v) ← Father(v, u)
– h2: Child(u, v) ← Parent(v, u)
• The process of augmenting the set of
predicates, based on background
knowledge, is often referred to as
constructive induction
 Learning task in terms of (IE)
Attractive features
• subsumes the common definition of learning as finding
some general concept that matches a given set of
training examples (no background knowledge B is
available)
• incorporating the notion of background information B,
this formulation allows a more rich definition of when a
hypothesis may be said to “fit” the data, allowing the
domain-specific background information B to become
part of the definition of “fit”
• this formulation invites learning methods that use the
background information B to guide the search for h,
rather than merely searching the space of syntactically
legal hypotheses.
 Learning task in terms of (IE)
Practical difficulties
• The requirement (IE) does not naturally accommodate noisy training
data.
– does not allow errors in the observed description of the instance xi or its
target value f(xi)
– errors can produce an inconsistent set of constraints on h
– most formal logic frameworks completely lose their ability to distinguish
between truth and falsehood once they are given inconsistent sets of
assertions
• Searching the space of hypotheses is intractable in the general case
– FOL is so expressive that the number of hypotheses that satisfy IE is
too large.
– recent work has sought restricted forms of first-order expressions, or
additional second-order knowledge, to improve the search tractability
• Despite our intuition that background knowledge B should help
constrain the search for a hypothesis, in most ILP systems the
complexity of the hypothesis space search increases as
background knowledge B is increased
 Relevance to the BPM field
• Process/Workflow mining = automatically
inferring a process model from execution
traces (history logs)
– www.processmining.org (Wil van der Aalst)
– A.J.M.M. Weijters and W.M.P. van der Aalst, In: Kröse, B., De Rijke, M.,
Schreiber, G, and Someren, M. van (Eds.), Process Mining: Discovering
Workflow Models from Event-Based Data, Proceedings of the 13th Belgium-
Netherlands Conference on Artificial Intelligence (BNAIC 2001). BNVKI,
Maastricht, pp 283-290, 2001
– Goedertier, S., Martens, D., Baesens, B., Haesen, R. and Vanthienen, J., A new
approach for discovering business process models from event logs. Research
Report KBI0716, ''K.U.Leuven'', 2007
– M. Bozkaya, J.M.A.M. Gabriels, and J.M.E.M. van der Werf, Process
Diagnostics: a Method Based on Process Mining, To appear in the proceedings
of International Conference on Information, Process, and Knowledge
Management (eKNOW), 2009
 References (foundations)
• 1994-1997
– N. Lavrac, S. Dzeroski, Inductive Logic Programming:
Techniques and Applications. Ellis Horwood, 1994
– S. Muggleton, L. Raedt, Inductive Logic
Programming: Theory and Methods. Journal of Logic
Programming, 19-20:629-679, 1994
– T.M. Mitchell, Machine Learning. McGraw-Hill,
chapter 10, Learning sets of rules, 1997
– S.-Hwei, N.-Cheng, R. de Wolf, Foundations of
Inductive Logic Programming. Lecture Notes in
Artificial Intelligence, LNCS, 1228, 1997
 References (present)
• 2006-2008
– A.J. Knobbe, Multi-Relational Data Mining. Frontiers
in Artificial Intelligence and Applications, 145, IOS
Press, 2006
– K. Kersting, An Inductive Logic Programming
Approach to Statistical Relational Learning. Frontiers
in Artificial Intelligence and Applications, 148, IOS
Press, 2006
– L. Raedt, Logical and Relational Learning. Cognitive
Technologies, Springer, 2008
– L. Raedt, P. Frasconi, K. Kersting, S. Muggleton,
Probabilistic Inductive Logic Programming. Theory
and Applications. Lecture Notes in Artificial
Intelligence, LNCS 4911, Springer, 2008