With many thanks to Jean Daunizeau, Guillaume Flandin, Karl Friston & Will Penny
Introduction and general principles | key words and definitions | inference techniques and outcomes
Introduction and general principles | key words and definitions | inference techniques and outcomes
Science 2011
Introduction and general principles | key words and definitions | inference techniques and outcomes
Applied statistics
Descriptive statistics
summary statistics, graphics
Inferential statistics
Data interpretations, decision making
(Modeling, accounting for randomness and unvertainty, hypothesis testing, infering
hidden parameters)
Probability
Introduction and general principles | key words and definitions | inference techniques and outcomes
Plausibility
You know that :
Introduction and general principles | key words and definitions | inference techniques and outcomes
P. de Fermat (1601-1665)
B. Pascal (1623-1662)
A.N. Kolmogorov (1903-1987)
Cox-Jaynes theorem:
1.Divisibility and comparability
2.Common sense
3.Consistency
=0
=1
= /
Introduction and general principles | key words and definitions | inference techniques and outcomes
Given a bag with twice more white balls than red balls, what is the
probability to draw 2 red balls ?
Deductive reasoning
P( R 2) ?
Given that we drawn 5 red and 15 white balls, what is the proportion
of red balls in the bag ?
Inductive reasoning
Introduction and general principles | key words and definitions | inference techniques and outcomes
Bayes theorem
~1774
~1748
prior belief
F
p)E)ppF
pCppCF
E
C
)C
C
C
p (p
C(pp
)((p
C
E
EE
p
CC
C
C
Joint probability :
P C F
P F C PC
P F
Conditional probability :
Marginal probability :
C: causes
F: facts
p ( F , C ) p F C pC
p F C
p ( F ) p F Ci p (Ci )
i
Introduction and general principles | key words and definitions | inference techniques and outcomes
Frequentist interpretation
Bayesian interpretation
- Probability = degree
measure of uncertainty
of
belief,
- Is meant to be Objective
Introduction and general principles | key words and definitions | inference techniques and outcomes
ML
MAP
Introduction and general principles | key words and definitions | inference techniques and outcomes
PY , M
likelihood
Causes
posterior distribution
Observations
P Y , M
Introduction and general principles | key words and definitions | inference techniques and outcomes
Bayesian inference
What I cannot create, I do not understand.
Richard Feynman (1918 1988)
Model/Hypothesis
Likelihood
P Y , M
Posterior
or conditional
Prior
PY , M P M
PY M
Marginal likelihood
or evidence
To be infered
Introduction and general principles | key words and definitions | inference techniques and outcomes
Likelihood function
Assumption
PY , M
= ()
=
= +
P 4 0.05
1 2
p exp 2
2
0
Data distribution, given the parameter value:
2
1
p y exp 2 y f
2
Introduction and general principles | key words and definitions | inference techniques and outcomes
Incorporating priors
P M
Likelihood
generative model M
Prior
= +
~ ,
~ 0,
Introduction and general principles | key words and definitions | inference techniques and outcomes
Incorporating priors
P M
Likelihood
generative model M
Prior
= +
~ ,
~ 0,
Introduction and general principles | key words and definitions | inference techniques and outcomes
Incorporating priors
P M
Likelihood
generative model M
Prior
= +
~ ,
~ 0,
Introduction and general principles | key words and definitions | inference techniques and outcomes
About priors
Shrinkage prior
~ 0,
Conjugate prior
when the prior and posterior distributions belong to the same family
Likelihood dist.
Binomiale
Beta
Multinomiale
Dirichlet
Gaussian
Gaussian
Gamma
Gamma
Likelihood
= 1 +
~ 0,
Prior
= 1 , 2 , . . , 1
1 ~ 2 , 2
2 ~ 3 , 3
1 ~ ,
causality
Introduction and general principles | key words and definitions | inference techniques and outcomes
p t H0
p y
P H0 y
P t t * H0
t t Y
t*
if P t t * H 0
then reject H0
classical SPM
if P H 0 y then accept H0
Bayesian PPM
Introduction and general principles | key words and definitions | inference techniques and outcomes
1 if 0
H 0 : p H 0
0 otherwise
p Y H 0
p Y H1
H1 : p H1 N 0,
if
PY M
0
< then reject H0
1
Y
space of all datasets
Introduction and general principles | key words and definitions | inference techniques and outcomes
Inference on models
if
PY M 1 PY M 2
, select model 1
BF12
PY M 1
PY M 2
Introduction and general principles | key words and definitions | inference techniques and outcomes
Model evidence
y = f(x)
Occams razor
Complex models should not be considered without necessity
y=f(x)
Data space
p(Y | M ) p(Y | , M ) p( | M )d
Usually no exact analytic solution !!
Introduction and general principles | key words and definitions | inference techniques and outcomes
sup PY , M 1
BIC 2 log
n 2 n1 log N
sup PY , M 2
sup PY , M 1
AIC 2 log
2n2 n1
sup P Y , M 2
Free energy F
Introduction and general principles | key words and definitions | inference techniques and outcomes
Introduction and general principles | key words and definitions | inference techniques and outcomes
Introduction and general principles | key words and definitions | inference techniques and outcomes
Introduction and general principles | key words and definitions | inference techniques and outcomes
Introduction and general principles | key words and definitions | inference techniques and outcomes
P Y , M
The posterior can be combined with the posterior model probabilities to compute a posterior
over parameters independent of model assumptions (within the chosen set)
PM Y
Introduction and general principles | key words and definitions | inference techniques and outcomes
Introduction and general principles | key words and definitions | inference techniques and outcomes
Introduction and general principles | key words and definitions | inference techniques and outcomes
Introduction and general principles | key words and definitions | inference techniques and outcomes
Introduction and general principles | key words and definitions | inference techniques and outcomes
Overview
References