DS2.

10

How many subjects for your sensory and consumer studies?

Gain your own expertise with a systematic bootstrap approach.
A. DESBIENS1, C. URBANO1, E. TEILLET1
1SensoStat, Dijon, France

Problematic
• The number of subjects needed for sensory and consumer tests is an important and redundant problematic for most companies using sensory analysis routinely.
• Anyone can find norms and tables in literature proposing guidelines in terms of panel size (e.g. AFNOR NF V 09-500, 2012), but these advices are finally empirical and
“a priori” defined. It cannot really be adapted to the specific objectives and products of each company and study.
• In this context, bootstrap (Efron, 1979) can be a very useful resampling technique enabling to extract “a posteriori” information of your data about the panel size.

How does it work ?

Data concerned What is Bootstrap? Statistical protocol
Resampling with replacement
Raw data
o Any sensory or consumer data: Original dataset
(e.g. 100
e.g. liking score (at least 20 consumers)
observations recommended) First stage: Choice of:
Assessment on
ANOVA (product + subject) + Multiple Comparison Tests: • α threshold
Possible calculation of: • Multiple
raw data  Product F-value
comparison test
Liking  Number of discriminated pairs
Subjects Product  Percentage of ”information” lost… • Parameter(s)
score calculated
Subject 1 Product A 7
Creation of Choice of:
Subject 2 Product A 8 Non parametric Non parametric Non parametric • α threshold
B= 1,000 Second stage: Confidence interval of Confidence interval of Confidence interval of
… … … • Confidence interval
“realistic” Bootstrap the selected parameters the selected parameters the selected parameters construction
Subject n Product A 3 new panels for n consumers for n-10 consumers for 10 consumers
(percentile or other
of 100, method)
… … … 90,…, 10
Upper bound of the
Subject 1 Product P 5 consumers. confidence interval
Subject 2 Product P 6 Third stage:
Graphical
… … … representation Lower bound of the Estimated mean of the
B boostraps B boostraps B boostraps confidence interval parameter (e,g: number
Subject n Product P 1 and conclusions of discriminated pairs)
of 100 of 80 … of 10
consumers consumers consumers

Example of Results
o Example for steaks: 6 products, 84 consumers, 12 discriminated pairs in the original dataset o Example for dairy products: 12 products, 102 consumers, 39
out of the 15 possible (percentage of discrimination: 80%) discriminated pairs in the original dataset out of the 66 possible (percentage of
discrimination: 59%)
Number of discriminated pairs
Good panel size?

o The mean number of discriminated pairs is not o With 95% confidence intervals:
linear. • A significant loss of information
o With 95% confidence intervals: occurs with a panel size of 40
• A significant loss of information occurs with a consumers.
panel size of 16 consumers only.
• But with less than 40 consumers, we could be o What about 50 or 60 consumers?
afraid to loose to much information… Significant loss
Significant loss
of information
of information
o With around 50 consumers (42% economy), we
would have good chances to reach the same
information in terms of discriminated pairs. If consumer tests are conducted
routinely, meta-analyses are
Product F-value possible to determine your own
guidelines in terms of panel sizes.

o The product F-value (from ANOVA product +

subject) is the ratio between information (Product
MS) and “noise“ (Error MS). o Approximately the same conclusions as
for the number of discriminated pairs
o The mean F-value decreases linearly (according to
the formula). o With 95% confidence intervals:
• Significant loss of information with
o With 95% confidence intervals: 50 consumers.
Significant loss • A significant loss of information occurs with 40 Significant loss
of information of information
consumers.

Discussion
Advantages Limits Perspectives
 The method may be applied to every kind of hedonic  Bootstrap is time-consuming and depends on your  Adaptation to other sensory and marketing
assessment, emotion measurements and so on… software and hardware protocols (discrimination and similarity tests,
 Bootstrap is a non-parametric method  Approximately 20 minutes for 1,000 bootstraps sensory profile, temporal methods…)
 no assumption on the distribution of the data  By now, only one dataset and one descriptor studied  Definition and optimization of cost/loss of
(specially in case of consumers segmentation) at each analysis information criteria
 Meta-analyses of several datasets and descriptors
 R-Package?