Detailed comparison of

ModelRisk, @RISK and Crystal Ball

Crystal Ball is a product of Oracle Corporation. @RISK is a product of Palisade Corporation.

Disclaimer: All reasonable attempts have been made to ensure that the comparison between ModelRisk and
competitive products described in this document are factually based, fair and accurate. All software was used
according to vendor specifications and versions tested were those available at the time of publication (July 2014).
If you are aware of any inaccuracies or omissions in this text that should be corrected in order to maintain a fair
and accurate comparison, please contact admin@vosesoftware.com.

Vose Software BVBA www.vosesoftware.com 1|Page

Introduction
This technical review compares the three most common risk analysis Excel add-in software products:

Product Link to official intro video Company web site

Crystal Ball: Here www.oracle.com
ModelRisk: Here www.vosesoftware.com
@RISK: Here www.palisade.com

Summary of conclusions
The best product choice depends on the factors you consider most important:

Lowest cost for similar level of features ModelRisk

Largest choice of distributions ModelRisk
Offers a choice of correlation tools ModelRisk
Most closely follows Excel protocol ModelRisk
Easiest to share results ModelRisk
Widest range of technical tools ModelRisk
Includes insurance and finance modeling ModelRisk
Widest range of results plots ModelRisk, @RISK
Linking to Microsoft Project in Excel - @RISK
Language other than English - @RISK or Crystal Ball

Approach
The primary data source for this review is this1 independent and up-to-date overview of Excel risk
analysis add-ins on Wikipedia. This review only includes the Wikipedia information where there is a
difference between the software products. Aside from which technical capabilities each product has,
user-experience is also extremely important and a strong measure of the quality of the software as a
whole so, where possible, links have been provided to official videos produced by each developer so you
can compare the approaches. Please do take a look at them. This review is split into the following
sections:

Functionality
Technical specifications
Simulation controls
Reporting results
Help file, technical support and training
Implementation within Excel
Precision of the algorithms used in the software
Pricing

1
Last viewed 13 July, 2014

Vose Software BVBA www.vosesoftware.com 2|Page

Functionality
The following chart is plotted from the Wikipedia article which gave a numerical summary of the
number of features each product offered2

The three types of random variables that you tend to see in risk analysis modeling are:

Distributions
Correlation structures
Time series

Distributions - there are something like 150 distribution types that have a practical use and are not
simply a rehashed or renamed version of another one. You could say that 90% of users only need to use
10% of the types of distribution that are known, but which 10% varies enormously between the types of
problems that the users focus on. You can view videos of how the different products fit distributions to
data here: Crystal Ball3, ModelRisk, @RISK.

Correlation structures (copulas) are a very important and much overlooked component of risk modeling.
Failure to incorporate appropriate correlation will usually result in an underestimation of the risk. All of
the products reviewed effectively use a Gaussian copula which allows simulating a correlation structure
between any number of variables4. ModelRisk also provides the Clayton, Gumbel, Frank and Student

2
The table was removed from general view because of Wikipedias policy of not including original research (you can track the discussion that
resulted in its removal on this page).
3
Official video not found
4
In fact, @RISK and Crystal Ball uses rank order correlation, a simulation technique from 1982 that closely resembles a Gaussian copula. See
Iman, R. L. and Conover, W. J., (1982). 'A Distribution-Free Approach to Inducing Rank Order Correlation Among Input Variables', Commun
Statist-Simula Computa 11(3) 311-334.

Vose Software BVBA www.vosesoftware.com 3|Page

copulas. Bivariate versions can be rotated and all can be statistically fit to data. ModelRisk also offers an
empirical copula that will statistically reproduce unusual data patterns across any number of variables.

Correlating two Uniform distributions using the Gaussian copula with, for example, a 90% correlation
would produce a scatter pattern that looks like this:

But the different copulas will produce markedly different patterns for the same 90% correlation, for
example (reading from the top left: Clayton, inverted Clayton, Frank, Gumbel, inverted Gumbel,
Student):

For example, the Clayton copula (top left plot) is often used by banks for loan defaults because these are
strongly correlated in times of recession, but not in better times. The selection of correlation shape can
have a very significant effect on the tails of results when the correlation level is moderate (e.g. 10% to
50%). You can view the way ModelRisk fits copulas to data here, and for unusual patterns here.

Time series models are used to forecast variables across several periods, like exchange rates, inflation,
sales volumes. They incorporate at a minimum the trend and the level of randomness around that trend.
More sophisticated models incorporate characteristics like sudden jumps, cycles of high and low periods
of volatility, etc. The more sophisticated models are usually fit to data using software. You can view
videos of how the different products fit time series to data here: Crystal Ball Suite, ModelRisk, @RISK.

Vose Software BVBA www.vosesoftware.com 4|Page

Technical specifications
This section compares the key differences in the modeling tools that each product offers.

@RISK @RISK @RISK Crystal Crystal Ball ModelRisk ModelRisk ModelRisk

Technical specification Standard Professional Industrial Ball Suite Standard Professional Industrial
Distribution fitting No Yes Yes Yes Yes No Yes Yes
Correlation fitting No No No No No No Yes Yes
Time series fitting No No Yes Yes Yes No Yes Yes
Optimizer included No No Yes No Yes No Yes Yes
Database connectivity No No No No No No No Yes
VBA calls to functions No Yes Yes Yes Yes Yes Yes Yes
C++ calls to functions No Yes Yes No No Yes Yes Yes
Six sigma supported Yes Yes Yes Yes Yes No No Yes
UDF error analysis No No No No No Yes Yes Yes
Extreme value modeling No No No No No No Yes Yes
Expert elicitation tools Yes Yes Yes No No No Yes Yes
Data previsualizer No No No No No No Yes Yes
ODE and numerical integration No No No No No No No Yes
Assumption library No Yes Yes No No No No Yes
Converters for CB CB CB No No CB, @RISK CB, @RISK CB, @RISK

Source: Wikipedia

Notes

Distribution fitting a fundamental requirement unless your risk analysis model is only based on subjective estimates of uncertainty
Correlation fitting another fundamental requirement unless your risk analysis model is only based on subjective estimates of
uncertainty, or all your variables are unconnected (independent)
Time series fitting a fundamental requirement if you want to make forecasts based on historic data
Optimizer allows you to determine the optimal values of variables you control based on some target(s) like maximizing the mean
profit. All three products use the industry leader OptQuest optimizer, and @RISK includes other optimizers
Database connectivity useful if you have databases with large datasets that you want to fit distributions, correlation structures or
time series and that will update as new data are added to the database
VBA calls useful if you want to automatically run simulations, or functions that will internally perform simulations or probability
calculations, etc.
C++ calls for very advanced modelers who want to build their own specialist, high performance simulation programs
Six Sigma functionality a moderately useful feature for Six Sigma specialists that analyzes simulation results using Six Sigma
metrics
Probability calculations a very useful capability for advanced probability modelers because, amongst other reasons, one can
combine calculation and simulation together to create more precise and faster models.
User Defined Function error analysis an error message appears in the spreadsheet cell. For example:

Extreme vale modeling very useful in certain circumstances - many risk analysis issues are driven by extremes, like predicting the
size of the largest wave or wind gust, the lowest temperature, the largest insurance claim, the longest blackout, etc.
Expert elicitation tools to help the modeler get the most precise uncertain estimates from subject matter experts. If this is useful
to you, you should investigate each tools capabilities because there is a great deal of variation in what they offer
Data previsualizer allows you to review and explore the data before you start fitting distributions, time series, or correlations.
Again, a large variation in what is offered, so worth investigating the differences if this is important
Differentiation and integration a very advanced capability for either (i) building a system of ordinary differential equations with
uncertain inputs; or (ii) performing one-dimensional numerical integrations. Useful for scientists, engineers and finance/insurance
people mostly

Vose Software BVBA www.vosesoftware.com 5|Page

 Assumption library allows the user to store key assumptions that are used in various models. Again, a very wide range of
capabilities offered, so worth investigating the differences if this is important
Converter used if one changes to another software product, and wishes to keep using models developed in the old product (CB =
Crystal Ball)

Simulation controls
This section compares the controls that are available to run simulations.

@RISK @RISK @RISK Crystal ModelRisk ModelRisk ModelRisk

Simulation controls Standard Professional Industrial Ball Standard Professional Industrial
Lock / unlock random variables Yes Yes Yes No Yes Yes Yes
Apply specific sample in model Yes Yes Yes No Yes Yes Yes
Random number generator Several Several Several MCG Twister Twister Twister
Spreadsheet interpreter No No No Yes No No No
Sampling method MC, LHS MC, LHS MC, LHS MC, LHS MC MC MC

Source: Wikipedia

Notes:

Lock/unlock sampling switches off the Monte Carlo simulation, replacing random values with a predictable value like the median.
Useful in model auditing as one can compare values between different versions of a model
Apply sample to model allows the user to review the results data and force Excel to show a particular selected sample. This lets
you check, for example, why a particularly high or peculiar value was produced by the model. It also demonstrates to an auditor that
specific scenarios appear within the results, which is especially useful for banks and insurance companies
Random number generator advances in random number generators have made this essentially irrelevant. Nearly every product
uses the Mersenne Twister by default.
Spreadsheet interpreter this creates a compiled version of the Excel model and then runs this instead of within Excel. The result is
a much faster simulation time. Mostly useful if one has a large model that needs optimizing. The drawback is that not all Excel
models can be compiled.
Sampling method MCS = Monte Carlo simulation, LHS = Latin Hypercube sampling. We describe in a LinkedIn blog why LHS is no
longer important here.

Reporting results
Risk analysis is a decision-making tool. That means that it is vital to be able to share the results of a risk
analysis effectively. The simulation results from a risk analysis model can be complex for a decision-
maker to understand, so a wide variety of customizable graphical reports is essential.

@RISK @RISK @RISK Crystal ModelRisk ModelRisk ModelRisk

Reporting results Standard Professional Industrial Ball Standard Professional Industrial
Free report viewer No No No No Yes Yes Yes
Spider Yes Yes Yes No Yes Yes Yes
Box Yes Yes Yes No Yes Yes Yes

The greatest difference between the products is how they organize the storage and presentation of
results, and particularly how these results can be shared with people who are not licensed users of their
products, which is explained below.

Vose Software BVBA www.vosesoftware.com 6|Page

 Crystal Ball
Crystal Ball opens a new window for each graph after clicking a ribbon button. Examples:

Image source: Oracle Axoft

Results files can only be opened by another Crystal Ball user, though the charts can be copied as bitmaps
into Word, PowerPoint, etc.

ModelRisk
Results are shown in a separate application called the Results Viewer which opens at the end of the
simulation. The user can create and save an electronic report and that file can then be distributed to
others, without the need to share the model itself. The file can be opened by anyone who has installed
the ModelRisk Results Viewer, which is available free of charge from Vose Softwares web site here.

Image source: Vose Software

Aside from saving on licenses, the main advantages of this approach is that the report is electronic and
completely customizable labels and color schemes can be edited by the reviewer, new graphs can be
created, plots can be interrogated for say the 95th percentile instead of the 90th, and the whole
report can be resaved if desired. Plots can also be copied into Word, PowerPoint and other applications.

Vose Software BVBA www.vosesoftware.com 7|Page

 @RISK
@RISK opens a new window for each graph after clicking a ribbon button. Examples:

Image source: Palisade

Results files can only be opened by another @RISK user, though the charts can be copied as bitmaps into
Word, PowerPoint, etc.

Help file, technical support and training

Technical support is a vital part of software delivery. Installation of software, particularly ad-ins to other
products, is not always simple because of access restrictions and security policies within a company and
sometimes requires liaising with the IT department. Moving software to a new computer can also be
very frustrating if one does not have ready access to technical support.

For the majority of risk analysis problems, the modeling techniques are not that complex, but some
basic training is really essential to avoid making the most common mistakes. Online videos that teach
the basics of driving the software are a good start, and a comprehensive help file saves a lot of time, but
if you are making multi-million dollar decisions based on a risk analysis model then we recommend
investing in 1-2 days of face-to-face, or at least online, training. We strongly recommend that the
training you purchase is not just about how to use the software. Instead, the course should focus on the
types of problems that you face, and the content tailored to show you the features of the software, the
particular distributions, etc. that you will need. The examples really need to be tailored to your industry
and problems to be effective, since people find it difficult to extrapolate from an example in another
field.

Help file, support @RISK @RISK @RISK Crystal ModelRisk ModelRisk ModelRisk
and training Standard Professional Industrial Ball Standard Professional Industrial
UDFs linked to help file Yes Yes Yes No Yes Yes Yes
Online videos Yes Yes Yes No Yes Yes Yes
Online training Yes Yes Yes No Yes Yes Yes
Onsite training Yes Yes Yes No Yes Yes Yes
ZH, EN, FR, ZH, EN, FR, DE, ZH, EN, FR,
Language versions EN, JA, ES EN EN EN
DE, JA, PT, ES JA, PT, ES DE, JA, PT, ES

Vose Software BVBA www.vosesoftware.com 8|Page

Notes

UDFs linked to help file this means that when you open a dialog for a function, there is a link that will take you directly to the
relevant help file topic. Excel makes this quite easy for developers to do, and automatically adds a hyperlink to the dialog box, so it is
hard to see why this feature should not be implemented

Implementation within Excel

Risk analysis add-ins for Excel are popular because Excel is essentially ubiquitous across businesses and
government agencies, and the modeling environment is intuitive and easy to use. At the same time,
there are many features like database connectivity, the VBA macro language, pivot tables, etc. that are
used by a small minority of power users but make it possible to develop very sophisticated models if
needed. Microsoft also encourages developers to produce add-ins and have a programming structure to
ensure that the add-ins will perform well with Excel. Excel add-ins for risk analysis therefore make a lot
of sense. Monte Carlo simulation add-ins for Excel reviewed here take two approaches to adding
uncertainty to the model.

Approach 1: Covering a cell with a distribution

The first approach, used in Crystal Ball, is to add an invisible layer to the spreadsheet. One selects a
spreadsheet cell that contains a value (not a formula), selects a distribution from the menu perhaps
with links to other cells, and clicks OK to enter the distribution. The cell background turns green, but the
original number that was in that cell remains unchanged. Some people like this because one can share
the model with non-Crystal Ball users and it still works as the original unrisked spreadsheet model,
although the simulation component is lost unless the other user has Crystal Ball too. It has its dangers,
however. You can enter a value of 100 in the cell, for example, and cover it with a uniform distribution
between 0 and 1. The incompatibility is not detected by the software and the distribution is only visible
if you click on a Crystal Ball button. The lack of visibility also makes it more difficult to audit and check a
model. Links within Crystal Ball are invisible to Excel, and Excels auditing tools do not work.

Approach 2: Inserting a random sampling function

The second approach is to provide user-defined functions, or UDFs. This is the standard method of
creating extra Excel functionality envisioned by Microsoft and employed by ModelRisk and @RISK. For
example, the following UDFs will take a random sample from a Triangular distribution with minimum,
most likely and maximum of 0, 40 and 100 respectively:

ModelRisk: =VoseTriangle(0,40,100)
@RISK: =RiskTriang(0,40,100)

The use of UDFs makes it possible to trace through the logic of the model using Excels Formula Auditing
tools as illustrated below:

Vose Software BVBA www.vosesoftware.com 9|Page

A fundamental Excel rule is that there is a hierarchy of calculation in any formula. For example, in this
formula:

=SUM(SQRT(A1), A2)

Excel will evaluate A1 and A2, then evaluate SQRT(A1), and finally evaluate the SUM.

These rules are critical to having a predictable calculation outcome. However, this presents problems for
developers in creating functions that perform some operation on a distribution. For example, consider
the following @RISK function:

=RiskCompound(5, RiskLognorm(10,1))

The function sums 5 independent samples from a Lognormal(10,1) distribution. However, if we follow
Excels rules, the RiskLognorm function which is a random sampling function - must be evaluated
before the RiskCompound function. So, wed get something like this as an intermediary calculation:

=RiskCompound(5, 1.032)

in which case the RiskCompound would just be adding five 1.032 values together. To avoid this, @RISK
suppresses the evaluation of the RiskLognorm function. This causes problems of predicting how the
function behaves. For example, it is not immediately evident how the RiskCompound function should
behave in these circumstances:

=RiskCompound(5, RiskUniform(RiskUniform(2,3),4))
=RiskCompound(5, A2), where A2: =RiskNormal(A1,2)

ModelRisks approach is to use different functions when defining a distribution, called Objects, not the
same function used for sampling. Their equivalent function works as follows:

= VoseAggregateMC(5, VoseLognormalObject(10,1))

The LognormalObject function is used to define the distribution we are intending to use. Object
functions allow ModelRisk to comply with Excels evaluation rules. They are used in a variety of
ModelRisks tools.

Vose Software BVBA www.vosesoftware.com 10 | P a g e

Pricing
All software vendors offer different editions of their products. In the following table, the products are
sorted according to price in US$ for a single perpetual (i.e. not time limited) license including the first
years maintenance and support. These prices appear on the following web sites, accessed on 10 July
2014: Crystal Ball5, ModelRisk6, @RISK7.

ModelRisk @RISK @RISK ModelRisk @RISK ModelRisk Crystal Ball

Price comparison Standard Standard
Crystal Ball
Professional Professional Industrial Industrial Suite

Perpetual license + 1st year support $895 $1,195 $1,213 $1,595 $1,645 $2,195 $2,295 $2,433
Cost of additional year's support $161 ? $218 ? $296 ? $413 $438
# CPUs supported All 2 ? 2 All All All ?
Single click function view Yes No No No Yes No Yes No
Distribution fitting No No Yes Yes Yes Yes Yes Yes
Links to Microsoft Project No No No Yes No Yes No No
Aggregate modeling No No No Yes Yes Yes Yes No
Markov Chain simulation No No No No Yes No Yes No
Multivariate time series No No No No Yes No Yes No
Dynamic sorting No No No No Yes No Yes No
Optimizer No No No No Yes Yes Yes Yes
Stochastic data sharing No No No No No No Yes No
Specialist financial tools No No No No No No Yes No
Specialist insurance tools No No No No No No Yes No
Bayesian model averaging No No No No No No Yes No

Notes

Some additional factors have been included that are not already described in previous sections. The choice is somewhat subjective
but focused on features useful to a modeler.

5
https://shop.oracle.com/pls/ostore/product?p1=oraclecrystalball&p2=&p3=&p4=&sc=ocom_crystalball
6
www.vosesoftware.com/purchasing.php
7
www.palisade.com

Vose Software BVBA www.vosesoftware.com 11 | P a g e