Key Characteristics
Decision Management Systems have three key characteristics – four if you count
their unique focus.
Decisions
Their unique focus, of course, is that they are designed to manage and automate
decisions. Not process, or events. Nor are they focused on managing data or being
a complete enterprise application. Decision Management Systems just make
decisions.
Agile
Because decision making changes all the time, often in response to external stimuli
over which you have little or no control, Decision Management Systems must be
designed for change and for the management of change, not just for execution. The
system is built from the very beginning on the basis that it will need to change. This,
of course, is one of the primary reasons for using business rules and a business rules
management system as the basis for building a Decision Management System.
Analytic
Good decisions—accurate or precise, profitable decisions—often rely on the
analysis of data to see what is likely to work and work well. Decision Management
Systems are therefore analytic, embedding such analysis into their decision-making.
They are designed to apply the content of what we know, the data we have, not just
its structure.
Adaptive
Finally Decision Management Systems are designed to improve over time, to adapt.
They are designed to learn, test and experiment. Both automated learning and
support for the kind of experimentation that helps humans learn how best to change
the system are involved. A Decision Management System rarely works as well on
day 1 as it will later. It is also timeless, continuing to adapt to new circumstances.
Becoming Analytic
Why Be Analytic?
When people make decisions they consider policies and regulations, they apply their
experience and best practices. Often they also conduct some analysis of data,
looking at transactions and historical records and changing their decision based on
what they find. When we automate a decision we can use business rules to do a
great job of representing policies, regulations, best practices and expertise. What we
lack is a way to automate this analysis.
Uses of Analytics
Analysis of data improves the precision or accuracy of decisions. By evaluating the
effectiveness of alternatives in the historical record and by consider the data
available, the effectiveness of decisions is improved. In general there are three kinds
of decisions that can be improved with analytics—risk decisions, fraud decisions and
opportunity or marketing decisions. These are all kinds of operational decisions, the
kind of high volume, transactional decisions that can be automated using Decision
Management Systems. Analytics have the most history in risk and fraud decision
making but there is tremendous energy behind using analytics in marketing and
other opportunity-centric decisions more recently.
Risk Decisions
Risk is acquired one customer, one transaction at a time: Credit risk is acquired by
making bad loans to consumers, supply chain risk by selecting the wrong supplier,
schedule risk by picking the wrong route. Many operational decisions can therefore
be improved if the decision is made based on an accurate assessment of risk.
In general there is a big gap between the value of a good risk decision and the value
of a bad one. If, for instance, an accurate decision is made about credit risk then a
bank stands to make some money from fees. If an inaccurate decision is made then
the bank could lose significantly more money in unrecoverable debt.
To assess risk in these decisions requires analytics that can predict the risk—
analytics that can answer the question “how risky is this person or transaction.”
Organizations often want to know why someone or something is risky and the use
of analytics in these circumstances is often regulated, limiting the kinds of analytics
that can be used.
Fraud Decisions
Fraud decisions are closely related to risk decisions except that there is generally no
upside in a fraud decision—a bad fraud decision will result in a loss but a good
decision does not automatically make money, it simply makes it possible to make
money from the transaction. Classic fraud decisions are those around accepting
credit cards, identifying someone for security purposes, and detecting tax or
insurance claims fraud.
Good fraud decisions require an accurate assessment of how likely it is that this
transaction, person or network is fraudulent. In general no-one really cares how this
is determined as long as few fraudulent transactions are missed and few legitimate
transactions are mislabeled as fraud.
Opportunity Decisions
Opportunity decisions are focused on how to maximize loyalty and revenue,
typically when interacting with customers or prospects. These very customer-
centric decisions include cross-sell and up-sell decisions for example. There is
generally little difference between good and bad decisions as these decisions are
operating at the margins. There is generally no downside at all—even a very bad
opportunity decision will not undermine the core transaction, it will simply mean no
additional value can be created.
Analytics can be used to predict which offers or actions will provoke a positive
response, to predict how likely there is to be an opportunity to provoke a response
or to estimate the potential available. These kinds of analytics must change rapidly to
take advantage of competitive and market circumstances and generally involve many
scenarios that must be modeled.
Types of Analytics
To improve our risk, fraud and opportunity decisions we need to develop more
analytic Decision Management Systems. Traditional analytic tools such as reporting
and query tools are not embeddable in Decision Management Systems and offer only
limited potential. To drive analytics into our Decision Management Systems, we
must move up the analytic spectrum. Success will require going beyond Business
Intelligence tools to data mining and predictive analytics. Data mining and predictive
analytics approaches in particular let you take the increasing volume of data available
to you and apply it to your Decision Management Systems.
Figure 3: The Analytics Spectrum
Thresholds
The first step is to use analytics to improve the thresholds used in your rules. As
business users develop business rules, or work with business analysts to do so, many
thresholds and conditions must be specified. These may be specified by regulations
but are often either based on best practices and experience or company policy.
Simple analytic tools such as query and visualization tools can be used to evaluate
these thresholds. The impact of a threshold can be assessed, for instance the
number of existing customers or transactions that would pass a rule can be
measured and this can be compared with the business’ expectations. Similarly rules
that intended to detect or flag something can be compared with historical results to
see if they actually do so. One tax authority, for instance, used this approach to
validate the rules their experts wrote for detecting fraudulent tax returns. Reports
and queries identified the tax returns that the rules identified and these were
assessed to see how many were actually fraudulent.
The end result of this step is the analytic confirmation of judgmental business rules.
Data Mining
The next step is to use mathematical techniques to find candidate business rules.
Many data mining and analytic techniques generate rules or decision trees for
segmentation and association. For instance data mining tools can be used to find
statistically similar clusters of customers or to find products that are often sold
together. While these techniques are mathematical in nature, and often require a
different tool that is more focused on an analytic user, the end result is very
explicable to someone familiar with business rules and with the use of decision trees
to represent business rules.
Because business users sometimes resist the use of analytics, being uncomfortable
with having their actions determined by an algorithm, this shared representation can
be very useful. For instance, the analytically derived business rules or decision tree
can be loaded into a business rules management system and then edited, allowing
business users to exert some control over the results. This will tend to degrade the
value of the analytics—the non-intuitive results of data mining are often the most
useful—but it might make the difference between implementation of some analytic
results and none. One organization for instance used data mining techniques to
identify parts that an engineer should take with them to an onsite visit because they
were likely to be required. By allowing business users to selectively implement these
decision trees in a business rules management system the organization overcame
opposition to the approach.
Some tools, including some business rules management systems, make this
combination of analytic and judgmental development more explicit. Analytic
techniques that support building decision trees, using mathematics to find the
correct branches in the tree, are combined with human judgment while editing a
decision tree. For instance the user might begin by using an algorithm to find the
segmentation approach that most clearly divides profitable and unprofitable
customers but then apply their own judgment or policy at the next level of the tree,
dividing customers by gender say.
The end result of this step is a set of new business rules that have been analytically
derived from your data.
Predictive Analytics
The final step is to use mathematical techniques to build analytic models that predict
risk, fraud or opportunity. These models typically involve expertise in analytics as
well as specialized tools for developing the models. The models don’t generally make
decisions but predictions that can be used in decision-making rules. For instance a
model might predict the likelihood that a particular transaction is fraudulent.
Business rules would then use this result or score along with the value of the
customer, location, amount etc to determine the best action to take, the right
decision to make.
Examples of these models include credit risk scorecards that use various factors
about a consumer to score their credit risk, neural networks that automatically
detect fraud by seeing how different a transaction or network is from regular
patterns, and regression models used to predict the propensity of a consumer to
accept a particular offer if it is made to them.
Building these models is generally a specialist task performed by people known as
data miners, data scientists or statisticians. They will use workbenches that allow
them to pull in large amounts of data; clean, integrate and analyze this data; find
predictive characteristics in this data; and develop and deploy a predictive analytic
model based on these characteristics. Some automated tools aimed at less technical
users exist, often taking advantage of machine learning techniques.
In general predictive analytics are mixed with business rules in Decision Management
Systems by making the result of the predictive analytic model, the score, available to
the business rules as an attribute. This might be stored in the database along with
other data, created dynamically in the database when required using an in-database
analytic engine, or calculated using code or business rules. The trend is towards
scoring, running the model, in real-time so that the predictive analytic result being
used in the decision is as up to date as possible but many scenarios only require that
the scores are available in the database as fields that the business rules management
system can access.
Becoming Adaptive
If agility is about being able to respond to known change, changes in policy or
regulation for instance, being adaptive is about responding to unknown changes. This
matters because decisions are difficult to manage over time as:
Decisions can take time to play out.
The outcome of a decision is not always immediately apparent so it may not be
clear what the best decision is when you have to make it.
Decisions are a moving target.
Thanks to constant change in the environment, “best” is not a static concept but
a dynamic one.
You can’t control many of the factors that affect a decision.
However, operational and repeatable tactical decisions are repeated over and over
again in your organization. This gives you the opportunity to manage them in the
long run by learning what might work better in the future by analyzing the decisions
you have already made. So experiment!
Why Experiment?
You experiment to continuously improve the way you make decisions.
Experimentation might allow you to see how to respond to changing business
conditions or it might help you make a decision better and better over time to
boost profits, reduce losses, or improve retention.
You constantly learn more about your customers and gather more information
about their behavior. New insights and market trends come from you, your
competitors and from third parties. A process for continual review and
improvement of how you take a decision allows you to detect and respond to
changes in the behavior of your customers without having to start a special project
and helps you show an ROI for the data you collect and analyze.
One of the challenges with decisions is that they take time to play out—the results
may not become obvious for some time. At the point you make an operation
decision you cannot know what the long-term outcome of that decision will be. So,
if you take a population and take a decision about how to treat them, you will only
see the results if you wait for some period of time—perhaps until they are next due
for a check-up or a renewal, perhaps sooner. It is important to track your results
and tie that back to your decisions to know what kind of impact you are looking for.
However this is not enough: by the time the results are clear it is way too late to
gather any new data about what might have worked better. If you use a single
decision-making approach (a single set of business rules and predictive analytic
models) for every decision you make then you will have no data about how other
actions might have resulted in better (or worse) results.
This is what makes experimentation so important. By experimenting you create data
about multiple decision-making approaches. These different approaches can then be
compared to see what the best approach is and this best approach can then be
applied to future decisions.
A/B Testing
A/B or A/B/C (or A/B…./X) testing compares multiple approaches where there is
no particular bias to one approach or another. Each approach is different in some
known ways from the others. These differences can be significant but it is more
common for them to be fairly modest. It is also common for the same kind of
change to be made in each approach so that they all vary along the same dimension
as it were.
A/B Testing is common in marketing where multiple offers or offer selection rules
are compared and in pricing where multiple pricing or discounting approaches are
considered. A/B Tests tend to work well when there is quick feedback on how well
something works (offers are accepted, products purchased immediately) and often
only run for a short period.
During the test transactions are generally allocated to one of the approaches
randomly with an even distribution between the approaches. Which approach was
used is recorded so that results can be compared across approaches. Once the test
is complete one of the approaches will be selected as the preferred approach,
because it generates better results, and that will then be used for the rest of the
campaign.
Sometimes one approach will work best for one subset of the transactions and
another approach for a different subset. In these circumstances an approach can be
developed that combines the two, with additional rules to determine which subset
the transaction is in.
Champion/Challenger Testing
Champion / Challenger testing is designed to handle a situation where you have a
preferred or established approach and wish to see if there is some alternative that
might be better. This is common in risk-based decisions where there is a significant
investment in the current approach and where it may be some time before the
results can be determined.
You take your main Figure 1:The Champion/Challenger Process
approach—the champion—and
apply it to most of your
population. You also create a
number of challengers or
alternatives and apply them to a
small percentage of the
population.
Each Challenger differs from
the Champion in some
measurable and defined way.
Perhaps it has different business
rules, perhaps it uses a different
risk model, perhaps it is more
aggressive about retaining
customers. Each Challenger will
therefore deliver different
results from the Champion.
These results may be better or
worse but only testing the
approaches with real
transactions, in a live environment, can really show.
This approach means that most customers or transactions are still handled using
your preferred or approved approach. This is important in decisions with a long
time to value as it ensures some stability for the majority of transactions. However,
as time passes, you are also collecting information about how these alternatives
work. You can extrapolate from their effectiveness of these challengers on these
subsets and determine if they work better or worse than the champion.
If one works better, you can make that the new champion and then apply it to all
subsequent decisions. As noted above you might find that a challenger is only better
in certain subsets and so merge the challenger into the champion. To ensure
ongoing improvement, you can create or promote a new champion and develop new
challengers to keep repeating the process. Ensuring that you always have challengers
running allows you to constantly verify your approach and continuously improve it.
There is a growing array of automated analytic technology that runs A/B and
Champion/Challenger testing “under the covers” and constantly updates predictive
analytic models based on the results of these experiments. These adaptive analytic
technologies are particularly powerful in marketing and other un-regulated
environments and are becoming more widespread and established.
Next Steps
Organizations have implemented decision management systems to improve fraud
detection, automate and streamline underwriting, improve marketing response
rates, increase customer loyalty and satisfaction measures, ensure GRC compliance,
improve process efficiency and more. More use cases can be found in free our
report on Decision Management Systems Platform Technologies.
As organizations move forward into the era of Big Data, effective systems will
become increasingly analytic. Analytic systems can no longer be kept separate from
operational systems, and analytic Decision Management Systems is a well proven
approach that enables you to close the loop by putting analytics to work improving
day to day decisions.
Increasing rates of change in business models, competitive landscape and more mean
that static applications, no matter how agile, will fall behind. Building adaptive
Decision Management Systems allows organizations to constantly experiment and
learn what works and what does not. Adaptive systems stay relevant and continue
to improve over time.
Works Cited
Taylor, J. (2012). Decision Management Systems Platform Technologies Report. Palo Alto
CA: Decision Management Solutions.
Taylor, J. (2012). Decision Management Systems: A Practical Guide to Using Business
Rules and Predictive Analytics. New York, NY: IBM Press.
Contact Us
If you have any questions about Decision Management Solutions or would like to discuss engaging us we
would love to hear from you. Email works best but feel free to use any of the methods below.
Email : info@decisionmanagementsolutions.com
Phone : +1 650 400-3029
Fax : +1 650 352-9247