Paul A. Jensen
All the models in this workbook are created with the DP Data add-in.
Modeling takes place in the DP Models add-in
Problems are solved with either the Markov Analysis add-in or the DP Solver add-in.
Use the Start command to add or replace buttons on a worksheet.
Use the Finish command to remove buttons from the worksheet.
Problems
The first three problems are about a birth-death process with purging. Each state has the same b
CTMC: Continous Time Markov Chain
DTMC: Discrete Time Markov Chain
MDP: Finds the optimum state for the purge operation
These problems have different birth and death rates at each state.
Diff: DTMC with different rates
Dif_MDP: Finds the optimum state for the purge operation
ramming
lver add-in.
CTMC Data
CTMC
Birth-Death
CTMC
Build Model
Maximum Population
Economic Measure
Time Measure
Cost per Birth
Cost per Death
Action
Fixed Purge Cost
Variable Purge Cost
Holding Cost Rate for Items in the System
Discount Rate
Birth Rate
Death Rate
Time Interval (Analysis Period)
Build/Modify Model
10
Cost
Hour
5
-10
Purge
100
-20
2
0.1
10
12
0.01
/unit purged
/Hour
/Hour
/Hour
/Hour
Hours
CTMC Model
CTMC
Birth-Death
1
0
2
0
0
3
Cost
Hour
10.000%
None
1
Change Model
Transfer to Markov A
List States
List Elements
State Vector
Index
Count
Feasible
Min
Max
Name
Cost
Final Cost
Population
State
10
10
11
11
1
1
1
1
0
10
P
(10)
P(10)
0
2
20
0
0
0
Definitions
s1: Number in Population
Transitions
State/Event Transition Equations, Rates and Costs
Population
State
10 P(10)
Transition Transition caused by purge
State Population
Feasible
1
1
1
Min
10
Max
10
Change State
-10
Population Index Feasible
Next State
0
1
1
Max
Change State
1
Population Index Feasible
Next State
11
###
0
1
Population Index Feasible
Next State
11
###
0
Event Vector
Summary
Name: P(10)/Birth
Birth
Event
Index
2
Count
4
Feasible
1
Min
Max
Name
Cost
0
Rates
0
1
2
2
1
0
1
Death
0
2
1
1
0
1
5
10
-10
12
Birth
Feasible:
Subset:
State
1
0
1
Transition
Feasible:
1
Subset:
1
Birth
5
10
Definitions
e1: Birth Event (1 if birth, 0 if not)
e2: Death Event (1 if death, 0 if not)
Transition Summary
Name: P(10)/Birth
Feasible:
1
Subset:
1
Feasible
Event
1
Min
Max
Birth
Death
0 Birth
Death
1
1
1
0
1
0
Transition 1 Summary
Feasible:
1
Cost:
-120
Rate:
10
Next State:
1
Exit Model:
0
Index:
1
Name
P(0)
ystem is empty
Feasible
Event
1
Min
Birth
1
0
Death
1
0
Transition 2 Summary
Feasible:
0
Cost:
5
Rate:
10
Max
Next State:
Exit Model:
Index:
###
0
9999
Name
Err:502
Name
Err:502
Event
1
Min
Max
Birth
Death
1
1
0
0
1
1
Transition 3 Summary
Feasible:
0
Cost:
5
Rate:
10
Next State:
###
Exit Model:
0
Index:
9999
CTMC
CTMC
Step
CalculateMeasure
Change
Hour
Analyze
State
Index Names
Economics
0
P(0)
1
P(1)
Transient
2
P(2)
3
P(3)
Steady State
4
P(4)
5
P(5)
n-step Probabilities
6
P(6)
7
P(7)
First Pass
8
P(8)
9
P(9)
Simulate
10
P(10)
Absorbing States
11
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
Sum
Matrix Analyzed.
11 Recurrent States
1 Recurrent State Class
0 Transient States
0
1
2
P(0)
P(1)
P(2)
0
10
0
12
0
10
0
12
0
0
0
12
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
10
0
0
22
22
22
3
P(3)
0
0
10
0
12
0
0
0
0
0
0
22
4
P(4)
0
0
0
10
0
12
0
0
0
0
0
22
5
P(5)
0
0
0
0
10
0
12
0
0
0
0
22
6
P(6)
0
0
0
0
0
10
0
12
0
0
0
22
7
P(7)
0
0
0
0
0
0
10
0
12
0
0
22
8
P(8)
0
0
0
0
0
0
0
10
0
12
0
22
9
P(9)
0
0
0
0
0
0
0
0
10
0
12
22
10
P(10)
0
0
0
0
0
0
0
0
0
10
0
10
Sum
10
22
22
22
22
22
22
22
22
22
22
Status
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
0
1
2
3
4
5
6
7
8
9
10
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
6
P(6)
7
P(7)
8
P(8)
9
P(9)
10
P(10)
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.454545
0
0
0
0
0 0.454545
0
0
0
0.545455
0 0.454545
0
0
0 0.545455
0 0.454545
0
0
0 0.545455
0 0.454545
0
0
0 0.545455
0
Economic Data
Type:
Title:
CTMC
CTMC
Calculate
Matrix
conomic Measure
Discount Rate
Cost
0.1
0
1
2
3
4
5
6
7
8
9
10
State
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
State
Cost
Rate
0
2
4
6
8
10
12
14
16
18
20
Combine
d Cost
Rate
50
-68
-66
-64
-62
-60
-58
-56
-54
-52
-1300
Combine
d Cost
per Step
5
-3.09091
-3
-2.90909
-2.81818
-2.72727
-2.63636
-2.54545
-2.45455
-2.36364
-59.0909
P(0)
0
-10
0
0
0
0
0
0
0
0
-120
P(1)
5
0
-10
0
0
0
0
0
0
0
0
P(2)
0
5
0
-10
0
0
0
0
0
0
0
P(3)
0
0
5
0
-10
0
0
0
0
0
0
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
0
0
0
5
0
-10
0
0
0
0
0
0
0
0
0
5
0
-10
0
0
0
0
0
0
0
0
0
5
0
-10
0
0
0
0
0
0
0
0
0
5
0
-10
0
0
0
0
0
0
0
0
0
5
0
-10
0
0
0
0
0
0
0
0
0
5
0
-10
10
P(10)
0
0
0
0
0
0
0
0
0
5
0
CTMC
CTMC
0
1
2
3
4
Inverse
P(0)
P(1)
P(2)
P(3)
P(4)
Steady State (Time) 0.2331 0.1882 0.1508 0.1196 0.0937
Transition Time
Matrix
5
P(5)
0.072
6
P(6)
0.054
7
8
P(7)
P(8)
0.0389 0.0264
9
10
P(9)
P(10)
0.016 0.0073
-510 -523.7
-540
Expected
Transition
Time
0.052077
Average
Cost
rate
-45.8737146
Count
0
1
2
3
4
5
6
7
8
9
10
Name
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
Populatio
n
0
1
2
3
4
5
6
7
8
9
10
Cost
0
2
4
6
8
10
12
14
16
18
20
Final Cost
0
0
0
0
0
0
0
0
0
0
0
Trans.
Range
1
3
5
7
9
11
13
15
17
19
21
Event List
Index
1
2
Count
1
2
Nam
e
Death
Birth
Birth Death
0
1
1
0
Cost
-10
5
Transitions
State
Index Index
1
1
2
2
3
2
4
3
5
3
6
4
7
4
8
5
9
5
10
6
11
6
12
7
13
7
14
8
15
8
16
9
17
9
18
10
19
10
20
11
21
11
Event
Index
Name
2
P(0)/Birth
1
P(1)/Death
2
P(1)/Birth
1
P(2)/Death
2
P(2)/Birth
1
P(3)/Death
2
P(3)/Birth
1
P(4)/Death
2
P(4)/Birth
1
P(5)/Death
2
P(5)/Birth
1
P(6)/Death
2
P(6)/Birth
1
P(7)/Death
2
P(7)/Birth
1
P(8)/Death
2
P(8)/Birth
1
P(9)/Death
2
P(9)/Birth
1
P(10)/Death
2
P(10)/Birth
Cost
5
-10
5
-10
5
-10
5
-10
5
-10
5
-10
5
-10
5
-10
5
-10
5
-10
-120
Next
Next
Rate State Name
10
2 P(1)
12
1 P(0)
10
3 P(2)
12
2 P(1)
10
4 P(3)
12
3 P(2)
10
5 P(4)
12
4 P(3)
10
6 P(5)
12
5 P(4)
10
7 P(6)
12
6 P(5)
10
8 P(7)
12
7 P(6)
10
9 P(8)
12
8 P(7)
10
10 P(9)
12
9 P(8)
10
11 P(10)
12
10 P(9)
10
1 P(0)
DTMC Data
DTMC
Birth-Death
DTMC
Build Model
Maximum Population
Economic Measure
Time Measure
Cost per Birth
Cost per Death
Action
Fixed Purge Cost
Variable Purge Cost
Holding Cost Rate for Items in the System
Discount Rate
Birth Rate
Death Rate
Time Interval (Analysis Period)
Holding Cost for Items in the System
Discount Rate
Birth Probability
Death Probability
Build/Modify Model
10
Cost
Hour
5
-10
Purge
100
-20
2
0.1
10
12
0.01
0.02
0.001
0.1
0.12
/unit purged
/Hour
/Hour
/Hour
/Hour
Hours
/Period
/Period
/Period
/Period
DTMC Model
DTMC
Birth-Death
1
0
2
0
0
3
Cost
Hour
0.100%
None
1
Change Model
List States
Transfer to DP Solver
List Elements
State Vector
Index
Count
Feasible
Min
Max
Name
Cost
Final Cost
Population
State
10
10
11
11
1
1
1
1
0
10
P
(10)
P(10)
0
0.02
0.2
0
0
0
Definitions
s1: Number in Population
Transitions
State/Event Transition Equations, Probabilities and Cost
Population
State
10 P(10)
Transition Transition caused by purge
State Population
Feasible
1
1
1
Min
10
Max
10
Change State
-10
Population Index Feasible
Next State
0
1
1
Max
Change State
1
Population Index Feasible
Next State
11
###
0
1
Population Index Feasible
Next State
11
###
0
Event Vector
Summary
Name: P(10)/Birth
Birth
Event
Index
2
Count
4
Feasible
1
Min
Max
Name
Cost
0
Probability
0
1
2
2
1
0
1
Death
0
2
1
1
0
1
5
0.1
-10
0.12
Birth
Feasible:
Subset:
State
1
0
1
Transition
Feasible:
1
Subset:
1
Birth
5
0.1
Definitions
e1: Birth Event (1 if birth, 0 if not)
e2: Death Event (1 if death, 0 if not)
Transition Summary
Name: P(10)/Birth
Feasible:
1
Subset:
1
Feasible
Event
1
Min
Max
Birth
Death
0 Birth
Death
1
1
1
0
1
0
Transition 1 Summary
Feasible:
1
Cost:
-120
Probability:
0.1
Next State:
1
Exit Model:
0
Index:
1
Name
P(0)
ystem is empty
Feasible
Event
1
Min
Birth
1
0
Death
1
0
Transition 2 Summary
Feasible:
0
Cost:
5
Probability:
0.1
Max
Next State:
Exit Model:
Index:
###
0
9999
Name
Err:502
Name
Err:502
Event
1
Min
Max
Birth
Death
1
1
0
0
1
1
Transition 3 Summary
Feasible:
0
Cost:
5
Probability:
0.1
Next State:
###
Exit Model:
0
Index:
9999
DP Solver
Type
Title
Model Name
Goal
States
Actions
Actions/State
Events
Events/Action
Iteration Type
DTMC
DTMC
DTMC_Model
None
11
0
0
2
3
Value
Policy Steps
600
Stop Dif.
0.00001
Value Error
252.085253
Prob. Error
1.0874E-007
Time Measure
Hour
Economic Measure
Cost
Discount Rate
0.100%
Discount Factor
99.9%
Step Interval
1
Solve
Change
Make LP Model
Equations
State List
Index
1
2
3
4
5
6
7
8
9
10
11
Name
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
Populati
on
0
1
2
3
4
5
6
7
8
9
10
Cost
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Final
Cost
0
0
0
0
0
0
0
0
0
0
0
Gain
Expected
Transitio Transitio
n Range n Value
2
0.5
2
-0.7
2
-0.7
2
-0.7
2
-0.7
2
-0.7
2
-0.7
2
-0.7
2
-0.7
2
-0.7
2
-13.2
Step
Value
0.5
-0.68
-0.66
-0.64
-0.62
-0.6
-0.58
-0.56
-0.54
-0.52
-13
Expected
Next
Value
-184.087
-192.275
-201.177
-209.785
-218.324
-227.069
-236.346
-246.556
-258.188
-271.841
-275.746
Discount
Value
-183.403
-192.762
-201.636
-210.215
-218.726
-227.442
-236.69
-246.87
-258.47
-272.09
-288.47
Exp. Value
0 -0.45872
Next
Value
-183.151
-192.51
-201.384
-209.963
-218.474
-227.19
-236.438
-246.618
-258.218
-271.838
-288.218
State
Prob.
0.233135
0.188236
0.15082
0.119641
0.093658
0.072005
0.053961
0.038925
0.026394
0.015953
0.007251
Last
Prob.
0.233135
0.188236
0.150821
0.119641
0.093658
0.072005
0.053961
0.038925
0.026394
0.015953
0.007251
Event List
Index
1
2
3
4
Name
Death
Birth
Null
NA
Transition List
Birth
Event
Cost
Death
0
1
0
0
1
0
0
0
0
0
0
-999999
Prob.
1
0
0
0
Index
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
Transition List
State
Index
1
1
2
2
2
3
3
3
4
4
4
5
5
5
6
6
6
7
7
7
8
8
8
9
9
9
10
10
10
11
11
11
Event
Index
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
Next
State
2
1
1
3
2
2
4
3
3
5
4
4
6
5
5
7
6
6
8
7
7
9
8
8
10
9
9
11
10
10
1
11
Next
Name
P(1)
P(0)
P(0)
P(2)
P(1)
P(1)
P(3)
P(2)
P(2)
P(4)
P(3)
P(3)
P(5)
P(4)
P(4)
P(6)
P(5)
P(5)
P(7)
P(6)
P(6)
P(8)
P(7)
P(7)
P(9)
P(8)
P(8)
P(10)
P(9)
P(9)
P(0)
P(10)
Next
Cost
-192.51
-183.151
-183.151
-201.384
-192.51
-192.51
-209.963
-201.384
-201.384
-218.474
-209.963
-209.963
-227.19
-218.474
-218.474
-236.438
-227.19
-227.19
-246.618
-236.438
-236.438
-258.218
-246.618
-246.618
-271.838
-258.218
-258.218
-288.218
-271.838
-271.838
-183.151
-288.218
Next
Prob.
0.023313
0.209821
0.022588
0.018824
0.146824
0.018098
0.015082
0.11764
0.014357
0.011964
0.09332
0.011239
0.009366
0.073053
0.008641
0.0072
0.056164
0.006475
0.005396
0.04209
0.004671
0.003892
0.030361
0.003167
0.002639
0.020588
0.001914
0.001595
0.012443
0.00087
0.000725
0.005656
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
-0.0999
0
0
0
0
0
0
0
0.220779
-0.0999
0
0
0
0
0
0
-0.11988 0.220779
-0.0999
0
0
0
0
0
0 -0.11988 0.220779
-0.0999
0
0
0
0
0
0 -0.11988 0.220779
-0.0999
0
0
0
0
0
0 -0.11988 0.220779
-0.0999
0
0
0
0
0
0 -0.11988 0.220779
-0.0999
0
0
0
0
0
0 -0.11988 0.220779
-0.0999
0
0
0
0
0
0 -0.11988 0.220779
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
1
1
1
1
1
1
1
1
1
1
1
0.119643
Reward
0.5
-0.68
-0.66
-0.64
-0.62
-0.6
-0.58
-0.56
-0.54
-0.52
-13
Exp.
Value
-0.1638
Index
1
2
3
4
5
6
7
8
9
10
11
Name
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
Populat
ion
0
1
2
3
4
5
6
7
8
9
10
C_L
C_U
Val. Error
Prob. Error
Final Step 20
Discoun State
t Value Prob.
0.65859 0.42238
-6.031 0.29237
-9.7866 0.16599
-11.468 0.07722
-12.007 0.02955
-12.217 0.00936
-13.036 0.00247
-16.189 0.00054
-24.938 0.0001
-43.98 2E-005
-77.288 2E-006
-1583
-160.71
1422.31
0.01961
Initial
Next
Value
0
0
0
0
0
0
0
0
0
0
0
Last
Prob.
1
0
0
0
0
0
0
0
0
0
0
Exp.
Value
0.5
Markov Chain Iterations
Step 1
Step 2
Discoun State
Discoun
t Value Prob.
t Value
0.5
0.9
0.88162
-0.68
0.1
-1.2159
-0.66
0
-1.3197
-0.64
0
-1.2798
-0.62
0
-1.2398
-0.6
0
-1.1998
-0.58
0
-1.1598
-0.56
0
-1.1198
-0.54
0
-1.0799
-0.52
0
-2.2886
-13
0
-23.142
-13000
500
13500
0.2
-10142
381.618
10523.9
0.156
Exp.
Value
0.382
Exp.
Value
0.29016
Exp.
Value
0.21728
State
Prob.
0.822
0.168
0.01
0
0
0
0
0
0
0
0
Step 3
Discoun State
t Value Prob.
1.1712 0.75996
-1.6536 0.21444
-1.962 0.0246
-1.9193
0.001
-1.8593
0
-1.7994
0
-1.7395
0
-1.6795
0
-1.7443
0
-4.7447
0
-31.219
0
Step 4
Discoun State
t Value Prob.
1.38783 0.7097
-2.0241 0.24621
-2.5788 0.04075
-2.5565 0.00324
-2.4787 0.0001
-2.3988
0
-2.3189
0
-2.2515
0
-2.5746
0
-7.5451
0
-37.778
0
-8077
289.581
8366.54
0.12408
-6559.2
216.634
6775.87
0.10053
Exp.
Value
0.15839
Exp.
Value
0.10999
Step 5
Discoun State
t Value Prob.
1.54559 0.66827
-2.3485 0.2679
-3.1675 0.0568
-3.1888 0.00661
-3.0976 0.0004
-2.998 0.00001
-2.8994
0
-2.8496
0
-3.5698
0
-10.482
0
-43.204
0
Step 6
Discoun
t Value
1.65503
-2.6411
-3.7283
-3.814
-3.7155
-3.5971
-3.4834
-3.4847
-4.7104
-13.432
-47.767
-5425.2
157.761
5582.92
0.08285
-4563.7
109.439
4673.15
0.06936
State
Prob.
0.63359
0.28261
0.07189
0.01089
0.00098
5E-005
1E-006
0
0
0
0
Exp.
Value
0.06958
Step 7
Discoun
t Value
1.72419
-2.9121
-4.2628
-4.43
-4.3317
-4.1963
-4.0737
-4.1635
-5.9701
-16.323
-51.666
-3898.8
69.1603
3967.99
0.05889
State
Prob.
0.60415
0.29242
0.08564
0.0158
0.00186
0.00014
6E-006
1E-007
0
0
0
Exp.
Value
0.03536
Step 8
Discoun
t Value
1.75931
-3.1683
-4.7733
-5.0357
-4.9457
-4.7961
-4.6733
-4.889
-7.3218
-19.116
-55.044
-3377.7
35.1148
3412.85
0.05065
State
Prob.
0.57882
0.29878
0.09794
0.02111
0.00304
0.00029
2E-005
6E-007
1E-008
0
0
Exp.
Value
0.00602
Step 9
Discoun
t Value
1.76528
-3.4148
-5.2623
-5.6303
-5.5566
-5.397
-5.2849
-5.6613
-8.7411
-21.792
-58.007
-2963.4
5.97417
2969.34
0.04406
State
Prob.
0.55679
0.30268
0.1088
0.02662
0.00452
0.00053
4E-005
2E-006
7E-008
1E-009
0
Exp.
Value
-0.0194
Step 10
Discoun
t Value
1.74603
-3.6549
-5.7324
-6.2132
-6.1639
-5.9995
-5.9107
-6.4782
-10.207
-24.344
-60.637
-2629.3
-19.249
2610.08
0.03872
State
Prob.
0.53744
0.30483
0.11833
0.03219
0.00625
0.00087
9E-005
6E-006
3E-007
8E-009
1E-010
Exp.
Value
-0.0417
Markov Chain Iterations
Step 11
Step 12
Discoun State
Discoun
t Value Prob.
t Value
1.70473 0.52027
1.64398
-3.8913 0.30571
-4.1258
-6.1856 0.12664
-6.6242
-6.7844 0.03769
-7.3441
-6.7672 0.0082
-7.3663
-6.6044 0.00132
-7.2121
-6.5521 0.00016
-7.2102
-7.3362 1E-005
-8.231
-11.702 8E-007
-13.212
-26.771 3E-008
-29.076
-62.993 9E-010
-65.125
-2426.6
-41.301
2385.27
0.03433
-2305.5
-60.752
2244.74
0.03075
Exp.
Value
-0.0614
State
Prob.
0.50493
0.30568
0.13387
0.04305
0.01032
0.00187
0.00026
3E-005
2E-006
1E-007
4E-009
Exp.
Value
-0.079
Step 13
Discoun
t Value
1.56594
-4.3596
-7.05
-7.8927
-7.9608
-7.8232
-7.8852
-9.1581
-14.727
-31.267
-67.068
-2190.5
-78.044
2112.42
0.02901
State
Prob.
0.49112
0.30499
0.14015
0.0482
0.01258
0.00252
0.00039
5E-005
4E-006
3E-007
1E-008
Exp.
Value
-0.0947
Step 14
Discoun
t Value
1.47241
-4.5937
-7.4646
-8.4306
-8.551
-8.438
-8.577
-10.113
-16.237
-33.349
-68.852
-2082.5
-93.525
1989.01
0.02736
State
Prob.
0.4786
0.30382
0.1456
0.05312
0.01494
0.00327
0.00056
8E-005
8E-006
6E-007
4E-008
Exp.
Value
-0.109
Step 15
Discoun
t Value
1.36494
-4.8287
-7.8695
-8.9584
-9.1367
-9.057
-9.2852
-11.09
-17.736
-35.331
-70.502
-1982.1
-107.47
1874.59
0.02582
State
Prob.
0.4672
0.30231
0.15033
0.05779
0.01735
0.00411
0.00077
0.00012
1E-005
1E-006
1E-007
Exp.
Value
-0.1219
Step 16
Discoun
t Value
1.24483
-5.0651
-8.2659
-9.4767
-9.7182
-9.6803
-10.009
-12.087
-19.219
-37.22
-72.036
-1889
-120.11
1768.87
0.02438
State
Prob.
0.45676
0.30056
0.15442
0.06219
0.01981
0.00503
0.00103
0.00017
2E-005
2E-006
2E-007
Exp.
Value
-0.1337
Step 17
Discoun
t Value
1.11322
-5.3034
-8.6549
-9.9862
-10.296
-10.308
-10.747
-13.098
-20.683
-39.023
-73.469
-1803
-131.61
1671.36
0.02305
State
Prob.
0.44715
0.29865
0.15797
0.06633
0.02227
0.00603
0.00133
0.00024
3E-005
4E-006
4E-007
Exp.
Value
-0.1446
Step 18
Discoun
t Value
0.97109
-5.5437
-9.0375
-10.488
-10.87
-10.94
-11.499
-14.121
-22.125
-40.747
-74.816
-1723.6
-142.13
1581.5
0.02182
State
Prob.
0.43827
0.29661
0.16104
0.07021
0.02473
0.00709
0.00167
0.00032
5E-005
7E-006
7E-007
Exp.
Value
-0.1546
Step 19
Discoun
t Value
0.81929
-5.7862
-9.4145
-10.981
-11.44
-11.577
-12.262
-15.152
-23.544
-42.397
-76.086
-1650.5
-151.8
1498.68
0.02067
State
Prob.
0.43004
0.29451
0.1637
0.07383
0.02716
0.0082
0.00205
0.00042
7E-005
1E-005
1E-006
Exp.
Value
-0.1638
Step 20
Discoun
t Value
0.65859
-6.031
-9.7866
-11.468
-12.007
-12.217
-13.036
-16.189
-24.938
-43.98
-77.288
-1583
-160.71
1422.31
0.01961
State
Prob.
0.42238
0.29237
0.16599
0.07722
0.02955
0.00936
0.00247
0.00054
0.0001
2E-005
2E-006
DTMC
DTMC
Step
CalculateMeasure
Change
Hour
Analyze
State
Index Names
Economics
0
P(0)
1
P(1)
Transient
2
P(2)
3
P(3)
Steady State
4
P(4)
5
P(5)
n-step Probabilities
6
P(6)
7
P(7)
First Pass
8
P(8)
9
P(9)
Simulate
10
P(10)
Absorbing States
11
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
Sum
Matrix Analyzed.
11 Recurrent States
1 Recurrent State Class
0 Transient States
0
1
2
3
4
P(0) P(1) P(2) P(3) P(4)
0.9
0.1
0
0
0
0.12 0.78
0.1
0
0
0 0.12 0.78
0.1
0
0
0 0.12 0.78
0.1
0
0
0 0.12 0.78
0
0
0
0 0.12
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.1
0
0
0
0
1.12
1
1
1
1
5
6
7
8
9
10
P(5) P(6) P(7) P(8) P(9) P(10) Sum
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
0.1
0
0
0
0
0
1
0.78
0.1
0
0
0
0
1
0.12 0.78
0.1
0
0
0
1
0 0.12 0.78
0.1
0
0
1
0
0 0.12 0.78
0.1
0
1
0
0
0 0.12 0.78
0.1
1
0
0
0
0 0.12 0.78
1
1
1
1
1
1 0.88
Status
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Economic Data
Type:
Title:
DTMC
DTMC
Calculate
Matrix
conomic Measure
Discount Rate
Cost
0.001
0
1
2
3
4
5
6
7
8
9
10
State
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
State
Cost
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0
Combine
d Cost
P(0) P(1) P(2)
0.5
0
5
0
-0.68
-10
0
5
-0.66
0
-10
0
-0.64
0
0
-10
-0.62
0
0
0
-0.6
0
0
0
-0.58
0
0
0
-0.56
0
0
0
-0.54
0
0
0
-0.52
0
0
0
-13 -120
0
0
10
DTMC
DTMC
Inverse
0
1
2
3
4
P(0)
P(1)
P(2)
P(3)
P(4)
Steady State Dist. 0.2331 0.1882 0.1508 0.1196 0.0937
5
P(5)
0.072
6
P(6)
0.054
7
8
P(7)
P(8)
0.0389 0.0264
Average
9
10
Cost
P(9)
P(10)
per period
0.016 0.0073 -0.45873619
Count
0
1
2
3
4
5
6
7
8
9
10
Name
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
Populatio
n
0
1
2
3
4
5
6
7
8
9
10
Cost
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Final Cost
0
0
0
0
0
0
0
0
0
0
0
Trans.
Range
2
5
8
11
14
17
20
23
26
29
32
Event List
Index
1
2
Count
1
2
Nam
e
Death
Birth
Birth Death
0
1
1
0
Cost
-10
5
Transitions
State
Index Index
1
1
2
1
3
2
4
2
5
2
6
3
7
3
8
3
9
4
10
4
11
4
12
5
13
5
14
5
15
6
16
6
17
6
18
7
19
7
20
7
21
8
22
8
23
8
24
9
25
9
26
9
27
10
28
10
29
10
30
11
31
11
32
11
Event
Index
Name
2
P(0)/Birth
3
P(0)/Null
1
P(1)/Death
2
P(1)/Birth
3
P(1)/Null
1
P(2)/Death
2
P(2)/Birth
3
P(2)/Null
1
P(3)/Death
2
P(3)/Birth
3
P(3)/Null
1
P(4)/Death
2
P(4)/Birth
3
P(4)/Null
1
P(5)/Death
2
P(5)/Birth
3
P(5)/Null
1
P(6)/Death
2
P(6)/Birth
3
P(6)/Null
1
P(7)/Death
2
P(7)/Birth
3
P(7)/Null
1
P(8)/Death
2
P(8)/Birth
3
P(8)/Null
1
P(9)/Death
2
P(9)/Birth
3
P(9)/Null
1
P(10)/Death
2
P(10)/Birth
3
P(10)/Null
Cost
5
0
-10
5
0
-10
5
0
-10
5
0
-10
5
0
-10
5
0
-10
5
0
-10
5
0
-10
5
0
-10
5
0
-10
-120
0
Next
Next
Prob. State Name
0.1
2 P(1)
0.9
1 P(0)
0.12
1 P(0)
0.1
3 P(2)
0.78
2 P(1)
0.12
2 P(1)
0.1
4 P(3)
0.78
3 P(2)
0.12
3 P(2)
0.1
5 P(4)
0.78
4 P(3)
0.12
4 P(3)
0.1
6 P(5)
0.78
5 P(4)
0.12
5 P(4)
0.1
7 P(6)
0.78
6 P(5)
0.12
6 P(5)
0.1
8 P(7)
0.78
7 P(6)
0.12
7 P(6)
0.1
9 P(8)
0.78
8 P(7)
0.12
8 P(7)
0.1
10 P(9)
0.78
9 P(8)
0.12
9 P(8)
0.1
11 P(10)
0.78
10 P(9)
0.12
10 P(9)
0.1
1 P(0)
0.78
11 P(10)
MDP Data
MDP
Birth-Death
MDP
Build Model
Maximum Population
Economic Measure
Goal for Optimization
Time Measure
Cost per Birth
Cost per Death
Action
Min. population to allow Purge
Max. population to allow Accept
Fixed Purge Cost
Variable Purge Cost
Holding Cost Rate for Items in the System
Discount Rate
Birth Rate
Death Rate
Time Interval (Analysis Period)
Holding Cost for Items in the System
Discount Rate
Birth Probability
Death Probability
Build/Modify Model
10
Cost
Min
Hour
5
-10
Purge
5
10
100
-20
2
0.1
10
12
0.01
0.02
0.001
0.1
0.12
/unit purged
/Hour
/Hour
/Hour
/Hour
Hours
/Period
/Period
/Period
/Period
MDP Model
MDP
Birth-Death
1
1
2
0
3
3
Cost
Hour
0.100%
Min
1
Change Model
Transfer to DP Solve
List States
List Elements
State Vector
Index
Count
Feasible
Min
Max
Name
Cost
Final Cost
Population
State
10
10
11
11
1
1
1
1
0
10
P
(10)
P(10)
0
0.02
0.2
0
0
0
Definitions
s1: Number in Population
Decisions
State/Action Interaction: Feasibility and Cost
Population
State
10 P(10)
Decision 1 States that must Accept arrivals
State Population
Feasible
1
0
0
Min
0
Max
4
Decision 2 States that allow arrivals and Purge
State Population
Feasible
1
0
0
Min
5
Max
9
Decision 3 States that require Purge
State Population
Feasible
1
Min
Max
1
10
10
Transitions
State/Action/Event Transition Equations, Probabilities
Population
State
10 P(10)
Transition Transition caused by purge
State Population
Feasible
1
1
1
Min
0
Max
10
Change State
-10
Population Index Feasible
Next State
0
1
1
1
Population Index Feasible
Next State
11
###
0
1
Population Index Feasible
Next State
11
###
0
Transfer to DP Solver
Action Vector
Index
Count
Feasible
Min
Max
Name
Cost
Event Vector
Hold/Purge
Action
1
1
2
2
1
1
1
1
0
1
Purge Purge
0
0
0
Definitions
a1: Purge if 1, Accept if 0
Hold/Purge
Action
1 Purge
ActionHold/Purge
1
0
Min
0
Max
0
ActionHold/Purge
1
1
Min
0
Max
1
ActionHold/Purge
1
2
2
1
0
1
Death
0
2
1
1
0
1
5
0.1
-10
0.12
Birth
Birth
5
0.1
Definitions
e1: Birth Event (1 if birth, 0 if not)
e2: Death Event (1 if death, 0 if not)
Feasible
Birth
Event
Index
2
Count
4
Feasible
1
Min
Max
Name
Cost
0
Probability
0
Feasible
1
Min
Max
1
1
1
Feasible
ActionHold/Purge
1
1
Min
1
Max
1
Birth
Event
Feasible
Feasible
Event
1
Min
Max
Birth
Event
1
Min
Max
Birth
Event
1
Min
Max
Birth
Death
0 Birth
Death
1
1
1
0
1
0
Name
P(0)
ystem is empty
Feasible
ActionHold/Purge
1
0
Min
0
Max
0
1
0
1
Death
1
0
0
Name
Err:502
Name
Err:502
ActionHold/Purge
1
1
Min
0
Max
1
Feasible
Death
1
1
0
0
1
1
Summary
Name: P(10)/Purge/Birth
Feasible:
Subset:
State
1
0
Decision
Feasible:
1
Subset:
3
Transition
Feasible:
1
Subset:
1
Decision Summary
Name: P(10)/Purge
Feasible:
1
Subset:
3
Decision 1 Summary
Feasible:
0
Cost:
0
Exit Model:
0
Index:
9999
Decision 2 Summary
Feasible:
0
Cost:
0
Exit Model:
0
Index:
9999
Decision 3 Summary
Feasible:
1
Cost:
Exit Model:
Index:
0
0
3
Transition Summary
Name: P(10)/Purge/Birth
Feasible:
1
Subset:
1
Transition 1 Summary
Feasible:
1
Cost:
-120
Probability:
0.1
Next State:
1
Exit Model:
0
Index:
1
Transition 2 Summary
Feasible:
0
Cost:
5
Probability:
0.1
Next State:
###
Exit Model:
0
Index:
9999
Transition 3 Summary
Feasible:
0
Cost:
5
Probability:
0.1
Next State:
###
Exit Model:
0
Index:
9999
DP Solver
Type
Title
Model Name
Goal
States
Actions
Actions/State
Events
Events/Action
Iteration Type
MDP
MDP
MDP_Model
Min
11
2
2
2
3
Policy
Policy Steps
Stop Dif.
0.00001
Value Error
5.684E-011
Prob. Error
3.542E-008
Time Measure
Hour
Economic Measure
Cost
Discount Rate
0.100%
Discount Factor
99.9%
Step Interval
1
Solve
Change
Make LP Model
Equations
State List
Index
1
2
3
4
5
6
7
8
9
10
11
Name
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
Populati
on
0
1
2
3
4
5
6
7
8
9
10
Cost
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Final
Cost
0
0
0
0
0
0
0
0
0
0
0
Gain
Decision Decision
Index
Range
1
1
1
1
1
1
1
1
1
1
2
1
2
1
2
1
2
1
3
1
1
1
Action
Index
1
1
1
1
1
1
1
1
1
1
1
Action
Name
Hold
Hold
Hold
Hold
Hold
Hold
Hold
Hold
Hold
Hold
Hold
Decision
Value
-435.508
-444.888
-453.783
-462.383
-470.916
-479.654
-488.926
-499.13
-510.755
-524.403
-540.791
Step
Value
0.5
0.52
0.54
0.56
0.58
-0.6
-0.58
-0.56
-0.54
-0.52
0.7
Discount
Value
-435.508
-444.868
-453.743
-462.323
-470.836
-479.554
-488.806
-498.99
-510.595
-524.223
-540.591
Exp. Value
0 0.325022
Next
State
Value
Prob.
-435.508 0.220921
-444.868 0.024547
-453.743
0
-462.323
0
-470.836
0
-479.554
0
-488.806
0
-498.99
0
-510.595
0
-524.223
0
-540.591
0
Exp. Value
Action List
Last
Prob.
0.245468
0.196676
0.156017
0.122134
0.093898
0.070368
0.05076
0.03442
0.020803
0.009456
1.6E-007
Index
1
2
3
4
Name
Hold
Purge
Null
NA
Hold/Pu
rge
0
1
0
0
Event List
Action
Cost
0
0
0
999999
Index
1
2
3
4
Name
Death
Birth
Null
NA
Birth
0
1
0
0
Decision List
Event
Cost
Death
1
0
0
0
0
0
0
999999
Prob.
1
0
0
0
Decision
Index
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
State
Index
1
2
3
4
5
6
6
7
7
8
8
9
9
10
10
11
Action
Index
1
1
1
1
1
1
2
1
2
1
2
1
2
1
2
2
Decision
Decision
Name
Cost
P(0)/Hold
0
P(1)/Hold
0
P(2)/Hold
0
P(3)/Hold
0
P(4)/Hold
0
P(5)/Hold
0
P(5)/Purge
0
P(6)/Hold
0
P(6)/Purge
0
P(7)/Hold
0
P(7)/Purge
0
P(8)/Hold
0
P(8)/Purge
0
P(9)/Hold
0
P(9)/Purge
0
P(10)/Purge
0
Transition List
Expected Expected
Transitio Transitio
Next
Decision
Last
n Range n Cost
Value
Prob.
Cost
2
0.5 -436.444 -435.508 0.245468
2
-0.7 -444.632 -444.888
0
2
-0.7 -453.536 -453.783
0
2
-0.7 -462.144 -462.383
0
2
-0.7 -470.686 -470.916
0
2
-0.7 -479.433 -479.654
0
2
-3.2 -474.103
-476.83
0
2
-0.7 -488.714 -488.926
0
2
-5.2 -482.366 -487.084
0
2
-0.7 -498.928
-499.13
0
2
-7.2
-491.42 -498.129
0
2
-0.7 -510.566 -510.755
0
2
-9.2 -501.694 -510.393
0
2
-0.7 -524.224 -524.401
0
2
-11.2 -513.716 -524.403
0
2
-13.2 -528.118 -540.791
0
Index
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Decision
Index
1
1
2
2
2
3
3
3
4
4
4
5
5
5
6
6
6
7
7
7
8
8
8
9
9
9
10
10
10
11
11
11
12
12
Event
Index
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
35
36
37
38
39
40
41
42
43
44
45
46
47
12
13
13
13
14
14
14
15
15
15
16
16
16
3
1
2
3
1
2
3
1
2
3
1
2
3
Transition Name
P(0)/Hold/Birth
P(0)/Hold/Null
P(1)/Hold/Death
P(1)/Hold/Birth
P(1)/Hold/Null
P(2)/Hold/Death
P(2)/Hold/Birth
P(2)/Hold/Null
P(3)/Hold/Death
P(3)/Hold/Birth
P(3)/Hold/Null
P(4)/Hold/Death
P(4)/Hold/Birth
P(4)/Hold/Null
P(5)/Hold/Death
P(5)/Hold/Birth
P(5)/Hold/Null
P(5)/Purge/Death
P(5)/Purge/Birth
P(5)/Purge/Null
P(6)/Hold/Death
P(6)/Hold/Birth
P(6)/Hold/Null
P(6)/Purge/Death
P(6)/Purge/Birth
P(6)/Purge/Null
P(7)/Hold/Death
P(7)/Hold/Birth
P(7)/Hold/Null
P(7)/Purge/Death
P(7)/Purge/Birth
P(7)/Purge/Null
P(8)/Hold/Death
P(8)/Hold/Birth
Transitio Transitio
n Cost
n Prob.
5
0.1
0
0.9
-10
0.12
5
0.1
0
0.78
-10
0.12
5
0.1
0
0.78
-10
0.12
5
0.1
0
0.78
-10
0.12
5
0.1
0
0.78
-10
0.12
5
0.1
0
0.78
-10
0.12
-20
0.1
0
0.78
-10
0.12
5
0.1
0
0.78
-10
0.12
-40
0.1
0
0.78
-10
0.12
5
0.1
0
0.78
-10
0.12
-60
0.1
0
0.78
-10
0.12
5
0.1
Next
State
2
1
1
3
2
2
4
3
3
5
4
4
6
5
5
7
6
5
1
6
6
8
7
6
1
7
7
9
8
7
1
8
8
10
Next
Name
P(1)
P(0)
P(0)
P(2)
P(1)
P(1)
P(3)
P(2)
P(2)
P(4)
P(3)
P(3)
P(5)
P(4)
P(4)
P(6)
P(5)
P(4)
P(0)
P(5)
P(5)
P(7)
P(6)
P(5)
P(0)
P(6)
P(6)
P(8)
P(7)
P(6)
P(0)
P(7)
P(7)
P(9)
Next
Next
Cost
Prob.
-444.868 0.024547
-435.508 0.220921
-435.508
0
-453.743
0
-444.868
0
-444.868
0
-462.323
0
-453.743
0
-453.743
0
-470.836
0
-462.323
0
-462.323
0
-479.554
0
-470.836
0
-470.836
0
-488.806
0
-479.554
0
-470.836
0
-435.508
0
-479.554
0
-479.554
0
-498.99
0
-488.806
0
-479.554
0
-435.508
0
-488.806
0
-488.806
0
-510.595
0
-498.99
0
-488.806
0
-435.508
0
-498.99
0
-498.99
0
-524.223
0
P(8)/Hold/Null
P(8)/Purge/Death
P(8)/Purge/Birth
P(8)/Purge/Null
P(9)/Hold/Death
P(9)/Hold/Birth
P(9)/Hold/Null
P(9)/Purge/Death
P(9)/Purge/Birth
P(9)/Purge/Null
P(10)/Purge/Death
P(10)/Purge/Birth
P(10)/Purge/Null
0
-10
-80
0
-10
5
0
-10
-100
0
-10
-120
0
0.78
0.12
0.1
0.78
0.12
0.1
0.78
0.12
0.1
0.78
0.12
0.1
0.78
9
8
1
9
9
11
10
9
1
10
10
1
11
P(8)
P(7)
P(0)
P(8)
P(8)
P(10)
P(9)
P(8)
P(0)
P(9)
P(9)
P(0)
P(10)
-510.595
-498.99
-435.508
-510.595
-510.595
-540.591
-524.223
-510.595
-435.508
-524.223
-524.223
-435.508
-540.591
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
-0.0999
0
0
0
0
0
0.220779
-0.0999
0
0
0
0
-0.11988 0.220779
-0.0999
0
0
0
0 -0.11988 0.220779
-0.0999
0
0
0
0 -0.11988 0.220779
-0.0999
0
0
0
0 -0.11988 0.220779
0
0
0
0
0 -0.11988 0.220779
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
0
0
0
0
0
0
0
0
0
0.1
-0.22
0.12
1
1
1
1
1
1
1
1
1
1
1
0.070368
0.05076
Reward
0.5
-0.68
-0.66
-0.64
-0.62
-0.6
-0.58
-0.56
-0.54
-11.02
-13
Gain
Index
1
2
3
4
5
6
7
8
9
10
11
Name
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
Populat
ion
0
1
2
3
4
5
6
7
8
9
10
Exp.
Value
0 -0.0195
Final Step 10
Discoun State
t Value Prob.
1.74359 0.53745
-3.6779 0.30483
-5.8999 0.11833
-7.1016 0.03219
-9.5681 0.00625
-15.468 0.00087
-26.22 8E-005
-38.722
0
-51.532
0
-64.379
0
-77.229
0
Initial
Next
Value
0
0
0
0
0
0
0
0
0
0
0
Last
Prob.
1
0
0
0
0
0
0
0
0
0
0
Exp.
Value
0
0.5
Value Iterations
Step 1
Discoun State
t Value Prob.
0.5
0.9
-0.68
0.1
-0.66
0
-0.64
0
-0.62
0
-3.1
0
-5.08
0
-7.06
0
-9.04
0
-11.02
0
-13
0
Gain
Exp.
Value
0.382
Step 2
Discoun
t Value
0.88162
-1.2159
-1.3197
-1.2798
-1.4895
-5.54
-9.3601
-13.12
-16.881
-20.641
-24.401
State
Prob.
0.822
0.168
0.01
0
0
0
0
0
0
0
0
Gain
Gain
Exp.
Value
0 0.29016
Step 3
Discoun State
t Value Prob.
1.1712 0.75996
-1.6536 0.21444
-1.962 0.0246
-1.9442
0.001
-2.4875
0
-7.5073
0
-12.95
0
-18.318
0
-23.678
0
-29.039
0
-34.4
0
Gain
Exp.
Value
0 0.21728
Step 4
Discoun State
t Value Prob.
1.38783 0.7097
-2.0241 0.24621
-2.5813 0.04075
-2.6387 0.00324
-3.5414 0.0001
-9.1311
0
-15.954
0
-22.769
0
-29.57
0
-36.37
0
-43.17
0
Gain
Exp.
Value
0 0.15839
Step 5
Discoun State
t Value Prob.
1.54559 0.66827
-2.3487 0.2679
-3.1776 0.0568
-3.3594 0.00661
-4.6081 0.0004
-10.501 0.00001
-18.467
0
-26.576
0
-34.672
0
-42.766
0
-50.86
0
Gain
Exp.
Value
0 0.10996
Step 6
Discoun
t Value
1.65501
-2.6423
-3.7532
-4.099
-5.6625
-11.681
-20.575
-29.828
-39.089
-48.346
-57.604
State
Prob.
0.63359
0.28261
0.07189
0.01089
0.00098
5E-005
0
0
0
0
0
Gain
Exp.
Value
0 0.06946
Step 7
Discoun
t Value
1.72405
-2.9155
-4.3109
-4.8496
-6.6906
-12.715
-22.347
-32.604
-42.909
-53.213
-63.516
State
Prob.
0.60415
0.29242
0.08564
0.0158
0.00186
0.00014
0
0
0
0
0
Gain
Exp.
Value
0 0.03503
Step 8
Discoun
t Value
1.75884
-3.1758
-4.8531
-5.6041
-7.6851
-13.638
-23.845
-34.972
-46.212
-57.456
-68.7
State
Prob.
0.57884
0.29878
0.09794
0.02111
0.00304
0.00029
0
0
0
0
0
Gain
Exp.
Value
0 0.00604
Step 9
Discoun
t Value
1.76411
-3.4286
-5.3822
-6.3564
-8.6426
-14.53
-25.12
-36.994
-49.066
-61.155
-73.245
State
Prob.
0.55681
0.30268
0.1088
0.02662
0.00452
0.00053
3E-005
0
0
0
0
Gain
Exp.
Value
0 -0.0195
Step 10
Discoun
t Value
1.74359
-3.6779
-5.8999
-7.1016
-9.5681
-15.468
-26.22
-38.722
-51.532
-64.379
-77.229
State
Prob.
0.53745
0.30483
0.11833
0.03219
0.00625
0.00087
8E-005
0
0
0
0
Index
1
2
3
4
5
6
7
8
9
10
11
Count
0
1
2
3
4
5
6
7
8
9
10
Name
P(0)
P(1)
P(2)
P(3)
P(4)
P(5)
P(6)
P(7)
P(8)
P(9)
P(10)
Populatio
n
0
1
2
3
4
5
6
7
8
9
10
Cost
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Final Cost
0
0
0
0
0
0
0
0
0
0
0
Action List
Index
1
2
Event List
Count
0
1
Nam Hold/Pur
e
ge
Hold
0
Purge
1
Cost
0
0
Nam
Index Count
e
1
1
Death
2
2
Birth
Birth
0
1
Decisions
Death
1
0
Cost
-10
5
State Action
Action Trans/
Index Index Index
Name
Cost Range
1
1
1
P(0)/Hold
0
2
2
2
1
P(1)/Hold
0
5
3
3
1
P(2)/Hold
0
8
4
4
1
P(3)/Hold
0
11
5
5
1
P(4)/Hold
0
14
6
6
1
P(5)/Hold
0
17
7
6
2
P(5)/Purge
0
20
8
7
1
P(6)/Hold
0
23
9
7
2
P(6)/Purge
0
26
10
8
1
P(7)/Hold
0
29
11
8
2
P(7)/Purge
0
32
12
9
1
P(8)/Hold
0
35
13
9
2
P(8)/Purge
0
38
14
10
1
P(9)/Hold
0
41
15
10
2
P(9)/Purge
0
44
16
11
2
P(10)/Purge
0
47
Transitions
Index
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
Transitions
Decisi
on
Event
Index Index
1
2
1
3
2
1
2
2
2
3
3
1
3
2
3
3
4
1
4
2
4
3
5
1
5
2
5
3
6
1
6
2
6
3
7
1
7
2
7
3
8
1
8
2
8
3
9
1
9
2
9
3
10
1
10
2
10
3
11
1
11
2
11
3
12
1
12
2
12
3
13
1
13
2
13
3
14
1
14
2
14
3
15
1
15
2
Name
P(0)/Hold/Birth
P(0)/Hold/Null
P(1)/Hold/Death
P(1)/Hold/Birth
P(1)/Hold/Null
P(2)/Hold/Death
P(2)/Hold/Birth
P(2)/Hold/Null
P(3)/Hold/Death
P(3)/Hold/Birth
P(3)/Hold/Null
P(4)/Hold/Death
P(4)/Hold/Birth
P(4)/Hold/Null
P(5)/Hold/Death
P(5)/Hold/Birth
P(5)/Hold/Null
P(5)/Purge/Death
P(5)/Purge/Birth
P(5)/Purge/Null
P(6)/Hold/Death
P(6)/Hold/Birth
P(6)/Hold/Null
P(6)/Purge/Death
P(6)/Purge/Birth
P(6)/Purge/Null
P(7)/Hold/Death
P(7)/Hold/Birth
P(7)/Hold/Null
P(7)/Purge/Death
P(7)/Purge/Birth
P(7)/Purge/Null
P(8)/Hold/Death
P(8)/Hold/Birth
P(8)/Hold/Null
P(8)/Purge/Death
P(8)/Purge/Birth
P(8)/Purge/Null
P(9)/Hold/Death
P(9)/Hold/Birth
P(9)/Hold/Null
P(9)/Purge/Death
P(9)/Purge/Birth
15
16
16
16
3
1
2
3
P(9)/Purge/Null
P(10)/Purge/Death
P(10)/Purge/Birth
P(10)/Purge/Null
0
-10
-120
0
0.78
0.12
0.1
0.78
10
10
1
11
P(9)
P(9)
P(0)
P(10)
DTMC Data
Diff
Birth-Death
DTMC
Build Model
Maximum Population
Economic Measure
Time Measure
Cost per Birth
Cost per Death
Action
Fixed Purge Cost
Variable Purge Cost
Holding Cost Rate for Items in the System
Discount Rate
Population
Birth Rate
Death Rate
Time Interval (Analysis Period)
Holding Cost for Items in the System
Discount Rate
Population
Arrival Probability
Departure Probability
/unit purged
/Hour
/Hour
1
2
9
8
12
11
Hours
/Period
/Period
1
2
0.09
0.08
0.12
0.11
10
4
3
10
9
9
12
1
3
10
7
4
2
6
12
2
21
3
0.04
0.03
4
0.1
0.09
5
0.09
0.12
6
0.01
0.03
7
0.1
0.07
8
0.04
0.02
9
0.06
0.12
10
0.02
0.21
DTMC Model
Diff
Birth-Death
1
0
2
0
0
3
Cost
Hour
0.100%
Change Model
List States
Transfer to DP Solver
List Elements
State Vector
Index
Step
Count
Feasible
Min
Max
Name
Cost
State
Population
11
0
1
11
11
1
1
1
1
0
10
P
_0
P_0
0
0.02
0
Definitions
s1: Number in Population
Transitions
State/Event Transition Equations, Probabilities and Cost
Population
State
0 P_0
Transition Transition caused by purge
State Population
Feasible
1
0
0
Min
10
Max
10
Change State
0
Population Index Feasible
Next State
0
1
1
Max
Change State
0
Population Index Feasible
Next State
0
1
1
0
Population Index Feasible
Next State
0
1
1
Event Vector
Index
Step
Count
Feasible
Min
Max
Name
Cost
Probability
Summary
0
1
4
0
Event
Birth
Death
0
0
2
2
2
1
1
1
0
0
1
1
Null
Name: P_0/Null
Feasible:
Subset:
0
Transition
Feasible:
0
Subset:
2
Null
0
0
5
0.05
State
1
0
-10
0
0
0
Definitions
e1: Birth Event (1 if birth, 0 if not)
e2: Death Event (1 if death, 0 if not)
Transition Summary
Name: P_0/Null
Feasible:
1
Subset:
2
Feasible
Event
1
Min
Max
Birth
Death
0 Null
Death
0
1
1
0
1
0
Transition 1 Summary
Feasible:
0
Cost:
80
Probability:
0
Next State:
1
Exit Model:
0
Index:
9999
Name
P_0
ystem is empty
Feasible
Event
1
Min
Birth
1
0
Death
1
0
Transition 2 Summary
Feasible:
1
Cost:
0
Probability:
0
Max
Next State:
Exit Model:
Index:
1
0
2
Name
P_0
Name
P_0
Event
1
Min
Max
Birth
Death
1
1
0
0
1
1
Transition 3 Summary
Feasible:
1
Cost:
0
Probability:
0
Next State:
1
Exit Model:
0
Index:
3
DTMC
Diff
Step
Calculate
Measure
Change
Hour
Analyze
State
Index Names
Economics
0
P_0
1
P_1
Transient
2
P_2
3
P_3
Steady State
4
P_4
5
P_5
n-step Probabilities
6
P_6
7
P_7
First Pass
8
P_8
9
P_9
Simulate
10
P_10
Absorbing States
11
P_0
P_1
P_2
P_3
P_4
P_5
P_6
P_7
P_8
P_9
P_10
Sum
Matrix Analyzed.
11 Recurrent States
1 Recurrent State Class
0 Transient States
0
1
2
P_0
P_1
P_2
0.95
0.05
0
0.12
0.79
0.09
0
0.11
0.81
0
0
0.03
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0.02
0
0
1.09
0.95
0.93
3
P_3
0
0
0.08
0.93
0.09
0
0
0
0
0
0
1.1
4
P_4
0
0
0
0.04
0.81
0.12
0
0
0
0
0
0.97
5
P_5
0
0
0
0
0.1
0.79
0.03
0
0
0
0
0.92
6
P_6
0
0
0
0
0
0.09
0.96
0.07
0
0
0
1.12
7
P_7
0
0
0
0
0
0
0.01
0.83
0.02
0
0
0.86
8
P_8
0
0
0
0
0
0
0
0.1
0.94
0.12
0
1.16
9
P_9
0
0
0
0
0
0
0
0
0.04
0.82
0.21
1.07
10
P_10
0
0
0
0
0
0
0
0
0
0.06
0.77
0.83
Sum
1
1
1
1
1
1
1
1
1
1
1
Status
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Class-1
Economic Data
Type:
Title:
DTMC
Diff
Calculate
Matrix
conomic Measure
Discount Rate
Cost
0.001
0
1
2
3
4
5
6
7
8
9
10
State
P_0
P_1
P_2
P_3
P_4
P_5
P_6
P_7
P_8
P_9
P_10
State
Cost
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Combine
d Cost
0.25
-0.73
-0.66
-0.04
-0.32
-0.65
-0.13
-0.06
0.16
-0.72
-4.3
P_0
0
-10
0
0
0
0
0
0
0
0
-120
P_3
0
0
5
0
-10
0
0
0
0
0
0
P_4
0
0
0
5
0
-10
0
0
0
0
0
P_5
0
0
0
0
5
0
-10
0
0
0
0
P_6
0
0
0
0
0
5
0
-10
0
0
0
P_7
0
0
0
0
0
0
5
0
-10
0
0
P_8
0
0
0
0
0
0
0
5
0
-10
0
10
P_9
0
0
0
0
0
0
0
0
5
0
-10
P_10
0
0
0
0
0
0
0
0
0
5
0
DTMC
Diff
Inverse
0
1
2
3
P_0
P_1
P_2
P_3
Steady State Dist. 0.2024 0.0829 0.0662 0.1709
4
5
P_4
P_5
0.074 0.0602
6
P_6
0.175
Average
7
8
9
10
Cost
P_7
P_8
P_9
P_10
per period
0.0225 0.1039 0.0332 0.0087 -0.19193373
-201.7 -205.9 -217.1 -231.2
Count
0
1
2
3
4
5
6
7
8
9
10
11
12
13
Name
P_0
P_1
P_2
P_3
P_4
P_5
P_6
P_7
P_8
P_9
P_10
P_11
P_12
Final
Populatio
n
0
1
2
3
4
5
6
7
8
9
10
11
12
13
Cost
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0.22
0.24
0
Trans.
Range
MDP Data
Dif_MDP
Birth-Death
MDP
Build Model
Maximum Population
Economic Measure
Goal for Optimization
Time Measure
Cost per Birth
Cost per Death
Action
Min. population to allow Purge
Max. population to allow Accept
Fixed Purge Cost
Variable Purge Cost
Holding Cost Rate for Items in the System
Discount Rate
Population
Birth Rate
Death Rate
Time Interval (Analysis Period)
Holding Cost for Items in the System
Discount Rate
Population
Arrival Probability
Departure Probability
Build/Modify Model
10
Cost
Min
Hour
5
-10
Purge
5
10
100
-20 /unit purged
2 /Hour
0.1 /Hour
0
1
5
9
0
12
0.01 Hours
0.02 /Period
0.001 /Period
0
1
0.05
0.09
0
0.12
8
11
4
3
2
0.08
0.11
3
0.04
0.03
10
9
4
0.1
0.09
9
12
1
3
5
0.09
0.12
6
0.01
0.03
10
10
7
4
2
6
12
2
21
0.1
0.07
8
0.04
0.02
9
0.06
0.12
10
0.02
0.21
MDP Model
Dif_MDP
Birth-Death
1
1
2
0
3
3
Cost
Hour
0.100%
Min
Change Model
Transfer to DP Solve
List States
List Elements
State Vector
Index
Step
Count
Feasible
Min
Max
Name
Cost
State
Population
10
10
1
11
11
1
1
1
1
0
10
P
_10
P_10
0
0.02
0.2
Definitions
s1: Number in Population
Decisions
State/Action Interaction: Feasibility and Cost
Population
State
10 P_10
Decision 1 States that must Accept arrivals
State Population
Feasible
1
0
0
Min
0
Max
4
Decision 2 States that allow arrivals and Purge
State Population
Feasible
1
0
0
Min
5
Max
9
Decision 3 States that require Purge
State Population
Feasible
1
Min
Max
1
10
10
Transitions
State/Action/Event Transition Equations, Probabilities
Population
State
10 P_10
Transition Transition caused by purge
State Population
Feasible
1
1
1
Min
0
Max
10
Change State
-10
Population Index Feasible
Next State
0
1
1
1
Population Index Feasible
Next State
11
###
0
1
Population Index Feasible
Next State
11
###
0
Transfer to DP Solver
Action Vector
Index
Step
Count
Feasible
Min
Max
Name
Cost
Event Vector
Action
Hold/Purge
2
2
1
3
3
1
1
1
1
0
2
Hold
Hold
0
0
0
Definitions
a1: Purge if 1, Accept if 0
Hold/Purge
Action
2 Hold
ActionHold/Purge
1
0
Min
0
Max
0
ActionHold/Purge
1
1
Min
0
Max
2
ActionHold/Purge
2
1
4
1
0
0
Birth
5
0.02
Definitions
e1: Birth Event (1 if birth, 0 if not)
e2: Death Event (1 if death, 0 if not)
Feasible
Index
Step
Count
Feasible
Min
Max
Name
Cost
Probability
Event
Birth
Death
1
0
2
2
2
1
1
1
0
0
1
1
Birth
5
-10
0.02
0.21
Feasible
1
Min
Max
0
1
1
Feasible
ActionHold/Purge
1
1
Min
1
Max
2
Birth
Event
Feasible
Feasible
Event
1
Min
Max
Birth
Event
1
Min
Max
Birth
Event
1
Min
Max
Birth
Death
0 Birth
Death
1
1
1
0
1
0
Name
P_0
ystem is empty
Feasible
ActionHold/Purge
1
0
Min
0
Max
0
1
0
1
Death
1
0
0
Name
Err:502
Name
Err:502
ActionHold/Purge
1
1
Min
0
Max
2
Feasible
Death
1
1
0
0
1
1
Summary
Name: P_10/Hold/Birth
Feasible:
Subset:
State
1
0
Decision
Feasible:
0
Subset:
9999
Transition
Feasible:
1
Subset:
1
Decision Summary
Name: P_10/Hold
Feasible:
0
Subset:
9999
Decision 1 Summary
Feasible:
0
Cost:
0
Exit Model:
0
Index:
9999
Decision 2 Summary
Feasible:
0
Cost:
0
Exit Model:
0
Index:
9999
Decision 3 Summary
Feasible:
0
Cost:
Exit Model:
Index:
0
0
9999
Transition Summary
Name: P_10/Hold/Birth
Feasible:
1
Subset:
1
Transition 1 Summary
Feasible:
1
Cost:
-120
Probability:
0.02
Next State:
1
Exit Model:
0
Index:
1
Transition 2 Summary
Feasible:
0
Cost:
5
Probability:
0.02
Next State:
###
Exit Model:
0
Index:
9999
Transition 3 Summary
Feasible:
0
Cost:
5
Probability:
0.02
Next State:
###
Exit Model:
0
Index:
9999
DP Solver
Type
Title
Model Name
Goal
States
Actions
Actions/State
Events
Events/Action
Iteration Type
MDP
Dif_MDP
Dif_MDP_Model
Min
11
3
3
2
3
Policy
Policy Steps
Solve
Change DP
Make LP Model
Equations
State List
Index
1
2
3
4
5
6
7
8
9
10
11
Name
P_0
P_1
P_2
P_3
P_4
P_5
P_6
P_7
P_8
P_9
P_10
Populati
on
0
1
2
3
4
5
6
7
8
9
10
Cost
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
kov Chain
Gain
Decision Decision
Index
Range
1
1
1
1
1
1
1
1
1
1
2
1
2
1
2
1
3
1
3
1
1
1
Action
Index
1
1
1
1
1
1
1
1
1
1
1
Action
Name
Hold
Hold
Hold
Hold
Hold
Hold
Hold
Hold
Hold
Hold
Hold
Decision
Value
-203.978
-213.083
-219.464
-222.715
-229.723
-235.126
-237.705
-256.146
-271.021
-286.197
-296.509
Step
Value
0.25
0.27
0.29
0.31
0.33
-0.65
-0.63
-0.61
-0.54
-0.52
0.45
State
Value
-203.978
-213.063
-219.424
-222.655
-229.643
-235.026
-237.585
-256.006
-270.861
-286.017
-296.309
Exp. Value
0 0.058377
Next
State
Value
Prob.
-203.978 0.267283
-213.063 0.014068
-219.424
0
-222.655
0
-229.643
0
-235.026
0
-237.585
0
-256.006
0
-270.861
0
-286.017
0
-296.309
0
Exp. Value
Action List
Last
Prob.
0.28135
0.110919
0.083868
0.198405
0.079766
0.060161
0.155241
0.011359
0.018932
-5E-008
-4E-008
Index
1
2
3
4
5
Name
Hold
Purge
Hold
Null
NA
Hold/Pu
rge
0
1
2
0
0
Event List
Action
Cost
0
0
0
0
999999
Index
1
2
3
4
Name
Death
Birth
Null
NA
Birth
0
1
0
0
Decision List
Event
Cost
Death
1
0
0
0
0
0
0
999999
Prob.
1
0
0
0
Decision
Index
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
State
Index
1
2
3
4
5
6
6
6
7
7
7
8
8
8
9
9
9
10
10
10
11
Action
Index
1
1
1
1
1
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
2
Decision Decision
Name
Cost
P_0/Hold
0
P_1/Hold
0
P_2/Hold
0
P_3/Hold
0
P_4/Hold
0
P_5/Hold
0
P_5/Purge
0
P_5/Hold
0
P_6/Hold
0
P_6/Purge
0
P_6/Hold
0
P_7/Hold
0
P_7/Purge
0
P_7/Hold
0
P_8/Hold
0
P_8/Purge
0
P_8/Hold
0
P_9/Hold
0
P_9/Purge
0
P_9/Hold
0
P_10/Purge
0
Transition List
Expected Expected
Transitio Transitio
Next
Decision
n Range n Cost
Value
Cost
2
0.25 -204.433 -203.978
2
-0.75 -212.545 -213.083
2
-0.7 -218.983 -219.464
2
-0.1 -222.838 -222.715
2
-0.4 -229.553 -229.723
2
-0.75 -234.611 -235.126
2
-3 -231.586 -234.355
2
-3 -231.586 -234.355
2
-0.25 -237.692 -237.705
2
-0.7 -237.172 -237.635
2
-0.7 -237.172 -237.635
2
-0.2 -256.202 -256.146
2
-6.7 -249.514 -255.965
2
-6.7 -249.514 -255.965
2
0
-271.17 -270.899
2
-3.4 -267.888 -271.021
2
-3.4 -267.888 -271.021
2
-0.9 -284.815 -285.431
2
-7.2 -279.276 -286.197
2
-7.2 -279.276 -286.197
2
-4.5 -292.301 -296.509
Last
Prob.
0.28135
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Index
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Decision
Index
1
1
2
2
2
3
3
3
4
4
4
5
5
5
6
6
6
7
7
7
8
8
8
9
9
9
10
10
10
11
11
11
12
12
Event
Index
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
12
13
13
13
14
14
14
15
15
15
16
16
16
17
17
17
18
18
18
19
19
19
20
20
20
21
21
21
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
Transition Name
P_0/Hold/Birth
P_0/Hold/Null
P_1/Hold/Death
P_1/Hold/Birth
P_1/Hold/Null
P_2/Hold/Death
P_2/Hold/Birth
P_2/Hold/Null
P_3/Hold/Death
P_3/Hold/Birth
P_3/Hold/Null
P_4/Hold/Death
P_4/Hold/Birth
P_4/Hold/Null
P_5/Hold/Death
P_5/Hold/Birth
P_5/Hold/Null
P_5/Purge/Death
P_5/Purge/Birth
P_5/Purge/Null
P_5/Hold/Death
P_5/Hold/Birth
P_5/Hold/Null
P_6/Hold/Death
P_6/Hold/Birth
P_6/Hold/Null
P_6/Purge/Death
P_6/Purge/Birth
P_6/Purge/Null
P_6/Hold/Death
P_6/Hold/Birth
P_6/Hold/Null
P_7/Hold/Death
P_7/Hold/Birth
Transitio Transitio
n Cost
n Prob.
5
0.05
0
0.95
-10
0.12
5
0.09
0
0.79
-10
0.11
5
0.08
0
0.81
-10
0.03
5
0.04
0
0.93
-10
0.09
5
0.1
0
0.81
-10
0.12
5
0.09
0
0.79
-10
0.12
-20
0.09
0
0.79
-10
0.12
-20
0.09
0
0.79
-10
0.03
5
0.01
0
0.96
-10
0.03
-40
0.01
0
0.96
-10
0.03
-40
0.01
0
0.96
-10
0.07
5
0.1
Next
State
2
1
1
3
2
2
4
3
3
5
4
4
6
5
5
7
6
5
1
6
5
1
6
6
8
7
6
1
7
6
1
7
7
9
Next
Name
P_1
P_0
P_0
P_2
P_1
P_1
P_3
P_2
P_2
P_4
P_3
P_3
P_5
P_4
P_4
P_6
P_5
P_4
P_0
P_5
P_4
P_0
P_5
P_5
P_7
P_6
P_5
P_0
P_6
P_5
P_0
P_6
P_6
P_8
Next
Next
Cost
Prob.
-213.063 0.014068
-203.978 0.267283
-203.978
0
-219.424
0
-213.063
0
-213.063
0
-222.655
0
-219.424
0
-219.424
0
-229.643
0
-222.655
0
-222.655
0
-235.026
0
-229.643
0
-229.643
0
-237.585
0
-235.026
0
-229.643
0
-203.978
0
-235.026
0
-229.643
0
-203.978
0
-235.026
0
-235.026
0
-256.006
0
-237.585
0
-235.026
0
-203.978
0
-237.585
0
-235.026
0
-203.978
0
-237.585
0
-237.585
0
-270.861
0
P_7/Hold/Null
P_7/Purge/Death
P_7/Purge/Birth
P_7/Purge/Null
P_7/Hold/Death
P_7/Hold/Birth
P_7/Hold/Null
P_8/Hold/Death
P_8/Hold/Birth
P_8/Hold/Null
P_8/Purge/Death
P_8/Purge/Birth
P_8/Purge/Null
P_8/Hold/Death
P_8/Hold/Birth
P_8/Hold/Null
P_9/Hold/Death
P_9/Hold/Birth
P_9/Hold/Null
P_9/Purge/Death
P_9/Purge/Birth
P_9/Purge/Null
P_9/Hold/Death
P_9/Hold/Birth
P_9/Hold/Null
P_10/Purge/Death
P_10/Purge/Birth
P_10/Purge/Null
0
-10
-60
0
-10
-60
0
-10
5
0
-10
-80
0
-10
-80
0
-10
5
0
-10
-100
0
-10
-100
0
-10
-120
0
0.83
0.07
0.1
0.83
0.07
0.1
0.83
0.02
0.04
0.94
0.02
0.04
0.94
0.02
0.04
0.94
0.12
0.06
0.82
0.12
0.06
0.82
0.12
0.06
0.82
0.21
0.02
0.77
8
7
1
8
7
1
8
8
10
9
8
1
9
8
1
9
9
11
10
9
1
10
9
1
10
10
1
11
P_7
P_6
P_0
P_7
P_6
P_0
P_7
P_7
P_9
P_8
P_7
P_0
P_8
P_7
P_0
P_8
P_8
P_10
P_9
P_8
P_0
P_9
P_8
P_0
P_9
P_9
P_0
P_10
-256.006
-237.585
-203.978
-256.006
-237.585
-203.978
-256.006
-256.006
-286.017
-270.861
-256.006
-203.978
-270.861
-256.006
-203.978
-270.861
-270.861
-296.309
-286.017
-270.861
-203.978
-286.017
-270.861
-203.978
-286.017
-286.017
-203.978
-296.309
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
Solution
-203.978
-213.063
-219.424
-222.655
-229.643
-235.026
-237.585
-256.006
-270.861
-286.017
-296.309
0
0
0.08
-0.07
0.09
0
0
0
0
0
0
0
0
0
0.04
-0.19
0.12
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
-0.0999
0
0
0
0
0
0.210789 -0.08991
0
0
0
0
-0.02997 0.040959 -0.00999
0
0
0
0 -0.06993 0.170829
-0.0999
0
0
0
0 -0.01998 0.060939
0
0
0
0
0 -0.11988 0.180819
0
0
0
0
0 -0.20979 0.230769
0
0
0
0
0.1
-0.21
0.03
0
0
0
0
0
0
0
0
0
0.09
-0.04
0.07
0
0
0
0
0
0
0
0
0
0.01
-0.17
0.02
0
0
0
0
0
0
0
0
0
0.1
-0.06
0.12
0
0
0
0
0
0
0
0
0
0
-0.18
0.21
1
1
1
1
1
1
1
1
1
1
1
-5E-008
-4E-008
Reward
0.25
-0.73
-0.66
-0.04
-0.32
-0.65
-0.13
-0.06
-3.24
-7.02
-4.3
Step
Calculate Measure
Change
Hour
Analyze
State
Index
Names
Economics
0
P_0
1
P_1
Transient
2
P_2
3
P_3
Steady State
4
P_4
5
P_5
n-step Probabilities
6
P_6
7
P_7
First Pass
8
P_8
9
P_9
Simulate
10
P_10
Absorbing States
11
P_0
P_1
P_2
P_3
P_4
P_5
P_6
P_7
P_8
P_9
P_10
Sum
Matrix Analyzed.
9 Recurrent States
1 Recurrent State Class
2 Transient States
0
1
2
P_0
P_1
P_2
0.95
0.05
0
0.12
0.79
0.09
0
0.11
0.81
0
0
0.03
0
0
0
0
0
0
0
0
0
0
0
0
0.04
0
0
0.06
0
0
0.02
0
0
1.19
0.95
0.93
3
P_3
0
0
0.08
0.93
0.09
0
0
0
0
0
0
1.1
4
P_4
0
0
0
0.04
0.81
0.12
0
0
0
0
0
0.97
5
P_5
0
0
0
0
0.1
0.79
0.03
0
0
0
0
0.92
6
P_6
0
0
0
0
0
0.09
0.96
0.07
0
0
0
1.12
7
P_7
0
0
0
0
0
0
0.01
0.83
0.02
0
0
0.86
8
P_8
0
0
0
0
0
0
0
0.1
0.94
0.12
0
1.16
9
P_9
0
0
0
0
0
0
0
0
0
0.82
0.21
1.03
10
P_10
0
0
0
0
0
0
0
0
0
0
0.77
0.77
Sum
Status
1 Class-1
1 Class-1
1 Class-1
1 Class-1
1 Class-1
1 Class-1
1 Class-1
1 Class-1
1 Class-1
1 Transient
1 Transient
Economic Data
Type:
DTMC
Title: Dif_MDP
Calculate
Matrix
conomic Measure
Discount Rate
Cost
0.10%
0
1
2
3
4
5
6
7
8
9
10
State
P_0
P_1
P_2
P_3
P_4
P_5
P_6
P_7
P_8
P_9
P_10
State
Cost
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Combine
d Cost
0.25
-0.73
-0.66
-0.04
-0.32
-0.65
-0.13
-0.06
-3.24
-7.02
-4.3
P_0
0
-10
0
0
0
0
0
0
-80
-100
-120
P_3
0
0
5
0
-10
0
0
0
0
0
0
P_4
0
0
0
5
0
-10
0
0
0
0
0
P_5
0
0
0
0
5
0
-10
0
0
0
0
P_6
0
0
0
0
0
5
0
-10
0
0
0
P_7
0
0
0
0
0
0
5
0
-10
0
0
P_8
0
0
0
0
0
0
0
5
0
-10
0
10
P_9
0
0
0
0
0
0
0
0
0
0
-10
P_10
0
0
0
0
0
0
0
0
0
0
0
-235 -237.6
7
8
P_7
P_8
0.0114 0.0189
-256 -270.9
Average
9
10
Cost
P_9
P_10
per period
0 -7E-019 -0.22075359
-286 -296.3
Index
1
2
3
4
5
6
7
8
9
10
11
Action List
Count
0
1
2
3
4
5
6
7
8
9
10
Name
P_0
P_1
P_2
P_3
P_4
P_5
P_6
P_7
P_8
P_9
P_10
Populatio
n
0
1
2
3
4
5
6
7
8
9
10
Cost
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Index
1
2
3
Action List
Event List
Count
0
1
2
Nam Hold/Pur
e
ge
Hold
0
Purge
1
Hold
2
Cost
0
0
0
Nam
Index Count
e
1
1
Death
2
2
Birth
Birth Death
0
1
1
0
Decisions
Cost
-10
5
State Action
Action Trans/
Index Index Index
Name
Cost Range
1
1
1
P_0/Hold
0
2
2
2
1
P_1/Hold
0
5
3
3
1
P_2/Hold
0
8
4
4
1
P_3/Hold
0
11
5
5
1
P_4/Hold
0
14
6
6
1
P_5/Hold
0
17
7
6
2
P_5/Purge
0
20
8
6
3
P_5/Hold
0
23
9
7
1
P_6/Hold
0
26
10
7
2
P_6/Purge
0
29
11
7
3
P_6/Hold
0
32
12
8
1
P_7/Hold
0
35
13
8
2
P_7/Purge
0
38
14
8
3
P_7/Hold
0
41
15
9
1
P_8/Hold
0
44
16
9
2
P_8/Purge
0
47
17
9
3
P_8/Hold
0
50
18
10
1
P_9/Hold
0
53
19
10
2
P_9/Purge
0
56
20
10
3
P_9/Hold
0
59
21
11
2
P_10/Purge
0
62
Transitions
Decisi
on
Index Index
1
1
2
1
3
2
4
2
5
2
6
3
7
3
8
3
9
4
10
4
11
4
12
5
13
5
14
5
15
6
16
6
17
6
18
7
19
7
20
7
21
8
22
8
23
8
24
9
25
9
26
9
27
10
28
10
29
10
30
11
31
11
32
11
33
12
34
12
35
12
36
13
37
13
38
13
39
14
40
14
41
14
42
15
43
15
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
15
16
16
16
17
17
17
18
18
18
19
19
19
20
20
20
21
21
21
Event
Index
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
Name
P_0/Hold/Birth
P_0/Hold/Null
P_1/Hold/Death
P_1/Hold/Birth
P_1/Hold/Null
P_2/Hold/Death
P_2/Hold/Birth
P_2/Hold/Null
P_3/Hold/Death
P_3/Hold/Birth
P_3/Hold/Null
P_4/Hold/Death
P_4/Hold/Birth
P_4/Hold/Null
P_5/Hold/Death
P_5/Hold/Birth
P_5/Hold/Null
P_5/Purge/Death
P_5/Purge/Birth
P_5/Purge/Null
P_5/Hold/Death
P_5/Hold/Birth
P_5/Hold/Null
P_6/Hold/Death
P_6/Hold/Birth
P_6/Hold/Null
P_6/Purge/Death
P_6/Purge/Birth
P_6/Purge/Null
P_6/Hold/Death
P_6/Hold/Birth
P_6/Hold/Null
P_7/Hold/Death
P_7/Hold/Birth
P_7/Hold/Null
P_7/Purge/Death
P_7/Purge/Birth
P_7/Purge/Null
P_7/Hold/Death
P_7/Hold/Birth
P_7/Hold/Null
P_8/Hold/Death
P_8/Hold/Birth
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
P_8/Hold/Null
P_8/Purge/Death
P_8/Purge/Birth
P_8/Purge/Null
P_8/Hold/Death
P_8/Hold/Birth
P_8/Hold/Null
P_9/Hold/Death
P_9/Hold/Birth
P_9/Hold/Null
P_9/Purge/Death
P_9/Purge/Birth
P_9/Purge/Null
P_9/Hold/Death
P_9/Hold/Birth
P_9/Hold/Null
P_10/Purge/Death
P_10/Purge/Birth
P_10/Purge/Null
0
-10
-80
0
-10
-80
0
-10
5
0
-10
-100
0
-10
-100
0
-10
-120
0
0.94
0.02
0.04
0.94
0.02
0.04
0.94
0.12
0.06
0.82
0.12
0.06
0.82
0.12
0.06
0.82
0.21
0.02
0.77
9
8
1
9
8
1
9
9
11
10
9
1
10
9
1
10
10
1
11
P_8
P_7
P_0
P_8
P_7
P_0
P_8
P_8
P_10
P_9
P_8
P_0
P_9
P_8
P_0
P_9
P_9
P_0
P_10