Bates & others v Post Office

Supplemental Report of Dr Robert Worden

01 February 2019

Table of Contents

1. Introduction and summary 2

1.1 Context 2
1.2 Document structure and summary 3

2. Response to Claimants’ evidence 5

2.1 Subpostmaster witness statements 5
2.2 Mr Roll's second witness statement 12
2.3 Other factual evidence from the Claimants 24

3. Further Analysis of KELs 25

4. Review of documents referenced by Mr Coyne 27

5. New data and analysis 29

5.1 Size of Claimants' branches 29
5.2 Maximum number of bugs which might have financial impact 31
5.3 Impact of bugs in Horizon on Claimants' branches 33

6. Other new evidence 36

6.1 Management of robustness 36
6.2 The Dalmellington incident 38
6.3 Permission controls - Horizon Issue 11 44

7. Horizon Issue 15 - Transaction Corrections (TCs) 46

Bates & others v Post Office Limited Supplemental Report Page 1 of 47

D3/6/1
1 Introduction and summary

1. INTRODUCTION AND SUMMARY

1.1 Context
1. This supplemental report is issued approximately three months after my first report

for the Horizon trial in the case of Bates and others v Post Office.

2. In this time, I have received no further reports from Mr Coyne, and the new

evidence that has become available to me is:

2.1. responses to further requests for information made by Mr Coyne and me;

2.2. the Second Witness Statement of Mr Roll1; and

2.3. the Second Witness Statement of Mr Parker2.

3. In the course of preparing this report, I have examined several documents which had

been previously disclosed, but which I had not examined before issuing my first

report. The effect of this has been (with a few minor exceptions, noted in this

report) to reinforce the opinions expressed in my first report, and to give me better

grounds for holding those opinions.

4. This report contains the results of further work on certain issues which, as I

indicated in my first report3, I was not able to complete in the time available for that

report. This has resulted in no significant changes to my opinions but has resulted in

some minor changes and additions described here.

1 Second Witness Statement of Mr Richard Roll, Horizon Issues Trial, 16 January 2019. {E1/10}

2 Second Witness Statement of Mr Stephen Paul Parker, Horizon Issues Trial, 29 January 2019. {E2/12}
3 See paragraphs 17, 18, 114, 430.3, 568, 695.1, 696, 715, 723, 869, 879 and 880

Bates & others v Post Office Limited Supplemental Report Page 2 of 47

D3/6/2
1 Introduction and summary

5. Shortly before this report was due, Mr Coyne provided me with a list of 127 Peaks

that he intends to comment on in his report. I have not commented on those Peaks

in this report as I will first need sight of his comments.

1.2 Document structure and summary

6. Section 2 contains my responses to the Claimant Subpostmasters' witness statements

which I had not time to address in my first report, and to Mr Roll's second witness

statement that I received on 17 January 2019. In broad terms, I have found most of

this inconclusive of problems in Horizon and it has not materially altered my

opinions.

7. Section 3 summarises the results of my further analysis of KELs in relation to the

robustness of Horizon. I have found no further KELs that describe a bug that

would introduce lasting errors in branch accounts. The detailed results of examining

these KELs are in Appendix A to this report.

8. Section 4 summarises the results of my examination of documents cited by Mr

Coyne in his first report, which I had not time to respond to in my first report, and a

few related documents which I have needed to examine. Examining these

documents has added further support to my previous opinions. Detailed responses

to points made by Mr Coyne are contained in Appendix B to this report.

9. Section 5 summarises the results of some further analysis I have made in support of

the quantitative results in section 8 of my first report. The result of this analysis is to

provide better support for these results, and not to change them materially. The

most important change is to move one numerical result by about 20% in a direction

which favours the Claimants - but which does not alter any of my qualitative

opinions which flow from it.

Bates & others v Post Office Limited Supplemental Report Page 3 of 47

D3/6/3
1 Introduction and summary

10. Section 6 contains the results of further examination I have made of previously

disclosed documents relevant to Horizon issues 3 and 11. The effect of this

examination has been not to alter my previous opinions significantly, but to identify

further evidence in support of them.

11. Section 7 addresses Horizon Issue 15 on Transaction Corrections (TCs). It

summarises a revision to the calculation I have made in section 9 of my first report,

of an upper limit on the mean magnitude of the impact of errors in TCs on

Claimants' branch accounts. The effect of this revision is not materially to alter the

limit I derived in my first report, but to increase my confidence in it. The underlying

calculation is described in Appendix C.

12. Appendix D describes the annex to my report, which is an Excel spreadsheet of my

numerical calculations, which the reader may use to alter any of the input

assumptions and see the changed results.

Bates & others v Post Office Limited Supplemental Report Page 4 of 47

D3/6/4
2 Response to Claimants’ evidence

2. RESPONSE TO CLAIMANTS’ EVIDENCE

2.1 Subpostmaster witness statements

13. In preparing my first report, I did not have time to respond in detail to the witness

statements of six individuals, all of whom were Subpostmasters or assisted them, and

four of whom are Claimants. These are the witness statements of Mr Latif4, Anup

Patny5 and Akash Patny6, Ms Burke7, Mr Tank8 and Mr Singh9. I here address these

witness statements, sometimes with reference to the second witness statement of

Angela Van Den Bogerd10, in which she responded to them.

14. I first offer general comments on these witness statements, then address each one in

turn.

15. Each of these Claimant witness statements describes some small number of incidents

involving Horizon, based on the Claimant's recollection of those incidents - which I

assume is usually not supported by any contemporary notes made by the Claimant, as

these have not been provided in evidence. Typically, there is only one paragraph or a

few paragraphs describing each incident.

16. By its nature, this Claimant evidence mainly gives the branch view of those incidents.

It contains only occasional and indirect evidence about the other information

recorded in Horizon and used by the Horizon support staff - as relayed to the

Subpostmaster though the helpline or in other ways, and then as understood and

4 Witness Statement of Mr Adrees Latif, Horizon Issues Trial, 28 September 2018. {E1/1}

5 Witness Statement of Ms Anup Kumar Patny, Horizon Issues Trial, 28 September 2018. {E1/2}
6 Witness Statement of Mr Akash Kumar Patny, Horizon Issues Trial, 28 September 2018. {E1/3}
7 Witness Statement of Ms Angela Burke, Horizon Issues Trial, 28 September 2018. {E1/4}
8 Witness Statement of Mr Jayesh Tank, Horizon Issues Trial, 28 September 2018. {E1/6}
9 Witness Statement of Mr Septal Singh, Horizon Issues Trial, 28 September 2018. {E1/8}
10 Second Witness Statement of Ms Angela Van Den Bogerd, Horizon Issues Trial, 17 November 2018. {E2/5}

Bates & others v Post Office Limited Supplemental Report Page 5 of 47

D3/6/5
2 Response to Claimants’ evidence

recalled by him some time later. Thus, it presents mainly the branch picture of

events. The picture is subject to the limitations of human memory.

17. In contrast, at the time an incident happens, Fujitsu support staff have at their

disposal extensive electronic information, such as transaction data and event logs,

which they can combine with the Subpostmaster's account of events to attempt to

understand the causes and potential effects of the incident. These are the Horizon

countermeasures of RDS and MID. As described in my first report11, in my opinion,

the investigation of any incident is best done by a cooperative process between the

Subpostmaster and the support staff, with each contributing the information they

know best.

18. Typically incidents of this kind may give rise to a KEL and associated Peaks - and an

incident is highly likely to do so if it has any risk of introducing errors in branch

accounts, which is the focus of Horizon Issues 1 and 3. Each KEL contains

information about both sides of the picture (the branch viewpoint and the Horizon

viewpoint), using many different sources of information, recorded at the time the

information was current in everyone's mind or available electronically, together with

the analysis of knowledgeable Fujitsu support staff.

19. Thus, for an expert assessing the evidence for this trial, through no fault of the

Claimant witnesses, a typical Claimant Subpostmaster witness statement is much less

informative than a typical KEL. Unlike a Subpostmaster’s witness statement, a KEL

is contemporary, is based on electronic and human evidence, reflects both the branch

view and the Horizon view, already incorporates in-depth analysis by knowledgeable

support staff, and is not subject to the limitations of human memory.

11 Paragraph 89.6

Bates & others v Post Office Limited Supplemental Report Page 6 of 47

D3/6/6
2 Response to Claimants’ evidence

20. In my opinion, the several hundred KELs which I have examined have informed me

much better about Horizon and its limitations than the six Subpostmaster witness

statements have done. The six Subpostmaster witness statements do not significantly

alter the opinions I have formed on the basis of several hundred KELs, Peaks, and

other evidence.

21. Much of the Subpostmaster witness evidence is disputed by Post Office and it is not

for me to decide whether the Claimants' or Post Office's factual evidence is correct -

that is a matter for the Court. Given this level of dispute, and as explained in more

detail, the evidence is often inconclusive of a problem in Horizon (as opposed to

some other root cause of problem e.g. user error) or it is too general to help me form

an opinion. Where possible, I have commented on those cases where the evidence

affirms or contradicts my understanding of Horizon.

22. In reviewing the evidence, I have not examined the electronic evidence cited in Ms

Van Den Bogerd's witness statement12, to confirm whether it supports her account

of the events as I believe that to be a matter for the Court.

23. In any event, if the Claimants' evidence is found to be correct at trial, I think it

unlikely that it would materially alter my overall opinions due to the inherent

limitations in this type of evidence as explained above and below.

Mr Adrees Latif
24. In paragraphs 6 - 8 of his Witness Statement, Mr Latif describes an incident in 2015

involving transfers between stock units, in which he says there was a Horizon-

induced discrepancy of £2000.

12 Second Witness Statement of Ms Angela Van Den Bogerd, Horizon Issues Trial, 17 November 2018. {E2/5}

Bates & others v Post Office Limited Supplemental Report Page 7 of 47

D3/6/7
2 Response to Claimants’ evidence

25. Ms Van Den Bogerd, in her second witness statement at paragraph 85 onwards, has

consulted the audit records of transaction data and says at paragraph 90 that she

believes Mr Latif has recalled events incorrectly.

26. The sequence of events described by Mr Latif appears common enough - and in the

absence of further information such as event logs, is not indicative of a problem

Horizon. There are other possible explanations, such as that given by Ms Van Den

Bogerd. So, I regard this evidence as inconclusive of a problem with Horizon.

27. Similarly, the incident involving scratch cards at paragraphs 9 - 14 of Mr Latif's

Witness Statement seems common enough, and in the absence of other information,

does not indicate that Horizon could not handle user errors (or errors by Camelot)

correctly. According to Mr Latif, there was some anomaly handling a TC. But Mr

Latif does not mention TAs which are usually involved, and which were involved,

according to Ms Van Den Bogerd's analysis of the transaction data. Therefore, I am

unable to draw any conclusion from Mr Latif's evidence on its own. I note in

Appendix C of this report that Camelot is one of the most frequent source of TCs by

volume, many of them induced by user error.

28. His allegations from paragraph 15 onwards are too brief and general for me to

comment on.

29. Mr Coyne's paragraph 6.69, where he cites the witness statement of Mr Latif, does

not add any further analysis which might alter my opinion.

Bates & others v Post Office Limited Supplemental Report Page 8 of 47

D3/6/8
2 Response to Claimants’ evidence

Anup and Akash Patny

30. The Patnys' account of the events13 following a system outage on 9th May 2015

illustrates two points to me: that having only the branch side of the story is much less

informative than being able to tie together the branch history with electronic records;

and that for complex sequences of events over several days, personal recollection is

more fallible than electronic records.

31. For these reasons, I can infer little from the Patnys' account on its own. I also note

Ms Van Den Bogerd’s account, which suggests there may have been some confusion

between stocks of £1 coins and stocks of £10 notes, and one can readily see how

such an error could be made. I also note in Appendix C to this report on TCs that

remming of cash is one of the more common sources of TCs, many of which are

indications of counting errors in the branch. These seem to be frequent, and my

opinion remains that Horizon handles TCs correctly, and that in the great majority of

cases they are issued correctly.

32. At paragraph 18 of his witness statement, Akash Patny describes an incident

involving MoneyGram, and says 'I know that I did not make any mistake which would have

led to the shortfall.' Ms Van Den Bogerd's examination of the transaction records at

paragraph 71 of her Witness Statement suggests otherwise. In my opinion, in a

support role, one would generally require more than one Subpostmaster to assert that

'I did not make a mistake', before the balance of probabilities might start to point to

an error in the software.

33. Mr Coyne refers to Mr Patny's witness statement, at paragraph 5.185 of his report.

But he does not provide any further analysis which might alter my opinion.

13 As contained within the Witness Statement of Mr Akash Kumar Patny, Horizon Issues Trial, 28 September 2018 and the

Witness Statement of Ms Anup Kumar Patny, Horizon Issues Trial, 28 September 2018.

Bates & others v Post Office Limited Supplemental Report Page 9 of 47

D3/6/9
2 Response to Claimants’ evidence

Ms Burke
34. At paragraph 14 of her statement14, Ms Burke describes an incident arising from the

same Horizon outage on 9 May 2015 as that which affected the Patnys, in which she

later managed to track down the customer who had left the branch, and with his

support, apparently got an error of £150 corrected by a TC.

35. Ms Van Den Bogerd's evidence at paragraph 103 suggests that Ms Burke may have

made a mistake, by putting two customers in the same basket.

36. I would expect this mistake to occur frequently across the whole Post Office branch

network and would expect Horizon to have robustness countermeasures (UEC) to

correct it. In this case the countermeasure would involve TCs, and I would expect

the error to have been corrected by a TC, even if Ms Burke had not had the

customer's assistance. There is nothing in her evidence to indicate that the normal

UEC processes would not have corrected the error.

37. Mr Coyne's reference to Ms Burke at his paragraph 5.40 does not provide further

analysis which might alter my opinion. At his paragraph 6.65, referring to Ms Burke,

Mr Coyne says: ‘Whilst the receipt of the TCV would ultimately rebalance the discrepancy for

Mrs Burke it illustrates one example of error within the issuing of TCs.’ I have addressed the

level of errors in issuing TCs in section 9.6 of my first report, and further in section 7

of this report.

Mr Tank
38. At paragraph 6 of his witness statement15, Mr Tank describes an incident involving a

power failure and a discrepancy of £600.

14 Witness Statement of Ms Angela Burke, Horizon Issues Trial, 28 September 2018, paragraph 14. {E1/4/3}

15 Witness Statement of Mr Jayesh Tank, Horizon Issues Trial, 28 September 2018, paragraph 6. {E1/6/2}

Bates & others v Post Office Limited Supplemental Report Page 10 of 47

D3/6/10
2 Response to Claimants’ evidence

39. In my opinion, while Horizon has robust facilities for handling power failures and

ensuring that they do not introduce errors in branch accounts, nevertheless recovery

from a power failure is outside a Subpostmaster's normal experience of using

Horizon, and it can be complex, so there is extra scope for user errors. Horizon

needs to be robust against these as well.

40. As with the other Claimant witnesses, Ms Van Den Bogerd's analysis at paragraph 76

onwards of her Witness Statement suggests that Mr Tank made mistakes (in this

case, not following the recovery guide and not retaining some receipts). It also

suggests that she has found discrepancies between Mr Tank's account of the calls he

made to NBSC, and the NBSC call logs. So, I cannot draw any conclusions from Mr

Tank's evidence about this incident.

41. At paragraphs 15 - 19 Mr Tank mentions issues over label printing, which appear to

have occurred from 2007 onwards. When he says at paragraph 17: 'Horizon jumped

ahead to the end of the transaction and therefore no label could be printed', there is too little

detail for me to analyse what might have been happening, or whether it might have

arisen from a software bug.

Mr Singh
42. The events described by Mr Singh start in 2003, so there is little possibility of using

electronic records to corroborate his account (except for NBSC call records).

43. From paragraph 6 onwards16, he describes a large 'phantom transaction' and a

subsequent audit. Ms Van Den Bogerd from paragraph 35 of her Witness Statement

notes several apparent inconsistencies and gaps in Mr Singh's evidence about this.

16 Witness Statement of Mr Septal Singh, Horizon Issues Trial, 28 September 2018., paragraph 6 onwards. {E1/8/2}

Bates & others v Post Office Limited Supplemental Report Page 11 of 47

D3/6/11
2 Response to Claimants’ evidence

These, and the lack of electronic evidence, imply to me that I cannot draw any

conclusion from his evidence about the phantom transaction.

44. Generally, within a short space, Mr Singh mentions a large number of potential

problems with Horizon (such as paragraph 12, 'other shortfalls of smaller amounts on a

weekly basis', paragraph 13 ' Horizon was also prone to errors and mistakes occurred with every

type of product' and paragraph 15 ' Sometimes Horizon just stopped working'). However,

these are described so briefly, with so little specific detail, that it is not possible for

me to comment on them, and they do not alter my opinion.

45. At paragraph 16, Mr Singh says: 'issues with access to Horizon’s data and its

reporting functions were not sufficient to enable me to identify the alleged

discrepancies and therefore the cause for the shortfalls'. However, he does not say

what these issues were, so it is not possible for me to comment on them.

2.2 Mr Roll's second witness statement

46. I have recently received the second witness statement of Mr Roll17, in which he

addresses various points from my first report, including sections 4 and 11. I

comment here on Mr Roll's second witness statement.

47. I have also received on 29 January 2019 the second witness statement of Mr Parker18,

in which he responds to Mr Roll's second statement. As Mr Parker has had access to

contemporary relevant information including relevant Peaks, and as he has a much

deeper knowledge of Horizon than I do, Mr Parker's responses to Mr Roll are

generally more detailed and specific than my own. I have not yet had time to read all

the documents referred to by Mr Parker. However, I have found that Mr Parker's

17 Second Witness Statement of Mr Richard Roll, Horizon Issues Trial, 16 January 2019. {E1/10}
18 Second Witness Statement of Mr Stephen Paul Parker, Horizon Issues Trial, 29 January 2019, subject to the correction in

Womble Bond Dickinson's letter of 1 February 2019. {E2/12}

Bates & others v Post Office Limited Supplemental Report Page 12 of 47

D3/6/12
2 Response to Claimants’ evidence

views are generally consistent with my own views (formed before I saw Mr Parker's

statement). In places below, I comment on Mr Parker's statement - where it seems

to agree with my own views, or where it brings new information to me.

48. For the expert, Mr Roll's witness statement presents many of the same difficulties as

the Claimant Subpostmasters’ witness statements. It is based on personal

recollection (I assume without the benefit of contemporary notes); it generally

addresses each incident or type of incident in one or a few paragraphs; it represents a

single person's viewpoint on events which involved many people; and there may now

be no electronic evidence that I can corroborate it against. I have not had time to

request or seek such evidence, or to inspect the evidence cited by Mr Parker.

Furthermore, Mr Roll's evidence is contested, and I cannot anticipate what the court

will find.

49. Mr Roll's evidence about any incident or type of incident appears less informative to

me than a typical KEL. Nevertheless, where he comments on my first report, I have

responded in the following paragraphs.

50. At paragraph 5, Mr Roll addresses hardware failures. He says he was involved in a

hardware failure at least once a month. With hardware in approximately 13,000

branches, this does not imply a high failure rate. However, it reinforces the point

that Horizon had to have robustness countermeasures built in to ensure that

hardware failures did not introduce errors in branch accounts, as I have described in

my first report at paragraph 151.

51. Mr Parker's witness statement, at paragraph 3, is consistent with my opinion that Mr

Roll's evidence does not imply a high failure rate of hardware.

Bates & others v Post Office Limited Supplemental Report Page 13 of 47

D3/6/13
2 Response to Claimants’ evidence

52. Mr Roll says: 'these problems could and did affect branch accounts'. He provides no evidence

to distinguish between two different effects: (a) some temporary impact on branch

accounts, which is later corrected by one or more countermeasures, or (b) a

permanent inaccuracy in branch accounts. Events of type (a) are routine. Mr Roll

provides no evidence that the problems were of type (b) - and in my opinion, more

detailed evidence than his personal recollections would be required to change my

opinions.

53. In paragraphs 6 - 8, Mr Roll details various specific hardware failures. He does not

provide any evidence to imply that these caused lasting errors in branch accounts, or

in some way evaded the usual countermeasures. Therefore, my opinion - that

countermeasures were effective - is unchanged.

54. In paragraph 8, he describes a fault which could cause a Subpostmaster to

inadvertently switch off the power. But loss of power would in any event occur

across the Post Office network with such high frequency that Horizon had to be

robust against it, and in my opinion it was so.

55. Mr Parker's responses to these points by Mr Roll, at Mr Parker's paragraphs 4 - 7, are

different from my own and are more detailed. Nevertheless, they are consistent with

my more general view, that hardware faults would only very rarely affect branch

accounts.

56. At paragraph 9, Mr Roll addresses transactional integrity, which he says had

'limitations and errors' - citing cases where 'deficits doubled'. He provides no

evidence to imply that the doubling of deficits was caused by a failure in transactional

integrity. The incidents of deficits doubling which I have examined were not caused

Bates & others v Post Office Limited Supplemental Report Page 14 of 47

D3/6/14
2 Response to Claimants’ evidence

by failures in TIN (Transactional Integrity), which was built into Horizon at a deep

level. This paragraph does not alter my opinion on the TIN countermeasure.

57. Mr Parker's statement, at his paragraphs 9 and 10, that Mr Roll's evidence does not

provide any link between the problem he describes and transactional integrity, is

consistent with my opinion.

58. Mr Roll's paragraph 10 is a long description of a problem arising from concatenating

product codes, and possible duplicates in the results. I make the following

comments:

58.1. The consequence of the fault Mr Roll describes would be for Post Office to

make a payment to the wrong client organisation. This has no direct impact

on branch accounts, and Mr Roll offers no evidence that it would have had

any impact. Because client accounts were remote from branch accounts,

there is in my opinion no reason to expect any impact on branch accounts.

58.2. His statements that the fault was 'impossible to fix' or would involve costs

which were 'astronomical' need not, in my view, influence my opinions; it is

always necessary to consider the cost of any fix and its impact, and Mr Roll

does not address this trade-off.

59. In addressing this evidence from Mr Roll, at paragraphs 11 and 12 of his witness

statement, Mr Parker focuses on reference data issues. Mr Parker's statement at his

paragraph 12 that making a payment to the wrong client organisation does not imply

any effect on branch accounts is consistent with my opinion.

60. In paragraph 11, Mr Roll addresses recovery processes. In my first report, I

commented at paragraphs 156 and 235 - 236 that transaction recovery processes

Bates & others v Post Office Limited Supplemental Report Page 15 of 47

D3/6/15
2 Response to Claimants’ evidence

were unfamiliar to most Subpostmasters and were therefore more prone to human

error. I agree with Mr Roll that recovery processes were often complex. However,

in my opinion:

60.1. Recoveries from a failure are intrinsically more complex and diverse than

normal business transactions. The complexity of recovery processes was

intrinsic to their nature and was not a design fault in Horizon.

60.2. Because human errors were more common in recovery processes, Horizon

has to be robust against them, to ensure they do not introduce errors in

branch accounts. These countermeasures (part of UEC) have been described

in my first report at paragraph 60 and, in my opinion, they were successful.

61. Mr Parker says, at his paragraph 14, that recovery processes are more complex and

error-prone than normal transactions. He also says that these processes were

designed to be no more complex than they need be. These statements are consistent

with my opinions.

62. They are also consistent with the first sentence of Mr Roll's paragraph 11. I agree

with Mr Parker's comments on the rest of Mr Roll's paragraph 11, at Mr Parker's

paragraph 15 - where Mr Parker says that Fujitsu made efforts to identify any

software error, rather than just attribute it to human error. I addressed this topic at

paragraphs 973 -978 of my first report.

63. I do not understand the basis on which Mr Roll comments on TCs in his paragraphs

12-, because they were only introduced after he had left Fujitsu. Mr Parker confirms

this at his paragraph 16.

Bates & others v Post Office Limited Supplemental Report Page 16 of 47

D3/6/16
2 Response to Claimants’ evidence

64. In his paragraph 12, Mr Roll says: ‘I also think it is wrong to say that software errors would

occur uniformly across branches, as I explained in the example in para 10 above’. He is referring

to the errors described in paragraph 167 of my report, which are specifically a

hypothetical 'any such error' that 'gives rise to so many TCs' that it is easy to detect. Since

TCs are not in Mr Roll's experience, his comments on them do not alter my opinions

about them.

65. In paragraph 622 of my first report, considering actual errors rather than hypothetical

errors, I assumed (and justified the assumption in my Appendix F) that software

errors would occur at random, with a probability which is approximately uniform

across branches (dependent only on their sizes and volumes of transactions). This

probability may have been (and in my opinion19 was) very small, so that bugs would

occur sporadically, only in one or two branches in very few months. This pattern

would not appear to be 'uniform across branches'. The pattern is consistent with the

example Mr Roll gave in his paragraph 10, that 'some problems were very specific and arose

rarely '.

66. I agree with the first sentence of Mr Roll's paragraph 13, about testing, and said so in

my first report at section 6.6. Mr Parker also agrees at his paragraph 17.

67. In his paragraph 14, Mr Roll disagrees with paragraph 168 of my first report, where I

said: 'The likelihood of any software error in Horizon staying disguised as a human error, and thus

of not being detected, is extremely small'.

68. My opinion is unchanged. Mr Parker's statements, at his paragraphs 18 - 20, citing

Fujitsu's mechanism for detecting issues, provide further detail if they are accepted.

19 As described in paragraphs 632 - 635 of my first report.

Bates & others v Post Office Limited Supplemental Report Page 17 of 47

D3/6/17
2 Response to Claimants’ evidence

69. In my experience, the detection of software errors in live use (those that have not

been detected in testing) depends on two key capabilities: (a) spotting some repeated

pattern in a number of incidents and (b) reproducing an incident in test, where you

can trace precisely where the error arose, in software or in the inputs. In my opinion,

as evidenced by many KELs, Fujitsu were highly effective at both of these

techniques. Mr Parker's paragraphs 19 and 20 describe how they operated.

70. If a software error caused some incident to occur only once then it was, as Mr Roll

implies, possible for SSC to attribute it wrongly to human error, because of lack of

evidence or uncertainty surrounding how the single incident had occurred.

71. However, if a software error occurred more than once - even a few times - then the

KEL process was in my opinion very effective at spotting a repeating pattern, and

the balance of probability about its cause could shift from human error to some

other factor such as software or reference data. Then, in my opinion, Fujitsu would

have every incentive to identify the cause, to reduce their future support workload

and provide the service to Post Office that they had contracted to provide.

72. Fujitsu's next move would often be to try to reproduce the fault in test. If they could

do this (which to judge from a sample of KELs and Peaks I have seen, they usually

could), then this is in my experience an extremely powerful way to detect software

bugs. In a test configuration, you can step through the software line by line, keeping

track of how the relevant data values change as you do so. This facility, called

'debugging', is provided in all modern programming environments and is a very

powerful way to identify and correct software bugs. I use it regularly myself. As

soon as I can reproduce a fault in test, it is usually 'game over' for any software bug.

Bates & others v Post Office Limited Supplemental Report Page 18 of 47

D3/6/18
2 Response to Claimants’ evidence

73. Therefore, in my opinion the only software bugs in Horizon which could have

remained disguised as human errors were those which were small and occurred only

very few times. In terms of impact on branch accounts, these bugs, had they existed,

would have had very small impact.

74. While Mr Roll says, in his paragraph 14, that 'I do not believe that we were successful in

identifying and correcting all problems' he does not quantify how many problems remained

unsolved, or for how long. His statement does not alter my opinions.

75. In his paragraph 15, Mr Roll refers to pressures on the testing teams of time,

budgets, and staffing. In my experience, these pressures are a reality of IT

development. I have not seen any evidence that these pressures were excessive in

the case of Horizon. Rather, Horizon's good record in service20 indicates to me that

its testing was effectively carried out. I discuss Fujitsu's testing processes further in

section 6 below.

76. Mr Parker offers a different and more detailed, but in my opinion consistent, view in

his paragraphs 24 and 25.

77. In his paragraph 19, Mr Roll addresses my comment that ‘Horizon’s systems software

generates events whenever something unexpected happens’. I clarify here that I was referring to

two distinct uses of 'events' in event logs:

77.1. Some kinds of events are known in advance to be troublesome - so these

events are automatically brought to an operator's attention.

20 For instance, in that Horizon has produced over three million sets of branch accounts with very few errors, as described

at paragraph 619 of my first report; or in its high level of robustness as summarised at paragraph 518 of my first report.

Bates & others v Post Office Limited Supplemental Report Page 19 of 47

D3/6/19
2 Response to Claimants’ evidence

77.2. The event logs contain a great deal of other information - other types of

event which are not signs of trouble in themselves, but which can be

examined retrospectively when something unexpected happens (especially

when it is very novel or had not been anticipated), to help find out the cause.

78. The first kind of event is designed to address the 'known unknowns' - things which

should not happen, but which you can anticipate might happen (so it is worth

watching for them), whereas the second kind of event is useful for the 'unknown

unknowns' - things which really come out of the blue.

79. Mr Roll's paragraph 19 appears to say that events of the first type did not cover

everything. I agree. But events of the second type were then used.

80. Mr Parker's description at his paragraph 26 is consistent with my views above. He

reiterates a point from his first witness statement, which I cited in my first report at

paragraphs 421 and 491 - that any effect which was perceived to have the potential to

affect branch accounts was treated with high priority.

81. I also find plausible Mr Roll's statements from 'this was a developing area' and on to the

end of paragraph 19. It is, in my experience, often appropriate to assess the impact

of some bug as being so rare or small that it is not worth the effort of fixing it. No

IT development team has enough time, budget and manpower to do everything it

would like to do 'in a perfect world'. Trade-offs must be made. If they are not made,

projects fail.

82. In his paragraph 20, Mr Roll addresses a factual point about injection of transactions.

He says: 'Sometimes we had to ask for a specific person to log in to the counter before injecting

transactions so that the software would not detect any discrepancies. A transaction inserted in this

Bates & others v Post Office Limited Supplemental Report Page 20 of 47

D3/6/20
2 Response to Claimants’ evidence

way would appear to the subpostmaster as though it had been carried out through the counter in

branch'. He then goes on to disagree with my paragraph 1119.

83. It seems to me that I require further factual information before I can comment on

this evidence. Which 'specific person'? Under what circumstances? How frequently?

Until I have that information, it remains possible in my view that any transaction

which 'would appear to the subpostmaster as though it had been carried out through the counter in

branch' might only be a transaction that he had given his consent for, as the 'specific

person' - and which had in effect been made on his behalf.

84. Therefore, Mr Roll's new evidence does not cause me to alter the opinion expressed

at paragraph 1119 of my first report, when commenting on Mr Roll's first witness

statement, that he could not alter branch accounts without the Subpostmaster

knowing.

85. In his paragraphs 27 - 34, Mr Parker provides detailed and specific commentary on

Mr Roll's paragraph 20, using his knowledge and the appropriate contemporary

documents, where they have been found. Here he acknowledges that Fujitsu could

insert transactions into branches by a piggy back process. I am not yet able to

comment on Mr Parker's evidence or the documents he cites.

86. In his paragraph 21, Mr Roll refers to my statement that 'as for transferring money,

Horizon includes no functionality that allows payments to be made to external parties or account'.

Mr Roll appears to agree with this statement but implies later that SSC staff could

cause payments to be made through other Post Office systems.

87. This point does not materially alter my opinion, which was expressed in paragraph

1121 of my first report, when commenting on Mr Roll's first witness statement, that

Horizon itself cannot pay external parties, and that any payment to an external party

Bates & others v Post Office Limited Supplemental Report Page 21 of 47

D3/6/21
2 Response to Claimants’ evidence

(made through some other Post Office system) must leave traces of the payee behind

it. In my experience, for any system to pay an external party, that party must first be

identified to the system in an auditable way. Therefore, if some member of SSC staff

were, for instance, to initiate a payment to himself or some accomplice for nefarious

purposes, that accomplice would need to be identified in a Post Office financial

system. However, I note that security of financial systems is not an area of my

specialist expertise. My first report, and this report, do not address the issue of

possible sophisticated hacking exploits.

88. Mr Parker's comments, in his paragraph 35, on Mr Roll's paragraph 21, are more

detailed and specific than my own - but in my opinion are consistent with them.21

89. Mr Roll's statement in paragraph 21 that 'there was no limit to the type of transaction that we

could insert' is problematic to me. In my opinion, there would have been at least

practical limits to the kinds of transactions which could be inserted without serious

consequences. For instance, if one were to insert a transaction which violates the

double-entry accounting constraints, then in my experience all sorts of alarms would

be raised automatically - and almost the only way to undo the damage would be to

reverse out the transaction, if that could be done in time. In my experience, nobody

could do something like that and keep his job.

90. Mr Parker's paragraphs 33 and 34 describe some of the controls in place, including

requiring two staff to be present for some types of change.

91. In his paragraph 22, Mr Roll disagrees with my paragraph 1144 where I said that SSC

access to the counters is strictly controlled.

21 Subject to Womble Bond Dickinson's letter of 1 February 2019.

Bates & others v Post Office Limited Supplemental Report Page 22 of 47

D3/6/22
2 Response to Claimants’ evidence

92. Paragraph 1144 of my first report is supported by the first Witness Statement of Mr

Parker. When I say that access is ‘tightly controlled’, I mean that the facility is

limited to members of the SSC, and usage is controlled in at least two ways: (i) SSC

team members are not authorised to inject transactions without approval; (ii) Any

injected transactions are visible to the Subpostmaster in their accounts and tied to a

specific user. In section 6 of this report, I describe evidence from service audits of

Fujitsu that these controls were effective.

93. In his paragraph 23, Mr Roll addresses the rebuilding of transaction data. He says

that my description of the process at paragraph 1131 is a simplification. I agree. It is

always necessary to simplify the description of how any IT system works - firstly to

make it understandable to oneself, and secondly to explain it to the court within a

reasonable space. Horizon consists of several million lines of code; simplification is

essential.

94. In describing transaction injection, I simplified both the process itself (the 'happy

path') and all the countermeasures, designed to ensure that if anything went wrong, it

did not have unacceptable consequences (the 'unhappy paths'). Frequently the latter

part leads to more complexity and development effort than the former.

95. Mr Roll describes a partly manual process, which had the potential for manual errors.

So, while Mr Roll says that there was ‘room for error in this process', I agree; but there had

to be safety nets to catch any errors. In my opinion, Horizon has good safety nets

(countermeasures) and it is robust. Mr Roll's paragraph 23 does not contain any

evidence or detail to imply that this was not the case.

96. Mr Parker's second witness statement at paragraphs 36 - 38 addresses the rebuilding

of transaction data in more detail than I have done. I believe his evidence is

Bates & others v Post Office Limited Supplemental Report Page 23 of 47

D3/6/23
2 Response to Claimants’ evidence

generally consistent with my simplified account. I have not yet had time to assess in

more detail how his account compares with my knowledge of Horizon.

97. In his paragraph 24, Mr Roll addresses a factual point about prior knowledge of the

Subpostmaster. Without further evidence, I cannot comment on this.

2.3 Other factual evidence from the Claimants

98. I briefly note that in my opinion Mr McLachlan's factual witness statement has many

of the limitations of individual Subpostmasters' statements - being based mainly on

recollections of the branch view of each incident, without any corroborating

electronic evidence - and with the additional limitation of being at second hand.

99. Insofar as he once served as an expert on a trial related to Horizon, it would seem to

me that the time and information available to Mr McLachlan was less than that now

available to Mr Coyne; so, the factual information Mr McLachlan can now provide

has not altered my opinions.

100. Mr Henderson, in his role for Second Sight, has had access to more information,

including the XML raw transaction logs he refers to. Mr Coyne has had access to the

same information. To the extent that Mr Coyne has used this information in his first

report, or referred to Mr Henderson, I have addressed it. But the factual evidence

provided by Mr Henderson does not alter my opinions.

Bates & others v Post Office Limited Supplemental Report Page 24 of 47

D3/6/24
3 Further Analysis of KELs

3. FURTHER ANALYSIS OF KELS

101. In Appendix D of my first report, I included a table of 30 randomly selected KELs

and identified where, in my opinion, the KELs contained evidence of the application

of countermeasures. These were the first 30 of the 80 KELs I had inspected to

estimate an upper limit on the extent of bugs in Horizon.

102. I have since then extended the sample of 80 randomly selected KELs to a sample of

200. I wished to gather more evidence about the application of countermeasures

(relevant to Horizon Issue 3), and to gather more evidence about the extent of

impact of bugs on Claimants' accounts (Horizon Issue 1).

103. I have now examined 200 KELs, for each KEL asking: (a) what does it tell me about

the practical application of specific countermeasures? and (b) what is the likelihood

of the incident described in the KEL having introduced any inaccuracy in branch

accounts or any lasting inaccuracy that would not be picked up by one or more

countermeasures? The results of this analysis, for the 200 KELs, are contained in a

table in Appendix A.

104. The effect of this work has been not to alter my qualitative opinions about either

Horizon Issue 1 or Issue 3, but to increase the confidence with which I hold those

opinions, as they are now based on a larger sample of evidence.

104.1. The KELs demonstrate the depth and diversity of the countermeasures built

into Horizon and show that they worked well - particularly in preventing

errors in branch accounts. This further supports my opinion that Horizon

was a very robust system.

Bates & others v Post Office Limited Supplemental Report Page 25 of 47

D3/6/25
3 Further Analysis of KELs

104.2. In a small minority of cases, the problem noted in the KEL arose from the

application of a countermeasure. This is to be expected. A large proportion

of the effort in developing any IT system is devoted to developing

countermeasures, and these (like any other part of the software) cannot be

expected to be completely free of errors or unforeseen problems.

104.3. For the great majority of KELs, it can easily be seen that their effects were

unrelated to branch accounts, or to show otherwise that they would have no

other effects on branch accounts.

104.4. For a few KELs, the analysis of their possible impact on branch accounts is

more complex; but still out of the sample of 200 KELs, in my opinion none

of them introduces lasting inaccuracies in branch accounts.

105. The last of these points lends further support to my quantitative conclusions on

Horizon Issue 1, discussed in the section 5 of this report.

Bates & others v Post Office Limited Supplemental Report Page 26 of 47

D3/6/26
4 Review of documents referenced by Mr Coyne

4. REVIEW OF DOCUMENTS REFERENCED BY MR

COYNE

106. In section 8.14 of my first report, I responded to Mr Coyne's opinions but I said that

I had not had time to analyse in detail all the documents cited by Mr Coyne and his

interpretations of those documents, because to do so would involve a lengthy

analysis of each document and its context.

107. I have now examined the documents referred to by Mr Coyne, and the conclusions

he draws from them. I have also examined other documents which I felt are

necessary to provide context to those cited by Mr Coyne. Where it is necessary for

me to respond to his conclusions, I do so in a table in Appendix B. These comments

do not alter my previous responses to Mr Coyne's report.

108. In the table in Appendix B to this report, I give comments on about 40 of the

documents cited by Mr Coyne. This does not address all the documents he cited. In

cases where the implications he draws from a document are purely factual (e.g.

about how Horizon worked, or about the processes around it), or where I do not

disagree with what he says, or have already addressed it in my first report, there is no

entry in the table.

109. I summarise here the opinions drawn from the detail in the table:

109.1. In many respects, the documents reinforce the opinion in my first report that

Horizon is a very robust system22 as they provide further evidence of

countermeasures in action.

22 See paragraph 49.1

Bates & others v Post Office Limited Supplemental Report Page 27 of 47

D3/6/27
4 Review of documents referenced by Mr Coyne

109.2. I previously criticised Mr Coyne's report for undertaking limited analysis of

whether an issue related to a bug causing financial loss in branches23.

Instead, Mr Coyne had referred to many events that have no bearing on the

issues that the experts must address and do not go to the central question of

the robustness of Horizon. My deeper analysis of the documents cited by Mr

Coyne supports my earlier opinions of his report. In my opinion, only 3 of

the 37 documents examined in the table in Appendix B (numbers 14, 17, and

18) even come close to indicating any bug which might have affected

Claimants’ accounts: and even for those, any impact on Claimants is very

unlikely. The reader is referred to those rows of the table. Many of the

documents cited by Mr Coyne do not directly relate to Horizon at all.

109.3. Other documents in the table are cited in section 6 of Mr Coyne's report, in

relation to reconciliation and TCs. None of these documents contains

evidence which alters my previous opinions on these issues, as expressed in

section 9 of my first report.

23 See paragraph 871 of my first report

Bates & others v Post Office Limited Supplemental Report Page 28 of 47

D3/6/28
5 New data and analysis

5. NEW DATA AND ANALYSIS

110. In this section, I describe two updates to the calculations in section 8 of my first

report. The first is an update to the calculation of the relative size of Claimant

branches in comparison to all Post Office branches (section 8.5 of my first report

and section 5.1 below). The second is to factor the additional KELs I have reviewed

into the calculation of the upper limit of the maximum number of bugs in Horizon

that have a financial impact (section 8.7.7 of my first report and section 5.2 below).

111. I then explain at section 5.3 below how these two updates affect one of my main

quantitative conclusions, namely that in my opinion no more than 0.134% of the

Claimants’ shortfalls can be attributed to bugs in Horizon24. The effect of these

updates is insignificant, with the revised figure being 0.181%25. This calculation

therefore continues to support my overall conclusions that Horizon is very robust

and extremely unlikely to be the cause of shortfalls in branches.

5.1 Size of Claimants' branches

112. In section 8.5 of my first report26, I calculated a factor 0.37 which expressed the

average size of a Claimant's branch (defined in terms of customer transactions per

day) divided by the average size of any branch across the Post Office network

(defined in the same way).27

24 This calculation is summarised in Table 8.4, at paragraph 754 of my first report.

25 This calculation is summarised in Table 5.1, at paragraphs 127 – 129 of this report.

26 See paragraph 629.

27 See paragraph 629.2 of my first report.

Bates & others v Post Office Limited Supplemental Report Page 29 of 47

D3/6/29
5 New data and analysis

113. This calculation was based on a spreadsheet of customer sessions per week at

Claimants' branches28, attached to the second Witness Statement of Angela Van Den

Bogerd and was based on data taken from three separate years: 2001, 2007 and 2018.

In my first report, I made a calculation for each year of the size of Claimant branches

versus all branches, and then took an average across the three years. It is however

statistically more sound to aggregate the size of all Claimant branches (as measured

by customer transactions) across all three years and then compare this to the

aggregate size of all branches across all three years. This is a subtle statistical change

of method, but I believe it to be more reliable and I can explain it in more detail if

needed.

114. The result of this calculation is that the ratio (Claimant branch size/typical branch

size) is now 0.45, instead of the figure 0.37.

115. This figure has therefore increased by approximately 20%, in a direction which

favours the Claimants' case (as larger Claimant branches, doing more customer

transactions, would make them more prone to bugs in Horizon).

116. This update has a minor knock-on effect on the upper limit calculated in section 8.7

of my report, as explained below in section 5.3.

117. It also has minor knock-on effects on various other opinions in my first report. The

most important of these is the figure of 50,000 bugs in paragraph 635 of my first

report (the number of bugs similar to the Suspense Account bug, required to give as

much as a 10% chance of a £1000 error in a Claimant's branch accounts in one

month). This figure is now altered from 50,000 bugs to 40,000 bugs, which is an

28 Weekly customer sessions spreadsheet {POL-0444071}.

Bates & others v Post Office Limited Supplemental Report Page 30 of 47

D3/6/30
5 New data and analysis

insignificant change in the sense that there would still need to be an impossibly high

number of bugs.

118. There are similar 20% alterations implied in a number of paragraphs in my first

report. Because they are only 20% adjustments to very large numbers, and in no case

alter the qualitative impact of the number, I do not list them in this report. They are

straightforward to work out and can be provided to the court if required.

119. In short, this update is statistically more reliable, but the difference is immaterial to

my overall opinions.

5.2 Maximum number of bugs which might have financial impact

120. In section 8.7.10 of my first report, I estimated an upper limit on the number of

different bugs with financial impact in Horizon over its lifetime. The upper limit was

conservatively estimated at 672 bugs, and my central upper limit was 145 bugs. The

limit was restated in section 8.11.

121. This estimation was made using various pieces of data, one of which was a sample of

80 randomly selected KELs29. I have now expanded that random sample set to 200

KELs (i.e. I have reviewed 120 further KELs). This additional sample of KELs,

randomly selected from the 8394 available KELs, is included with my analyses of

them in a table in Appendix A to this report.

122. As before, for the great majority of KELs, it is obvious that they have no effect on

branch accounts, because the affected functionality is unrelated to branch accounts.

For a few of the KELs, it is necessary to analyse in more detail how countermeasures

acted on those occasions.

29 Explained in paragraph 717 of my first report.

Bates & others v Post Office Limited Supplemental Report Page 31 of 47

D3/6/31
5 New data and analysis

123. My confidence in this analysis has been increased since the preparation of my first

report. In that report, I analysed two sets of KELs (one set of KELs cited in Mr

Coyne's report30, and one set selected by me31) and subsequently Fujitsu analysed the

same KELs. When they did so, I understand that Fujitsu were aware of my analysis.

124. Since writing my first report, I have further compared my own analyses of KELs

with those done by Fujitsu. In cases where their conclusion differed from my own, it

was generally by applying their better knowledge of Horizon to eliminate the

possibility of impact on branch accounts, where I had been in doubt.

125. In the additional sample of 120 randomly selected KELs which I have analysed for

this report, I have found none which in my opinion would introduce errors in branch

accounts. This mirrors my review of the first set of 80 randomly selected KELs,

where I similarly found none that in my opinion would introduce errors in branch

accounts.32

126. Having found no more KELs, which would introduce errors in branch accounts,

lends weight to there being fewer bugs in branches and only drives my statistical

analysis further in that direction. This larger sample of 200 KELs has therefore

increased my confidence that the upper limit of bugs with financial impact in

Horizon over its lifetime is 672. I have left this conservative upper limit unchanged.

127. It remains my opinion that this conservative upper limit on the number of bugs is

very small indeed compared to the number required to support the Claimants' case -

which, as described above, is now 40,000 bugs.

30 Analysis contained in a table in section D.3 of Appendix D to my first report.

31 Analysis contained in two tables in sections D.2 and D.4 of Appendix D to my first report.

32 See paragraph 738 of my first report.

Bates & others v Post Office Limited Supplemental Report Page 32 of 47

D3/6/32
5 New data and analysis

128. It also remains the case that the two experts, having searched for bugs in Horizon for

almost a year, and having written three reports which I have seen, have not identified

more than a handful of bugs with definite or likely impact on branch accounts -

unless Mr Coyne's second report changes that conclusion.

5.3 Impact of bugs in Horizon on Claimants' branches

129. In section 8.7 of my report, I calculated an upper limit on the financial impact of

bugs in Horizon on the Claimants' branches, expressed as a percentage of their total

claimed shortfalls of £18.7 million.

130. Making conservative assumptions, designed to favour the Claimants, I calculated this

upper limit to be 0.134%. The calculation was summarised in table 8.4, at paragraph

754 of my first report. The effect of the conservative assumptions can be seen in

several rows of this table, and together they moved the result in the Claimants'

favour by more than a factor of 10.

131. This conservative upper limit now needs to be adjusted for the revised estimate of

Claimants' branch sizes as section 5.1 above.

132. As explained in section 5.2, I am keeping with my previous analysis of the upper limit

on the number of bugs which might affect branch accounts.

133. The revised calculation of upper limit on the financial impact of bugs in Horizon on

the Claimants' branches is shown in the table below.

Bates & others v Post Office Limited Supplemental Report Page 33 of 47

D3/6/33
5 New data and analysis

Item Label Central Conservative Source

Estimate Estimate

Mean number of branches in PO network, 1999 - 2018 A 13560 13560 Volume of Branches Table {POL-
0444070} as referenced in the Second
{F/1838}
Witness Statement of Ms Van Den
Bogerd
Years lifetime of Horizon B 19 19 2000 to 2018 inclusive
Total branch months (sets of monthly branch accounts across C 3091680 3091680 C=A*B*12
PO)
Claimants branch size/typical branch size D 0.45 0.5 Revised estimate in section 5.1 of my
supplemental report, raised
conservatively from 0.45 to 0.5
Scaling factor from all PO to one claimant branch month E 6870400 6183360 E=C/D

Total claimant claimed shortfall (pounds) F 18700000 18700000 Claims

Total claimant branch months (sets of monthly branch G 52000 52000 Claims
accounts for claimants)
Scaling factor from all claimants' sets of monthly accounts, to H 132 119 H=E/G
all PO branches sets of monthly accounts

Maximum number of KELs with potential impact on branch L 100 200 Section 5 of my supplemental report
accounts, based on limited sampling of KELs finds stronger support for these figures,
but I do not change them
Mean financial impact of KEL with potential impact (pounds) M 1000 6000 Finding from my inspection of KELs -
section 8.7
Maximum summed financial impact of KELs with potential N 100000 1200000 N=L*M
impact (pounds)

Probability that SPMs report a bug with financial impact, T 0.9 0.7 Evidence on SPM reporting of
given that the bug occurs anomalies - section 7.5
Probability that Fujitsu create a KEL, given that SPMs report a U 0.9 0.5 Evidence on Fujitsu processes for KEL
bug creation - section 7.5
Probability that Fujitsu create a KEL, given that a bug occurs V 0.81 0.35 V=T*U
Probability that a KEL is not archived W 0.85 0.85 Evidence on KELs archiving (Parker WS)
Probablity that a KEL is created and not archived, given that a X 0.69 0.30 X=V*W
bug occurs

Maximum summed financial impact of bugs on all PO Y 145243 4033613 Y = N/X

branches, corrected for KEL sampling,creation and retention
(pounds)
Maximum summed financial impact of bugs on all claimant Z 1099 33921 Z=Y/H
branches, corrected for KEL sampling,creation and retention
(pounds)

Maximum financial impact of bugs on claimants, as a E1 0.006 0.181 E1 = 100*Z/F

percentage of their claimed shortfalls

Maximum possible number of bugs, corrected for KEL E2 145 672 E2=R/X
sampling, creation, and retention

Table 5.1 - Estimates of maximum financial impact of all known bugs on Claimants'
branch accounts (as defined in row E1 of the table)
134. The net effect of this adjustment is to change the resulting number by an

insignificant amount - in either absolute or relative terms. The percentage upper

limit is now 0.181% rather than 0.134%. This makes no difference to any qualitative

conclusion based on the limit.

Bates & others v Post Office Limited Supplemental Report Page 34 of 47

D3/6/34
5 New data and analysis

135. The table above is also supplied as an Excel spreadsheet33, so any reader can alter the

inputs and see the result.

33 See Appendix D.

Bates & others v Post Office Limited Supplemental Report Page 35 of 47

D3/6/35
6 Other new evidence

6. OTHER NEW EVIDENCE

136. Since my last report, I have reviewed, and given further consideration to, a number

of additional documents (as well as further KELs, as described above) relating to

robustness of Horizon. I highlight the material ones below, but none of this new

evidence has changed my overall conclusions.

6.1 Management of robustness

Quality assurance and control
137. In section 6.6.2 of my first report, starting at paragraph 316, I introduced CMMI and

Fujitsu’s achievement of Level 3 status for its Post Office Account (POA). Since

then, I have seen the CMMI Appraisal Report34. This appraisal covered POA’s main

engineering and risk management process areas (including testing). It was led by a

certified external assessor. The accreditation was valid until December 2008 when it

was decided to let it lapse because the benefits were insufficient to justify the cost.

This additional evidence clarifies the timescale over which the CMMI accreditation

was relevant and gave assurance about the quality of Fujitsu's processes.

138. Since my last report, I have seen the risk-based testing strategy that Fujitsu devised

for Horizon Online.35 Its purpose was to optimise quality, effort and elapsed time by

focussing on areas of largest business and technical risk. In my opinion, this strategy

was an appropriate choice for a project like Horizon Online.

139. In section 6.6.4 of my first report, I presented my opinions on the topic of regression

testing. Since then I have seen further evidence, which confirms those opinions.

34 CMMI Class A Appraisal Report – December 2005, IA/REP/066 {POL-0084985}.

35 HNG-X Risk Based Testing Management Plan, TST/MGT/STG/0001, v2.0, 10 April 2008 {POL-0098383}.

Bates & others v Post Office Limited Supplemental Report Page 36 of 47

D3/6/36
6 Other new evidence

The Horizon Online Testing Strategy36 defined the regression testing policy. The

application of the policy was clearly documented in specific test plans.

140. During the implementation of Horizon Online, Fujitsu created more than one

hundred test plans and reports. I have examined a selection of these. For example,

the test plan for Release 14.60 Global Pay Integrity Fee Removal37 lays out how

regression testing was to be used for that release and the corresponding report38

confirms that 100% coverage and success were achieved. This means that the full

scope of that release was shown not to cause any issues with the existing system.

141. Another example is the Horizon Online Performance/Stress High Level Test Plan39,

which confirms how this NFR countermeasure was to be proven as well as the

intention to test for functional and non-functional regression - in layman’s terms,

that Horizon Online functions correctly with no adverse impact on Horizon

performance.

142. In my opinion, thorough regression testing is an important factor in preventing any

reductions in robustness of a system caused by new releases; and Fujitsu practised it.

This additional evidence supports my previous opinion that the testing regime for

Horizon was adequate.

36 HNG-X Testing Strategy, TST/GEN/STG/0001, v2.0, 10 April 2008, section 4.5 {POL-0098552}.

37 Horizon Online System Validation and Integration Release 14.60 Test Report, TST/SOT/HTP/3172, v1.0, 10 August

2016 {POL-0147088}.
38 Horizon Online Solution Validation and Integration Release 14.60 Test Report, TST/SOT/REP/3210, v1.0, 24 August

2016 {POL-0147204}.
39 HNG-X Performance/Stress High Level Test Plan, TST/SOT/HTP/0003, v1.0, 12 September 2008 {POL-0101204}.

Bates & others v Post Office Limited Supplemental Report Page 37 of 47

D3/6/37
6 Other new evidence

Risk management
143. In section 7.2 of my first report, I discussed risk management. I have now seen

evidence40, which confirms how both Post Office and Fujitsu managed Horizon risks

and issues. This appears to be consistent with best practice, as defined in the

PRINCE2 methodology.

6.2 The Dalmellington incident

144. In my first report I discussed the KEL relating to the Dalmellington incident, but

when doing so, I did not relate it specifically to Dalmellington. Since then, when

reviewing the evidence about Dalmellington, I have further clarified my

understanding of cash remittances (remming) in a way that I believe is useful and is

described below.

6.2.1 Context
145. The Dalmellington incident involved remming of cash between a main branch and

an outreach branch. Outreach branches mainly occur in areas of low population

density, such as Ayrshire.

146. I note that the remming of cash is a frequent area of human error, because it involves

counting of large amounts of cash into a pouch or out of it.

147. If a Subpostmaster makes such an error, for most cash remming there are two

different ways in which the error can be detected and corrected:

147.1. The Subpostmaster may find a discrepancy (Horizon cash does not match

physical cash) the next time he counts cash for the branch

40 IT risk register, 19 September 2011 {POL-0219381} and Fujitsu samples from IT risk registers {POL-
0449325}.

Bates & others v Post Office Limited Supplemental Report Page 38 of 47

D3/6/38
6 Other new evidence

147.2. The Post Office central Cash Management function (in Chesterfield) counts

the cash which it despatches or receives in each pouch. If this does not

match Horizon's version of the cash remmed (as entered by the

Subpostmaster) a TC is issued.

148. So normally, if a Subpostmaster makes an error, he can detect it when counting cash

in the branch daily and wait for the TC which will correct it. Alternatively, if he

receives a TC, he can check his cash and make sure the TC is correct. So, when the

error is corrected, usually two versions of the correction (from Post Office cash

management, and from the Subpostmaster) are checked against each other. This

makes the correction robust and likely to be correct in itself.

149. There is a high volume of cash remming TCs - approximately 21,000 per year, or

20% of the total TCs (as in Table 9.3, paragraph 938, of my first report). According

to Paul Smith's witness statement41, at paragraph 18, these TCs led to 680 disputes

(about 3%) of which 40 were upheld (i.e. the TC was incorrect, and the erroneous

TC was corrected, in 0.2% of cases). This low error rate is perhaps to be expected

because of the 'double error correction' process.

150. Remming to an outreach branch is a slightly different case, because Post Office

central cash management are not involved, so there is no central Post Office business

process to create the TC. But similar considerations apply. The Subpostmaster

should be counting cash:

150.1. When he puts it into the pouch, and he takes it out of the pouch (or when

somebody else counts it for him).

41 Witness Statement of Mr Paul Smith, Horizon Issues Trial, 16 November 2018, paragraph 18. {E2/9/4}

Bates & others v Post Office Limited Supplemental Report Page 39 of 47

D3/6/39
6 Other new evidence

150.2. When he checks cash at his main branch, and when he checks cash at his

outreach branch.

151. So, one can imagine a number of different error scenarios, involving possible mis-

counting on four different occasions. The Subpostmaster can detect any one mis-

count if his other counts are accurate. Furthermore, if the rem in does not match the

rem out, Horizon can detect it - as can the Subpostmaster. Two errors of counting

which cancel out are unlikely. So, the Subpostmaster has plenty of opportunities to

detect an error, to recount, and to ask for some TC in Horizon if that is needed to

match up the Horizon picture with his actual cash. In my opinion, remming to an

outreach branch is almost as robust and correctable as remming to Post Office cash

management.

152. I note that there are two main types of error possible when remming cash:

152.1. Mis-counting the contents of a pouch, which typically leads to a small

discrepancy;

152.2. Counting some pouch several times, or not at all, which typically leads to a

larger discrepancy - some multiple of the pouch contents.

153. I would expect errors of the second kind to be much more easily detectable by the

Subpostmaster, because some large amount is repeated or doubled or omitted, and

he has some incorrect number of receipts42. I understand that the Subpostmaster

can easily correct these himself, by making a further compensating rem, and that

42 Remittance pouches are barcoded, and Horizon directs the Subpostmaster to scan in the codes, which uniquely identify

each pouch remitted (see Post Office Onboarding Counter Guide, v6.0, 13 July 2018, pages 29 – 31 {POL-0440083}). In
this way, Horizon itself can check the balance of remming in and out for each pouch.

Bates & others v Post Office Limited Supplemental Report Page 40 of 47

D3/6/40
6 Other new evidence

many Subpostmasters knew how to do this. Dalmellington was an error of the

second kind.

6.2.2 The Dalmellington incident

154. The Dalmellington incident involved a sequence of events arising from a forced

shutdown in certain circumstances which could easily mislead the Subpostmaster to

making an error - recording a rem operation several times. This error would be

obvious to the Subpostmaster, and some Subpostmasters who experienced it were

able to cancel it out by a reverse rem. Others could ask Post Office centrally to

correct the discrepancy, and this appears to have happened at the Dalmellington

branch.

155. In my opinion on the KEL related to Dalmellington, as expressed in my first report

(analysis of KEL acha621p, in Appendix D.3), it was a straightforward example of

user error correction (UEC) which worked as intended - although at the time I did

not recognise this KEL as referring to Dalmellington. In my opinion, for the

reasons summarised above, there was probably no permanent error introduced in

branch accounts by the Horizon software.

156. Even so, Post Office and Fujitsu were concerned that Horizon should not induce the

user to make such an error, even in rare circumstances. So, a KEL and a Peak were

created, and this problem in Horizon was fixed.

157. To make this point more clearly: in my opinion, for many KELs, it would have been

obvious to Fujitsu that any error in branch accounts would have been corrected in

due course, by the normal processes of error correction such as monthly branch

balancing or TCs; but in spite of this, Fujitsu might still have decided to take special

action to fix the problem sooner - for instance, to assist the Subpostmaster in a

Bates & others v Post Office Limited Supplemental Report Page 41 of 47

D3/6/41
6 Other new evidence

visible manner, or to avoid the Subpostmaster being concerned for a longer period

than necessary.

158. Therefore, in my opinion, a note in a KEL or a Peak that Post Office would need to

fix something in a branch accounts does not imply that if Post Office did not fix it, a

permanent error in the accounts would ensue. Post Office were concerned to fix

problems quickly, so that the Subpostmaster would not be worried and distracted by

them; but even if the problem were not fixed quickly, some later, slower process of

error correction (such as a TC or monthly balancing of branch accounts) may have

fixed it. Without further investigation of each example, one cannot conclude either

way; but in many cases, it seems likely to me in most cases of remming that a later

correction would have occurred, through the normal mechanisms - which had been

designed to cope with the great majority of incidents. Dalmellington was such a case.

Torstein Godeseth's second WS to paragraphs 55 onwards, referring to

Dalmellington, reinforces my opinion that this was so. Similarly, the small maximum

financial impact that I have estimated of incorrect TCs reinforces my opinion.

159. Further, but weaker, support from this view comes from a note by Ann Chambers in

the Peak related to Dalmellington (PC0246949)43, where she had looked for other

affected branches. She said: ‘I can only check back two months; I've found 4 other instances

(outreach branches 214869, 106444, 110444, 207828) and all but the last removed the

discrepancy by completing a rem out for the excess, which corrected the system cash holding. Branch

224843 may be able to do the same but NBSC should advise on this’.

160. She had also said previously: ‘This is not an area that has changed for several years so it likely

to have happened before but we have no record of it having been reported to us’ and said: ‘any fix

43 {POL-0415840}

Bates & others v Post Office Limited Supplemental Report Page 42 of 47

D3/6/42
6 Other new evidence

will not retrospectively change the branch accounts’. This last statement seems to me a simple

comment on what Fujitsu were doing and not doing - that they were only fixing the

Horizon software to avoid future occurrences.

161. She did not say anything to imply: ‘we are going to have to find all affected branches

and correct their accounts’. In my opinion, from assessing many KELs by her, Ann

Chambers appears to have been diligent and knowledgeable; had it been necessary to

correct branch accounts in retrospect, in my opinion she would probably have said

so there. But as I have said above, this is only a weak inference by me.

162. Another example of this effect - that many errors are corrected after some delay -

comes from the recently disclosed document 'Horizon Issue Management - Incident

report and status update'44 which describes a similar issue, involving locked user

terminals, which could lead to errors after restart (unlocking), and had affected

several branches. This document noted: ‘Most branches have already balanced again and

therefore there is no point looking to make any correction to the data.’ This implies that in this

case, balancing corrects the data, and the only purpose of any other correction would

be to correct it sooner.

163. The reason I did not recognise that KEL acha621p related to Dalmellington was that

I was responding to Mr Coyne's reference to the KEL - and in the two paragraphs in

which he referenced the KEL (paras 5.23 and 5.130 of his report) he did not relate

either of the paragraphs to Dalmellington (addressed at his paras 5.16 - 5.19), or to

each other. It was not evident from Mr Coyne's report that these three separate

extracts all referred to the same incident.

44 {POL-0511065}

Bates & others v Post Office Limited Supplemental Report Page 43 of 47

D3/6/43
6 Other new evidence

6.3 Permission controls - Horizon Issue 11

164. In paragraph 1153 of my first report, when addressing Horizon Issue 11 about

permission controls on access to Horizon data by Fujitsu employees, I stated that

external audits of Horizon's permission controls had identified room for

improvement.

165. Since then I have seen a separate series of audit reports - the EY service audits of

Fujitsu - which gave Fujitsu a clean bill of health for several years on Control

Objective 10 (access control), and on all the other control objectives, with no

recommendations for improvement.

166. These audits increase my confidence that the SEC countermeasure was applied

effectively and modify the opinions I expressed in paragraph 1153 of my first report,

about Horizon issue 11. On balance, the audit reports say that permission controls

were adequate. Details of the service audit reports follow.

167. As well as their financial audits of Post Office, Ernst & Young (EY) also acted as an

independent service auditor of ’Fujitsu’s System of IT Infrastructure Services

supporting Post Office Limited’s POLSAP and Horizon Online applications’45. The

reports were prepared for Post Office and their independent auditors (who were also

EY; I do not know whether EY maintained a 'Chinese wall' between the two audit

teams).

168. The service audit for 2014 consists of a 'Description' by Fujitsu of the services,

signed off by Fujitsu senior management, and a number of tests of that description

carried out by the auditors. Section 7.2.10 of this document describes Control

45 Description of Fujitsu’s System of IT Infrastructure Services supporting Post Office Limited’s POLSAP and HNG-X

applications {POL-0218579}.

Bates & others v Post Office Limited Supplemental Report Page 44 of 47

D3/6/44
6 Other new evidence

Objective 10, which is that: 'Controls provide reasonable assurance that access to system

resources, including computing platforms and operating systems, is restricted to properly authorised

individuals.' In all seven tests of this control objective, there were 'no deviations noted'

by EY.

169. Across all 13 control objectives, EY gave Fujitsu a clean bill of health, with no

recommendations for improvement46. They noted in their summary that 'the controls

related to the control objectives stated in the Description were suitably designed throughout the period

1 April 2014 to 31 December 2014 if POL applied the complementary user entity controls

contemplated in the design of Fujitsu’s controls and subservice organisations.' and that 'the controls

tested ... operated effectively.'

170. The qualifying 'if' in the first sentence was further reinforced by EY noting: 'Because

of their nature, controls at a service organisation may not prevent or detect and

correct all errors or omissions in processing or reporting transactions.'

171. In my opinion, one should not read too much into the qualifications, or the contrast

between the service report (which addressed only Fujitsu and gave them a clean bill

of health) and the financial audit (which addressed both Fujitsu and Post Office, and

made observations); clearly the service audit needed to note carefully its restricted

scope, to auditing the service organisation alone. Nevertheless, the contrast raises a

question that coordination between Post Office and Fujitsu might have been seen by

an independent outside organisation as an area for improvement, more so than the

performance of Fujitsu itself. In any case, this evidence does not detract from my

overall opinion that Horizon is very robust.

46 I have also examined the service audit reports for 2012 {POL-0219219}, 2013 {POL-0218333} and 2016 {POL-

0219210}. All these reports found that the control objectives were suitably designed and that the controls operated
effectively.

Bates & others v Post Office Limited Supplemental Report Page 45 of 47

D3/6/45
7 Horizon Issue 15 - Transaction Corrections (TCs)

7. HORIZON ISSUE 15 - TRANSACTION CORRECTIONS

(TCS)

172. In my first report, section 9.7 should not have been a separate section but should

have been the continuation of section 9.6. With that correction, this section of my

supplemental report refers to the enlarged section 9.6 of my first report.

173. In his first report, Mr Coyne implied that errors in TCs might be a significant cause

of shortfalls in Claimants' branch accounts. In my first report, at paragraph 891, I set

out my view that this analysis was beyond the scope of the Horizon Issues but was

putting forward my own analysis for the sake of balance. I maintain that position

and so am keeping this section of this report brief.

174. In section 9 of my first report, I estimated an upper limit for the mean financial

impact per month of errors in TCs on a Claimant's branch account. This upper limit

was approximately £2 per month47.

175. This was a conservative calculation, using assumptions designed to favour the

Claimants. However, I now believe the calculation was not presented as clearly as it

could have been, and it could have been made clearer why it was conservative.

Furthermore, I have now identified other evidence which relates to the limit, and

which further supports the estimate. In this section, I describe the results of a new

estimate which supersedes my previous estimate.

176. To produce the revised estimate, I have looked again at the different categories of

TC, and evidence about their frequency and the numbers disputed. I have also

looked at call centre data, which records details about calls on a weekly basis, with

47 See paragraph 896 of my first report

Bates & others v Post Office Limited Supplemental Report Page 46 of 47

D3/6/46
7 Horizon Issue 15 - Transaction Corrections (TCs)

brief descriptions of each call. This has given me further information about the

numbers of disputed TCs depending on their type.

177. This has enabled me to make a revised estimate of the maximum expected mean

magnitude of errors in branch accounts caused by erroneous TCs. The revised upper

limit is smaller than the first limit, but not significantly so. Therefore, it does not

alter any qualitative conclusions which flow from the limit. However, I now have

increased confidence in the limit, because it is based on wider evidence and a clearer

analysis. This analysis is described in Appendix C.

Bates & others v Post Office Limited Supplemental Report Page 47 of 47

D3/6/47