In this study the general and editorial strength of seven leading accounting journals
is reported upon for the years 1983, 1985 and 1987. Using size-adjusted measures,
we conclude that the editorial boards of theJournal of Accounting and Economics and
the Journal of Accounting Research unambiguously dominate those of the other
journals and that these two journals, with the Accounting Review, consistently
outperformed expectation in attracting citations.
INTRODUCTION
Academic journals are an important vehicle for discourse between
researchers, as well as a means of communicating the results of research to
the community at large (see Latour & Woolgar, 1979; Barnes, 1977).
Publication and subsequent citation of academic research is taken as a
primary indicator of individual, scholastic and institutional reputation. In
the United Kingdom, for example, the University Grants Council (UGC)
and other funding bodies now use publication in ‘refereed’ journals as the
primary component of departmental ranking for funding decisions (see
Rothman, 1987).
In this article we examine the performance of the seven most highly
ranked accounting journals identified in Nobes (1985) peer-evaluation
study. Journals predominantly concerned with issues in finance are, there-
fore, excluded. The 1983 data reported in this study permits comparison
with Nobes (1985):
This list incorporates the top four accounting journals (*) identified by
Brown & Gardner (1985) using citation analysis. A number of other
studies have used citational analysis to evaluate the impact of journals in
accounting (see, for example, McRae, 1974; Dyckman & Zeff, 1984).
We have measured journal performance in two ways by examining: (i)
the reputation of each journal’s editorial board; and (ii) the reputation
accruing to the journal itself. We would argue that the quality of an
editorial board, revealed by the citational credit gained by the individuals
concerned, is an important determinant of the quality of a journal. Over
the longer run, improvement in the quality of an editorial board should
yield a corresponding improvement in the impact of articles published
and thus in the reputation of the journal concerned. Journal reputation is
measured using a performance index which relates actual-to-expected
cross-journal citations.
In the second part of this article we discuss the methods used in this
study and comment upon their rationale. In the third part we present the
results of our study. In the final part we enter a number of caveats
concerning this research and draw a number of conclusions.
METHODS
In order to analyse the citational strength of individuals and journals we
constructed an exhaustive data base ofbibliographies attached to all articles,
notes and research reports published in the seven journals for the years
1983,1985 and 1987. In our view, the Social Sciences Citation Index (SSCI)
was inappropriate for this task as citations in that index are retrievable only
under primary authorship. Garfield (1979) found that ‘failure to include
secondary-author papers in citation counts introduces a substantial error
at the very highest stratum of cited scientists’ (pp. 242-243). Given our
expectation that this study would focus attention on just that stratum of
accounting scholars, we concluded that use of the SSCI would be likely
to produce a severe and non-random bias in our results. Our detailed
procedures were as follows.
(a) For the selected years we used, as the citation source, all main articles,
notes, comments and research reports (assignable articles) appearing
in the seven journals listed above (only book reviews were omitted).
From these sources the authors of all bibliographic citations were
recorded, whether these referred to books, journal articles, mono-
graphs, dissertations or working papers. During this process self-
citations and articles with multiple authorships were noted.
(b) Self-citations were eliminated from the count. This is a common
PERFORMANCE INDICES IN ACCOUNTING JOURNALS 269
practice and is consistent with the definition of a citation in terms
of the impact of an item on others. In cases where the source and
reference citation had an author in common, the citations to all
coauthors were eliminated.
(c) Given the degree of uncertainty concerning the relative con-
tributions to a given article by joint authors, we have followed the
principle of indifference and credited authors with the appropriate
fractional citation following the practice adopted by Dyckman &
Zeff, 1984; Gray, Haslam & Prodhan, 1987.
(d) In a number of cases miscitations and ambiguity in attribution were
identified. These problems were rectified by recourse to the original
source material.
(e) Cited items which did not have a personal author (for example
editorial policy statements and statements from publishing organ-
izations) were not relevant to the study and were excluded.
No attempt was made to adjust for negative citations since it is now widely
accepted that the significance of a paper, in terms of its contribution to
intellectual progress, is not necessarily determined by its ‘correctness’ (Cole
& Cole, 1973; Downing & Stafford, 1981). No attempt was made to adjust
for the age of cited items. The rationale for this practice is to control for
the expanding total number of citations due to growth within the litera-
ture. This did not appear to be necessary as Cole & Cole (1967) found that
there was a very high correlation between citation counts weighted to
take into account age and unweighted counts.
The journal performance index is calculated as the ratio of actual-to-
expected citations received for each journal. Expected citations are derived
from a naive model which assumes an even distribution of citations by
assignable article count over the preceding five years of publication for
each journal. We adopted a five-year span to reduce any unsystematic
annual distortions which may occur, rather than the one-year period
used in other studies (see Garfield, 1979). This model assumes that each
assignable article is equally likely to attract citations and is designed to
reduce size bias between journals (cf. Roche & Smith, 1978).
Corrections to the data, adjusting for the relative preponderance of
notes and short research reports (minor items), did not significantly alter
the expectations for each journal and are, therefore, not reported. The
insensitivity of the expectation measure to minor items can be explained
by three factors: (i) only JAE and Ab did not produce some minor items
containing bibliographies but not counted as full articles; (ii) minor items
attract heavy citations in their own right; and (iii) in some journals the
distinction between full articles and certain minor items is difficult to
justify in terms of length.
270 V. A. BEATTIE AND R. J. RYAN
TABLE 1
Citations attributable to editorial board members
RESULTS
In Table 1 we show the totals collected from the seven journals in each of
the three years studied, the citations attributable to editorial board mem-
bers and the relevant percentages. On average, editorial board members
collect 3.6 times as many citations than would be anticipated on the basis
of their numerical strength amongst the total number of cited scholars.
This multiplier would be even higher if the number of non-publishing
and uncited scholars within the community were to be included. Any
explanation for this high ratio is necessarily ambiguous, although it is clear
that highly cited academics do dominate the editorial boards of the journals
studied. This evidence should also be considered in conjunction with the
tendency, reported upon later in this article, of academics to highly cite
the principal editors of their target journals (see Table 8).
For each of the three years of this study, cumulative citation functions
(against ranked order of each editorial board) were prepared. Figure 1
shows the cumulative function for 1987. Functions were prepared for 1983
and 1985 but for the sake of brevity are not reported here.
On the basis of these functions, rankings were prepared using both first
and second-order dominance criteria. Ordered tuples, decided by the
second-order criterion, are demarcated by square parentheses [ 1. For
second-order estimation, upper bounds on the respective functions were
defined by the size of the smaller editorial board. Definition by size of the
larger editorial board reverses the ranking of AOS and AR in 1983 and
ABR and AR in 1985. Total, mean and median citations count for each
editorial board are also reported in Table 2.
Two points are striking about the relationships shown in Figure 1 and
Table 2: (i) the high stability of the rankings over the years studied and
(ii) the unambiguous domination by JAE and JAR of the other journals
in terms of the quality of their editorial boards. Cross correlations between
the median scores for each editorial board are shown in Table 3.
PERFORMANCE INDICES IN ACCOUNTING JOURNALS 271
In
.E 300
E
0
.-P
H
z 200
3
100
ROS
0 IO 20 30 40 50
Ranked order of editorial hard members (by citations)
Figure 1. Cumulative function of editorial citations (1987). 0 = editor (average ifjoint).
TABLE 2
Rankings of citational strengthsof editorial boards
1983 dc* UAE > JAR] > [AOS > AR] > Ab > ABR > JBFA
total 38551 423.07 162.40 208.83 97.67 73.50 50.16
median IO.58 7.50 4.50 1.83 2.50 2xu.I 0.42
mean 13.77 11.13 5.80 2.78 4.65 3.34 1.39
1985 dc DAR > JAE] > [AOS > ABR > AR] > [Ab > JBFA]
total 376.58 338.00 133.00 116.11 145.57 74.75 43.50
median 6.58 9.00 3.00 3.12 2.19 3.00 1.oo
mean 9.18 11.27 4.16 4.47 3.03 3.56 3.11
1987 dc UAE > JAR] > [AOS > AR] > ABR > Ab > JBFA
total 390.22 405.14 219.29 221.63 88.57 129.13 51.67
median 9.17 7.92 3.25 1.50 3.17 2.50 0.84
mean 13.01 Il.25 5.77 4.14 4.61 4.40 3.69
* dc = dominance criterion.
272 V. A. BEATTIE AND R. J. RYAN
TABLE 3
TABLE 4
Researchers attracting ten or more citations
1983 1987
l lndnidual w&s an edmxinl board member on at least one of the seven~aumals that year.
274 V. A. BEATlIE AND R. J, RYAN
TABLE 5
Institutional ranking by editorial board person-years on the top seven accounting journals jbr the
years 1973-87 and 197841
same number of citations (in total) as any of the three journals appearing
in the top group.
The position of AOS is surprising as it is the only member of this lower
group which is cited by Brown and Gardner (1985) as being a leading
journal. It is clear from our statistics that AOS is very highly regarded by
authors writing for AOS, although it has yet to make any significant
impact elsewhere. Of all the journals studied, it appears to have formed
the strongest clientele effect within the accounting literature. Such clientele
effects may be associated with the relative newness of a particular journal
(which does not appear to be significant in the case of JAE) and with the
relative isolation ‘of the intellectual universe in which a journal operates’
(Garfield, 1979). G’lven the rather unexpected performance of AOS, we
examined journals in cognate disciplines to AOS. Apart from the seven
journals included in this study AOS authors most heavily cite: Admin-
istrative Science Quarterly (24 citations in 1987), New Left Review (13),
Sociology (10) and]oumal of Management Studies (10). Only three citations
to AOS arose from these sources in that year.
Finally, in Table 8 we offer an index of the tendency of academics
writing in these journals to appeal to the work of the editors. The expected
citations to each of the journal editors assumes an equilikelihood of citations
in the respective journal adjusted for journal size (in terms of the total
PERFORMANCE INDICES IN ACCOUNTING JOURNALS 275
TABLE 6
Performance indexfir the Seven leading accounting journals
TABLE 7
Cross correlation of pelformance indices between each of‘
the years studied (pearson rho-lower /eff, significance
levels-upper right)
CONCLUSION
Our principal conclusions from this research are as follows.
(i) Editorial board members are much more heavily cited than other
researchers in this set ofjournals.
(ii) In terms of editorial strength, JAR and JAE unambiguously domi-
TABLE 8
Index oftendency to cite principal editor by authors
8522[= (iv)]
PERFORMANCE INDICES IN ACCOUNTING JOURNALS 277
nate the group of journals studied and this performance is reflected
in their general citational strength.
(iii) Only JAR, JAE and AR exceeded expectations in terms of citational
strength.
(iv) AOS appears to exhibit a strong clientele effect with only a small
proportion of its citations arising from other journals.
REFERENCES
Barnes, S. B. (1977). interest and the Growth ofKnowledge, London: Routledge Kegan Paul.
Brown, L. D. & Gardner, J. C. (1985). ‘Using citation analysis to assessthe impact of
278 V. A. BEATTIE AND R. J. RYAN