6.

Data Collection and Methods of Analysis

6 Data Collection and Methods of Analysis

This chapter describes the data that was collected for the empirical analysis and the methods that were applied to analyze them. Multiple methods were used to collect data on the variables described in the previous chapter. The data collection was designed in a way that interrupted working processes and patient care as little as possible. This means that we collected as much data as possible from documentation, observation, and interviews. Additionally, we used questionnaires and forms for staff members to fill in, following Hackman and Oldhams (1980) suggestion to combine questionnaires, interviews, and observations in order to draw an extensive picture of an investigated object. An overview of the sources and measurement methods of all variables is given in Table 6.1 for the independent variables and in Table 6.2 for the dependent variables. This chapter first depicts the main means of the data collection, which are interviews, observation, documentation, patient registration, questionnaires, and forms. Second, it describes details of the measurement of the indicators per variable.
Table 6.1: Measurement of independent variables
Element under investigation Task environment Variables Characteristics of demanded care Professional resources Managerial and political setting Indicators Input volume Input variation Input predictability Occupational structure Volume of education Development and implementation of EPs Integration of ECU in hospital Task allocation Formal coordination Sources and measurement methods Hospital information system Structured interviews with ECU management Structured interviews with ECU management Semi-structured interviews with ECU management Document analysis Observation Document analysis Semi-structured interviews with ECU management Observation Questionnaires for staff members Observation

Organizational structure

Formal structure

Relational structure

Supervision Informal coordination

93

6. Data Collection and Methods of Analysis

Table 6.2: Measurement of dependent variables
Element under investigation Organizational performance Variables Organizational quality of care Indicators Waiting time Treatment time Number of diff. contact persons per patient Number of unscheduled returns Number of diagnostic tests Number of consultations Twelve dimensions of working climate (see below) Sources and measurement methods Hospital information system Analysis of patient records

Costs

Working climate

Medical quality

Correctness of diagnosis made by residents Appropriateness of diagnostic tests, consultations, suggested treatment

Hospital information system Analysis of patient records Analysis of ECU documentation Questionnaire for staff members Semi-structured interviews with ECU management Observation Repetitive questionnaire for supervising specialists to judge a sample of treated patients

94

6. Data Collection and Methods of Analysis 6.1 Interviews, observations, documents and patient registration

In total, 69 interviews and informal conversations were conducted, all lasting between 30 minutes and 1,5 hours. In most participating cases, an introductory interview was conducted with general managers which served both sides in getting to know each other, and to find out if the case was suitable for participation. Further interviews with ECU managers, medical mangers, and senior EPs in cases where these were employed- were subsequently scheduled. Interviews conducted in case -EP_L(I) (EP_L(I)) during the pilot-study were conducted by researchers other than the author. These interviews were transcribed and were hence available for use as a source of information. Interviews conducted by the author were tape-recorded and transcribed. The guide for structured interviews can be found in Appendix II. Besides structured questions, parts of the interviews were conducted in a semi-structured way. This provides the advantage of covering all the questions that need to be answered according to the interview guide, but to also provide the freedom for interviewees to report their own view on things. Informal conversations were mostly conducted during work breaks on observation days. They proved to be a valuable source of information on detailed peculiarities of the cases. Informal conversations were recorded by taking notes, if not during the conversation then directly after. Table 6.3 provides an overview of the interviews and conversations conducted and the interviewees addressed per case.

95

6. Data Collection and Methods of Analysis

Table 6.3: Interviews and informal conversations per case Introductory and (Semi-) structured preparatory (group-) recorded interviews interviews 2 General manager 3 Specialists -EP_L(I) 3 Discussion rounds 1 ECU manager (pilot) with specialists 6 Residents 2 General manager 1 Specialist -EP_L(I) 1 ECU manager 2 EPs* 1 ECU manager 1 ECU manager -EP_L(II) 1 ECU team leader +EP_L_EAR 1 ECU med. manager 1 ECU manager 1 ECU med. manager 1 ECU manager 1 EP education manager 1 ECU med. Manager 2 ECU manager 1 Senior EP

Informal conversations Not recorded 15

+EP_L_ADV

1 ECU med. manager 1 Med. manager, senior EP, ECU manager 1 General manager, ECU manager 1 ECU committee

1 ECU manager 5 1 GP resident 1 ECU manager 6 +EP_S_EAR 2 EPs 1 two senior EPs 1 ECU manager 7 +EP_S_ADV 1 Senior EP 1 Senior EP 2 EPs Total number of interviews and conversations 69 * Please note that case -EP_L(I) intended to implement permanent EPs but stopped the development shortly after the first EPs had started working. Therefore, although interviews were conducted with the EPs while they were working at the ECU, the case is treated as a non-EP case, using mostly data from the time before the trained EPs were employed. -EP_S

3 Specialists 1 EP* 2 Specialist 1 ECU manager 1 Resident 1 EP 1 EP in training 1 Secretary 1 ECU nurse 1 EP education manager 1 EP 1 Intermediary 1 Nurses training coordinator 1 GP resident 1 ECU nurse 1 ECU med. manager 1 ECU nurses 1 ECU nurse

10 7

10

In addition, non-participative observation of work processes was conducted in all cases. Most observations were carried out by the author. Two observation days were conducted and well documented by other researchers during the pilot study. The observant accompanied ECU employees a doctor or a nurse whom she was allocated to- during shifts. Table 6.4 provides an overview on the number of observation days per case.

Table 6.4: Number of observation days +EP_L_ADV 2 x 8h +EP_L_EAR 2 x 8h -EP_L(I) 2 x 8h, 2 x 4h -EP_L(II) 1 x 8h, 1x 4h +EP_S_ADV 1 x 6h, 1 x 3h -EP_S 1 x 6 h, 1 x 4h +EP_S_EAR 2 x 4h Total observation days

2 2 3 2 2 2 2 15

96

6. Data Collection and Methods of Analysis Wherever available, ECU documents were analyzed. We were mainly interested in documents relating to the introduction of EPs, which were only available in two cases (+EP_L_EAR and -EP_L(I)). Besides this, job descriptions and process guidelines in use were analyzed. Again, however, these documents did not exist in every participating case. Table 6.5 provides an overview of the available documents per case.

Table 6.5: Available documents per case Job descriptions Process guidelines -EP_L(I) -EP_L(II) Not available ECU manager ECU team leader ECU nurse ECU secretary ECU manager ECU nurse ECU secretary ECU manager ECU nurse ECU triage nurse ECU secretary ECU intermediary ECU nurses Partly derivable from general Howto Guide for interns Not available Not available Not available Available

Documents on EP implementation Available Not applicable

+EP_L_EAR

Not available

Extensively available

+EP_L_ADV

ECU strategy papers

Not available

-EP_S

Not available

Not applicable

+EP_S_EAR +EP_S_ADV

Not available Not available

Not available Not available

Data on patient characteristics were mainly collected from the hospitals information systems. Until recently, most hospitals used a registration system where doctors recorded information on patients manually in a paper record which was subsequently transferred into a computer based system by hospital secretaries. Data from some cases result sometimes partly- from such registration (-EP_L(I), +EP_L_EAR, -EP_L(II)). Since the introduction of a new accounting system in Dutch hospitals, the so-called DBC system, most hospitals have started to implement integrated hospital information systems in which ECU nurses and doctors directly enter information on patients conditions, conducted diagnostic tests, and treatments (which subsequently forms the base for the hospitals financial administration). Both types of registration are assumed to provide reliable and complete data, and should be comparable across cases. In some cases (+EP_S_EAR, +EP_L_EAR, +EP_L_ADV),

97

6. Data Collection and Methods of Analysis data collection from the integrated hospital information system took place only shortly after its implementation at the ECU. In these cases, ECU employees did not always seem to register completely and consistently. This was no reason to exclude these cases from the study, but it explains why we sometimes had to deal with relatively large amounts of missing values.

98

6. Data Collection and Methods of Analysis

6.2 Questionnaire A questionnaire was used to collect data on three variables: first, coordination methods in use, supervision, and consultation that ECU employees experience in patient care; second, relational structure of patient care; third, the experienced working climate at the ECU. Questions on working climate were based on a questionnaire used by the Dutch Institute Work and Stress (Instituut Werk en Stress) and on Anderson and West (1994). Questions on supervision/ consultation and on the relational structure were developed following logbooks that had been used for data collection in the case where the pilot study had been conducted. The questionnaire was tested with several nursing experts, discussed with experienced scholars in the field, and finalized after several revisions. The complete questionnaire can be found in Appendix III.

Relational structure Data on the relational structure of the patient care process was measured by means of three questions in the questionnaire (questions 7A, 7B, 7C). ECU employees were asked about consultation habits by phone and face-to-face within their own and with other professional groups for patient care during day, evening, and night shifts. The first question covered the frequency of contact within and between professional groups on a 3-point scale: no contact (0 times per shift), occasional contact (1-4 times per shift), regular contact (> 4 times per shift). The scale of the frequency definitions were derived from the pilot study, where sufficient variation occurred between 0 to more than 4 contacts between professional groups per shift. The second question covered the initiative for the contact. It was measured on a 3-point scale (mainly own initiative equal initiative mainly the other occupations initiative). The third question asked about the average importance of the contact for the actual patient care. Again, a 3-point scale was used (high-middle-low importance). Questionnaire data on relational structure was analyzed by using measures from social network analysis. Social network analysis enables the researcher to derive patterns of relationships -a system of relationships- from raw relational data, often collected by means of surveys in which individuals describe their relationship with other members of an organization. Three different kinds of network levels can be differentiated: ego networks, group networks, and social system networks (Monge and

99

6. Data Collection and Methods of Analysis Eisenberg 1987; Burt 1980). In the present study, we aggregate data from individual respondents per occupational group, hence analyzing the group network. Aggregating the data per professional group helps to reduce the danger of missing values, to which network analysis is very sensitive (Burt 1987: p. 63) as they can create huge holes in the who-to-whom data matrix (Rogers and Kincaid 1981). In the analysis of the data, we describe the different professional groups in the ECU as the networks nodes, between which relations occur represented by means of ties between the nodes. Existing ties are linked with a value which is based on the frequency of contact. The initiative for the contact is embodied by the direction of the tie from one node to another (mutual tie or single-sided). We calculated the degree centrality of the network and of the professional groups as a measure to analyze differences across the cases. Centrality is regularly used to identify network leaders. We selected Freemans degree centrality. It measures the overall activity of network actors, who in this case are occupational groups. It is based on the extent to which a node is linked to other nodes. Wasserman and Faust (1994) define central network actors as have ing the most ties to other actors in the network (p. 178). Degree centrality can be calculated per node, indicating the in-degree (received ties) and the out-degree (outgoing ties) per node, and for the complete network as a percentage of the existing of all possible ties. In particular the in-degree, based on received ties, is associated with power and influence of a certain node within a network, as it is based on the frequency of network actors contacting a specific network actor (Hagen et al. 1997). On an aggregated level, network centralization indicates the degree to which a network is focused around a few central nodes (Scott 1991) and is thus particularly interesting for revealing patterns of coordination and leadership (Irwin and Huges 1992; Tichy 1980; Turk 1977). The raw network data was transformed in order to be analyzable by means of UCINET 6 for Windows (Borgatti et al. 2002). The following steps were taken for transposition: 1. Data was entered on the basis of individual employees. The following codes were used for contact: Never contact = 1, sometimes contact = 2, regularly contact = 3. The following codes were used for the initiative of the contact: Mostly you = 1, fifty-fifty=2, your colleague=3. The following codes were used for the importance of the contacts: high = 1, middle=2, low =3.

100

6. Data Collection and Methods of Analysis 2. Employees were grouped according to their occupation and sums of the scores were calculated per professional group. The sums were divided by the number of respondents from the occupation. This results in an average score per professional group. 3. Occasionally, several occupational groups needed to be summed (e.g. residents of internal medicine and residents of surgery were combined to the occupational group residents). In that case taking the average would not have yielded correct results (e.g. 1= never contact with internal medicine residents and 3=regular contact with surgical residents should result as a 3 (= regular contact with residents in general) instead of the mean, which is 2. 4. In case of very low average scores (1-1.49), no tie was assumed to exist and was therefore deleted. For 1.5-2.49, the tie was recoded to 1. For 2.5-3.0, ties were recoded to 2. 5. Occupational groups were arranged symmetrically, i.e. the same occupational groups which receive and give ties. 6. The remaining values, which indicate the existence of a low or high frequency of contact, were connected with the direction of the tie according to who was indicated as the main initiator of the contact. In the original data, the direction ranged from 1 (= contact is mostly initiated by oneself) to 3 (contact is mostly initiated by the colleague). The averages of all members of a profession were used to direct the ties. 7. In case of missing values for an occupational group, incoming and outgoing contacts were derived from the other respondents answers. For example: The number of contacts that nurses experienced to receive from specialists was estimated as the number of outgoing contacts from specialists to the nurses.

101

6. Data Collection and Methods of Analysis Experienced working climate Survey questions on working climate questions 1 through 6, see Appendix III- were based on a questionnaire used by the Dutch Institute Work and Stress (Instituut Werk en Stress) and on Anderson and West (1994). Anderson and West developed a questionnaire according to the shared perceptions approach (e.g. Koys and DeCottis 1991; Uttal 1983; Payne, Fineman, and Wall 1976). This approach emphasizes the importance of shared perceptions underpinning the notion of the working climate. Following Reichers and Schneider (1991, p. 22), we define the experienced working climate as shared perceptions of the way things are within ECUs in terms of practices, policies, and procedures. In the data reduction phase we conducted factor analyses per a priori scale and subsequently calculated Cronbachs alpha. We derived 12 dimensions of working climate. The dimensions are described as follows (see also Table 6.6):

Table 6.6: Dimensions of working climate and applied scales Dimension

Interpersonal atmosphere Interpersonal support Interpersonal harmony Opportunity for personal development Commitment Challenging work Work Autonomy Work Pressure Team Learning Cooperation among colleagues Clarity of work rules Clarity of role expectations

N of items 4 2 3
7 6 5 2 4 11 4 3 3

Cronbachs
.75 .74 .96 .90 .84 .59 .70 .81 .89 .64 .64 .66

Variance explained by factor

59% 80% 94% 62% 57% 38% 77% 64% 48% 48% 58% 60%

Interpersonal atmosphere is defined as the overall atmosphere that is experienced among colleagues. Colleagues in this respect means all the people working at the ECU regardless of their profession. It is measured by 4 items on a 5-point Likert-type scale from 1= never to 5= always. Interpersonal support is the support that employees experience overall from their ECU colleagues, regardless of professional groups and hierarchies. It is measured by means of 2 items on the same scale as interpersonal atmosphere. Interpersonal harmony refers to the conflicts that employees experience with their colleagues. It is high if little conflict occurs. It is measured by 3 items on a 5-point Likert-type scale from 1= never to 5= always. Opportunity for personal development is defined as the experienced opportunities to individually develop in work. It is measured by 7 items on a 5-point scale from 1= strongly disagree to 5= 102

6. Data Collection and Methods of Analysis strongly agree. Commitment is defined as an employees individual attitude to external parties and as the solidarity felt with the ECU. It is measured by means of 6 items on the same Likert-type scale as personal development. Challenging work refers to employees experience in terms of broadening their personal abilities and the individual discretion of using these abilities in fulfilling tasks. It is measured by 5 items on a scale from 1= never to 4= always. Work autonomy refers to the discretion someone experiences in fulfilling his/her tasks. Work pressure refers to the time pressure employees experience. The former is measured by 2 items on a scale from 1= never to 4= always, the latter is measured by 4 items on the same scale. Team learning and cooperation among colleagues reflect the climate felt within the whole team. Team learning covers innovation and assesses how much support employees feel they get for suggesting new and better working methods that deviate from the common daily routine. Cooperation among colleagues asks to what extent team work really exists. Team learning is measured by means of 11 items, cooperation among colleagues by 4 items, both on the same Likert-type scale as personal development. Finally, the two dimensions clarity of work roles and clarity of expectations towards employees measure the degree to which employees know who does what, respectively what they are expected to do, e.g. by means of fixed procedures. Both dimensions are measured by means of 3 items, on the same Likert-type scale as commitment.

Open questions The questionnaire contained, besides questions on the dimensions described above, the possibility for respondents to note their opinion on developments in their ECU. To that end we added space for possible annotations and we asked the following open questions: What do you think of todays functioning of the ECU? And, what expectations do you have of EPs?

103

6. Data Collection and Methods of Analysis

6.3

Forms per patient

The medical quality that an ECU delivers was measured by means of a form that was to be filled in per ECU patient by a supervising specialist. The basic assumption underlying the form is that the main task of doctors working at ECUs is to correctly diagnose patients and to manage adequate medical treatment by either providing it themselves or by referring the patient to the appropriate specialist. The appraisal of whether diagnosis and treatment were issued appropriately exceeds the ability of a social science researcher; we therefore developed a form in close cooperation with medical professionals. The form asked medical specialists to assess -per ECU patientthe appropriateness of ECU doctors suggested diagnoses, treatment, ordered diagnostic tests, ordered consultations, and it finally asked whether the ECU doctor would have needed specialist supervision on the patient at all. For the complete form see Appendix IV. Like most medical issues, a golden standard the one correct way - is missing in the appraisal of medical quality delivered by ECU doctors: while some specialists may attribute young doctors discretion in treating patients, others may always ask to be consulted. Opinions may differ among specialists on diagnoses and on how to appropriately treat certain complaints; and even medical specialists are not immune to medical errors. We hence acknowledge that results derived from the described method can depend on the opinion of participating specialists. However, given the importance that especially ECU practitioners give to including medical quality as an indicator of organizational performance and the extensive discussions with experts in the field of medical research concentrating on measuring it, we found the depicted method the most valuable and most feasible way of assessing the diagnosing and treatment quality that doctors at ECUs deliver.

104

6. Data Collection and Methods of Analysis

6.4

Data collection and analysis per variable

Having broadly described the methods of data collection, the following section provides detailed information on the indicators that were collected per variable.

6.4.1 Task Environment and Context As described earlier, we characterize task environment and context by characteristics of the demanded care, professional resources and the managerial/ political setting. Characteristics of demanded care were operationalized as the input volume (=number of patients per day/hour), input variation (=diversity of patients complaints in terms of severity and specialties), and input predictability (=deviation of input volume and variation per day). Most of this data was collected from the cases hospital information systems, where information per patient was available on the date and time of a patients visit, the attending specialty, type of patient referral (general practitioner, self-referring patients, specialist, police, etc.) and the type of admission (admission as in-patient or not). Two cases (-EP_L(II) and +EP_L_ADV) also register information about the severity of a patients complaint. Table 6.7 gives an overview of the data characteristics provided by the cases to analyze the characteristics of demanded care. Besides calculating the standard deviation of the total number or daily patients and the number of patients per specialty per day, ECU managers and employees were asked in interviews to describe the input predictability in their ECU. The collected and analyzed data will be described per variable below.

105

6. Data Collection and Methods of Analysis Table 6.7: Data collection of characteristics of demanded care (for the results per category see chapter 7.1.1, pp. 117)
+EP_L_ADV +EP_L_EAR -EP_L(I) 1.10.0131.10.01 2164 0 2164 1.10.0131.10.01 2164 3 2161 1.10.0131.10.01 2164 0 2164 1.10.0131.10.01 2164 0 2161 1.10.0131.10.01 2164 0 2164 1.10.0131.10.01 2164 3 2161 -EP_L(II) 1.7.0431.12.04 9777 0 9777 1.1.0331.12.03 19634 0 19634 1.1.0331.12.03 19634 0 19634 1.1.0331.12.03 19634 0 19634 +EP_S_ADV 1.1.0531.7.05 8353 0 8353 1.1.0531.7.05 8353 0 8353 1.1.0531.7.05 7948 21 0 7948 1.1.0531.7.05 7948 321 7948 1.1.0531.7.05 7948 0 7948 1.1.0531.7.05 7948 0 7948 -EP_S 19 1.11.0430.9.05 9285 0 9285 1.11.0430.9.05 9285 0 9285 1.11.0431.10.05 9835 178 9657 1.1.0530.4.05 2740 0 2740 1.1.0530.4.05 2740 0 2740 1.1.0530.4.05 2740 6 2734 +EP_S_EAR 20

Number of patients per day Measurement 1.1.041.7.04period 30.9.04 15.10.04 n (patients) 19210 5698 missing 0 0 valid 19210 5698 Number of patients per specialty Measurement 1.1.041.7.04period 30.9.04 15.10.04 n (patients) 19210 5698 missing 303 0 valid 18907 5698 Source of referral Measurement 1.1.041.1.03period 30.9.04 31.12.03 n (patients) 19210 28843 missing 106 1 valid 19104 28842 Admission rate: Overall Measurement 1.1.041.7.04period 30.9.04 15.10.04 n (patients) 19210 28842 missing 344 0 valid 18896 28842 Admission rate: GP referred admissions Measurement 1.1.041.7.04period 30.9.04 15.10.04 n (patients) 19210 28842 missing 370 1 valid 18840 28842 Admission rate: Admissions per specialty Measurement 1.1.041.1.03period 30.9.04 31.12.03 n (patients) 19210 28842 missing 471 5 valid 18725 28837 Urgency of complaints (triage color) Measurement 1.1.04period 31.8.04 Not n (patients) 17084 registered missing 2.786 valid 14298

1.1.05-28.03.05 1408 0 1408 1.1.05-28.03.05 1408 19 1389 1.1.05-28.03.05 1408 25 1383 1.1.05-28.03.05 1408 15 1393 1.1.05-28.03.05 1408 15 1393 1.1.05-28.03.05 1408 0 1408

Not applicable

1.7.0431.12.04 9777 1694 8083 8.11.0428.11.04 1103 0 1103

Not registered

Not registered

Not registered

Not registered

As described above, professional resources were operationalized as the occupational structure (= staffing structure of the ECU) and by the volume of education (= young doctors who need to be trained at the ECU). Data on this variable was collected by
19

In this case, all policlinical admissions are handled by SEH staff. Patients may hence appear in the hospital information system as ECU patients, who were sent by specialists to the SEH purely for admission. To exclude these patients from our calculations, we ignored all patients referred to the SEH by a specialist with subsequent admission from the admission analyses. 20 Note that for analyses of admission rate per specialty for this case, patients registered as admitted for the observatory were neither calculated as admitted nor as not admitted (coded as not applicable and hence excluded from the analysis). 21 From 25-27 May 2005, general practitioners in the Netherlands were on strike, causing many more ECU visits from self-referrals without admission. These days were hence excluded from the analysis. The total patient n taken into account (=7.948) is thus lower than the real n (=8.354).

106

6. Data Collection and Methods of Analysis means of structured interviews with ECU managers (see interview guide, Appendix II).

The managerial and political setting of the ECU comprises the managerial situation of the ECU and its integration into the hospital. In cases that employ EPs, it also comprises the history of EPs implementation and introduction. Information was largely gathered by means of semi-structured interviews with ECU managers and EPs (see interview guide, Appendix II). Besides interviews, hospital documents on the EP introduction were analyzed where available and observations were conducted.

6.4.2 Organizational structure As described in chapter 5.3.2, we investigate the differentiation and the coordination in ECUs, and we split coordination into programming and feedback (personal and group mode).

Differentiation is operationalized as the splitting of tasks and their allocation to professionals. Information was gathered by means of document analysis (job descriptions, process descriptions) where available (see Table 6.5 above). In addition, semi-structured interviews were conducted with ECU managers and additional detailed information was collected through informal conversations with ECU employees and observations. Coordination by programming refers to the use of formal coordination methods. Information on the experienced use of formal coordination methods was collected by means of semi-structured interviews with ECU managers, informal conversations with ECU employees, and observations. Coordination by feedback (group mode) is operationalized as scheduled meetings for the ECU staff. Information on this aspect was collected by means of semistructured interviews with ECU managers and informal conversations with ECU employees. Coordination by feedback (personal mode) refers to the relational structure with regard to patient care. Information on these indicators was collected during observation and, even more important, by means of survey questions. As described above, employees were asked about their contacts with other staff members during

107

6. Data Collection and Methods of Analysis patient care, the latter yielding relational network data. Data on supervision and consultation were analyzed using SPSS. Network features were calculated using UCINET 6 for Windows (Borgatti et al. 2002).

108

6. Data Collection and Methods of Analysis

6.4.3 Organizational Performance As described earlier, we investigated four aspects of organizational performance: organizational quality of care, cost benefits, working climate, and medical quality.

Organizational quality of care was operationalized by indicators that relate to the service that patients receive, i.e. waiting time, treatment time, the number of different nurses and doctors (faces) a patient sees, and the number of patients who return to the ECU without being scheduled for a follow up visit. Data on these indicators were collected by means of the hospital information systems. If it was not possible for the information system to produce a report on one of the indicators we drew a patient sample on which we analyzed the detailed patient records. These samples were drawn in the following way: On day 1 of the analyzed period the 5th patient was selected, on day 2 the 10th, on day 3 the 15th and so forth. Following this procedure we took variation in registration time into account while leaving all other characteristics at random. The waiting time was defined as the time between the registered arrival of a patient at the ECU and the registered time the treatment started, mostly by a nurses investigation. The triage by a nurse, who assesses the state of urgency and serves as an indicator for the maximum reasonable waiting time, was not regarded as the beginning of treatment. Waiting times after the beginning of treatment for instance waiting time for results of laboratory tests- were not considered as waiting time but as a part of the treatment time. Treatment time was defined as the time between the registered beginning of the treatment the end of the waiting time- and the registered time of the patient leaving the ECU. The number of different faces a patient has contact with was derived from the number of medical and nursing employees who were registered with a patient, either by means of their registration as a user of the hospital information system (user name) with a patient, or by means of registration by a different user (e.g. nurses who register consulting specialists who come and see the patient). In one case, the case of the pilot study, observations on the different contacts per patient had been conducted in addition to analyzing the data registered. Despite the comparatively small n, the observational data is considered to be highly valid. We therefore use it to supplement the data from the hospital registration. An unscheduled return of a patient was defined as a patient who visits the ECU, is subsequently sent

109

6. Data Collection and Methods of Analysis home, and returns to the ECU within a week for a complaint that is related to the previous one. In case +EP_S_EAR, data on complaints per patient was not available. In this case, however, scheduled returns were not re-registered in the information system; we therefore were able to derive the number of unscheduled returns by counting patients returning to the ECU within one week for the same specialty. Table 6.8 provides on overview of the data used for analyses.

110

6. Data Collection and Methods of Analysis

Table 6.8: Data collection of indicators of organizational quality

Measurement period WAITING TIME +EP_L_ADV 1.6.04-30.9.04 Means of collection n (patients) missing valid

Patient registration: Data on registration 8725 time and time of begin treatment 1.7.04Patient registration: Data on registration 5472 +EP_L_EAR 15.10.04 time and time of begin treatment Patient registration (main specialties): 1.10.01Data on registration time and time of 2907 -EP_L(I) 30.11.01 begin treatment 8.11.04Patient registration: Data on registration -EP_L(II) 1101 28.11.04 time and time of begin treatment Patient sample: The selected sample showed no significant difference in overall throughput time to the complete 122 +EP_S_ADV 1.4.05-31.7.05 registration. The sample has been controlled for measurement errors, which occur in the complete registration. Patient registration: Data on registration -EP_S 1.1.05-30.4.05 3403 time and time of begin treatment +EP_S_EAR No data on waiting time available (see throughput time below instead) TREATMENT TIME Patient registration: Time of treatment +EP_L_ADV 1.6.04-30.9.04 8725 begin and patients leave 1.7.04Patient registration: Time of treatment +EP_L_EAR 5472 15.10.04 begin and patients leave 1.10.01Patient registration: Time of treatment -EP_L(I) 2907 30.11.01 begin and patients leave 8.11.04Patient registration: Time of treatment -EP_L(II) 1101 28.11.04 begin and patients leave Patient sample: The selected sample showed no significant difference in overall throughput time to the complete +EP_S_ADV 1.4.05-31.7.05 122 registration. The sample has been controlled for measurement errors, which occur in the complete registration. Patient registration: Time of treatment -EP_S 1.1.05-30.4.05 3403 begin and patients leave +EP_S_EAR No data on treatment time available (see throughput time below instead) THROUGHPUT TIME Patient registration: Data on registration 8725 +EP_L_ADV 1.6.04-30.9.04 time and time of leave. 1.7.04Patient registration: Data on registration +EP_L_EAR 5472 15.10.04 time and time of leave. Registration of patients for main 1.10.01specialties (traumatology, surgery, -EP_L(I) 2907 30.11.01 internal medicine, pulmonology): Data on registration time and time of leave. 8.11.04Patient registration: Data on registration -EP_L(II) 1101 28.11.04 time and time of leave. Patient sample and detailed analysis of the patient record (registration time and +EP_S_ADV 1.4.05-31.7.05 122 time of leave). Patient registration: Data on registration -EP_S 1.1.05-30.4.05 3.374 time and time of leave. 1.1.06Patient registration: Data on registration +EP_S_EAR 1408 28.03.06 time and time of leave.

572 339 102 121

8153 5133 2805 980

122

1149

2254

570 609 36 114

8153 5089 2871 988

122

80

3323

572 386

8153 5086

121

2786

121 0

980 122

18

1390

111

6. Data Collection and Methods of Analysis

Measurement Means of collection period NUMBER OF DIFFERENT CONTACTS Patient registration: Registered doctors, consulting doctors and nurses per patient 1.4.2004+EP_L_ADV (residents and nurses according to user 30.4.2004 name; consultants as registered by ECU nurses) +EP_L_EAR No reliable data available Patient registration: Registered doctors, 1.10.01-31.10.01 consulting doctors and nurses per patient -EP_L(I) Observations conducted in Nov. 2001: 13.11.01Patients were followed during their visit to 27.11.01 the SEH, different contact persons noted. 1.7.2004Patient sample and detailed analysis of the 31.12.2004 patient record (registered nurses, residents, -EP_L(II) (total n=9.777) and consulting specialists) Patient sample and detailed analysis of the 1.1.05-31.7.05 patient record: Registered nurses, EPs, and +EP_S_ADV (total n=7.203) consulting specialists Patient registration: Registered nurses, -EP_S 1.9.05-26.10.05 doctors, and, consulting doctors per patient Patient registration: Registered consulting doctors, registered nurses shift overlap, +EP_S_EAR 1.1.06-28.03.06 combined with expert opinions on specialists visits to ECU patients UNSCHEDULED RETURNS 1.4.2004Patient registration: Data on date, +EP_L_ADV 30.6.2004 admission, source of referral, complaint 1.7.2004Patient registration: Data on date, +EP_L_EAR 30.09.2004 admission, source of referral, complaint Patient registration: Data on date, -EP_L(I) 1.10.01-31.10.01 admission, source of referral, complaint 1.7.2004Patient sample and detailed analysis of the 31.12.2004 patient record: Data on date, admission, -EP_L(II) (total n=9.777) source of referral, complaint Patient registration: Data on date, +EP_S_ADV 1.1.05-30.4.05 admission, source of referral, complaint Patient registration: Data on date, -EP_S 1.1.05-30.4.05 admission, source of referral, complaint Patient registration: Data on date, +EP_S_EAR 1.1.06-28.03.06 admission, source of referral, complaint

missing

valid

2263

1508

755

2164 51

0 0

2164 51

184

184

181 1583

0 0

181 1583

1408

1399

6785 4740 2164 184 4435 3403 1408

4265 2704 0 1 481 530 16

2519 2036 2164 183 3954 2873 1392

112

6. Data Collection and Methods of Analysis As described earlier, costs were operationalized by the number of diagnostic tests that a doctor at the ECU orders, and by the number of consultations that were ordered from other specialties. Table 6.9 provides an overview of the available data used for analyses.
Table 6.9: Data collection of indicators of costs
Measurement period DIAGNOSTIC TESTS +EP_L_ADV +EP_L_EAR -EP_L(I) -EP_L(II) 1.7.04-19.10.04 1.10.01-31.10.01 1.7.04-31.12.04 (total n=9.777) 1.1.05-31.7.05 (total n=7.203) 1.1.05-30.4.05 1.1.06-28.03.06 Means of collection No reliable data available Patient sample: One patient per day. Detailed analysis of the patients registered diagnostic tests Patient registration: Data on registered diagnostic tests per patient Patient sample: One patient per day. Detailed analysis of the patients registered diagnostic tests Patient sample: One patient per day. Detailed analysis of the patients registered diagnostic tests Patient registration: Data on registered diagnostic tests per patient. Patient registration: Minimum diagnostic tests according to protocols in use per specialty. Registered data on consultations per patient Complete patient registration containing consultations per patient Patient registration: Registered data on consultations per patient Patient registration: Registered data on consultations per patient Patient sample. Registered consultations of all patients seen in one random week per month (=1790 patients on 46 days in 7 weeks). Patient registration: Registered data on consultations per patient Patient registration: Registered data on consultations per patient n missing valid

110 2164 184

3 0 0

107 2164 184

+EP_S_ADV -EP_S +EP_S_EAR

181 3403 1408

0 0 0

181 3403 1408

CONSULTATIONS +EP_L_ADV +EP_L_EAR -EP_L(I) -EP_L(II) 1.4.04-31.7.04 1.1.03-31.12.03 1.10.01-31.10.01 8.11.04-28.11.04 9087 28843 2164 1102 5171 0 0 2 3916 28843 2164 1100

+EP_S_ADV

1.1.05-31.7.05

1790

1790

-EP_S +EP_S_EAR

1.1.05-30.4.05 12.5.06-1.7.06

3443 1093

0 0

3443 1093

Working climate, as described earlier, was measured by means of 12 dimensions derived from a questionnaire that was distributed among all ECU- and affiliated employees. We used SPSS to calculate descriptive statistics and correlations, and to conduct ANOVA analyses (i.e. the comparison of mean scores between cases). In addition we included data in the analysis from answers to the two open questions in the questionnaire and data that were derived from semi-structured interviews and conversations with ECU employees as well as from observation.

113

6. Data Collection and Methods of Analysis As described above, medical quality here refers to the quality of diagnosing, including ordering diagnostic tests and consultations, and suggesting initial treatment by the doctors who regularly work at the ECU, as assessed by supervising specialists. As this data collection was only feasible with the support and extra work of medical specialists, appropriate mechanisms to collect the data were arranged individually with the cases so as to minimize interruption of the work routines while securing a reasonable return. Table 6.10 provides an overview of the details of data collection per case. The gathered data was subsequently analyzed using SPSS. We conducted ANOVA analysis for comparisons between cases and within cases. Table 6.10: Data collection of medical quality forms per case
Measurement period +EP_L_ADV 05.11.04 to 04.02.05 25.07.05 to 19.08.05 Doctors assessed and n of returned forms per kind of doctor 22 EP residents (n=96) Specialty of patients in the sample Surgery (n=89) Int. medicine (n=7) Internal medicine n (forms) 96 Handling of filling in

+EP_L_EAR

-EP_L(I)

21.05.03 to 02.09.03 Int. medicine: 14.03.05 to 05.07.05 Surgery: 29.11.05 to 12.12.05 11.05.06 to 04.06.06

Int. med. residents (n=59) EP residents (n=18) EPs (n=3) Int. med. residents (n=219) Int. med. residents / Int. med. interns (n=90) Surgery residents/ Surgery interns (n=24) EPs (n=225)

80

Specialists fill out forms on previous days patients during the morning meeting Specialist fills out forms on previous days patients during the morning meeting Specialists fill out forms directly when residents call for consult Specialists fill out forms directly when residents call for consult

Internal medicine

219

-EP_L(II)

Int. medicine (n=90) Surgery (n=25)

115

+EP_S_ADV

All specialties

225

Specialists fill out forms either directly when EPs call for consult or when no consult took place- patients are discussed afterwards Specialists fill out forms on previous days patients when checking the patient status

-EP_S +EP_S_EAR

It was not possible to set up a data collection on this variable. 05.04.06 to EPs (n=104) All specialties 28.05.06 GP residents (n=91) Interns (n=77)

287

TOTAL

1022

22

Differences between the sum of different doctors assessed and the number of returned questionnaires may be due to missing values.

114