Children and Youth Services Review 33 (2011) 19301938

Contents lists available at ScienceDirect

Children and Youth Services Review

j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / c h i l d yo u t h

Assessment of the needs of youth in residential care: Development and validation of an instrument
Maria Manuela Calheiros a,, Diniz Lopes a, b, Joana Nunes Patrcio c
a b c

Lisbon University Institute (ISCTE-IUL), CIS-IUL, Lisboa, Portugal Universit Paris Descartes, France CIS-IUL, Lisboa, Portugal

a r t i c l e

i n f o

a b s t r a c t
This article presents the development of a Residential Care Youth Needs Assessment (RCYNA) instrument. In two studies, initial evidence regarding the RCYNA psychometric properties was obtained using a sample of youth in residential care. Results show adequate levels of content and face validity, reliability and concurrent validity. 2011 Elsevier Ltd. All rights reserved.

Article history: Received 2 February 2011 Received in revised form 9 May 2011 Accepted 10 May 2011 Available online 14 May 2011 Keywords: Needs assessment Residential care Development of a measuring instrument

1. Introduction In the realm of psychosocial work with risk groups, the importance of creating specic services based on user needs has been discussed in order to increase the effectiveness of the response provided (e.g. Bullock, Little, & Millham, 1993). One of the services where this issue has been addressed is residential care for children and youth at risk. Various studies have shown that the services provided are, above all, determined by their availability (i.e., supply), and not so much by the needs of their users (i.e., demand), and thus do not effectively satisfy them (Aldgate & Statham, 2001). In fact, the literature on residential care cites several limitations of this response method, suggesting: a) the lack of care centered on the specic needs of youth and children (Bullock et al., 1993; Casas, 1993; Valle, 1998); and b) residential care's inability to efciently promote skill building so that, consequently, most youth leave institutions without the psychosocial resources needed for selfsufciency (Colca & Colca, 1996). A change, therefore, seems essential from services dened as broad and categorical to specic services called needs-led or community-based services (Calheiros, Seabra, & Fornelos, 1993; Taylor, 2005) so that care becomes more personalized (Axford & Little, 2004), specic, exible, multifaceted and differentiated (McCoy, McMillen, & Spitznagel, 2008). This may increase the likelihood that these services will be more effective (Axford, Little, Morpeth, & Weyts, 2005), since they are more oriented toward results

and based on empirical evidence involving the characteristics, needs and development phases of their users (Axford & Little, 2004). In this context, the assessment of needs is seen as a key aspect for the progress of social services for children and youth at risk (Bullock et al., 1993; Calheiros, Garrido, & Rodrigues, 2009), and entails the existence of specic instruments to assess the needs of youth in residential care. However, to our knowledge, no instrument has been specically developed and published to measure the needs of youth in residential care. Specic measures are of outmost importance since developmental approaches to the study of youth in residential care emphasize these environments as complex systems with specic structural and functional characteristics (e.g., high young/caregiver ratios, rotating staff and institutional structures and routines) in which youth experience accumulated institutional practices (Daining & DePanlis, 2007; Raymond & Heseltine, 2008). Furthermore, these youth are in a period where signicant developmental changes occur and have specic developmental needs not usually assessed by generic measures. Therefore, in this article, we present the Residential Care Youth Needs Assessment (RCYNA) instrument that aims at evaluating youth needs and has practical implications at the level of guideline planning and program design for this age group (Calheiros, 2008). Also, this article aims at presenting initial psychometric evidence regarding the RCYNA. 2. Denition and assessment of needs Contemporary perspectives dening the concept of needs tend to sustain that they are indeed universal, but what is required to satisfy these needs (i.e., satisers) maybe universal or relative and culturally

Corresponding author. E-mail address: maria.calheiros@iscte.pt (M.M. Calheiros). 0190-7409/$ see front matter 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.childyouth.2011.05.020

M.M. Calheiros et al. / Children and Youth Services Review 33 (2011) 19301938

1931

variable (Gough, 2003). These perspectives can be tied to the development context of children and youth at risk, and constitute the viewpoint adopted in this article. In this sense, needs are dened as the result of accumulated risk and protective factors in the development of children and youth in various areas of life (Little, Axford, & Morpeth, 2004). By a risk factor we mean a factor in the individual or in the environment which predispose individuals to specied social or psychological problems (Little & Mount, 1999, p. 49); risk factors are therefore dened as correlative to weak or negative results in development and well-being (Nettles & Pleck, 1994). On the other hand, protective factors are long lasting or passing conditions or attributes that moderate the negative effect of risk factors (Cicchetti & Rizley, 1981), protecting the development and well-being of individuals. Generally, these explain why some individuals are more resistant to a set of risk factors than others (e.g. Werner & Smith, 1982), and thus should be identied and understood. According to these perspectives, individuals with more risk and less protective factors will be in a situation of greater need. Thus, in a context of needs assessment, it is important to understand what may act as a protective or as a risk factor (Department of Health, 2000a). As a consequence, in the realm of services for children and youth at risk, the need to create structured models sustaining the assessment of needs is crucial. The instrument that we propose to develop (the RCYNA) is based on a specic needs assessment model: the Framework for the Assessment of Children in Need and their Families (FACNF; Department of Health, 2000b). The purpose of the FACNF is to provide a common language to the various entities and professionals working in the area of children and youth at risk. It is based on what children and youth need to achieve for successful development, and the factors that may positively or negatively affect their development (Ward & Rose, 2002). This model asserts that at least three domains must be considered in assessing child and youth needs: development needs; parents' or caregivers' ability to provide an appropriate response; and environmental and family factors. Within these domains, various dimensions or areas of life can be considered in which the risk and protective factors interact. In this way, the FACNF takes psychological and social dimensions of child and youth into account, along with diverse contextual variables, incorporating a developmental, ecological and systemic perspective (Bronfenbrenner, 1979) where different ecological systems impact varyingly according to their proximity to the individual (Swenson & Swenson, 2002). Thus, in the development of a specic instrument to measure the needs of youth in residential care it is essential to highlight this context as a primary area of assessment and to assess risk and protective factors in areas of life comprised in the three domains of the Needs Assessment System (Department of Health, 2000b). Moreover, while assessing the needs of youth in residential care the involvement of practitioners is essential, since their perceptions will inuence how they work with them. However, the participation of youth is now being viewed as equally important, since their needs' perceptions can trigger specic types of behavior and decisions (Little et al., 2004), and support a more effective response to youth problems through planned change and action. 3. Overview and objectives In this article, we present two studies involving the development and validation of a Residential Care Youth Needs Assessment (RCYNA) instrument. In Study 1, we describe the development of this instrument based on different information sources, and present content and face validity evidence regarding the RCYNA. In Study 2, we determine some of its initial psychometric qualities with a sample of 101 youth in residential care, namely its reliability and predictive validity (in its variant of concurrent validity).

4. Study 1 In this study we present the different stages of building the RCYNA. The aims of this study are: a) to identify dimensions of needs assessment based on existing methodologies in social work with children and youth at risk; b) to identify dimensions and variables relevant to youth in assessing their needs, using focus groups. These focus groups were conducted in order to incorporate in the RCYNA items that would derive from the specic characteristics of the context and population that this instrument aims to assess; c) to develop the RCYNA, taking into account the dimensions and variables identied by youth, instruments for assessing needs in social services, Portugal's Education Guardian Law (LTEOfcial Gazette [Dirio da Repblica], Law 166/99 of 14 September), content validation provided by researchers in the area of youth studies, and the opinions gathered in discussion groups with practitioners in this area. 4.1. Methodologies of assessing needs and identifying conceptual dimensions for operationalization With a view to improving the effectiveness of services for children and youth, the assessment of needs requires a systematic approach using a conceptual map or system for gathering and analyzing information on the child and his/her family, and that effectively discriminates between different types and levels of needs (Department of Health, 2000b). Various conceptual and methodological approaches have been developed within the scope of services for children and youth at risk, with at least four approaches of this type identied in the literature: a) LACLooking After Children (Department of Health, 1995); b) CLACommon Language Approach (Dartington Social Research Unit, 1998); c) FACNFFramework for the Assessment of Children in Need and their Families (Department of Health, 2000b); d) ICSIntegrated Children's System (Department of Health, 2002). These approaches are part of an ecological perspective with a holistic vision of the child, taking the interaction and inuence of various areas of his/her life and the various systems in which he/she is integrated into account. All of them underscore the importance of assessing needs in terms of risk and protective factors, in order to plan and implement care programs, thereby ensuring their effectiveness. However, these approaches have not been adapted to assessing the needs of youth in residential care. In fact, the existing tools have clinical assessment purposes, focusing on children and youth outcomes (e.g., Assessment and Action RecordsLAC) or do not assess specically the youth in residential care (e.g., Aggregated Data Form CLA), which leads to some gaps in the assessment of key aspects to this population (e.g., evaluation of the youthpractitioners relationship and of the residence living situation). In this article, we focus on the Aggregate Data Form (ADF), belonging to the CLA methodology, and on the Core Assessment Record (CAR) of the FACNF methodology, since they represent quantitative assessment methodologies that analyze needs taking risk and protective factors into account. The ADF, developed by the Dartington Social Research Unit (Little & Mount, 2003), identies groups of youth with similar needs based on information gathered on risk and protective factors in the ve areas of life included under the methodology to which it belongs (CLA), together with the risk thresholds of children (Little, Axford, & Morpeth, 2002). It is comprised of 312 variables, nearly all of them with a dichotomous response scale, and is completed by the technicaleducational team to make its content more objective and reliable. The CAR (Department of Health, 2000b) was developed to operationalize the FACNF methodology, assessing all of its areas through 298 items. The response scale is generally dichotomous, and is completed by practitioners together with the child/young person and his/her family members.

1932

M.M. Calheiros et al. / Children and Youth Services Review 33 (2011) 19301938

In addition to these two instruments, we also considered other measurements used to assess the effects of different residences of care on youth (Green & Ellis, 2007; Lemon, Hines, & Merdinger, 2005). Green and Ellis measure evaluates the satisfaction and productivity of structural (e.g., home appearance) and process (e.g., relationships between staff and residents) aspects of service. For the RCYNA, we focused mainly on the process aspects linked to the interaction between staff and residents. Lemon et al. measure covers areas such as educational history, employment history and social support, history of foster care system, skills training, and current life satisfaction, among others. We considered mostly the area of skills training, namely psycho-emotional/social skills. In selecting the dimensions to be included in this study, our principal criteria included the fact that they were common to the different needs assessment methodologies, and covered most dimensions of the Framework for the Assessment of Children in Need and their Families (Department of Health, 2000b), thereby making them consensual and integrating dimensions. Five dimensions were thus identied: 1) living situation; 2) social and family relationships; 3) social and antisocial behavior and skills; 4) physical and psychological health; and 5) education and employment. Living situation refers to the functional, relational, structural and physical conditions of the place of residence and neighborhood in addition to the household. It also refers to the economic conditions, stability and atmosphere experienced in these contexts. Social and family relationships refers to the quality of the young person's relationship with his/her family and non-family members. This dimension also portrays family history and function, such as abuse and domestic violence, the relationships with caregivers/adults in terms of emotions and education, and also refers to the stability of the relationship, guidance and limits in upbringing. The dimension of social and antisocial behavior and skills refers to an assessment of behavior of the young person and his/her family members, the relationship with peers and practitioners, the young person's practical skills (e.g. cleaning his/her room) and cognitive skills (e.g. goal setting and decision making), and institutional aspects such as the young person's involvement in different institutional issues. The dimension of physical and psychological health refers to the physical and psychological state of health of the young person and his/ her family members, risk behaviors and emotional problems. It also refers to health services, the young person and adults' ability to cope with problems, the youth's ability to adapt and his/her identity. Finally, the dimension of education and employment refers not only to the status of youth and family members in terms of education and employment, but also illustrates the young person's status in terms of his/her relationship with colleagues and teachers/employers, issues of satisfaction, motivation and conicts and disciplinary problems in these contexts. It also refers to educational needs, stability and the young person's interests and skills. These dimensions are assessed, in general, by reference to community social contexts and the family. Within the scope of this study, they have been incorporated into the context of residential care and used in the script that guides the focus groups. 4.2. Identication of dimensions relevant to youth 4.2.1. Method 4.2.1.1. Participants. In this study, four focus groups were held including a total of 21 participants, all of them in residential care. These youth were aged 1518 years (M = 16; SD = 1.07); 52% were female. The participants came from 20 residences of care in the Lisbon metropolitan area. To ensure the heterogeneity of the sample and a greater representativity of the data (Krippendorff, 1980), the

participants were chosen according to the following criteria: a) minimum age of 15 years; b) participants of male and female gender; c) participants with and without recorded behavioral problems; and d) at least three years of care. 4.2.1.2. Procedure. The youth who took part in this study were contacted directly and asked about their availability. The focus groups were conducted by three individuals from outside the institutions, two with the role of moderators and the other with the role of outside observer. The focus groups lasted from one to two hours, with the number of participants in each session ranging from four to six. In all of the sessions, the same introductory instructions were given, and the questions were asked in the same order, thereby preventing any biases. In addition, the participants were ensured that all data furnished would be kept condential and anonymous. The participants were also asked for their consent to record the focus groups, which were then transcribed and analyzed. 4.2.1.3. Focus group script. The script used in the focus group includes the ve dimensions described above, including topics involving the living situation (e.g. Tell us a bit about your living conditions), social and family relationships (e.g. What is your relationship like with the residential care practitioners?), physical and psychological health (e.g. What do you think is important to ensure your physical health?), behavior and skills (e.g. Tell us about your independence Describe your characteristics in this area), and education and employment (e.g. We would like to know how you feel about school). The script also included several topics on the young persons' living situation: capacity, privacy, physical space, stability, functioning (rules and services) and neighborhood (community resources); topics involving their social and family relationships: peer group, practitioners and the family or other relationships outside of the institution; and topics involving their behavior and skills: independence, support, etc. It should also be noted that the language used in formulating the questions was adjusted to the participants' age, and the focus groups were conducted so as to identify positive and negative aspects to allow an assessment of the dimensions examined in terms of risk and protective factors. 4.3. Results The data resulting from the focus groups were content analyzed. The rst step in this analysis was to select 834 enumeration and semantic register units of analysis; the second step was to dene a mixed system of categories, i.e. with categories dened a priori and a posteriori (Bardin, 2007). In this regard, we dened a system with 7 dimensions of needs, 25 macro-categories, 56 categories and 24 subcategories; the 7 dimensions of needs were determined a priori by the dimensions common to the different needs assessment methodologies referred to in the script, while with regard to the macrocategories and categories comprising them only 9 macro-categories and 6 categories were dened a priori. The third step was to verify this categorization system's reliability by determining inter-rater agreement at two levels: sequential attribution of the system's lower levels to those immediately above (Cohen's kappa = 0.968; p b 0.000) and attribution of one fourth of the units of analysis of each focus group (241 units) to the lower level of the categories system (Cohen's kappa = 0.814; p b 0.000). The fourth and last step entailed a structural and occurrence analysis of the categories system. Of the seven dimensions of needs identied, the most often recurring were living situation (residence of care; 48.6%) and social and family relationships (25.9%). After this, came education (10.6%), behavior and skills (6.9%), psychological health factors (3.4%), employment (2.6%), and, nally, physical health factors (2%).

M.M. Calheiros et al. / Children and Youth Services Review 33 (2011) 19301938

1933

The dimension of living situation is comprised of seven macrocategories, including the perception of the institution (41.2%), the physical space (22.5%) and the services of the residence of care (12.3%). Less important were the rules at the residence of care (8.6%), the atmosphere of safety at the residence of care (6.7%), the history of care (change of residence) (5%) and the desire to leave the residence of care (3.7%). In terms of the dimension of social and family relationships, the most recurring macro-categories are educators (48.2%) and peer group (37.5%), followed by practitioners 1 (8.3%) and family (6%). Three macro-categories were identied in the dimension of education: adaptation/difculties at school (56.8%), followed by academic learning (21.6%), and nally change of school (frequency) (21.6%). The dimension of behavior and skills includes three macrocategories: independence (84.5%), behavioral problems (8.6%) and search for/acceptance of support (6.9%). Two macro-categories were identied in the dimension of psychological health factors: one regarding factors of the institution (53.6%) and another regarding factors of the youth (46.4%) with inuence on psychological health. The dimension of employment has four macro-categories, including personal development at work (50%), followed by a macro-category referring to work as a positive experience (22.8%), another referring to the functional aspects of work (13.6%) and, nally, a macro-category pointing to difculties at work (13.6%). Finally, the dimension of physical health factors includes two macro-categories: healthy practices (70.6%) and health services (29.4%). In summary, the results of this content analysis underscore the high frequency of references to the living situation at the residence of care, and social relationships with practitioners and peers. This reinforces the importance of the subsystems closest to the youth's ecology as having the greatest potential to impact their life, and being the areas that should be prioritized for the specic instrument to be developed. 4.4. Development of the Residential Care Youth Needs Assessment (RCYNA) instrument The purpose of the RCYNA is to assess the needs of youth in ve areas of their life, namely living situation, social and family relationships, social and antisocial behavior and skills, physical and psychological health and education and employment. In fact, in terms of the focus groups as well as the different needs assessment methodologies, these areas are consensual and capable of integrating different dimensions of the Framework for the Assessment of Children in Need and their Families. In developing the RCYNA, in addition to the data gathered in the focus groups, the Aggregate Data Form (ADF; Little et al., 2002) and the Core Assessment Record (CAR; Gray, 2000) were used as reference instruments in adapting items. Other sources for the operationalization of items included the Education Guardian Law (LTEOfcial Gazette [Dirio da Repblica], Law 166/99 of 14 September), instruments used in assessment studies on residences of care (Green & Ellis, 2007; Lemon et al., 2005), literature on the subject (Bullock et al., 1993; Casas, 1993; Department of Health, 2000b; Valle, 1998) and the results of the discussion group with practitioners in this area, i.e. the operationalization of items that they considered relevant. Table 1 summarizes the origin of the RCYNA's items, according to the different information sources. The majority of the items originate from the focus groups with youth. 110 items resulted from the 68 macro-categories and categories identied. Secondly, the instruments
1 In this context, youth and professionals tend to refer to practitioners as psychologists and social workers, distinguishing them from educators.

analyzed contributed with 95 items. The contributions of the Education Guardian Law and of practitioners (5 items) had the least weight. However, it should be emphasized that some of the variables originating from the focus groups are shared by the other information sources. Of the 110 items operationalized from the focus groups, 59 are similar to the ADF, 11 are similar to the CAR and 6 are referred to in the Education Guardian Law, in other instruments or by the practitioners. In this way, it can be concluded that 34 items originate exclusively from the focus groups, essentially including the dimensions of living situation and social and family relationships, which were the most often cited by the youth. 4.4.1. Content and face validity of an initial version of the RCYNA In view of the goal to assess needs considering the presence of risk and protective factors in the ve dimensions mentioned, a content assessment of the RCYNA was conducted by a group of researchers in this area. In all, 109 risk items, 68 protective items and 33 neutral items were classied (see Table 2). The results of this assessment clearly show a greater preponderance of items involving risk factors compared to items involving protective factors, namely in the dimension of living situation (e.g., family is dependent on benets) and in the dimension of physical and psychological health (e.g., child pleasant to spend time with). Items involving protective factors appear to be essentially tied to the dimension of behavior and skills. Finally, the dimension of education and employment includes the highest number of items considered as neutral. In order to ensure face validity, the instrument's practicality, the pertinence of the items and the instrument's suitability to the context, the RCYNA was also discussed with a group of practitioners with different occupations (psychologists, social workers and educators) and performing different duties (direct work with young person or coordination and direction). The majority of practitioners expressed a high degree of comprehension of the RCYNA items and its adaptability to the context of residential care. Nevertheless, practitioners proposed some changes to the makeup of the original items, and these were taken into consideration in the nal version of the instrument. In addition, the instrument was pre-tested with a sample of practitioners from social/educational teams and care units evaluating 20 youth in residential care. The results of this pre-testing allow arguing for the RCYNA's suitability for assessing the needs of youth in this context. In its nal version, the RCYNA should be completed by the technical/educational team responsible for the young person, not only to minimize potential errors with regard to the assessment's subjectivity, but above all because youth are assessed in areas entailing specic areas of care by practitioners with different functions, which may result in a unique and specialized contribution in lling in the instrument. 4.4.2. Description of the nal version of the RCYNA The RCYNA is comprised of two parts: one in reference to the socio-demographic description of the young person, and another in reference to the assessment of his/her needs. As regards the sociodemographic description, in addition to age, ethnic group and gender, information is gathered on the young person's duration of care, the reason for admission to the institution and who reported the young person. The RCYNA's needs assessment component is comprised of 211 items: 210 items refer to risk and protective factors, and are distributed among the ve dimensions mentioned above, and one item (adapted from the ADF) refers to risk thresholds to measure the gravity or seriousness of the young person's problem (Is the impairment signicant or is it likely to be signicant if the situation persists?).

1934

M.M. Calheiros et al. / Children and Youth Services Review 33 (2011) 19301938

Table 1 Origins of the different items composing the different dimensions of the RCYNA. Source Number of items per dimension Living situation Focus groups FDA CAR Other instruments LTE and technicians 15 21 0 2 0 Social and family relationships 16 15 0 2 0 Behavior and skills 34 8 0 5 4 Physical and psychological health 31 15 2 1 0 Education and employment 14 23 0 1 1 110 82 2 11 5 Total

The rst dimension, living situation, comprised of 38 items, provides an assessment of the family's household and the residence of care in terms of residents, physical and economic conditions, location, stability, atmosphere and functioning, among other aspects (e.g. The residence of care's schedule is very strict; The residence of care functions much like a family environment; Overall, the institution satisfactorily addresses the young person's living needs). The dimension of social and family relationships considers the young person's relationship with family members and other significant people in his/her life, the family's functioning and history, the relationship with the technical/educational team, stability, the ability of the family and the young person to cope with problems, and other aspects. This dimension is comprised of 33 items (e.g. There is turnover in the technical/educational teams responsible for the young person; Young person recently ill-treated; and Overall, the institution promotes positive relationships between the young person and his/her family). The dimension of social and antisocial behavior and skills assesses the young person's relationship with peers and professionals, and the existence of behavioral problems of the young person and his/ her family members. It also includes a number of personal skills, from cognitive to practical/functional, and aspects of the residence of care, such as participation. This dimension incorporates 51 items (e.g. Has money beyond the monthly allowance; Is capable of making his/her own decisions; Makes requests, complaints and/or claims). The dimension of physical and psychological health incorporates 49 items, and assesses, for the young person as well as adults, the existence of physical or mental illnesses, risk and health behaviors and psychological or emotional problems. It also considers overall well-being, identity and several skills of the young person, among other factors (e.g. Tries to adapt or is adapted to his/her situation of care; Is capable of managing his/her emotions; Overall, the institution satisfactorily addresses the physical and psychological health needs of the young person). Finally, the dimension of education and employment describes the academic and professional situation of the young person and of adults, special education needs, absenteeism, integration problems and conicts, relationships with teachers and colleagues, the family's involvement in education, the young person's motivation and abilities, etc. This dimension is comprised of 39 items (e.g. Isolated at school or at work; Involved in extracurricular activities; and Overall, the institution satisfactorily addresses the young person's education and employment needs). By applying this instrument, a score can be obtained for each dimension assessed, involving the overall sum of the number of risk factors and the overall sum of protective factors present in the dimension. Total risk and protective scores can also be calculated, resulting from the sum of these factors in all of the dimensions. The scores obtained with this instrument can identify areas with greater urgency of care, thereby providing a comparison of the seriousness of different groups. Moreover, risk and protective factors may be analyzed independently to ascertain and implement youth care. This analysis may be undertaken by combining data and

identifying needs proles, as determined by the existence and absence of risk and protective factors. 5. Study 2 The purpose of this second study was to determine several psychometric qualities of the RCYNA, namely reliability, face validity and concurrent validity. The correlations between the dimensions comprising this instrument were also analyzed. To this end, the instrument was applied to teams of practitioners using a sample group of youth in residential care as a reference. Is should be noted that a specic analysis testing the factorial structure of the RCYNA was not performed. In fact, and due to the dichotomous nature of the RCYNA items, a latent class analysis should have been run (McCutcheon, 1987). Nevertheless, sample specicities and especially sample size prevented us from presenting this kind of evidence (it is known that small samples cause great instability to solutions provided by conventional latent class analysis; Yang, 2006). We are thus basing our assumptions regarding the RCYNA structure on the content analysis presented in Study 1, and we are further reinforcing the internal consistency of its dimensions through the calculation of internal consistency indexes (presented further down). Moreover, and as it can be seen from the evidence presented in Study 1, the dimensions underlying the RCYNA do not depart much from other instruments widely used in this area of study (e.g., ADF and CAR). 5.1. Method 5.1.1. Participants 101 youth from 22 residential units, aged 1423 years and with an average age of 16 years (SD = 1.8) were evaluated via the RCYNA. 43.7% were female. The majority of these youth were of Portuguese descent (69.9%), and the remaining of Portuguese African descent. These youth had lived in their respective residences of care for an average of eight years (SD = 3.56). These youth were assessed by a group of 47 practitioners, mostly female (70.2%), aged 2853 years, and with an average age of 37 years (SD = 7.94). With regard to the occupations of these practitioners, more than half were educators (55.3%), followed by social workers (23.4%) and psychologists (19.1%). In terms of their academic

Table 2 Number and type of items per dimension of the RCYNA (risk and protecting factors). Dimensions Factors Risk Living situation Social and family relationships Behavior and skills Physical and psychological health Education and employment Total 23 15 20 37 14 109 Protective 9 12 26 12 9 68 Neutral 6 6 5 0 16 33

M.M. Calheiros et al. / Children and Youth Services Review 33 (2011) 19301938

1935

background, 74.5% of the practitioners were degree holders, and 14.9% completed secondary education. It is important to note that these practitioners had worked at the institutions for an average of 9.51 years (SD = 6.90). Each young person was assessed by a team of practitioners ranging in size from 2 to 3 team members. 5.1.2. Instrument 5.1.2.1. Residential Care Youth Needs Assessment (RCYNA) instrument. In addition to the RCYNA, comprised of its two main parts (sociodemographic description and needs assessment), a third part was introduced to determine its concurrent validity. This third part is comprised of 32 items involving the services provided to the young person (12 items; e.g., youth mental health, and special education) and to his/her family (20 items; e.g., adult medication and nancial support). Two scores were calculated for these items: one for the services provided to the family, and another for the services provided to youth. Both stemmed from the total number of services provided, thereby ranging from 0 to 20, and 012, respectively. Finally, an item was introduced to assess the consistency/ agreement of the technical/education team while lling in the RCYNA How often did the team that completed the form disagree with the responses? with a response scale ranging from never (0 times) to frequently (more than 20 times). The items identied by the group of researchers as neutral (see Study 1) were not taken into consideration in the nal version of the RCYNA. Indeed, the aim of this instrument is to measure youth needs by identifying risk and protective factors. In this sense, neutral items do not directly contribute to the operationalization of these types of factors underlying the main dimensions of the RCYNA, and are therefore negligible. For this reason, the RCYNA instrument analyzed in the present study comprises 177 items, instead of its original 210 items. 5.1.3. Procedure Since the questionnaire is completed by practitioners instructions were given on how to ll out the RCYNA. Next, the questionnaires were handed out to the practitioners in charge of the residential care units to be subsequently distributed to, and completed by, the young persons' technical/education teams. To ensure the condentiality and anonymity of the data, instructions were given not to disclose the young persons' and practitioners' identities. All the questionnaires were put into a sealed envelope, which was then collected, obtaining a return of 100% of the questionnaires. 5.2. Results 5.2.1. Analysis of the psychometric qualities of the RCYNA 5.2.1.1. Reliability. The reliability of the RCYNA was assessed to demonstrate the internal consistency of risk and protective factors within each needs dimension assessed by calculating Cronbach's alpha coefcients. These analyses resulted in the removal of some items, since these contributed to the dimensions' inconsistency. In the dimension living situation (protective factor variant), 3 items were removed (e.g., the residence of care functions much like a family environment and youth prots from the available community resources); in the dimension social and family relationships (protective factor variant), 2 items were removed (youth understands family's problems, members of family have skills to deal with the problems faced by the youth); and in the dimension physical and psychological health (protective factor variant), 1 item was removed (youth perceives the institution is worried with his/her well-being); and nally, in the dimension education and employment (risk factor variant), 3 items were removed (e.g., permanently excluded from school and not serious special education needs). In

this way, the nal version of the RCYNA included 168 items, which represents a difference of around 5% in relation to all items in the initial version resulting from Study 1. 2 In the end, risk and protective factors were obtained with acceptable to high internal consistency: living situation (risk factor, = 0.71; protective factor, = 0.66); social and family relationships (risk factor, = 0.70; protective factor, = 0.63); behaviors and skills (risk factor, = 0.71; protective factor, = 0.75); physical and psychological health (risk factor, = 0.72; protective factor, = 0.66); education and employment (risk factor, = 0.60; protective factor, = 0.75). 5.2.1.2. Face validity. In order to demonstrate the RCYNA's face validity, we analyzed the item measuring the disagreement among practitioners at the time of lling out this instrument. By analyzing all of the points of this item's underlying scale, we can see that in 11.7% of the cases there was no disagreement, in 52.4% of the cases disagreement was rare (15 times), in 31.1% of the cases it was occasional (610 times), in 1 case it was sporadic (1115 times), and at no time was it regular (1620 times) or frequent (more than 20 times). We can therefore conclude that the completion of this item raised little disagreement among the evaluators, thereby demonstrating its comprehensibility, clarity and ease of completion. 5.2.1.3. Descriptive analysis of dimensions of the RCYNA. Having established the internal consistency of the dimensions of the RCYNA, along with its face validity, indicators were created for each of these dimensions. These indicators resulted from the total sum of participants' individual scores in the items comprising each of these dimensions. The range of these scores can be seen in Table 3. Note that higher risk factor values and lower protective factor values indicate situations of greater need. An analysis of the scores obtained in the RCYNA (see Table 3) in terms of risk and protection reveals that, on average, these youth have higher scores in protective factors (M = 9.35), and lower scores in risk factors (M = 4.56), F (1,71) = 160.41, p b 0.000, = 0.69. We can also 2 see that, in a general sense, the participants' scores in the various dimensions differ signicantly, F (4284) = 391.92, p b 0.000, = 0.85. 2 We also analyzed the factors in terms of their distributions (see Table 3). Looking at the skewness and kurtosis values for each dimension, and the ratio between these and the respective errors, we can see that the majority of the dimensions (whether risk or protective factors) have negative symmetry values, i.e. with distributions slightly biased to the right. The dimensions of behavior and skills and physical and psychological health (variant of risk factors) deviate from this pattern with positive symmetry values, i.e. with scores slightly biased to the left. In addition to this, when we look at the ratios between symmetry and the respective standard error, we can see that some of these dimensions deviate slightly from normality. In fact, the dimensions of living situation and social and family relationships (variant of protective factors), and the dimension of education and employment (variant of risk factor) show high ratios that bear out this fact. In the latter dimension we can also see that mean responses value come close to its standard deviation, which may be a sign of deviation from normality. When we analyze the values obtained for the kurtosis, we can see that the majority of the dimensions are slightly platykurtic, i.e. with negative kurtosis values. The dimensions of living situation and family relationships (variant of protective factors), as well as the dimension education and employment (variant of risk factor), show positive kurtosis values, i.e. their distributions are slightly leptokurtic.

The RCYNA is available upon request to the rst author of this article.

1936

M.M. Calheiros et al. / Children and Youth Services Review 33 (2011) 19301938

Table 3 Descriptive measures of the RCYNA (risk and protective factors). Dimensions Living situation Social and family relationships RF 6.68 2.85 0.00 12.00 0.36 1.29 0.53 0.94 PF 8.35 1.63 3.00 10.00 1.09 3.84 0.93 1.67 Behavior and skills RF 3.10 2.67 0.00 11.00 0.74 2.60 0.16 0.29 PF 21.03 3.39 13.00 26.00 0.52 1.82 0.62 1.12 Physical and psychological health RF 5.99 3.89 0.00 15.00 0.43 1.51 0.51 0.92 PF 7.32 2.00 3.00 11.00 0.14 0.49 0.42 0.76 Education and employment RF 1.13 1.28 0.00 5.00 1.26 4.47 1.15 2.06 PF 5.04 2.15 0.00 8.00 0.50 1.77 0.55 0.99 22.79 9.32 4.00 42.00 0.05 0.18 0.49 0.88 46.75 8.28 25.00 59.00 0.69 2.45 0.35 0.62 Total risk Total protective

Factors Mean Standard-deviation Minimum Maximum Symmetry Ratio symmetry/standard error Kurtosis Ratio kurtosis/standard error

RF 5.90 3.24 0.00 14.00 0.14 0.50 0.61 1.10

PF 5.01 1.26 2.00 6.00 1.15 4.07 0.22 0.39

Note: RF = Risk factors; PF = Protective factors.

Overall, the distribution behavior of the RCYNA's dimensions, in the areas of risk factors and protective factors alike, was acceptable with regard to deviations from normality, with their associated descriptive statistics falling within expected values. 5.2.1.4. Correlations between risk factors and protective factors in dimensions of the RCYNA. With regard to inter-dimension correlations, as shown in Table 4, the risk factors have moderate to high correlations, with an identical pattern seen in correlations between protective factors. However, and in the domain of risk factors, the associations between living situation, education and employment and social and family relationships, together with associations between this last factor and the dimension of education and employment, were not signicant. As expected, the associations between risk and protective factors, overall, were negative, i.e. the greater the presence of certain types of risk factors, the lower the presence of protective factors. In general, we can claim that these correlations attest to the solidity of this instrument's underlying construct, together with its ability to discriminate with regard to risk factors and protective factors. Finally, as shown in Table 5, when we analyze the correlations between each dimension (variants of risk or protective factors) with the total risk or protection score. We can see, as expected, that risk factors correlate in a high and positive manner with total risk score, and that protective factors correlate positively with the total protection score. Conversely, we can see negative correlations between risk factors and the total score in terms of protection, and between protective factors and the total risk score. In addition to their direct interpretation, these correlations also bear out the solidity of the operationalization of the needs construct, showing that this

instrument has sufcient sensitivity to differentiate scores obtained in terms of risk factors versus protective factors, and to reconcile factors that share the same conceptual base. 5.2.1.5. Concurrent validity. The criterion-related validity of the RCYNA (in its variant of concurrent validity) was analysed through the relationship between the intensity of risk factors present in the young person's life (total risk factor score) and the number of services provided to the young person (total sum of scores obtained for items involving the number of services used by the young person), since, theoretically, it can be expected that youth with more risk factors will be subject to more care. According to this hypothesis, we can see that the variable involving total risk factors has a positive and moderate correlation with the total services provided to the young person, r = 0.38, p = 0.000. In this way, high risk factor scores in the RCYNA allow us to predict that youth will likely use a higher number of care services. 6. Discussion The purpose of this work was to build a Residential Care Youth Needs Assessment (RCYNA) instrument containing the key dimensions for assessing needs, reecting the methodological approaches already laid out in the literature and, above all, including the viewpoint of youth to tailor the instrument to this context and to its target of assessment. In addition, efforts were made toward the initial assessment of the RCYNA by analyzing some of its psychometric characteristics. Although the instrument has been developed based on different conceptual and operational sources, as demonstrated in Study 1, the majority of the items comprising the RCYNA originated in the focus

Table 4 Correlation matrix of the dimensions of the RCYNA (risk and protective factors). Risk factors Risk factors: Living situation (LS) Social and family relationships (SFR) Behavior and skills (BS) Physical and psychological health (PPH) Education and employment (EE) Protective factors: Living situation (LS) Social and family relationships (SFR) Behavior and skills (BS) Physical and psychological health (PPH) Education and employment (EE) p b 0.05. p b 0.01. p b 0.000. LS SFR 0.11 BS 0.28 0.39 PPH 0.36 0.25 0.43 EE 0.07 0.22 0.47 0.27 Protective factors LS 0.20 0.43 0.43 0.39 0.30 SFR 0.01 0.33 0.46 0.40 0.26 0.57 BS 0.22 0.22 0.52 0.52 0.28 0.52 0.60 PPH 0.05 0.23 0.43 0.52 0.30 0.26 0.49 0.57 EE 0.24 0.34 0.60 0.48 0.38 0.41 0.53 0.52 0.60

M.M. Calheiros et al. / Children and Youth Services Review 33 (2011) 19301938 Table 5 Correlation matrix of the dimensions of the RCYNA and the total scoring on risk and protective factors. Total risk factors Risk factors Living situation (LS) Social and family relationships (SFR) Behaviour and skills (BS) Physical and psychological health (PPH) Education and employment (EE) Protective factors Living situation (LS) Social and family relationships (SFR) Behaviour and skills (BS) Physical and psychological health (PPH) Education and employment (EE) p b 0.000. p b 0.01. 0.62 0.59 0.75 0.78 0.47

1937

Total protective factors 0.19 0.36 0.63 0.60 0.37 0.65 0.78 0.88 0.77 0.78

0.53 0.44 0.55 0.47 0.61

groups with youth in residential care, which underscores the importance of gathering information from the assessment's targets. This methodology allowed variables to be identied, giving greater specicity and singularity to the instrument, and making it suitable to the interests of those it assesses. In fact, despite the existence of several instruments that may be applied to these types of youth (e.g. CAR), none were specically developed for this type of population, thus not assessing particular aspects of youth in care, such as those involving the residences of care or the youths' relationships with practitioners. In Study 2, we analyzed the psychometric qualities of the RCYNA. The results show suitable levels of reliability, face validity, construct validity and concurrent validity. With regard to reliability, we were able to show that the internal consistency of risk and protective factors in the various dimensions of needs was high, despite the removal of some items to obtain internally consistent factors. Indeed, this procedure generated some item mortality vis--vis the initial version, although quite small and therefore negligible. Moreover, a descriptive analysis of these dimensions showed that they follow a normal (or near normal) distribution. The exceptions to this were the dimensions of living situation and social and family relationships (variant of protective factors) and the dimension of education and employment (variant of risk factor), which showed slightly asymmetrical distributions. In a future application of this scale, it would be useful to review the items that comprise it in order to surpass this situation. In addition, the majority of the distributions show a negative skewness, indicating quite high risk and protective factor averages alike. Nonetheless, and as we noted, the youth who took part in the second study show higher average scores in protective factors than in risk factors. The averages of most of these dimensions (variants of risk and protective factors) also differ between themselves, which shows this scale's sensitivity to measure different facets of the needs of these youth. With regard to the inter-dimension correlations, in general, we observed positive associations among the risk factors and negative associations between the risk and protective factors. As we expected, these correlation standards show that youth with more risk factors in a dimension are more likely to have more risk factors and less protective factors in the other dimensions. These results are in line with that indicated in the literature (e.g. Little et al., 2004), in that it is not only a risk factor, but the interaction between various risk factors that affects the development of children and youth, via dynamic processes and chains of direct and indirect effects, while protective factors generally reduce or eliminate the negative effect of risk factors, thereby with the expectation of a negative relationship between these. Ultimately, the correlation patterns that we found follow the theoretical postulates involving the needs of youth, thereby ensuring

the validity of the underlying constructs of the different dimensions of the RCYNA. The RCYNA's concurrent validity was shown via the positive correlation between the total number of risk factors present in the youth's life and the amount of care to which the youth is subjected. This indicates that when youth are affected by more risk factors, they are subject to more care, or given more support services. This relationship proves the instrument's ability to predict situations of future care, in addition to assessing the seriousness of the youth's situation considering his/her needs in terms of the presence/absence of risk and protective factors. Despite the merit and practical pertinence we have given to this work, we must still consider some of its limitations and make several recommendations for future research with this instrument. In general terms, it should be emphasized that, although the instrument incorporates aspects that youth consider important in assessing their needs, its completion by practitioners does not allow us to assess the youths' perspective. Therefore, one recommendation arising from this work will be to develop a version of the RCYNA to be completed by youth. The development of such a version will provide insights into their perspectives, and will allow a comparison between self-assessments and technician assessments to gauge the congruence between both perspectives. However, it is important to note that this participation may occur after the assessment of needs, in terms of establishing specic goals for the design and development of programs, as it has proven essential for it to achieve change more effectively (Kirby, Lanyon, Cronin, & Sinclair, 2003). Furthermore, various studies have stated that children/youth generally have a positive view of the opportunity to take part and discuss the various issues involving them (Cashmore, 2006). Second, it is important to note that the response to the items is dichotomous, which may lead to some constraints in terms of analyzing and interpreting data (namely in conducting a more indepth test of this instrument's factorial structure and subsequent conrmation), and in forming proles. It would also be advisable to condense the number of items in the instrument (to lessen the potential effects of fatigue in completing it), as well as useful to ensure that the various dimensions of needs are assembled using the same number of risk and protective factors (to facilitate comparison). Finally, we stress the importance of obtaining more evidence concerning its predictive value, since it is important for this instrument to provide a basis for developing services for youth in residential care, both in terms of care, prevention, and planning. In fact, RCYNA criterion-related validity should be tested against criteria such as evaluation of outcomes in different areas of youth life and well-being or the assessment of youth success in educational, professional, and social integration areas.

1938

M.M. Calheiros et al. / Children and Youth Services Review 33 (2011) 19301938 Dirio da Repblica I Srie A. Lei Tutelar Educativa - Lei 166/99, de 14 de Setembro. Gough, I. (2003). Lists and thresholds: Comparing the Doyal-Gough theory of human need with Nussbaum's capabilities approach. WeD Working Paper #1. UK: University of Bath. Gray, J. (2000). Core assessment record. London: The Stationery Ofce Books. Green, R., & Ellis, P. (2007). Linking structure, process, and outcome to improve group home services for foster youth in California. Evaluation and Program Planning, 30, 307317. Kirby, P., Lanyon, C., Cronin, K., & Sinclair, R. (2003). Building a culture of participation: Involving children and young people in policy, service planning, delivery and evaluation. Nottingham: Department for Education and Skills Publications. Krippendorff, K. (1980). Content analysis: An introduction to its methodology. Beverly Hills, CA: Sage. Lemon, K., Hines, A., & Merdinger, J. (2005). From foster care to young adulthood: The role of independent living programs in supporting successful transitions. Children and Youth Services Review, 27, 251270. Little, M., Axford, N., & Morpeth, L. (2002). Aggregating data: Better management information and planning in children's services. Totnes: Warren House Press. Little, M., Axford, N., & Morpeth, L. (2004). Risk and protection in the context of services for children in need. Child and Family Social Work, 9, 105118. Little, M., & Mount, K. (1999). Prevention and early intervention with children in need. Aldershot: Ashgate Pub Ltd.. Little, M., & Mount, K. (2003). Paperwork: The clinical assessment of children in need. Totnes: Warren House Press. McCoy, H., McMillen, J., & Spitznagel, E. (2008). Older youth leaving the foster care system: Who, what, when, where, and why? Children and Youth Services Review, 30, 735745. McCutcheon, A. C. (1987). Latent class analysis. Beverly Hills: Sage. Nettles, S., & Pleck, J. (1994). Risk, resilience, and development: The multiple ecologies of black adolescents in the United States. In R. J. Haggerty, L. R. Sherrod, N. Garmezy, & M. Rutter (Eds.), Stress, risk, and resilience in children and adolescents: Processes, mechanisms, and interventions (pp. 147181). Cambridge, NY: Cambridge University Press. Raymond, I., & Heseltine, K. (2008). What does it mean to be an adult? Perceptions of young men in residential care. Child and Youth Care Forum, 37, 197208. Swenson, M., & Swenson, C. (2002). Panormica da investigao em terapia multissistmica [An overview on multisystemic therapy research]. In A. Castro Fonseca (Ed.), Simpsio internacional comportamento anti-social: Escola e famliaInternational symposium on antisocial behaviour: School and family. Coimbra, Portugal: Universidade de Coimbra. Taylor, K. (2005). Understanding communities today: Using matching needs and services to assess community needs and design community-based services. Child Welfare League of America, 84, 251264. Valle, J. (1998). Manual de programation y evaluacin para los centros de proteccin a la infancia [Handbook of programming and evaluation of child care centers]. Salamanca: Junta de Castilla y Len. Ward, H., & Rose, W. (2002). Approaches to needs assessments in children's services. London: Jessica Kingsley Publishers. Werner, E., & Smith, R. (1982). Vulnerable but invincible: A longitudinal study of resilient children and youth. New York: McGraw-Hill. Yang, C. (2006). Evaluating latent class analysis models in quantitative phenotype identication. Computational Statistics & Data Analysis, 50, 10901104.

References
Aldgate, J., & Statham, J. (2001). The children act now: Messages from research. London: The Stationery Ofce. Axford, N., & Little, M. (2004). Meeting needs or protecting rights: Which way for children services? Totnes: Dartington Social Research Unit. Axford, N., Little, M., Morpeth, L., & Weyts, A. (2005). Evaluating children's services: Recent conceptual and methodological developments. British Journal of Social Work, 35, 7388. Bardin, L. (2007). L'analyse de contenu [Content analysis]. Paris: Presses Universitaires de France. Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge: Havard University Press. Bullock, R., Little, M., & Millham, S. (1993). Residential care for children: A review of research. London: HMSO. Calheiros, M. M. (2008). Program design and evaluation for at-risk children and youth: Preventing developmental impairment. In C. Canali, T. Vecchiato, & J. K. Whittaker (Eds.), Assessing the evidence-based of intervention for vulnerable children and their families (pp. 538540). Padova: Fondazione Zancan. Calheiros, M., Garrido, M., & Rodrigues, L. (2009). Percorsi di autonomia: Una ricercaintervento portoghese [A route to autonomy: A Portuguese research-intervention]. In S. Premoli (Ed.), Verso l'autonomia: Percorsi di sostegno all'integrazione sociale di giovani [Towards autonomy: Pathways to support the social integration of youth] (pp. 96126). Milano: Franco Angeli. Calheiros, M. M., Seabra, D., & Fornelos, M. (1993). The nature and research of residential and foster care provision in Portugal. In M. Colton, & W. Hellinckx (Eds.), Child care in countries of the European community (pp. 177195). Avebury: Gower Publishing Group. Casas, F. (1993). Changing paradigms in child residential care. Paper presented at the Third European Scientic Congress on Residential and Foster Care. Lneburg, Germany. Cashmore, J. (2006). Ethical issues concerning consent in obtaining children's reports on their experience of violence. Child Abuse & Neglect, 30, 969977. Cicchetti, D., & Rizley, R. (1981). Developmental perspectives on the etiology, intergenerational transmissions, and sequelae of child maltreatment. New Directions for Child Development, 11, 3155. Colca, L., & Colca, C. (1996). Transitional independent living foster homes: A step towards independence. Children Today, 24, 711. Daining, C., & DePanlis, D. (2007). Resilience of youth in transition from out-of home care to adulthood. Children & Youth Services Review, 29, 11581178. Dartington Social Research Unit (1998). Towards a common language. Totnes: Dartington Social Research Unit. Department of Health (1995). Looking after children: Assessment and action records, essential information records, care plans, placement plans and review forms. London: HMSO. Department of Health (2000a). Assessing children in need and their families: Practice guidance. London: The Stationery Ofce Ltd.. Department of Health (2000b). Framework for the assessment of children in need and their families. London: The Stationery Ofce Ltd.. Department of Health (2002). Integrated children's system working with children in need and their families. London: Consultation Document.