The Role of Child and Youth Care Practitioners in Evidence - Based Practice in Group Care: Executive Summary
By Dr. Carol Stewart and Dr. Larry Saunders

The purpose of this project was to understand the factors affecting the use of Evidence-Based Practice (EBP) and Evidence-Based Treatment (EBT) in residential group care programs in the province of Ontario. The strategies and interventions used by Child and Youth Care (CYC) practitioners in the milieu were linked to EBT and theoretical models of intervention through a review of current literature and a survey of all group care programs serving children and youth with emotional and behavioural difficulties was undertaken. The analysis included consideration of differences in service delivery between child welfare, children’s mental health programs and the children’s private residence sector.

Overall the results of the survey indicate that:

• Interventions that are used by CYC practitioners are theoretically linked to EBTs that have demonstrated positive outcomes for children, youth and families.

• Manuals and specific procedures for implementing and evaluating these interventions in group care are not clearly developed in most programs although there is strong potential to develop these supports.

• An understanding of EBP and EBT has permeated into residential group care programs with most programs following standardized case planning procedures.

• There is differential knowledge and implementation of EBP within the various sectors that provide residential services for children and youth. Implementation of EBP was affected by organizational and individual factors, such as funding, access to information, and educational preparation of front-line workers. There were differences in these factors between children’s mental health and child welfare programs as well as among the private children’s residence programs.

• More research to demonstrate the specific connections between group care models, CYC practitioner interventions and client outcomes would lead to the development of EBT models in group care that could be replicated.

• More research on the factors supporting implementation and replication of essential ingredients in group care is required for group care to demonstrate the relationship between practitioner interventions and client success (outcomes).

Design of the Project

A survey research design was used to explore the ways in which front-line CYC practitioners understand and use EBT and EBP as well as to describe the scope of practice of CYC practitioners in group care programs. Program managers and front-line CYC practitioners in group care programs received a survey in the mail. Differences among child welfare (CAS), children’s mental health (CMH), and private operator (PCR) programs were assessed. The survey replicated previous work (Barwick et al, 2005) that described to what extent EBP had been adopted in the children’s mental health sector and extended the analysis to child welfare and private operators.

The survey sample was structured at the program level. All staffed group care programs in the province were invited to participate, and the survey was distributed to CYC practitioners and program managers in residential treatment centres and group homes that were: (a) licensed as a children’s residence, (b) worked with clients with mental health needs or diagnoses, (excluding homes whose primary focus was children who were medically fragile, had moderate developmental disabilities, autism or were placed under the Criminal Youth Justice Act) and (c) were staffed by CYC practitioners. Foster care programs were excluded. A total of 262 programs received the surveys and 178 programs returned at least one survey. Each program received a survey for the program manager and 2 to 4 surveys for the front-line CYC practitioners. A total of 1218 surveys were distributed and 495 were returned.

Responses from the program manager of each group care program were used to divide the group care programs into 2 groups: Those that were committed to EBP and already implementing the key factors involved in such an approach (the high uptake group) and those programs that were struggling to implement EBP (the low uptake group). Statistical differences between High and Low Uptake programs were explored to understand how case management practices and CYC interventions differed among these two groups. Statistical analysis also explored the differences in scope of practice among CYC practitioners as a result of education, experience, and employment sector.

Results and Recommendations

1. CYC Practitioner’s Scope of Practice has a knowledge base which needs to be developed into a documented EBT through additional research related to client outcomes.

Recommendation: Further research should be undertaken with group care programs with a High Uptake of EBP who have CYC trained practitioners to document client outcomes as they are related to CYC interventions.

2. The project was successful in piloting an instrument which measures the frequency of child and youth care intervention strategies that have demonstrated connections to the EBT and child and youth care literature. It also demonstrated our ability to measure the use of specific case management practices (EBP) demonstrated to be effective in enhancing children’s outcomes.

Recommendation: A more detailed analysis of the responses could uncover those specific aspects of the CYC interventions that are influenced by organizations or developed with additional years of experience. If indeed certain interventions have a greater likelihood of influencing children’s outcomes and they are mediated by organizational factors or by education and experience, such an understanding could guide organizational hiring practices.

Recommendation: The instrument has the potential to be used to bring consistency to the adaptation of EBT strategies in the group care sector (similar to the design of Wraparound, Multi-systemic Therapy, and Teaching Family Model). Further work is needed to refine the instrument and to develop a mechanism for assessing the outcomes of children and youth consistently across programs that are using these group care models.

3. EBP is used by CYC Practitioners in Group Care and EBT strategies were reported. Program managers and front-line CYC practitioners are familiar with the terminology of evidence-based practice and evidence-based treatment in all sectors.

Recommendation: The examples provided by respondents should be developed into questions to add to the instrument to capture the nature of behavioural and relationship oriented interventions.

Recommendation: It would be useful to determine which group care programs have specific planned curriculums for life skills or social skills, how these are integrated with daily living interventions (for transfer of training) and to assess specifically the outcomes of these curriculums using pre-and post existing measures such as OnLAC or built-in plan of care assessments to determine the impact and the essential ingredients. These would then meet the issues and concerns that need to be addressed. The Ontario Association of Children’s Aid Societies (OACAS), Ontario Association of Residences Treating Youth (OARTY) and MCYS should work together to identify those agencies that need extra support and to provide training and additional support for research.

Recommendation: Observing more specifically what techniques are being implemented in both individual and group sessions using strategies from Narrative Therapy, Solution Focused Therapy, SNAP, and COPE etc. would more clearly define how these approaches are being modified (if at all) and the effect on client outcomes.

Recommendation: Whittaker (2004) strongly recommended the co-location of family and residential programs to bring these aspects together and given the informal work that CYC practitioners are already doing with families, such co-location offers great benefits to group care programs and enhances the communication between family therapists and CYC practitioners working with children and youth. Programs engaged in this practice should be evaluated for client outcomes.

4. Case Management Practice is slightly different in programs that are ready to apply research-based knowledge

Recommendation: Refine the instrument to identify both common and different elements of case management and then use the survey with the selected High Uptake of EBP programs and collect children’s outcomes to identify which items are good practice that reflect a more intensive case management process and therefore contribute to enhanced outcomes. High Uptake EBP programs already have the capacity to undertake this type of assessment and data collection and the relative contributions of the particular case management factors can therefore be determined.

5. Cross Sector analysis indicates there are more similarities than differences, particularly in the organizational characteristics related to the Uptake of Evidence-Based Practice

Recommendation: Given the Ministry of Children and Youth Services (MCYS) direction toward EBP/EBT and the results of this survey among group care programs, CAS sector programs need significant support to implement EBP but are aware of the issues and concerns that need to be addressed. The Ontario Association of Children’s Aid Societies (OACAS), Ontario Association of Residences Treating Youth (OARTY) and MCYS should work together to identify those agencies that need extra support and to provide training and additional support for research.

6. In spite of the similarities in types of clients served, general services provided, referral sources etc. there are significant differences in the type of treatment interventions used by agencies operating staffed group care programs based on their funding source and primary service population.

Recommendation: The survey should be repeated with parented group care, treatment foster care and young offender facilities, with a more supportive methodology (e.g. phone survey) to determine differences in support, interventions, case management, and types of clientele as well as examining how families are involved in the lives of children that are placed in out-of-home care in order to maximize the potential for successful outcomes following return home.

Recommendation: More specific investigation of how staffed group care programs incorporate family focused EBT’s and how they make use of informal family contact is warranted. Given that these services are less likely to be "prescribed" on the plan of care, there is a significant difference between the sectors in their reported provision of family counselling and that family involvement is highly recommended in the literature as one of the factors influencing successful outcomes further investigation is essential.

7. There are some differences in human resources between the private and quasi-public sector of group care service providers. Employee’ education and experience varies and is modified by the nature of the sector that the group care practitioner is employed in.

Recommendation: Factors such as education and experience of group care staff should be assessed and considered in program’s ability to implement EBP and EBT. Additional support to programs in the PCR and CAS sectors may be required.

Conclusion

The purpose of this project was to understand the factors (systemic, organizational, and human) affecting the use of EBP and EBT in group care settings in the province of Ontario and to document the strategies and interventions that CYC practitioners use in the milieu which are supported by EBT and theoretical models of intervention. Both aspects of this purpose have been partially accomplished. The instrument developed has good face validity and the scope of practice that it assesses appears to be differentially affected by CYC education, years of experience, and employment sector. The instrument needs to be revised, but has the potential to provide a tool for assessing both case management practice and the interventions used in group care programs and identifying which ones are most clearly connected with client outcomes.

We have a beginning understanding of the factors affecting the use of EBP and EBT in group care programs. It is clear that the MCYS policy and encouragement of the use of EBT for service provision in the children’s mental health sector has influenced both the CMH agencies receiving transfer payments to deliver group care services and the agencies receiving per diem funding, generally known as the private operators but more appropriately termed Private Children’s Residences (PCR). It is also clear that being a PCR versus a CMH centre does not affect the programs ability to access, assess, adapt, or apply evidence-based practices in group care. Programs that are already engaged in EBP are poised to be able to help us describe and further identify clusters of milieu-based interventions which can be demonstrated to affect the outcomes of children and youth in group care and to be replicated in other programs to enhance the quality of care and service overall.

Organizational factors such as the ability to seek consultation, to identify and implement appropriate modifications to research-based knowledge, and the capacity of organizations to engage in change vary across the employment sectors. The education and experience levels of both program managers and CYC practitioners also varied across the employment sectors making system wide implementation of an EBP/EBT policy difficult without addressing these inequities and providing support to programs. It is essential that group care programs develop an understanding of the implications of a research-based approach to group care and the requirements of implementing the organizational change necessary to systematize procedures; collect assessment data as youth enter and exit programs; and maintain a system of care that remains individualized. Whittaker (2006) speaks eloquently to these challenges within a single agency. Ontario has adopted a vision for all children’s mental health services which is evidence-based and accountable (Government of Ontario, 2005). This demands additional supports for research and outcome measurement for the group care programs struggling to implement this vision.

For additional detail on this project please see the full report:

Stuart, C., Sanders, L., Fulton, R., Kroll, T., Rapuano, G., McMillan, C., (2008). Child and Youth Care Practitioners Contributions to Evidence-based Practice in Group Care. Toronto, ON: Ryerson University. Available from Dr. Carol Stuart, School of Child and Youth Care.

About the Author

Dr. Carol Stuart is an Associate Professor at the School of Child and Youth Care at Ryerson University.

Previous article: Mental Health of Young People in Care: Comparing Canadian Foster Youth with British and American General Population Youth

Next article: Outcomes of a Supplemental Learning Program for Children in Care at Family and Children's Services of Renfrew County