Standardised assessment tools in investigations of children and adolescents within the child protection service

Aim

The aim of this project was to investigate the psychometric properties of standardised assessment tools used on children and adolescents in contact with child welfare services. The project is based on a report with the title “Kartleggingsverktøy for utsatte barn og unge”, from the Norwegian Folkehelseinstituttet. In this project, an update of the literature search used in the Norwegian report was carried out in order to identify more recent published relevant systematic reviews. Finally, the result were presented from a Swedish perspective in an evidence map.

Method

A study protocol for this project was made a priori.

Inclusion criteria

Population: Children and adolescents 0-21 years of age.

Intervention: Standardised assessment tools measuring any of the following: exposure to violence and neglect, parent-child interaction, general mental health, behaviour problems, substance abuse, bullying, violence, crime, social competences, or quality of life.

Control: Only applicable for the evaluation of criterion validity.

Outcome: Psychometric properties, validity, and reliability.

Study design: Systematic reviews.

Language: English, Swedish, Norwegian, and Danish.

Search period: From 2000 to 2021. Final search September 2021.

Databases searched: Medline (Ovid), PsycInfo (EBSCO), Embase (Wiley), Cochrane library (Embase.com), Cinahl (EBSCO), Sociological abstracts (ProQuest), Epistemonikos, COSMIN database and Campbell library.

Client/patient involvement: No patient involvement.

The inclusion criteria for this project, as well as the categories used to classify the content in the evidence map, were outlined by the project group.

The systematic literature search from the Norwegian report was updated by an information specialist at Folkehelseinstituttet in Oslo. After the literature search was performed, two reviewers within the project independently screened the abstracts and full text articles and selected the relevant ones. The risk of bias in the included systematic reviews was assessed independently by two reviewers using the ROBIS tool. The systematic reviews included in the Norwegian report were also critically appraised using ROBIS. Any disagreements regarding the relevance or the appraisal of the risk of bias was resolved by discussions within the project group.

Depending on the research question addressed in the identified systematic reviews, they were classified into the prespecified categories and presented in the evidence map.

The report was reviewed by SBU’s internal quality assurance group, SBU’s scientific advisory board as well as by an external reviewer.

Results

A total of 31 relevant systematic reviews were identified and critically appraised. Of these, 7 were considered to have low risk of bias, 5 to have moderate risk of bias and 19 to have a high risk of bias. The 12 systematic reviews with low or moderate risk of bias are presented, with a link to each article, in the evidence map. In these 12 systematic reviews, a total of 53 assessment tools are included that can be accommodated within the Swedish social services' child and youth care, based on their assignments. All 53 assessment tools are presented in Appendix 4. Of the 53 assessment tools, 11 were judged as particularly relevant on the basis that they are currently in use in the social services' child and youth care in Sweden or that they have been translated into Swedish and used in a research context. These 11 assessment tools are divided into the following seven assessment areas:

Abuse and neglect: CTQ (Childhood Trauma Questionnaire).

Parent-Child interaction: No assessment tools in the included systematic reviews were considered as particularly relevant for a Swedish context.

Violence and crime: ESTER (Evidensbaserad Strukturerad Bedömning av Risk och Skyddsfaktorer), SAVRY (Structured Assessment of Violence Risk in Youth), YLS/CMI (Youth Level of Service/Case Management Inventory).

Substance use problems: AUDIT (Alcohol Use Disorders Identification Test), AUDIT-C (Alcohol Use Disorders Identification Test-Consumption), CRAFFT (Car, Relax, Alone, Forget, Friends, Trouble), ASSIST (Alcohol, Smoking and Substance Involvement Screening Test).

General mental health including behavior problems: CBCL (Child Behavior Checklist), SDQ (The Strengths and Difficulties Questionnaire).

Social competences: ASQ:SE-2 (Ages and Stages Questionnaires: Social-Emotional, Second Edition).

Quality of life: No systematic reviews were identified.

The results from the systematic reviews show that five of these 11 assessment tools (CRAFFT, SAVRY, YLS / CMI, SDQ and ASQ:SE-2) have been evaluated in primary studies investigating psychometric aspects. The results in the primary studies are mainly linked to validity evidence (if the method measures what is intended to be measured) and less to reliability (the precision of the measurement) estimates.

For the other six assessment methods, CTQ, ESTER, AUDIT, AUDIT-C, ASSIST and CBCL, no or few primary studies were included in the systematic reviews. This does not necessarily mean that there are no primary studies on these six assessments tools. It could be due to the systematic reviews' selection criteria (for example regarding population, outcome, or area of ​​activity). This made it difficult to assess the psychometric properties for these standardized assessment tools in children and adolescents. To acquire more knowledge, new systematic reviews are therefore needed, as well as more primary studies in this field. There is also a need for an increased focus on psychometric properties of assessment tools in the context of social child and adolescent care.

Conflicts of interest

In accordance with SBU’s requirements, the experts and scientific reviewers participating in this project have submitted statements about conflicts of interest. These documents are available at SBU's secretariat. SBU has determined that the conditions described in the submissions are compatible with SBU's requirements for objectivity and impartiality.