Psychometric Properties of the Norwegian Version of the Cognitive Therapy Adherence and Competence Scale (CTACS) and Its Associations With Outcomes Following Treatment in IAPT Norway

Abstract

Background: No studies have examined the underlying structure or predictive validity of the Cognitive Therapy Adherence and Competence Scale (CTACS). Examining the structure of the CTACS is of great relevance because it could provide information on what constitutes competence in CBT, and whether some underlying factors are more important for predicting treatment outcomes than others. This study investigates the psychometric properties of the Norwegian version of CTACS and its associations with treatment outcomes in a sample of primary care clients who received CBT for anxiety and/or depression.

Method: Independent assessors rated audiotaped therapy sessions (early, mid and late in treatment) in a sample of 132 primary care clients (mean [SD] age = 34.8 [11.8], 63.6% women), participating in the Prompt Mental Health Care trial. Outcomes were symptoms of anxiety and depression assessed by patient self-report questionnaires. Structural validity was examined by means of confirmatory and exploratory factor analyses (CFA/EFA), whereas longitudinal associations with treatment outcome were explored by adopting multilevel modeling.

Results: No evidence was found for the divergent validity of the constructs competence and adherence as indicated by a very high correlation between these two subscales in CTACS (0.97). Regarding reliability, ICCs for the mean score of the full competence scale and its associated subscales were generally good to excellent (0.70–0.80), although the subscale measuring the quality of the therapeutic relationship was relatively low (0.44). Internal consistency was overall acceptable, but our CFA models did not provide an acceptable fit for the pre-specified one-factor and four-factor solutions. EFA results were difficult to interpret, with a sub-optimal three-factor solution providing best model fit and only two meaningful factors [CBT specific skills (α = 0.82) and session structure (α = 0.59)]. Overall, the results indicated no evidence for the scales' predictive validity.

Conclusion: Our findings point to several psychometric problems of the CTACS that may limit both its research and clinical utility. The importance of providing empirical evidence for both reliability and validity aspects of scales are discussed and suggestions for future research are provided.