6 Comments
User's avatar
Zofia Kozak's avatar

I would love to hear more of your thoughts about the limitations of psychometrics in capturing therapy outcome data. I practice in the US and am seeing increasing use of weekly/monthly PHQ9, GAD questionnaires to attempt to capture patient success. Your point about the fact that such data collection may not take into account unconscious defenses that maintain illness, negative transference that could influence how they answer the questions, let alone myriad changes in their thinking that a 9-point questionnaire doesn’t capture.

Kate O'Connor's avatar

Thanks for asking about this - I’ve had something half written on outcome measures for ages that I might try and finish with this nudge! I agree with everything you’ve spoken to, and there are so many other variables that undermine the usefulness of those kinds of measures. Positive/idealising transferences can also skew results or reflect superficial improvements that don’t hold over time. Equally, if someone is really digging around and unearthing long denied anger, resentments, or trauma, they can feel a lot worse even if they’re getting better. Arbitrarily measuring random stuff makes clinicians feel better, but I don’t think it’s always a reflection of what’s going on.

Don Salmon's avatar

My goodness, thankful that someone has the patience to go into this much depth with the statistical/methodological analysis.

I started my grad schooling in 1990, skeptical but still trying to be open minded about quantitative research of any kind.

After 3000 psychological evaluations and roughly 20,000 individual tests as part of the battery - i will say this: I"m VERY impressed how precisely some cognitive tests (even a few subtests of the WAIS - NOT the information test!!!) can point to certain limitations (of all things, I found, and therapists who referred patients to me found, that the Digit Symbol test alone, when significantly low, led to direct changes in treatment, to account for the slowed processing speed, that really made a difference.

On the other hand, I rarely see a study of therapeutic modalities that impresses me. Now that consciousness is getting close to being one of the hottest topics in the sciences, and we are maybe coming to see that it cannot be quantified and it may be the foundation of all good therapy (I don't mean consciousness in contrast to the subconscious, I mean consciousness as the substance of everything in the universe), i think we're at a point wher ea radically different (alchemical, perhaps!!!) research paradigm is needed.

Kate O'Connor's avatar

Thanks 🙏🏼 I agree with you about cognitive testing vs therapy outcomes. It kind of feels like medicine for me - if I break my leg or get diagnosed with cancer, I’m going to be very grateful for Western medicine, but it doesn’t mean my GP knows how to deal with more chronic or ambiguous symptoms.

Amelia Scott's avatar

It also is my experience that, when you dig behind many of the popular catch phrases from seminal articles, what lies underneath is something much more tentative and nuanced. I feel that incentive structures for researchers - to publish, get grants, get media attention - actively suppress their search for the truth.

Kate O'Connor's avatar

Thanks Amelia. You probably have more experience, and more to say about this, than most!