How do we know if an intervention to address burnout is useful? In creating interventions to assist with burnout, it is important to assess satisfaction and the acceptability of those interventions. High satisfaction will facilitate engagement with the intervention, recommendations, and uptake of the intervention.
The Client Satisfaction Questionnaire-18 (CSQ-18) developed by Attkinsson and Zwick (1982) was designed to measure satisfaction with medical interventions, and may be useful to assess the success of burnout interventions. However, there are limitations that must be kept in mind when utilizing the CSQ-18, such as the fact it is proprietary, only available on request, and incurs a cost.
Attkisson CC, Zwick R. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. 1982;5(3):233-237.
The shorter Client Satisfaction Questionnaire-8 (CSQ-8) performs as well with some populations and may be substituted. The questions measure 8 items on various scales. While the following is not the actual scale, it helps clarify what kind of questions are included in the scale and the ratings used.
| 1. Receipt of the desired service:
2. Willingness to recommend:
3. Willingness to repeat program if necessary:
4. The quality of the service received:
| 5. The extent the program met needs:
6. Satisfaction with help:
7. Impact on effectively dealing with problems:
8. Overall satisfaction:
|
Other aspects to measure for this population include:
- Non-intrusive integration with practice and lifestyle
- Example: A 20-hour training program in mindfulness may be beneficial, but may be perceived as a burden if it is required and conflicts with personal activities
- A more specific measurement of desirable outcomes
- Example: Reducing workload, stress, or another parameter the intervention targeting
A weakness is that the scale has mostly been tested with interventions targeting non-professionals. In one study with residents, results showed an absolute ceiling effect with all program directors rating the program perfectly (32) and residents/fellows rating (30 to 31).
Ey Sydney, Moffit Mary, Kinzie J Mark, Brunett Patrick H. Feasibility of a Comprehensive Wellness and Suicide Prevention Program: A Decade of Caring for Physicians in Training and Practice. Journal of Graduate Medical Education. October 10, 2016;8(5):747-753. doi:10.4300/JGME-D-16-00034.1.
In this audience, it is unclear if the CSQ-8 is a valid means to discriminate high satisfaction from low satisfaction due to limited research and this ceiling effect. Hopefully, additional research will create a scale specifically designed for adult learners with health professional expertise.
Alternatively, to avoid a ceiling effect, one strategy may be to anchor responses by asking respondents to compare the experience to their expectation. Physicians and medical students are used to training and educational efforts so they should be able to compare this experience to similar experiences. Thus the scale might be:
- (1) Far below my expectation
- (2) Below my expectation
- (3) What I expected
- (4) Above my expectation
- (5) Far above my expectation
Such a scale increases the opportunity to identify areas where there is an opportunity for further improvement. Remodeling existing interventions and creating individualized interventions will likely be essential to exceed expectations.
Our burnout intervention is still in development, but alongside our existing scales, our plan is to produce a publicly-available, free to use intervention assessment and satisfaction scale specifically targeting medical students, residents, and practicing physicians. If we are going to address burnout, we need more than interventions. We need interventions that users rate highly and have a positive impact.
