Validation of Korean Version of Coma Recovery Scale-Revised (K-CRSR)

Hee Jun Han; Ee Jin Kim; Hae Jin Lee; Sung Bom Pyun; Kyung Lim Joa; Han Young Jung

doi:10.5535/arm.2018.42.4.536

Abstract

Objective

To determine the validity and reliability of the Korean version of the Coma Recovery Scale-Revised (K-CRSR) for evaluation of patients with a severe brain lesion.

Methods

With permission from Giacino, the developer of the Coma Recovery Scale Revised (CRSR), the scale was translated into Korean and back-translated into English by a Korean physiatrist highly proficient in English, and then verified by the original developer. Adult patients with a severe brain lesion following traumatic brain injury, stroke, or hypoxic brain injury were examined. To assess the inter-rater reliability, all patients were tested with K-CRSR by two physiatrists individually. To determine intra-rater reliability, the same test was re-administered by the same physiatrists after three days.

Results

Inter-rater reliability (k=0.929, p<0.01) and intra-rater reliability (k=0.938, p<0.01) were both high for total K-CRSR scores. Inter- and intra-rater agreement rates were very high (94.9% and 97.4%, respectively). The total K-CRSR score was significantly correlated with K-GCS (r=0.894, p<0.01), demonstrating sufficient concurrent validity.

Conclusion

K-CRSR is a reliable and valid instrument for the assessment of patients with brain injury by trained physiatrists. This scale is useful in differentiating patients in minimally conscious state from those in vegetative state.

Keywords: Vegetative state, Minimally conscious state, Coma Recovery Scale Revised, Consciousness

INTRODUCTION

It is essential to evaluate the brain function of patients with severe brain injury when making decisions regarding their treatment and prognosis.

A patient’s vegetative state (VS) is defined by a level of arousal with no sign of awareness [1], and the minimally conscious state (MCS) is defined by non-continuous but reproducible goal-directed behaviors [2]. VS and MCS represent very different prognoses, and therefore, accurate differentiation is a diagnostic challenge [3]. Therefore, several assessment tools have been developed to evaluate these conditions.

The Coma Recovery Scale (CRS) was developed in 1991 by Giacino to evaluate the consciousness disorder and the Coma Recovery Scale-Revised, (CRSR) was introduced in 2004 [4]. Since then, the CRSR has served as a useful tool for differentiation between MCS and VS with high reliability, validity, and sensitivity [4]. It has been translated into various languages and is standardized in many countries [5-8].

The present study was conducted to evaluate the validity and reliability of the Korean version of the CRSR (K-CRSR) in patients with a severe brain lesion and disturbed consciousness, for its standardization.

MATERIALS AND METHODS

Subjects

This trial was approved by the Inha University Hospital Institutional Review Board (No. IUHIRB-2012023). All the enrolled patients provided informed consent. Patients with severe brain injuries and disorders of consciousness were included. Patients who were admitted to the intensive care unit of Inha University Hospital with acquired brain injuries such as traumatic brain injury (TBI), stroke, and hypoxic brain injury were enrolled. Patients who were younger than 18 or who were sedated within 24 hours of evaluation were excluded. The patient’s age, gender, laterality, and type of brain lesion were investigated.

Translation

To translate the CRSR into K-CRSR, a Korean rehabilitation physician who was an expert in English was engaged with permission from Giacino, the author of the CRSR from Harvard Medical School. A draft of the K-CRSR was then back-translated into English and verified by the author. After appropriate modifications, the final form of the K-CRSR was administered to patients.

Procedures

Two physiatrists were trained to use the K-CRSR aided by a video instruction package prepared by another rehabilitation physician. The procedures described are similar to those used in the 2004 revision of the CRSR [4].

On the first day, the K-CRSR and K-GCS were administered to all patients by two randomly assigned raters (A and B), who were physiatrists. These two physicians submitted their assessments individually. Based on the K-CRSR results, patients were diagnosed with VS, MCS, or emergence from the minimally conscious state (MCS+) [3]. These patients were diagnosed with VS with the following scores: auditory area ≤2, visual area ≤1, motor area ≤2, oral function/language ≤2, communication=0, and arousal ≤2. If a patient scored 3 or 4 in the auditory area, 2–5 in the visual area, 3–5 in the motor area, 3 in the oral function/language area, 1 in communication, or ≤2 in alertness, they were defined as MCS. If the score for the motor area was 6 or that for communication was 2, the patient was defined as MCS+. Three days after the first assessment, a second assessment with the same protocol was submitted by rater A. The cross-correlation of rater A scores on day 3 with rater B scores on day 1 was used to determine the effect of concomitant changes on agreement between patients and raters.

Statistical analysis

Statistical analysis was conducted using IBM SPSS version 19 (IBM, Armonk, NY, USA). To assess inter-rater reliability, the assessment score of each rater on day 1 was compared. To test intra-rater reliability, the assessment score on day 1 and the score on day 3 were analyzed using Cohen’s kappa. The internal consistency and reliability were evaluated using the total CRSR score, and its subscales were analyzed using Cronbach’s alpha. The level of agreement between the examiners regarding diagnosis of each case was assessed. To test the concurrent validity of the K-CRSR, the scale and the Korean version of the Glasgow Coma Scale (K-GCS) were examined and analyzed using the Spearman rank correlation. Null hypotheses of no difference were rejected if p-values were <0.05.

RESULTS

Demographic data

From 2012 to 2015, 39 patients with a mean age of 56.9±16.9 years (range, 18–81 years) were assessed, including 22 males (mean age, 54.1±18.5 years; range, 18–81 years) and 17 females (mean age, 61.0±13.6 years; range, 38–79 years). The mean time from a patient’s brain injury to his or her assessment was 125.5±128.8 days (range, 8–463 days). Causes of the brain injuries were stroke (21 including hemorrhage in 17 cases and infarction in 4 cases), hypoxic brain injury, and TBI. The laterality of the brain lesion was right in 11 patients, left in 8, and bilateral in 20 (Table 1).

Statistical results

K-CRSR total scores ranged from 0 to 19. The mean KCRSR scores for days 1 and 3 did not differ significantly (Table 2). In the first assessment, the mean score for KCRSR was 7.0±6.2, and there was no significant difference between the raters (7.0±6.2 vs. 7.0±6.3). At the second evaluation, the mean score for K-CRSR was 7.2±6.4, and there was no significant difference (Table 2).

The mean score for K-GCS was 7.4±3.4, and there was no significant difference between the raters (7.5±3.4 and 7.4±3.5) (Table 2).

Concurrent validity

The total score for K-CRSR was strongly correlated with the total score for K-GCS, and the Spearman correlation coefficient was significant (r=0.894, p<0.01). The strong correlation between these two scales established the concurrent validity of K-CRSR.

Reliability

The inter-rater reliability of the total score for K-CRSR and its subscales was very high (Ƙ=0.93, p<0.01)—motor (Ƙ=0.84, p<0.01), oromotor/verbal (Ƙ=0.90, p<0.01), communication (Ƙ=1.00, p<0.01), arousal (Ƙ=0.90, p<0.01), auditory (Ƙ=0.95, p<0.01), visual (Ƙ=0.86, p<0.01) (Table 3). Agreement between the scores of rater B on day 1 and the scores of rater A on day 3 was very high (Ƙ=0.908, p<0.01)—motor (Ƙ=0.84, p<0.01), oromotor/verbal (Ƙ=0.91, p<0.01), communication (Ƙ=1.00, p<0.01), arousal (Ƙ=0.85, p<0.01), auditory (Ƙ=0.95, p<0.01), visual (Ƙ=0.90, p<0.01) (Table 3). The intra-rater reliability of the total score for K-CRSR and its subscales was very high (Ƙ=0.94, p<0.01)—motor (Ƙ=0.84, p<0.01), oromotor/verbal (Ƙ=0.95, p<0.01), communication (Ƙ=1.00, p<0.01), arousal (Ƙ=0.95, p<0.01), auditory (Ƙ=0.95, p<0.01), visual (Ƙ=0.91, p<0.01) (Table 3). Reliability was in perfect agreement with the score for the communication subscale.

Diagnostic agreement

In the first assessment, rater A diagnosed 20 patients with VS, 12 patients with MCS, and 7 patients with MCS+. On the same day, rater B diagnosed 18 patients with VS, 14 patients with MCS, and 7 patients with MCS+. Three days later, rater A diagnosed 19 patients with VS, 13 patients with MCS, and 7 patients with MCS+.

The corresponding rate of agreement in diagnostic impression between the two raters using K-CRSR was very high (94.9%; 37 of 39 cases). Two cases were diagnosed with VS by rater A, and as MCS by rater B.

The level of test-retest agreement was also very high (97.4%; 38 of 39 cases). A divergent case between VS and MCS was observed.

Internal consistency

The Cronbach’s alpha showed a high degree of internal consistency. Specifically, it was 0.9 for rater A on day 1, 0.9 for rater B and 0.9 for rater A on day 3. Spearman test was conducted to assess intercorrelations between subscale scores and across all observations of K-CRSR. K-CRSR showed a high correlation with neurobehavioral assessment (Table 4).

DISCUSSION

Severe brain damage can trigger major disturbances in consciousness, and prognosis of patients with moderate to severe impairment of consciousness may be occasionally difficult to predict. A few studies reported that only 3% of TBI patients diagnosed with VS showed recovery to moderate disability, whereas 50% of those diagnosed with MCS recovered to a moderate level of disability; however, TBI-related VS still appears to be associated with a better prognosis, in general, compared with hypoxic brain injury [9].

In previous reports, the rate of misdiagnosis between VS and MCS was 37%–43% suggesting inappropriate medical care [10-13]. Despite this difficulty, distinguishing VS from MCS is essential not only to improve prognosis but also for decision-making involving intensive rehabilitation. Contrary to patients in VS, those in MCS exhibit partial capacity for cognitive processing because of the remnant neuronal network or neuroplasticity. Additionally, MCS might be more amenable to rehabilitation strategies such as multi-sensory stimulation. Although it is still a challenge to discriminate between MCS and VS based on these definitions, early discrimination between MCS and VS facilitates varying diagnostic, therapeutic, and prognostic interventions.

As defined by Plum and Posner [14], coma is a state in which there is neither wakefulness nor awareness, while VS implies wakefulness without awareness. Specifically, VS is defined by the following features: (1) Absence of awareness of the self or environment, and inability to communicate with others; (2) lack of evidence of continuous, reproducible, purposeful, and voluntary response to visual, auditory, or touch stimulation; (3) absence of evidence supporting understanding or expression in language; (4) periods of sleep and awakening, with intermittent awakening; (5) possible life support without medical management because of the preserved function of hypothalamus and pons; (6) fecal and urinary incontinence; and (7) preserved cranial-nerve reflex [1].

In contrast, MCS was defined by Giacino et al. [2] as discontinuous but definitive evidence of consciousness. However, these activities may not always be reproducible, and the patient may follow simple commands, answer yes/no questions, make partially intelligible verbalization, engage in minor purposeful behavior, and perform minor movements other than reflex actions.

To date, a few neurobehavioral tools to assess the level of consciousness have been developed including GCS, Full Outline of Unresponsiveness (FOUR) [15], and CRS. Jennett and Bond [16] reported an objective description of brain injury recovery but provided only a simple description of this recovery in five stages (death, VS, severe injury, moderate injury, and recovery), complicating any discrimination between MCS and VS. Schnakers et al. [15] compared FOUR, GCS, and CRSR and verified that CRSR was superior at discriminating MCS from VS. Furthermore, they reported difficulty in discriminating between MCS and VS using FOUR, although all the three scales are clinically important in diagnosis or prognosis.

The original CRSR facilitated differentiation of MCS from VS with high reliability, validity, and sensitivity [4]. The scale has been translated into various languages and standardized in many countries [5-8]. As in the case of other translated forms of CRSR [5,7], the authors translated the CRSR into a Korean version. Therefore, we validated K-CRSR based on a significantly high correlation between K-CRSR and K-GCS and demonstrated its high inter and intra-reliability. Our study showed a relatively higher reliability compared with other studies [4-8] because the testers who participated in this study appeared to have received pre-training to enhance their skills through paper documentation and video instruction to administer the K-CRSR. The reliability of the CRSR was reportedly influenced by the experience of the evaluator [17]. Therefore, we recommended that new testers receive appropriate training before using the K-CRSR in clinical practice.

In the present study, disagreement between VS and MCS was observed in only two cases. In one case, rater A observed visual fixation (denotes MCS) and rater B interpreted visual startle. In another case, rater B observed localization to painful stimulation (denotes MCS) and rater A interpreted flexion withdrawal. Both of these cases were diagnosed as MCS because of their behavioral interaction with the environment. Other researchers found it difficult to distinguish between reflexive and voluntary movements under specific circumstances. Giacino et al. [4] previously reported inappropriate scores in specific subscales. For example, visual fixation and localization to painful stimulation were underrepresented despite their location in the visual and motor subscales. These findings suggest that the visual and motor subscales should be used attentively. As in previous studies [4], the correlation between rater A2 and rater B1 showed values intermediate between intra-rater and inter-rater agreement. These findings represent the combined effect of rater variability and patient fluctuation.

This study is associated with a few limitations. The sample size was small, and patients with only a limited etiology of brain lesions were enrolled. In particular, there were few TBI cases enrolled and investigated. In this regard, future investigations should be based on a larger sample of patients’ metabolic disorders, tumors, and other conditions. Our testers represent a single rehabilitation discipline; accordingly, the study would be improved by experts from a broader range of disciplines.

K-CRSR is a valid and reliable neurobehavioral assessment tool for evaluating patients with consciousness disorders and facilitates the diagnosis and discrimination of MCS or MCS+ from VS.

CONFLICTS OF INTEREST

No potential conflict of interest relevant to this article was reported.

ACKNOWLEDGMENTS

This study was supported by a research grant from Inha University. The authors are grateful to Prof. Joseph Giacino (original author of the CRS-R) for his feedback regarding the translation of the scale and his helpful comments.

Table 1.

Patients’ demographics (n=39)

Characteristic	Value
Age (yr)	56.9±16.9 (18–81)
Man (n=22)	54.1±18.5 (18–81)
Woman (n=17)	61.0±13.6 (38–79)
Days after injury	125.6±128.8 (8–463)
Etiology
Stroke	21
Hemorrhage	17
Infarction	4
TBI	10
Anoxia	8

Values are presented as mean±standard deviation (range) or number.

TBI, traumatic brain injury.

Table 2.

K-CRSR and K-GCS scores at initial assessment and reassessment

	1st day	3rd day
K-CRSR	7.0±6.2
Rater A	7.0±6.2	7.0±6.3
Rater B	7.0±6.3
K-GCS	7.4±3.4	-
Rater A	7.5±3.4	-
Rater B	7.4±3.5	-

Values are presented as mean±standard deviation.

K-CRSR, Korean version of the Coma Recovery Scale-Revised; K-GCS, Korean version of the Glasgow Coma Scale.

Table 3.

Reliability of total and subscale K-CRSR scores

K-CRSR subscale	Cohen’s kappa	95% CI	p-value	Agreement (%)
Inter-rater agreement
Auditory	0.95^*	0.91–0.98	<0.001	93
Visual	0.86^*	0.81–0.91	<0.001	87
Motor	0.84^*	0.80–0.89	<0.001	80
Oromotor/verbal	0.90^*	0.81–0.95	<0.001	90
Communication	1.00^*	1.00–1.00	<0.001	100
Arousal	0.90^*	0.81–0.95	<0.001	90
Intra-rater agreement
Auditory	0.95^*	0.91–0.98	<0.001	93
Visual	0.91^*	0.84–0.95	<0.001	90
Motor	0.84^*	0.80–0.89	<0.001	80
Oromotor/verbal	0.95^*	0.91–0.98	<0.001	93
Communication	1.00^*	1.00–1.00	<0.001	100
Arousal	0.95^*	0.91–0.98	<0.001	93
Agreement between rater B1 and rater A2
Auditory	0.95^*	0.91–0.98	<0.001	93
Visual	0.90^*	0.81–0.95	<0.001	90
Motor	0.84^*	0.80–0.89	<0.001	80
Oromotor/verbal	0.91^*	0.84–0.95	<0.001	90
Communication	1.00^*	1.00–1.00	<0.001	100
Arousal	0.85^*	0.81–0.89	<0.001	87

K-CRSR, Korean version of the Coma Recovery Scale-Revised; CI, confidence interval.

^* p<0.05.

Table 4.

Intercorrelations of K-CRSR subscales

	Auditory	Visual	Motor	Oromotor/verbal	Communication	Arousal
Auditory	1.000
Visual	0.899^*	1.000
Motor	0.763^*	0.781^*	1.000
Oromotor/verbal	0.779^*	0.755^*	0.698^*	1.000
Communication	0.713^*	0.783^*	0.708^*	0.728^*	1.000
Arousal	0.667^*	0.640^*	0.680^*	0.685^*	0.547^*	1.000

K-CRSR, Korean version of the Coma Recovery Scale-Revised.

^* p<0.05, Spearman correlation.

REFERENCES

1. Multi-Society Task Force on PVS. Medical aspects of the persistent vegetative state. N Engl J Med 1994;330:1499-508.

2. Giacino JT, Ashwal S, Childs N, Cranford R, Jennett B, Katz DI, et al. The minimally conscious state: definition and diagnostic criteria. Neurology 2002;58:349-53.

3. Schnakers C, Vanhaudenhuyse A, Giacino J, Ventura M, Boly M, Majerus S, et al. Diagnostic accuracy of the vegetative and minimally conscious state: clinical consensus versus standardized neurobehavioral assessment. BMC Neurol 2009;9:35.

4. Giacino JT, Kalmar K, Whyte J. The JFK Coma Recovery Scale-Revised: measurement characteristics and diagnostic utility. Arch Phys Med Rehabil 2004;85:2020-9.

5. Schnakers C, Majerus S, Giacino J, Vanhaudenhuyse A, Bruno MA, Boly M, et al. A French validation study of the Coma Recovery Scale-Revised (CRS-R). Brain Inj 2008;22:786-92.

6. Sacco S, Altobelli E, Pistarini C, Cerone D, Cazzulani B, Carolei A. Validation of the Italian version of the Coma Recovery Scale-Revised (CRS-R). Brain Inj 2011;25:488-95.

7. Tamashiro M, Rivas ME, Ron M, Salierno F, Dalera M, Olmos L. A Spanish validation of the Coma Recovery Scale-Revised (CRS-R). Brain Inj 2014;28:1744-7.

8. Di H, He M, Zhang Y, Cheng L, Wang F, Nie Y, et al. Chinese translation of the coma recovery scale-revised. Brain Inj 2017;31:363-5.

9. Giacino JT, Zasler ND. Outcome after severe traumatic brain injury: coma, the vegetative state, and the minimally responsive state. J Head Trauma Rehabil 1995;10:40-56.

10. Gill-Thwaites H. Lotteries, loopholes and luck: misdiagnosis in the vegetative state patient. Brain Inj 2006;20:1321-8.

11. Childs NL, Mercer WN, Childs HW. Accuracy of diagnosis of persistent vegetative state. Neurology 1993;43:1465-7.

12. Andrews K, Murphy L, Munday R, Littlewood C. Misdiagnosis of the vegetative state: retrospective study in a rehabilitation unit. BMJ 1996;313:13-6.

13. Bender A, Jox RJ, Grill E, Straube A, Lule D. Persistent vegetative state and minimally conscious state: a systematic review and meta-analysis of diagnostic procedures. Dtsch Arztebl Int 2015;112:235-42.

14. Plum F, Posner JB. The diagnosis of stupor and coma. Contemp Neurol Ser 1972;10:1-286.

15. Schnakers C, Giacino J, Kalmar K, Piret S, Lopez E, Boly M, et al. Does the FOUR score correctly diagnose the vegetative and minimally conscious states? Ann Neurol 2006;60:744-5.

16. Jennett B, Bond M. Assessment of outcome after severe brain damage. Lancet 1975;1:480-4.

17. Lovstad M, Froslie KF, Giacino JT, Skandsen T, Anke A, Schanke AK. Reliability and diagnostic characteristics of the JFK coma recovery scale-revised: exploring the influence of rater’s level of experience. J Head Trauma Rehabil 2010;25:349-56.