The Korean Version of the Oxford Cognitive Screen (K-OCS) Normative Study
Article information
Abstract
Objective
To generate a Korean version of the Oxford Cognitive Screen (K-OCS) and obtain cutoff scores that determine the impairment of each subdomain. Post-stroke cognitive impairment (PSCI) negatively impacts the rehabilitation process and independence in daily life. Its obscure manifestations require effective screening for appropriate rehabilitation. However, in most rehabilitation clinics, psychological evaluation tools for Alzheimer’s dementia have been used without such considerations. The OCS is a screening assessment tool for PSCI and vascular dementia that can evaluate the cognitive domains most often affected by stroke, including language, attention, memory, praxis, and numerical cognition. It comprises 10 subtasks and enables quick and effective cognitive evaluation.
Methods
The K-OCS, which considers Korea’s unique cultural and linguistic characteristics, was developed with the approval and cooperation of the original author. Enrollment of participants without disabilities was announced at Duksung Women’s University, Yongin Sevrance Hospital, CHA Bundang Medical Center. The study was conducted between September 2020 and March 2022 on 97 male and female participants aged ≥30 years.
Results
All the 97 participants completed the task. In this study, the 5th percentile score was presumed to be the cutoff value for each score, and the values are provided here. The cutoff score for each OCS subtask was similar to that of the original British version.
Conclusion
We suggest the usability of the K-OCS as a screening tool for PSCI by providing the cutoff value of each subtask.
INTRODUCTION
In Korea, one in forty adults is reported to have a history of stroke; in addition, 232 subjects per 100,000 have a stroke every year [1]. Stroke is the leading cause of disability worldwide, and patients manifest various neurological symptoms according to their brain lesions. Although paralysis and severe aphasia are relatively common, post-stroke cognitive impairment (PSCI) remains highly prevalent and disabling [2]. Previous reports on cognitive deficits after stroke show inconsistent results, reporting prevalence in approximately 10%–91.5% of all the patients with a history of stroke [3-6]. This wide range in the prevalence of PSCI is attributable to various evaluation times and differences in the demographic characteristics of participants between the studies. However, the lack of a global standard evaluation tool for PSCI has been identified as the main cause of discordant outcomes [7].
According to the World Health Organization’s 2018 Wilson-Jungner criteria guidelines, all stroke survivors should undergo cognitive and emotion screening [8]. However, there has been little consensus on assessment tools to evaluate PSCI [9]. To date, the most frequently used cognitive evaluation tools for PSCI are the Mini-Mental State Examination [10] and Montreal Cognitive Assessment (MoCA) [11], which were primarily developed to assess Alzheimer’s disease [12,13]. Consequently, the specificity and sensitivity of these tools cannot be used to determine the cognitive status of stroke survivors [14]. Conventional screening tools for dementia not only fail to evaluate typical PSCI but may also present deviated reports affected by frequent impairments in stroke, including language and visual perception. Therefore, a specialized screening tool is urgently required to identify PSCI.
The Oxford Cognitive Screen (OCS) was developed in the United Kingdom (UK) to detect PSCI and vascular dementia. It evaluates five cognitive domains, namely, attention, language, number, praxis, and memory, and is composed of 10 tasks [15,16]. The tasks are described in detail in Table 1. The OCS has several strengths that make it more suitable for post-stroke patients than other options: (a) it can be administered briefly and is available at bedside; (b) the material provides multiple-choice questions, allowing patients with speech impairment to answer by pointing; and (c) a stimulus is simultaneously linked to several tasks, which maximizes time efficiency [17]. Another advantage of this tool is the resultant “visual snapshot” (Fig. 1) of the cognitive profiles of a patient, which enables an understanding of overall cognitive characteristics and communication within a multidisciplinary team. In addition, it can facilitate figuring out the cognitive status of a patient for the family. The OCS was validated in the British population with high specificity [15]. Subsequently, its validation studies have been conducted in Italy, Hong Kong, Russia, Denmark, Portugal, Belgium, and Australia [18-24].
Description of tasks of each domain in Korean version of the Oxford Screening (K-OCS)
Visual snapshot Korean version of the Oxford Cognitive Screen (K-OCS) cognitive profile. EXEC, executive.
The primary aim of this study was to develop a Korean version of the OCS (K-OCS) that considers the cultural and linguistic features of Korea. We also aimed to provide a cutoff score for each task by obtaining percentile data to screen for cognitive impairment in the domain [25]. Additionally, the criteria were compared with those of previous large-scale validation studies to verify the validity of the K-OCS.
METHODS
Characteristics of the OCS scoring system
The OCS was comprehensively designed for aphasia and neglect symptoms. This was achieved by adopting short high-frequency words, forced-choice testing procedures, vertical layouts, and multimodal presentations. It uses some tasks intended to assess more than one domain. For example, the “reading” task is used not only for memory but also for neglect symptoms. This feature was attributed to the short administration time of the OCS (within 15 minutes) to complete. It can also be administered from the acute phase of stroke, three days after onset. The original author, Nele Demeyere, specified minimum target number of subjects as 60 for statistically valid standards in the translation and linguistic validation process [26].
As for the criteria to determine impairment, the OCS uses 5th-percentile cutoffs. Mostly, the scores of 5th percentile represent impairment. However, according to the task, the scores of ≥95th percentile in the following three tasks also indicate impairment: (1) “space asymmetry,” (2) “object asymmetry,” and (3) “executive function total” scores. In the “space and object asymmetry” tasks, which detect neglect, high positive and negative scores represent left and right spatial neglect, respectively. Thus, both the 5th percentile and 95th percentile could be used as criteria for the norm in these tasks.
In “executive function” tasks, which have four sorts of scores, stimuli are composed of seven circles and triangles each, which might distract examinees. One point is assigned to each correct connection. The next point is given if the subject correctly completes the connection on the next try, even for existing errors at some points in the previous line connection drawing. Therefore, the maximum number of points was six each for circle and triangle connections. Moreover, the alternative circle and triangle connecting task, the “executive (mixed)” task (maximum of 13 points) is to be followed. Then, the “executive function total” score can be obtained by subtracting the “executive (mixed)” score from the sum of scores in the “executive (circles)” and “executive (triangles)” tasks. The higher the score of the task, the greater the loss of executive function; thus, the score of the 95th percentile is the cutoff score.
Development of the K-OCS test
Two licensed clinical psychologists in their psychology PhD program, who were proficient in English, translated the original OCS [15], and another clinical psychology specialist who received a PhD in the United States conducted a reverse translation without reading the original version. The original writer, Nele Demeyere at Oxford University, assisted with the entire translation process and provided the final approval.
The basic structure of the test and the scoring rules of the original version were applied to the K-OCS without any changes. However, as this study aimed to adapt the OCS to Korean culture and language characteristics, some objects and words were replaced. A pilot study was conducted with 15 adult participants without disabilities, and a few nationality-adapted modifications were made. The first modification was required for the “picture naming” task in the language domain. The objects provided for the naming test should be challenging enough for patients with cognitive impairments but, simultaneously, familiar enough to be recognized. Some of the objects in the original version, such as “pear” and “filing cabinet,” were reported to be unfamiliar to most of the participants, especially to people older than 40 years who were expected to be the main subjects of this test. To select valid figure objects as options, we performed a validation study on the major candidate age population. Correct answer rates for the five candidate objects in the pilot test, “hippopotamus,” “watermelon,” “axe,” “pomegranate,” and “fire extinguisher” were 87.5%, 100%, 100%, 72.4%, and 87.5%, respectively (n=15). As the test is supposed to measure one’s naming function in usual daily living, rather than knowledge level, the word with accuracy under 80%, that is, “pomegranate” was excluded. The final selected number of the items was four—“hippopotamus,” “watermelon,” “axe,” and “fire extinguisher”—similar to the original version.
The second modification was made for the “reading” task of the language domain and the “recall and recognition” task in the memory domain, both of which used the same sentence. The principles of the sentence construction proposed by the original author were as follows: (a) placing the high neighborhood words (i.e., words that can change meaning by changing or deleting one letter, such as “cat” and “pat” in English) leftmost in four lines of the sentence; (b) the sentence consisting of 14 words including particles, and 4 of those should be infrequent words; and (c) phonically irregular words, such as “islands,” “quay,” “colonel,” and “yacht” should be included. To generate a new Korean sentence based on these principles, several adjustments and agreements with the original author were required to address the phonetic and syntactic differences between Korean and English. First, as there are few phonetically irregular words in Korean, infrequently used words were adopted instead of irregular words based on the frequency of modern Korean word usage [27]. The second adjustment was to make sentences in Korean similar to the original sentence in speech duration rather than word count itself, owing to mismatches in the syntax between the two languages. To match the speech duration, we added one more syllable for the pilot trial. Thus, 15 syllabi were included in the sentence, while the standard number of syllables in the original English sentence was 14. In a preliminary study (n=15), when the number of words in a sentence was 14, the mean value of the “sentence reading” performance was 13.97. When the number of included words was 15, its mean value was 14.65, indicating that an increase in the number of words in a sentence exerted no significant effect on the results of the “recall and recognition” tasks. Accordingly, a 15-syllable sentence is created.
Both the OCS and K-OCS include the following materials for testing: (1) a test booklet for multiple-choice items and examples, (2) a patient pack of paper-and-pencil task worksheets, (3) an easy scoring template, and (4) a user manual. The patient pack contains a visual snapshot report (Fig. 1), where impaired domains are marked by coloring in the blank spaces of the task for abnormal scores. Furthermore, detailed information about the patient can be shared by leaving comments adjacent to the figure, such as comments about the patient’s mood, physical state, or attitude that deserve consideration. The normative data and additional materials of the K-OCS can be downloaded from https://process.innovation.ox.ac.uk/clinical/p/ocs/questionnaire/1.
Participants and evaluations
The study protocol was approved by the Institutional Review Board of Duksung Women’s University (no. 2020-007-015-A) in Seoul and Yongin Sevrance Hospital, and the CHA Bundang Medical Center (no. 2020-07-0202) in Gyeonggi Province. Written informed consent was obtained from all participants in the study. Participants without known disabilities were recruited from September 2020 to March 2022 via notice board announcements at Duksung Women’s University, Yongin Sevrance Hospital, and CHA Bundang Medical. A total of 97 volunteers who were more than 30 years old, and lived in Seoul or Gyeonggi Province, participated in this study. All participants were personally interviewed and screened using the Christensen Health Screening criteria [28], Korean version of MoCA (K-MoCA) [29], and the Beck Depression Inventory the 2nd version (BDI-II) [30]. Patients with neurological diseases, brain damage, visual and speech impairments, or psychiatric history were excluded. The K-OCS was administered to those who were finally enrolled after screening for eligibility. When the K-MoCA and K-OCS were conducted on the same day, the time interval was set to at least 30 minutes.
Data and statistical analysis
For data analyses, IBM SPSS Statistic 21, R4.1.3 version (IBM Corp.) was used. Descriptive statistics were used to demonstrate the demographic features of the study population, as well as the mean, median, standard deviation, and range of each K-OCS task score. To verify the goodness of fit, Kolmogorov–Smirnov and Shapiro–Wilk normality tests were conducted. Correlations between K-OCS scores and demographic variables, such as gender, age, and education level, were analyzed using Spearman’s correlation (one-sided test) due to the distribution of K-OCS values that deviated from normality. Finally, we computed the percentile values of each task to obtain the cutoff scores by performing a frequency analysis [25]. In addition, we performed logistic regression analysis to verify whether the participants’ age and education level could predict the results of each task.
RESULTS
Subject characteristics
Among 135 voluntary participants, 8 scored below the normal cutoff value of the K-MoCA according to age [29], and 15 participants were determined to have depression with BDI-II>18 points (i.e., the cutoff score) and were excluded from the final analyses. Therefore, the data from 97 participants (33 male [34.0%] and 64 female [66.0%]) were included in the final analysis. Among them, 15 were enrolled in both the pilot and main studies. The age ranged from 35 to 74 with a mean age and standard deviation of 54.3±9.7. Their education level ranged from 5 to 18 years, and the mean education level and standard deviation were 13.8±2.7 years. Demographic characteristics and the results of MoCA and BDI-II by age group are shown in Table 2. Significant differences were observed in the MoCA scores between the <50- and >59-year age groups (p=0.001), whereas no differences were found in education level or BDI-II scores.
Characteristics of the participating subjects
Normative data of the K-OCS
According to interim analysis after assessing 30 cases, the score of the “reading” task was found to be lower than that of the original English version [15] and other reports [18,21]. Thus, we edited the sentence, which was the main stimulus of the task, to increase its score level in the middle of the study. Consequently, the data from 30 participants, who were examined before the sentence amendment were excluded, and those from the other 67 participants were included for analyses of “reading” and verbal memory “recall and recognition” tasks where the same revised sentence was used. The mean scores for each age and years of education group for the K-OCS task are presented in Table 3. Kolmogorov–Smirnov and Shapiro–Wilk normality tests showed that the tasks, except for “semantics” and “executive task (triangles),” did not meet the normality criteria (p<0.001). Finally, the score distribution and cutoff value with criteria of the 5th or 95th percentile for each task were obtained (Table 4).
Mean scores (points) of age and years of education in each Korean version of the Oxford Cognitive Screen (K-OCS) task
Score distribution of each task on Oxford Cognitive Screen (OCS) based on the whole sample (5th percentile and 95th percentile)
Comparison of cutoff values in other countries
The cutoff scores of previous studies conducted in Denmark, the UK, and Italy are presented in Table 5. Most task scores in the current study were comparable to those of large-scale studies conducted in other countries. Tasks that showed different cutoff scores with ≥1 point from all of the other countries were “sentence reading” and “imitation.”
Cutoffs compared across studies: 5th percentile (95th percentile)
Prediction of the K-OCS score according to age and years of education
We conducted a logistic regression analysis to verify whether years of education could predict the K-OCS scores. Cutoff scores of “semantics” and “executive task (circles)” had the maximum values, and the cutoff score of “recall” had the minimum value, of the score ranges; thus, these were excluded from the analysis. According to the regression model, all the tasks, except for the “imitation” task (odds ratio>upper confidence interval, p=0.05), did not predict task scores regarding education level and age.
DISCUSSION
In stroke clinics, PSCI, including attention, spatial recognition, memory, and executive function, may be obscure and easily neglected [7,8]. By applying effective screening assessments, patients with PSCI may receive timely interventions and early screening may help to understand longer term cognitive impairments [31]. This study aimed to set the measurement standards of the K-OCS by screening healthy normal participants for PSCI as the first essential step. The K-OCS was adapted to be applied to Koreans based on their unique language and cultural characteristics while maintaining the essentials of the OCS.
The resultant percentile scores of the Korean participants were compared with those provided in the Danish [21], Italian [18], and British [15] validation studies that enrolled larger samples (Table 5). The resultant cutoff scores of the K-OCS were similar to those in large-scale studies in other countries, indicating that this Korean version was adapted well enough to maintain its significant validity when applied to South Koreans. Only two tasks, “sentence reading” and “imitation” were found to show little difference in the cutoff scores. First, the cutoff scores of “sentence reading” were 15, 14, and 14.1–15 points in Denmark, the UK, and Italy, respectively. In the present study, it was 13 points, a relatively lower cutoff score than in the compared countries. The sentence in this task was not simply translated from the original OCS questionnaire but was newly constructed based on the basic principles of the original version. After a thorough analysis of the errors in the sample population, we found a relatively low cutoff score.
The cutoff score of “imitation” was higher than the previous studies. The “imitation” task requires mirroring the set of meaningless movements and detecting the signs of apraxia with a score ranging from 0 to 12. It may be possible to apply looser criteria depending on the position and the angle of the arm or hand, which are not precisely described, including degrees or if the gestures are easier for Koreans. Moreover, the possibility of deviation from interpretation seems low, as the verbal component is not significantly involved in this task. Thus, more practice could help understand the score of the “imitation” task.
While the cutoff values of the other tasks were not different from all of the previous reports, the value of “recognition” might be regarded to have a lower score compared with those in Denmark and the UK. The “recognition” task is evaluated as remembering the “sentence reading” stimulus performed beforehand, the cutoff score of which was also lower as previously mentioned. The relatively low “sentence reading” performance might have affected the subsequent “recognition” score.
The cutoff score for “object asymmetry” was similar to that in studies with a small number of participants, such as those in the UK and Denmark [15,21]. However, the value was different in large-scale studies such as in Italy [18], which necessitates further validation studies with larger populations to investigate the effect of sample size. In the MoCA test—a cognitive function screening test—there is a language fluency task in which participants are given a single letter and asked to tell as many as possible words starting with the letter. Due to differences between languages, eleven words as the cutoff number in North America correspond to six words in Korean [29]. Similarly, the cutoff scores for Korean language impairment may differ. However, the relatively low “recognition” scores did not lead to the floor effect; thus, it is not regarded to have affected as a limitation in applying this tool.
The cutoff score for “executive function total” in the K-OCS was identical to that in Denmark but was different to those of the UK and Italy. In this study, we examined whether the outlier values affected the cutoff score calculation and found that the cutoff score was still identical, even when the extreme values were included in the calculation.
This study has a few limitations. First, 97 normal participants enrolled in this study. Although this study met the minimal criteria to demonstrate equivalence with the original test, a larger Korean normative study is required to achieve an even more representative normative sample. Second, the super-aged group is not included. The subjects of the elderly over 75 years of age had difficulty participating in the study due to the impact of coronavirus disease 2019, which needs to be supplemented in follow-up research. Third, although age did not correlate with results other than performance score, some tasks showed correlations between education level and performance score in previous studies. In this study, the cutoff scores according to age and education level could not be calculated because of the relatively small number of cases. Future studies should focus on enrolling a larger number of participants to analyze the correlation between demographic factors and the K-OCS task performance, cutoff scores, and prediction results.
In conclusion, this study is significant in that it has developed and validated a K-OCS through a normative study, which has real-world applications for improving clinical practice.
Considering the current limitations in assessing PSCI, the development of the K-OCS will contribute to detecting PSCI and may facilitate appropriate interventions.
Notes
CONFLICTS OF INTEREST
No potential conflict of interest relevant to this article was reported.
FUNDING INFORMATION
This study was supported by a Special Research Fund for ARM Development of the Korean Academy of Rehabilitation Medicine in 2023 and IITP (2021-0-00742), funded by the Ministry of Science and ICT of Korea.
AUTHOR CONTRIBUTION
Conceptualization: Cho E. Methodology: Cho E. Formal analysis: Cho E. Funding acquisition: Kim MY. Project administration: Cho E, Kim MY, Choi S. Visualization: Cho E. Writing – original draft: Cho E, Kim MY, Hwang SSS. Writing – review and editing: Cho E, Kim MY, Demeyere N, Choi S. Approval of final manuscript: all authors.