Brain lesions can lead to deficits in single domains of cognitive function, such as memory, learning, attention, visuospatial perception, or language, but they can also lead to more complex deficits in higher-order cognitive ability i.e., executive function [1]. These cognitive deficits can have a negative impact on a patient's motivation for rehabilitation and on the learning process required for regaining motor skills, which in turn may lead to further functional decline and a delay in their return to society.

Traditionally, the clinical consequences of frontal lobe damage, such as a difficulty in maintaining goal-directed behavior, impulsivity, emotional labiality, and apathy are collectively referred to as "frontal lobe syndrome" or executive dysfunction [2,3]. However, according to Goldberg and Bougakov [4], the executive dysfunction is not only restricted due to frontal lobe damage, but also appeared in various areas as a result of the anatomical complexity of the brain. Therefore, a more functional approach is required for sufficient assessment of executive function.

Executive functions are integrative, higher level, cognitive processes that control and regulate other cognitive processes in order to set up strategy and to generate and maintain goal-directed behavior according to everyday life priorities [5,6]. In other words, it is a comprehensive term used to describe a variety of processes, such as initiation, organization and planning, hypothesis formulation, making decisions, self-monitoring and adjustment, and utilizing feedback [5,6]. Executive function also encompasses a person's ability to control their behavior, thus enabling them to be socially competent, while executive dysfunction can lead to adverse behavior, such as outbursts of emotion or declining motivation. This in turn can interfere with their social and occupational roles and impair their quality of life [6]. Patients with stroke, dementia, and traumatic brain injury frequently experience executive dysfunction. Therefore, proper assessment of executive function is of particular importance in the field of neurorehabilitation, especially when it comes to the planning of appropriate treatment programs. However, due to the difficulties in conceptualization of executive function, a clinically valid test for assessing patients with executive dysfunctions has not been developed enough.

Conventional tests of executive function, like the Wisconsin Card Sorting Test were not originally developed to assess deficits in real-world activities [7]. Several studies criticize these conventional tests as lacking sensitivity to identify everyday life problems that can occur in patients with clinically significant executive dysfunction, and thus highlight the potential invalidity of using these tools [7-9]. This criticism arises from the fact that conventional laboratory-based tests are designed to assess specific individual components of executive function while the patient is in a relatively well-supported and structured environment, far different from community settings [7-9]. In daily life, individuals encounter more complicated, novel, and challenging situations that require the active participation of both goal-directed and multi-tasking abilities.

For this reason, the benefits of assessing problem-solving abilities and performances in an open-ended naturalistic setting that reflects the real-world functioning of the individual has been emphasized [7,9]. Recently, there have been many attempts to develop such tests that can better identify everyday life difficulties that can arise in patients with executive dysfunction. For example, the Multiple Errands Test (MET), originally developed by Shallice and Burgess [10], requires participants to multitask efficiently in a shopping precinct. Additionally, a simplified version of the MET for a hospital setting has been developed, as well as site-specific versions for different hospitals [11,12]. The Multi-Level Action Test (MLAT) is a laboratory-based test that assesses an individual's ability to multitask [13]. However, due to the lack of standardization, its time-consuming nature, and the difficulty in scoring the MLAT, Schwartz et al. [14] developed the Naturalistic Action Test (NAT) by reducing the test time and simplifying the scoring of the MLAT. The NAT is a standardized, performance-based measure that assesses naturalistic action impairments that occur in everyday activities due to higher cortical dysfunction. The NAT discriminates well between participants with acquired brain injury (i.e., traumatic brain injury and stroke) and their matched controls, with adequate reliability and validity, and thus emphasizes the appropriateness of its use as a screening tool [14].

However, in Korea, there is currently no test available to evaluate executive function in a real-world setting, despite the above-mentioned critical clinical implications. The Kims Frontal-Executive Neuropsychological Test (K-FENT) [15] and subtest of frontal-executive functions in the Seoul Neuropsychological Screening Battery are frequently used to assess executive function in a clinical setting. However, these tests are a collection of traditional measures of executive function.

Thus, an assessment tool that can reflect and characterize an individual's performance of executive dysfunction in the everyday world is needed. In this study, we developed the modified Naturalistic Action Test (m-NAT) by adapting the NAT test for the Korean cultural background. In addition, we evaluated the psychometric properties of the m-NAT in Korean patients with impaired cognition.

In this study, we developed the m-NAT in order to adapt the NAT for Korean cultural background; this modified test discriminated well between patients with cognitive impairment and controls. The validity of the m-NAT was supported by the demonstration of significant correlation between neuropsychological measures of executive function and attention tasks, and daily life assessments. Furthermore, ICCs were high, illustrating very good inter-rater reliability.

Our finding that the m-NAT has good discriminative ability coincides with findings from previous studies that have assessed executive function under naturalistic conditions [10,11,14]. In 1991, the MET was proposed as a prototype for the assessment of real-world executive function; in this study, three traumatic brain injury patients with relatively high-level functioning displayed disorganized performances in terms of rule breaks and inefficiencies when compared with controls [10]. The hospital version of the MET (MET-HV), developed thereafter, also discriminated well between patients and controls, with 85% sensitivity and 95% specificity [11]. Furthermore, the NAT developed by Schwartz et al. [14] displayed good validity and inter-rater reliability when conducted on 100 patients with acquired brain injury and 28 matched controls.

However, these tests were developed to suit Western culture and required transcultural modification in order to be appropriate for Korean culture and lifestyle. The first task derived from the original NAT, "make toast and instant coffee," was applied in exact accordance with the original script since this is a part of the general dietary habit in Korea. However, the third task of "prepare a lunchbox with a sandwich" was modified to "prepare a lunchbox with rice and side dishes" due to the differences in Western and Korean dietary habits. Additionally, the task of "fill thermos with juice" was modified to "fill thermos with water." Through these modifications, m-NAT was simplified, but still displayed good discriminative ability. Moreover, the mean accomplishment score obtained from both groups was decreased in task 2 compared to task 1, thus demonstrating that the m-NAT possessed a gradation in task difficulty, like the original NAT.

In this study, the total number of errors committed by the patient group (mean, 6.9) was significantly higher than that of the control group (mean, 3.4). The number of omission errors was remarkably higher in the patient group when compared to controls; however, the commission error distribution was similar between groups. Of note, the commission errors in the patient group were more distinct and exaggerated than those observed in the control group. This finding may be partially explained by the theories of supervisory attentional system (SAS) and working memory model [8,20,21]. The SAS is involved in novel non-routine situations, and enables a subject to execute proper actions while suppressing improper actions, according to the priority of the actions [20,21]. The error frequency in our study corresponded to the complexity of the task. In addition, errors may also occur when subjects have difficulty in executing target actions without any external clues. This is usually a result of damage to their working memory, which temporarily saves and uses information.

However, errors were also observed in the control group that had no history of neurological or psychiatric conditions. According to Schwartz et al. [13], such results can be explained by the hypothesis of limited-capacity commodity. Other related general constructs, such as effort, attention, and cerebral activation can also be invoked. Individual performance variations in information processing and language tasks have been attributable in previous studies to the individual differences in available resource capacity [13,22]. Hence, a healthy person can still make errors because of their individual limited-capacity resources. Since the unique capacity resources of cognitive impaired patients is more restricted than that of controls, more significant and excessive errors can be observed. Of note, such errors tend to be omission errors when resource is severely restricted [13].

The construct validity of the m-NAT was examined against pre-evaluated neuropsychological tests. All measures of attention and executive functions demonstrated a significant correlation between accomplishment and m-NAT scores. The ecological validity of the m-NAT as a measure of deficits affecting functions in a real-world setting was supported by its significant correlation with measures of activities in daily living, such as FIM, K-IADL, EBS, and K-CFQ. In addition, its correlations with the results from the Stroop test, SART commission error, and Trail Making Test indicate that the m-NAT is an effective index for the measurement of selective attention, inhibitory capacity, and cognitive flexibility aspects of executive function. The original NAT developed by Schwartz et al. [14], was statistically significantly correlated with FIM, Dual Task Test, Self-Ordered Pointing Test, and Star Cancellation Test, thus validating the NAT as a measure of functional disability, arousal, visuospatial attention, and working memory. The Baycrest Multiple Errands Test (BMET), standardized by Dawson et al. [12], is an ecologically valid measurement of cognitive function, supported by a good correlation with measures of IADL and daily life functioning.

Classifying the various types of errors committed by individuals while performing the m-NAT and analyzing the characteristics of the errors may help highlight deficits in the problem-solving capacity of patients with cognitive impairment, in terms of issues that may arise during their daily life. The discriminative ability and ecological validity of traditional executive function tests, such as the Wisconsin Card Sorting Test have recently been considered insufficient since they have little relevance to executive function in actual life [7,9]. Direct observation of the strategy used by patients under naturalistic performance-based assessments will not only assist in diagnosing problems at the impairment level, but will also provide valuable information to help clarify an individual's functionality at the activity and participation level [8]. Ultimately, this information can assist health care providers in designing and tailoring more effective rehabilitation interventions for individual patients with difficulties in everyday life functioning. Hence, it is necessary to carry out these tests in real life settings.

The m-NAT, like the original NAT, is an advantageous performance-based test in which the test environment, materials, and cueing procedures are all standardized. Physical assistance due to factors, such as limited movement capabilities of the patient and/or weakness does not affect the score. It is also a suitable test for subjects with aphasia, as long as they can demonstrate comprehension of the task instructions. Thus, in a broad sense, the m-NAT can be administered to any subjects presenting with brain injury.

The inter-rater reliability study demonstrated high agreement on all components. This finding replicates those of previous studies regarding the NAT and BMET [12,14]. In this study, the standardized instruction manual was translated into Korean to ensure the examiner provided the same instructions to all participants. Therefore, the m-NAT is suitable for use by any medical practitioners in a clinical setting once they have received a simple tutorial and have been acquainted with the instruction manual.

This study had some limitations. First, our participants were relatively old and their level of education was low. Controls were mostly recruited from caregivers or family members of the patients. They were matched as closely as possible with the patient group, regarding age, gender, and duration of education. The mean age of the all participants was over 60 years and the mean education duration was 8.7 years. There was no statistical difference between patient and control groups for the number of errors in task 1. Therefore, we cannot exclude the possibility that both groups, comprised mainly of elderly subjects, were unfamiliar with this task due to cultural aspects. Nevertheless, the tasks were configured to ensure that the difficulty level increased in a stepwise manner, similar to the tasks described by Schwartz et al. [14]. For this reason, we speculate that the errors obtained during the first task were not remarkable for both groups. More importantly, the accomplishment and the m-NAT scores revealed a statistically significant difference between the two groups. Accordingly, we suggest that first task had sufficient discriminative ability to differentiate between the patients and the control participants.

Another limitation of this study is that the pre-evaluation neuropsychological assessment could have been more comprehensive. We did not assess individual cognitive domains, such as language, visuospatial function, and memory, which may have affected our results.

Lastly, our sample size was small and made up of heterogeneous brain lesions. A further study with a larger sample size is required to discern any characteristic findings of the m-NAT with regard to specific pathologies or disease status.

In conclusion, the m-NAT, which was modified from the NAT in order to assess everyday action impairments in a Korean population, discriminated well between control individuals and patients with acquired brain lesions. The validity of the m-NAT was supported by the demonstration of significant correlation with various measures of attention, executive functions, and daily life. In addition, inter-rater reliability was good. Based on these results, we suggest that the m-NAT is a useful tool for investigating real-world performance difficulties that are related to executive dysfunction in Korean patients. Finally, further research is required to promote the m-NAT, recruiting a larger number of subjects with various backgrounds.