Validation of the Portuguese version of Addenbrooke ’ s Cognitive Examination III in mild cognitive impairment and dementia

Results. The reliability of ACE-III was very good (α = 0.914). ACE-III significantly differentiated the 3 groups. The receiver operating characteristic (ROC) curves significantly favored ACE-III in comparison to another screening test – MoCA. ACE-III presented higher levels of sensitivity and specificity. Its total score correlated positively with the results on MoCA (ρ = 0.912; p < 0.001) and negatively with a depression scale (ρ = –0.505; p < 0.001).


Introduction
Dementia and cognitive impairment constitute a serious worldwide problem.Dementia is in the top 10 of the leading causes of the burden of disease in high-income countries. 1he prevalence of dementia in Western Europe in people over 60 years of age is estimated at 7.3% and increases exponentially with age, doubling every 6.3 years. 2 Cognitive assessment is central to the diagnoses of cognitive impairment and dementia, and should be performed in all patients in the early stages of the disease. 3Furthermore, patients with mild cognitive impairment (MCI) should be recognized due to an increased risk of subsequent dementia. 4herefore, the use of screening tests with high diagnostic utility, simplicity and celerity, and with normative data for the target population, is important in a time-constrained environment. 5,6ecently, the 3 rd version of Addenbrooke's Cognitive Examination (ACE-III) has been developed in order to improve certain weaknesses in some domains, such as repetition, comprehension and visuospatial items. 7With a mean administration time of 15 min, ACE-III meets the requirements of a screening test, evaluating different cognitive dimensions and enabling an overall picture of a subject's neurocognitive functioning.
The 3 rd version of Addenbrooke's Cognitive Examination is scored out of 100 and assesses 5 cognitive domains: attention (max score = 18 points), assessed through orientation, immediate verbal evocation of words and serial subtraction tasks; nemory (max score = 26 points), evaluated by verbal delayed recall (free recall and recognition), verbal learning and semantic memory tasks; verbal fluency (max score = 14 points), including phonetic and semantic fluency tasks; language (max score = 26 points), assessed through comprehension, repetition, naming, reading, and writing tasks; visuospatial (max score = 16 points), including visuoconstructive tasks (e.g., copying a cube and drawing a clock) and spatial perceptive tasks (counting dots and identifying incomplete letters). 7][9][10] However, there is a need to further determine the validity of the test in different clinical populations and to compare it with other tests. 7,10,11his study aims to determine the psychometric properties of the Portuguese version of ACE-III, namely: reliability and discriminant validity (sensitivity and specificity) in the identification of MCI and dementia, in comparison to other neuropsychological screening tests, as well as to establish its concurrent and divergent validity.

Participants
The study included a sample of 90 participants distributed into 3 groups: the Control group, made up of 30 subjects ranging between 61 and 81 years of age from the ACE-III Portuguese normative study, without any subjective complaint of memory loss and completely independent in everyday activities; the MCI group, composed of 30 subjects ranging between 47 and 79 years of age and fulfilling Petersen's criteria for MCI; and the Dementia group, made up of 30 subjects ranging between 59 and 87 years of age, with dementia according to National Institute of Neurological and Communicative Disorders and Stroke (NINCDS) and the Alzheimer's Disease and Related Disorders Association (ADRDA) criteria (Table 1). 10,12,13Individuals with a prior history of neuropsychiatric or systemic pathologies, liable to directly interfere with neurocognitive functioning, and those who are illiterate, were excluded.
Groups did not differ according to age (p = 0.673), schooling (p = 0.593) or gender (p = 0.679).The participants included in the MCI and Dementia groups were recruited from the neurology outpatient clinics of 3 different Portuguese hospitals.Clinical diagnoses were established by those services and complemented with the results of the Clinical Dementia Rating (CDR) to determine the severity of MCI and dementia.

Neuropsychological assessment
5][16] The MoCA test was selected to determine convergent validity of ACE-III due to its high specificity and sensitivity in detecting MCI and dementia. 14Geriatric Depression Scale was applied in order to establish divergent validity of ACE-III.
The scoring criteria of ACE-III followed the norms of the English version of the test. 17

Procedures
This study was approved by the ethical committees of the Alto Minho Hospital Center (Viana do Castelo, Portugal), the Alto Ave Hospital Center (Guimarães, Portugal) and the Trás-os-Montes and Alto Duoro Hospital Center (Vila Real, Portugal).All participants gave their informed consent.
The assessment occurred in a single session in a private room.The CDR and GDS tests were administered first, followed by ACE-III and MoCA.The order of the 2 neurocognitive tests alternated for all the participants in order to control for fatigue.

Statistical analysis
Statistical analysis was carried out using the program IMB Statistics v. 22 for Windows (IBM, Armonk, USA).
Reliability was determined by Cronbach's alpha.The comparison of the groups' performance from the tests was made through univariate analysis of variance (ANOVA) with subsequent comparisons by the Scheffe's test.The sensitivity and specificity of ACE-III in the distinction of the 3 groups were determined by the receiver operating characteristic (ROC) curves.Pearson's correlations were used to establish concurrent and divergent validity of ACE-III.
Results with p ≤ 0.05 were considered significant.

Results
The value of Cronbach's alpha for the ACE-III total score is considered very good (α = 0.914).
Table 2 shows the results obtained and the comparisons between the groups from the neuropsychological tests.The ANOVA was significant for each test and for the subscales of ACE-III.The Control and MCI groups' results were significantly different on most of the tests, except the Attention and Visuospatial subtests.The MCI and Dementia groups did not differ regarding the level of depression.
Table 3 presents the characteristics of the ROC curves generated by the ACE-III and MoCA scores (Fig. 1-3).All the generated curves are significant for both tests, but the area under the curve for the 3 clinical settings favors ACE-III.Cut-off scores and corresponding values of sensitivity and specificity were extracted (Table 4).A cut-off score of 82 on ACE-III differentiates the Control and MCI groups with a sensitivity of 87.5% and a specificity of 57.14%.A cut-off score of 66 on ACE-III differentiates the MCI and Dementia participants with a sensitivity of 89.9% and a specificity of 71.43%.A cut-off score of 74 on ACE-III discriminates the Control and Dementia group patients with a sensitivity of 100% and a specificity of 78.57%.Therefore, ACE-III holds sensitivity and specificity values higher than MoCA in all the domains.The ACE-III total score correlated positively with the results on MoCA (ρ = 0.912; p < 0.001) and negatively with the GDS scores (ρ = -0.505;p < 0.001).

Discussion
The present study shows that the Portuguese version of ACE-III has very good reliability and high diagnostic capacity in the context of MCI and dementia, even when compared with another widely used screening    Reliability determined by Cronbach's alpha was high and very similar to that obtained in a Spanish study of the test (α = 0.927), and also slightly higher than in a study with the original version (α = 0.88) and in a Portuguese normative study (α = 0.732). 7,8,10he ACE-III results were significantly different between the 3 groups.The MCI group obtained lower results in comparison to the Control group in the majority of the domains, except for the Attention and Visuospatial domains.The differentiation of the Memory, Fluency and Language domains could be related to the fact that the majority of the participants in the MCI group had a multi--domain form of MCI.In addition to the registered differences between the Control and MCI groups, the generated ROC curve was significant and with an area under the curve greater than the one generated by the MoCA test.Unfortunately, there was no possibility to compare this remark regarding the Portuguese version of ACE-III to other versions, since, to our knowledge, this is the first study to analyze the discrimination capacity of ACE-III in MCI.However, in comparison to other widely used screening tests -like the Mini-Mental State Examination (MMSE) with a sensitivity level for MCI of 64.6%, and the MoCA test with a sensitivity level of 84.1% -the sensitivity values obtained for different ACE-III cut-off points are encouraging. 14,18Future studies must determine whether the specificity values found in our study are due to the characteristics of our MCI sample or to the specifications of the instrument.
In our study, ACE-III also demonstrated good discrimination capacity between the MCI and Dementia groups.In direct comparison with MoCA, the cut-off scores were associated with more balanced levels of sensitivity and specificity.
The discrimination between the Control and Dementia groups was very satisfactory, with a significant area under the curve and values of sensitivity and specificity close to those of other versions of the test.In the original version, a cut-off score of 82 revealed a sensitivity of 93% and a specificity of 100%. 7The high level of specificity found in this study could be due to the inclusion of a subgroup of participants with a diagnosis of primary progressive aphasia; since the test is highly weighted toward language tasks, it could have positively affected discrimination capacity.A study of the Spanish version of ACE-III determined levels of sensitivity (83.1%) and specificity (80.4%) closer to ours. 8In fact, the Dementia group in that study was mainly composed of participants diagnosed with Alzheimer's disease and vascular dementia in the mild stage, as in our study.However, the cut-off score in the Spanish study was lower (65.5).This may be related to the fact that the participants in that study were older, as it is known that age is a determinant for performance in ACE-III. 8,10nce more, discrimination capacity of ACE-III in direct comparison to MoCA was slightly higher.One of the desirable properties of a cognitive assessment tool is its sensitivity to the deficits of aging and dementia. 19The inclusion of "true positives" and the exclusion of "true negatives" can accelerate the diagnostic process through the identification of cases for further testing.However, since further assessment requires more time and effort, specificity must be also observed.Thus, the suggested ACE-III cut-off scores were based on the equilibrium of this ratio (sensitivity/ specificity).
Concurrent validity was also established, which is in line with other language versions of ACE-III.The original version found high correlations between the domains of ACE-III and standard cognitive tests (e.g., the Rey Osterrieth Complex Figure Test), and the Spanish version established convergence through the Kappa index with MMSE. 7,8The negative correlation between ACE-III and the measure of depression (GDS) was expected, since the inverse relation between depressive mood and cognitive functioning is well-established. 20

Conclusions
The current study has some limitations.The reduced number of participants makes it hard to determine the diagnostic utility of ACE-III in the differentiation between various types of MCI and dementia.Concurrent validity could have been determined by using a more comprehensive neuropsychological assessment battery; however, it would have been very time-consuming, and since the standard neuropsychological assessment was so diverse in the 3 hospitals, we were unable to present additional neuropsychological data common to all the participants.Therefore, in conclusion, the Portuguese version of ACE-III has proven to be a valid instrument in the context of MCI and dementia.In comparison to MoCA, ACE-III shows better discrimination capacity.This is especially relevant for the prompt identification of patients that need a more extensive neuropsychological and neurological assessment.

Fig. 1 .
Fig. 1.ROC for the discrimination between the Control and MCI groups on ACE-III and MoCA ROC -receiver operating characteristic; MCI -mild cognitive impairment; ACE-III -Addenbrooke's Cognitive Examination III; MoCA -Montreal Cognitive Assessment.

Fig. 2 .
Fig. 2. ROC for the discrimination between the MCI and Dementia groups on ACE-III and MoCA ROC -receiver operating characteristic; MCI -mild cognitive impairment; ACE-III -Addenbrooke's Cognitive Examination III; MoCA -Montreal Cognitive Assessment.
, ACE-III also holds concurrent and divergent validity.

Fig. 3 .
Fig. 3. ROC for the discrimination between the Control and Dementia groups on ACE-III and MoCA ROC -receiver operating characteristic; ACE-III -Addenbrooke's Cognitive Examination III; MoCA -Montreal Cognitive Assessment.

Table 1 .
Sociodemographic and clinical characteristics of the sample

Table 2 .
Differences between the groups on the GDS, the MoCA and ACE-III

Table 3 .
Characteristics of the ROC curves generated by ACE-III and MoCA ROC -receiver operating characteristic; ACE-III -Addenbrooke's Cognitive Examination III; MoCA -Montreal Cognitive Assessment; MCI -mild cognitive impairment.

Table 4 .
Sensitivity and specificity of ACE-III and MoCA between the groups and for different cut-off scores ACE-III -Addenbrooke's Cognitive Examination III; MoCA -Montreal Cognitive Assessment; MCI -mild cognitive impairment.