External quality assessment (EQA) of the performance of laboratories participating in the European Antimicrobial Resistance Surveillance Network (EARS-Net) 2023

The aims of the EARS-Net EQA exercises are: 1) to assess the accuracy of species identification reported by individual participating laboratories; 2) to assess the accuracy of qualitative AST results reported by individual participating laboratories; and 3) to evaluate the overall comparability of routinely collected test results, between laboratories and EU/EEA countries. In EARS-Net EQA exercises, eligible laboratories are identified by National EARS-Net EQA Coordinators, designated by the Coordinating Competent Body in each EU/EEA country. Participating laboratories identify the species of six bacterial strains and submit AST results for the antimicrobial agents included in EARS-Net surveillance, using the methods that they apply routinely .

In 2023, the panel of six EQA strains consisted of Escherichia coli , Klebsiella pneumoniae (two strains), Enterococcus faecalis , Enterococcus faecium and Acinetobacter baumannii (Table 1). The E. coli strain, and one of the two K. pneumoniae strains had been included in previous EARS-Net EQA exercises. The E. coli strain (‘2023 EARS-Net 1’) was the most challenging strain in the 2022 EQA panel (strain ‘2022 EARS-Net 2’) − i.e. the strain with the most incorrect results. The K. pneumoniae strain (‘2023 EARS-Net 4’) was the strain with the highest concordance of AST results in the 2021 EQA exercise (strain ‘EARS-Net KPN 21.1’). It was included in the 2023 EQA panel to facilitate comparison of the performance of AST methods with the more challenging K. pneumoniae strain in the 2023 panel (strain '2023 EARS-Net 2'). 

On 12 June 2023, the six strains were distributed via the National EARS-Net EQA Coordinators to 951 laboratories in all 30 EU/EEA countries. An EQA webpage was opened to receive submission of results between 14 June and 11 August 2023. As in previous EARS-Net EQA exercises [2-4], concordance of species and AST interpretations with the expected results was defined as ‘excellent’ (≥95% of interpretations in concordance with expected results), ‘very good’ (>90% to <95%), ‘good’ (>85 to ≤90%) or ‘satisfactory’ (>80 to ≤85%). 

Results were submitted by 871 laboratories and two were excluded from the analysis because they did not submit data on AST interpretation. Species identification was evaluated for 869 laboratories, and 5 150 (99.1%) of the 5 197 reported species were correct. There was ‘excellent’ concordance for each of the six strains (98.8 to 99.3% concordance). Four laboratories reported the wrong species for all submitted strains. 

Interpretation of AST results was only evaluated if the species had been correctly identified. The evaluation was performed according to the clinical breakpoints in the European Committee on Antimicrobial Susceptibility Testing (EUCAST) Clinical Breakpoints Tables v13.0 [6], with the EUCAST categories ‘susceptible, standard dosing regimen (S), ‘susceptible, increased exposure’ (I), and ‘resistant’ (R). In the 2023 EARS-Net EQA exercise, the scoring system for the evaluation of interpreted results included an  assessment of the ‘level of difficulty’ and the ‘severity of error’ of the submitted AST result for each speciesantimicrobial agent combination. The scoring system was similar to that applied in the 2022 EARS-Net EQA exercise, with the exception that missing results did not generate a negative score. 

The ‘level of difficulty’ had two levels (‘easy’ and ‘difficult’), reflecting the magnitude of the risk of getting the AST result wrong. ‘Easy’ results were those with expected AST results far from the breakpoint, where the categorisation was obvious. Conversely, ‘difficult’ results were those close to the breakpoint or inside the area of technical uncertainty (ATU), or those for which breakpoints had been recently changed or added. Consequently, the scoring system allocated a higher score to ‘difficult’ results than ‘easy’ results, and penalised errors for ‘easy’ results more severely than errors for ‘difficult’ results.

The severity of error was divided into three levels: very major error (VME), which indicated reporting false susceptibility (i.e. reporting S or I, instead of R); major error (ME), which indicated reporting false resistance (i.e. reporting R, instead of S or I) and no error. The scoring system penalised VMEs more severely for ‘easy’ results than for ‘difficult’ results and did not penalise MEs if the test was considered ‘difficult’.
The reported interpretations of AST results were evaluated for 865 laboratories (excluding the two laboratories that did not submit data on AST interpretation and the four laboratories that reported the wrong species for all submitted strains).

Among the 53 272 evaluated AST results, the most frequently reported methods for AST had very good or excellent concordance with the expected result (Table 13). These were automated systems (53.9% of all tests; of which 94.2% were correct), followed by disk or tablet diffusion (27.3% of all tests, of which 95.2% were correct) and minimum inhibitory concentration (MIC) methods, including broth microdilution (10.7% tests, of which 95.6% were correct).

Overall, the submitted AST interpretations were in ‘very good’ concordance with the expected results, with 94.7% (50 441 out of 53 272) being correct. Otherwise, MEs and VMEs were observed for 3.2% and 2.2% of interpretations, respectively. At country level, 17 countries (Austria, Belgium, Bulgaria, Croatia, Cyprus, Denmark, Finland, Iceland, Italy, Liechtenstein, Luxembourg, Malta, Norway, Poland, Slovakia, Slovenia, and Sweden) achieved an ‘excellent’ level of concordance with the expected interpretation of AST results, and 13 countries
(Latvia, Czechia, Estonia, France, Germany, Greece, Hungary, Ireland, Lithuania, Netherlands, Portugal, Romania, and Spain) achieved a ‘very good’ level concordance. At laboratory level, 50.7% (n=439) of the laboratories achieved an ‘excellent’ level of concordance, 39.8% (n=345) achieved a ‘very good’ level of concordance, 7.7% (n=67) achieved a ‘good’ level of concordance, 1.2% (n=10) achieved a ‘satisfactory’ level, and 0.6% (n=5) were below the ‘satisfactory’ level.

There were 74 species-antimicrobial agent combinations tested for antimicrobial susceptibility in the 2023 EARSNet EQA exercise, and the vast majority had results in ‘excellent’ concordance with the expected results (n=58 or 78.4% of the combinations). A ‘very good’ level of concordance was achieved for eight combinations (10.8%). The species-antimicrobial agent combination with the lowest level of concordance was amikacin for the E. coli strain, with only 29.2% of correct interpretations and deviations that were MEs (S → R) reported for all of the frequently used methods. Low concordance was also observed for AST of other antimicrobial agents for the E. coli strain (piperacillin-tazobactam with 40.6% of concordance, cefepime with 83.4%, ceftazidime with 87.5%), for one of the K. pneumoniae strains (strain ‘2023 EARS-Net 2’) (amikacin with 66.7%, cefepime with 78.9% and imipenem with 82.5%), and the A. baumannii strain (amikacin with 70.0%). All remaining species-antimicrobial agent combinations achieved at least a ‘very good’ concordance (>90%).