Departments of Large Animal Clinical Sciences,1 Pathobiology, College of Veterinary Medicine, University of Florida, Gainesville, Florida 32610,2 Department of Pathobiology, University of Guelph, Guelph, Ontario, Canada N1G2W1,3 Department of Animal Hygiene, School of Veterinary Medicine and Animal Sciences, Kitasato University, Towada, Aomori 034-8628, Japan,4 Equine Medical Center of Ocala, Ocala, Florida 344745
Received 17 June 2002/ Returned for modification 16 October 2002/ Accepted 16 November 2002
| ABSTRACT |
|---|
|
|
|---|
| INTRODUCTION |
|---|
|
|
|---|
The farm-to-farm variation in the incidence of the disease likely reflects differences in environmental and management conditions as well as differences in the virulence of isolates. Unlike most environmental R. equi, isolates from pneumonic foals typically contain an 80- to 90-kb plasmid encoding a family of seven closely related virulence-associated proteins, designated VapA and VapC to VapH (4, 20, 23-26). Plasmid-cured derivatives of virulent R. equi strains lose their ability to replicate and survive in macrophages and fail to induce pneumonia in foals, confirming that the large plasmid is required for the virulence of R. equi (8, 27).
R. equi pneumonia of foals has traditionally been diagnosed based on culture or PCR amplification of the microorganism from a tracheobronchial aspirate. However, obtaining a tracheobronchial aspirate is not recommended for foals that present with severe respiratory distress, and the technique is not practical for routine diagnosis on large farms where the disease is enzootic and several foals must be sampled. There is therefore a need for a less invasive and more practical means of diagnosis. Although several assays detecting antibody to R. equi have been developed and are commonly used by practicing veterinarians for the diagnosis of R. equi infections, the diagnostic value of these assays has never been assessed in a clinical setting of heavy natural challenge. The objective of this study was to assess the performance of four currently available enzyme-linked immunosorbent assays (ELISAs) and an agar gel immunodiffusion test for diagnosis of R. equi pneumonia in foals.
| MATERIALS AND METHODS |
|---|
|
|
|---|
A tracheobronchial aspirate for bacterial culture and serum sample were obtained from each foal with clinical signs or radiographic or ultrasonographic evidence of pneumonia. For each foal diagnosed with bronchopneumonia, a serum sample was also obtained from the most closely age-matched clinically healthy pasturemate. Serum samples were frozen at -20°C until the end of the calendar year. Only serum samples from foals that remained clinically healthy throughout the entire breeding season were used as samples from clinically healthy foals (n = 24). If a foal bled as a healthy pasturemate subsequently developed respiratory, gastrointestinal, or joint disease, the sample was discarded.
Tracheobronchial aspiration was performed with the transtracheal approach or by passing a sterile, double-guarded aspiration catheter through the biopsy channel of a 1.2-m fiber optic endoscope, as described before (5). Bacteriologic culture of tracheobronchial aspirates obtained by use of this technique correlates closely with the results from the transtracheal approach (5). The tracheobronchial aspirate fluid was kept refrigerated until submitted to a local laboratory for aerobic bacteriologic culture between 5 min and 5 h after collection. Tracheobronchial aspirate fluid was inoculated onto standard bacteriologic culture media (Trypticase soy agar with 5% sheep blood and MacConkey agar) as well as on R equi-selective nalidixic acid-novobiocin-actidione (cycloheximide)-potassium tellurate medium (28). Bacterial pathogens were identified by use of standard identification procedures.
Serology by ELISA. Anti-R. equi antibody concentrations were measured with four ELISAs (ELISA-6939, ELISA-33701, ELISA-VapA, and ELISA-California) as previously described (11, 16, 21, 22). Antigens used were Tween 20 extract from a virulent plasmid-containing (ELISA-33701) or an avirulent (ELISA-6939) R. equi strain, supernatant antigen from clinical isolates (ELISA-California), and recombinant VapA (VapA-ELISA). Results for ELISA-6939 and ELISA-33701 are reported as optical density values (21, 22). ELISA-California was performed by the California Animal Health and Food Safety Laboratory System (Davis, Calif.),and results are reported as the ratio of positive serum or test serum absorbance to the negative control absorbance value (11). Results of the ELISA-VapA are reported as the last serum dilution giving a reading twice the optical density of a negative control serum diluted 1:10 (14, 16). Serum samples were coded so that the health status and culture results of the foals were unknown to the laboratory personnel.
Agar gel immunodiffusion. Fresh isolates of virulent R. equi were grown in dialysates of brain heart infusion broth for 30 days at 37°C. Bacteria were removed by high-speed centrifugation, and the resultant supernatant consisted of a concentrated composite of soluble R. equi antigens, including exoenzymes, cell wall components, capsular polysaccharides, and virulence-associated proteins, that served as a polyvalent antigen in the assay (6). Positive control serum samples were collected from naturally occurring microbiologically confirmed cases of R. equi-induced bronchopneumonia (3, 7).
Wells (4 mm) were punched into borate-buffered agar in plastic petri dishes. Serum samples (50 µl) from foals to be tested were added to wells arranged in hexagonal patterns around central wells that contained antigen (35 µl) so that they alternated with wells containing positive control serum (45 µl). Up to three serum samples could be tested per individual hexagonal pattern. The test wells were incubated in a humidified chamber at room temperature for 48 h, and development of precipitin lines was then assessed by viewing with indirect light against a dark background. The control serum samples produced up to four to five precipitin lines with the antigen, and the test serum samples formed lines of identity of varying intensity. Test serum samples were assessed as negative when the adjacent positive control precipitin line(s) passed directly into the test serum well.
A weakly positive reaction was defined as a line of identity of the test serum with the control serum line(s) that formed immediately in front of the test serum well. A positive reaction was defined as a line of identity of the test serum with the control serum line(s) that was roughly equidistant between the test serum and antigen wells. A strongly positive reaction was defined as a line of identity of the test serum with the adjacent positive control serum line(s) that formed closer to the antigen well than did the positive control serum line(s) (Fig. 1). Serum samples were coded so that the health status and culture results of the foals were unknown to the laboratory personnel.
|
The dependence of the diagnostic sensitivity and specificity on selected cutoff values was considered for test evaluation and comparison. The area under the curve (AUC) is a summary statistic of overall diagnostic performance. Receiver-operating characteristic plots for diagnostic tests with perfect discrimination between negative and positive reference samples have an AUC of 1.00 (100% sensitivity and 100% specificity). One can distinguish between a noninformative (AUC = 0.50), less accurate (AUC = 0.50 to 0.70), moderately accurate (AUC = 0.71 to 0.90), and perfect (AUC = 1.00) test (10). Comparison of the AUC between ELISAs was conducted with computer software from MedCalc (Mariakerke, Belgium).
While sensitivity and specificity are fixed characteristics of a test, the predictive values of positive and negative tests will vary with the prevalence of the disease. To simulate the diagnostic performance of each assay, the predictive values of positive and negative test results were estimated based on prevalences of R. equi pneumonia of 10% and 40%. Comparison of ELISA results between foals with R. equi pneumonia and clinically healthy foals was done with the Mann-Whitney U test. Results were considered significant if the value of P was <0.05.
For the agar gel immunodiffusion assay, sensitivity, specificity, and predictive values were also calculated at each cutoff point (weakly positive, positive, and strongly positive) with culture of R. equi from a tracheobronchial aspirate as the reference standard test. Predictive values of positive and negative test results were also estimated based on prevalences of R. equi pneumonia of 10% and 40%.
| RESULTS |
|---|
|
|
|---|
With receiver-operating characteristic analysis, the overall diagnostic performance of ELISA-California (AUC = 0.68; 95% confidence intervals = 0.56 to 0.79) was significantly higher (P = 0.03) than that of ELISA-VapA (AUC = 0.55; 95% confidence intervals = 0.44 to 0.65) for diagnosis of R. equi pneumonia (Table 1). Differences in the AUC of the receiver-operating characteristic analysis between other ELISAs were not statistically significant.
|
|
| DISCUSSION |
|---|
|
|
|---|
Before investigating the value of serology for early identification of foals with R. equi pneumonia on farms where the disease is enzootic, the first step was to determine if such an assay could differentiate foals with clinical R. equi pneumonia from apparently healthy animals. In the present study, it is likely that many foals used as clinically healthy controls may have had a subclinical infection with R. equi. In one study conducted on a farm with enzootic R. equi pneumonia, 77 (35%) of 216 foals sampled had tracheobronchial aspirate cultures positive for R. equi but no signs of respiratory disease (2). For this reason, bacteriologic culture or PCR amplification of R. equi from a tracheobronchial aspirate should always be interpreted in the context of clinical findings, and although they are considered gold standards for diagnosis in pneumonic animals, these assays should never be used as screening tests for clinically healthy animals.
Although anti-R. equi antibody concentrations in foals with R. equi pneumonia were significantly higher than those of healthy controls with ELISA-California and ELISA-6939, the extensive overlap in individual titers between groups indicated the limited value of these assays for diagnostic purposes (Table 2). To evaluate the diagnostic performance of each assay, sensitivity, specificity, and predictive values were calculated at various cutoffs. For most ELISAs, selection of a low cutoff resulted in good sensitivity but poor specificity (Table 1). Increasing the cutoff value resulted in better specificity but at the detriment of sensitivity (Table 1).
The choice of the most appropriate cutoff for a diagnostic test should include a consideration of (i) the distribution of results in two different populations, normal animals and animals with the disease (Table 1), (ii) the prevalence of disease in the population to be tested, and (iii) the consequences of false-positive and false-negative tests results (17). On a farm with enzootic R. equi infections, the consequence of missing the diagnosis of an infected animal (false-negative) may be, in the worst-case scenario, the death of the infected animal. Therefore, a very sensitive test is essential.
In the present study, at low cutoffs providing adequate sensitivity (>80%), specificity was less than 50% regardless of the assay considered, resulting in poor predictive value of a positive test. At such low cutoff values, the best negative predictive values on a farm where the prevalence of the disease is high were obtained with ELISA-California and ELISA-6939 (Table 1). The best diagnostic performance was achieved with ELISA-California at a cutoff of 40%, resulting in a sensitivity of 59% and a specificity of 88%. At a prevalence of 40%, this would result in positive and negative predictive values of 76%.
In one study, serological surveillance at 30 and 45 days of age with ELISA-6939 and a cutoff value of 0.3 was proposed for early detection of R. equi-infected foals on farms with enzootic infections (13). In another study with ELISA-6939, 20 (95%) of 21 foals with culture-confirmed R. equi pneumonia had an optical density value of 0.3 or greater, indicating good sensitivity of the assay (12). However, the diagnostic performance of the assay (sensitivity, specificity, and predictive values) on a farm where R. equi infections are enzootic was not determined. In the present study, ELISA-6939 with a cutoff of 0.3 gave a sensitivity of 41.5% and a specificity of 69.6%, resulting in poor positive and negative predictive values.
The poor diagnostic performance of serological assays in the present study is likely because widespread exposure of foals to R. equi at a young age leads to antibody production without necessarily inducing clinical disease (11). In addition, maternally derived antibody causes positive reactions with ELISAs, which further confounds their interpretation (11, 16).
Even on farms where the disease is enzootic, most R. equi isolates in the environment lack the VapA-encoding plasmid and are avirulent (18). In one study, foals experimentally infected with virulent R. equi seroconverted with ELISA-VapA 2 weeks postinfection, whereas foals administered the avirulent plasmid-cured derivative did not, suggesting that a diagnostic advantage of ELISA-VapA might be the failure to detect exposure to nonvirulent isolates (9). In the present study, the 95% confidence intervals of the area under the receiver-operating characteristic curve for ELISA-VapA included 0.5, indicating that the assay's performance was no better than flipping a coin. The poor diagnostic performance of ELISA-VapA in the present study confirms that many clinically healthy foals have antibody to virulent R. equi (19).
Agar gel immunodiffusion assays, such as the one used in the present study, are less sensitive than ELISA (21). As a result, adult horses are negative for precipitating antibody on agar gel immunodiffusion tests, and there are no positive reactions from colostral transfer of anti-R. equi antibody (7). In the present study, the agar gel immunodiffusion test with a strongly positive result as the cutoff resulted in poor sensitivity but good specificity.
In conclusion, the present study shows that currently available serological assays do not permit distinction between foals with R. equi pneumonia and clinically healthy pasturemates when performed on a single sample. Using these serological assays for screening or diagnosis of R. equi pneumonia in foals would result either in unnecessary therapy of many foals or in missing several infected foals, depending on the cutoff value used.
| ACKNOWLEDGMENTS |
|---|
We thank Vivian Nicholson for technical assistance.
| FOOTNOTES |
|---|
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Antimicrob. Agents Chemother. | Clin. Microbiol. Rev. | Infect. Immun. |
|---|---|---|
| J. Clin. Microbiol. | J. Virol. | ALL ASM JOURNALS |