| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Previous Article | Next Article ![]()
Clinical and Vaccine Immunology, October 2006, p. 1125-1130, Vol. 13, No. 10
1071-412X/06/$08.00+0 doi:10.1128/CVI.00236-06
Copyright © 2006, American Society for Microbiology. All Rights Reserved.
Department of Veterinary Population Medicine, College of Veterinary Medicine, University of Minnesota, St. Paul, Minnesota 55108,1 School of Veterinary Medicine, University of Wisconsin, 2015 Linden Drive, Madison, Wisconsin 53706-1102,2 University of Minnesota Veterinary Diagnostic Laboratory, 1333 Gortner Ave., St. Paul, Minnesota 55108,3 Minnesota Board of Animal Health, St. Paul, Minnesota,4 University of Pennsylvania, Kennett Square, Pennsylvania5
Received 23 June 2006/ Returned for modification 4 August 2006/ Accepted 18 August 2006
| ABSTRACT |
|---|
|
|
|---|
| INTRODUCTION |
|---|
|
|
|---|
Diagnosis of M. avium subsp. paratuberculosis infection is challenging because of the pathogen's slow growth and the lack of diagnostic tests sensitive enough to detect most subclinically infected cattle, many of which intermittently shed the pathogen, thus serving as sources of infection of susceptible cattle. Detection of the pathogen itself is the most definitive method of diagnosis since the pathogen can often be detected during both subclinical and clinical stages of the disease, but the typical method of pathogen detection (bacterial culture of feces) requires up to 16 weeks of incubation and is labor-intensive. Contamination of cultures is an added problem because of the frequent inability of current decontamination protocols to inactivate all of the nonmycobacterial microflora in feces, resulting in contamination of some cultures during the lengthy incubation period. These issues have led to the widespread use of diagnostic methods to detect the host immune response to M. avium subsp. paratuberculosis. Available serologic assays, however, are problematic as well, both in detecting infected cattle and in misclassifying uninfected cattle (23). Because no rapid diagnostic test to detect most subclinically infected cattle prior to fecal shedding is currently available, test-and-cull strategies for control of JD are not, by themselves, cost-effective (12) and cattle producers must implement lengthy (up to 10 years) herd control programs. Currently needed are high-volume diagnostic tests suitable for detection of this pathogen in dairy and beef herds to facilitate efficient operation of state and national control programs, including animal movement controls. As new diagnostic tests and testing methods become available, rational design of herd "certification" programs for noninfected herds and paratuberculosis control programs for M. avium subsp. paratuberculosis-infected herds require precise, objective estimation of the sensitivity and specificity of available diagnostic tests to define which tests are most cost-effective for use in these programs (5, 6).
Cattle shed M. avium subsp. paratuberculosis in feces at various levels, depending upon the stage of disease of individual animals (14). Quantification of fecal shedding is routinely performed by certain laboratories, especially those using Herrold's egg yolk (HEY) medium, and animals are commonly categorized into levels (light, moderate, and heavy fecal shedding) corresponding to the number of colonies observed per tube. This quantification provides an estimate of the risk of transmission from cattle at various stages of clinical disease through the fecal-oral route. In addition, this information helps to estimate the risk of transmission through other routes (milk, colostrum, and placenta) because cattle at later stages of infection are more likely to be infectious to susceptible cattle than are cattle at earlier stages. Rapid identification of these high-risk cattle enables management of cattle by risk category to reduce transmission, thereby contributing to overall JD control.
High-throughput PCR tests have the potential to provide rapid (less than 1 week) detection of M. avium subsp. paratuberculosis at a cost comparable to or less than that of conventional culture. Estimation of the sensitivity and specificity of these assays is necessary, however, before they are implemented routinely for JD diagnosis. The objective of this study was to evaluate the validity of a TaqMan-based PCR assay compared to those of other available assays for detection of M. avium subsp. paratuberculosis in infected and uninfected dairy cattle.
| MATERIALS AND METHODS |
|---|
|
|
|---|
Two types of controls were used for the fecal PCR assay. Extraction controls were tested once per extraction. For negative extraction controls, DNA was extracted from fecal samples from cattle from dairy herds known to be uninfected (level 4 of the Voluntary Johne's Disease Test Negative Program for Cattle) and tested with the MAV2 TaqMan PCR assay. Positive extraction controls were created by extracting DNA from fecal samples from cattle known to be uninfected that was then spiked with M. avium subsp. paratuberculosis and tested with the MAV2 TaqMan PCR assay. In the MAV2 TaqMan PCR assay, positive template controls consisted of dilutions of M. avium subsp. paratuberculosis DNA (previously extracted), with two positive template controls per plate. Eight no-template controls (NTC) consisting of nuclease-free water were tested per test plate. To determine the threshold above which diagnostic samples were considered positive, the high and low NTC values were first deleted and the mean of the remaining six values was determined for use in analyses described below.
The reactions in the plates were preread at 518 nm (6-carboxyfluorescein [6-FAM]). After the reactions were analyzed by PCR as outlined above, the reactions were postread at the same wavelength. ABI 7000 System Detector Software normalized the reporter dye (6-FAM) and calculated Rn, the fluorescence emission intensity of the reporter dye divided by the fluorescence emission intensity of the passive reference dye. The degree of fluorescence due to probe hydrolysis (interpreted as the degree of amplification) was determined by calculating the delta Rn, the difference between the Rn value after the PCR took place and the Rn value before the PCR for each sample. The cut point was calculated from the following equation: 6 x (standard deviation of the NTC x t) + average NTC value. The value t (5.894) was provided by ABI. The Johne's ISMAV2 TaqMan PCR assay detection limit was 7.5 x 103 CFU/ml of isolated M. avium subsp. paratuberculosis, estimated at 7.5 x 103 CFU/g of feces.
Sample collection for validation of PCR assay. The sensitivity and specificity of the PCR test for detection of M. avium subsp. paratuberculosis were estimated by using fecal samples collected from dairy cattle in U.S. dairy herds and sampling methods previously described by the study investigators (7). To estimate test specificity, we used fecal samples from dairy herds known to be free of paratuberculosis. The study herds included seven dairy herds at level 4 of the Voluntary Johne's Disease Test Negative Program for Cattle (n = 347 cows). Fecal samples from each cow were tested by three bacterial culture methods to provide further evidence of lack of infection. To estimate test sensitivity, we collected fecal samples from 1,808 cattle in seven dairy herds known to be infected. Criteria for herd selection included an M. avium subsp. paratuberculosis seroprevalence of >10% and no regular testing and culling for JD, to provide a natural spectrum of infected cattle in a study population not artificially influenced by prior culling of test-positive cattle.
From each cow that had calved at least once within the study herds, fecal, blood, and milk samples were collected simultaneously, labeled, and transported with refrigeration for processing. Fecal samples were submitted fresh, within 48 h of collection, to three different laboratories for laboratory testing, including the MVDL, the University of Wisconsin School of Veterinary Medicine, and the University of Pennsylvania. The MVDL performed conventional bacterial culture by the HEY agar sedimentation method (21) and performed the fecal PCR assays. The University of Wisconsin School of Veterinary Medicine cultured fecal samples by the BACTEC filter concentration method (BACTEC) (4), and the University of Pennsylvania performed M. avium subsp. paratuberculosis culture by the HEY-centrifugation technique (CENT) (9, 22, 23). Blood samples were centrifuged, and sera were harvested to perform ELISAs at the University of Wisconsin Johne's Testing Center with two different kits for serum antibody detection (ELISA A [IDEXX Laboratories, Inc., Westbrook, ME] and ELISA B [CSL/Biocor, Omaha, NE]). Milk samples were refrigerated and sent to a testing laboratory to perform a milk ELISA (ELISA E [Antel Biosystems, Inc., Lansing, MI]). All antibody assays were performed by each laboratory according to the manufacturers' instructions and interpreted as prescribed.
Classical analysis. PCR assay specificity was defined as the percentage of samples yielding a negative PCR assay result among the M. avium subsp. paratuberculosis-free fecal samples from noninfected herds. PCR assay relative sensitivity was estimated (with 95% confidence intervals [CIs]) within the infected cattle population by comparing fecal PCR results to the M. avium subsp. paratuberculosis status of each sample defined by all three culture methods collectively. Differences in relative specificity and sensitivity between the PCR assay and the ELISAs were evaluated with McNemar's tests of association. Because of the variability in fecal culture results among laboratories and culture methods, we defined a fecal sample as positive for M. avium subsp. paratuberculosis if at least one of three laboratories performing a fecal culture procedure designated the sample positive.
The relative sensitivity of the PCR assay was also estimated by M. avium subsp. paratuberculosis fecal shedding level by comparing the mean M. avium subsp. paratuberculosis culture score of each fecal sample to the PCR assay result. Each laboratory using solid medium ranked the number of M. avium subsp. paratuberculosis bacteria recovered as 1+ (fewer than 10 colonies per tube), 2+ (mean of 10 to 49 colonies per tube), 3+ (mean of 50 to 99 colonies per tube), or 4+ (>100 colonies per tube), considering all of the culture tubes inoculated. The laboratory using liquid medium also developed a comparable ranking system (1+, 2+, 3+, and 4+) for test results by categorization of time to detection. A composite M. avium subsp. paratuberculosis score based on reports from the three independent laboratories was calculated from the arithmetic mean of scores. PCR test results were compared to results of fecal culture and serum and milk serologic assays. Standard errors (SE) for CI estimation were calculated with the formula SE = pq/n, where p is the proportion of samples test negative, q is the proportion of samples not test negative, and n is the sample size. We evaluated assay differences in relative sensitivity between fecal shedding levels with McNemar's test of association after collapsing data into categories based on the fecal culture shedding levels described above (no shedding, light-to-moderate shedding [1+ to 2+], and heavy shedding [3+ to 4+]).
Bayesian analysis.
The true sensitivity and specificity of the PCR assay, CENT fecal culture, serum ELISA A, and serum ELISA B were estimated by the Bayesian method. The Bayesian method can be used to estimate the accuracy of diagnostic tests in the absence of a "gold standard." The method takes into account the uncertainty in the estimated sensitivities and specificities of the tests and allows prior information or expert knowledge to be incorporated into the analysis. Moreover, the parameter estimate based on Bayesian analysis has a probability distribution which allows direct probability interpretation (13). The analysis by the Bayesian method involves three main components: likelihood function L(y|
), prior distribution g(
), and posterior distribution f(
|y). These three components are combined through Bayes' rule, f(
|y)
L(y|
) x g(
), where
denotes proportionality (3). In the present study, the likelihood function was derived from the test results from the seven infected dairy herds, the prior distributions presented prior knowledge of the sensitivity and specificity of each testing method, and the posterior distributions combined all information about the parameters of interest (sensitivity and specificity) from the likelihood function and the prior distributions. Then, the estimation of the parameters of interest from the posterior distributions was carried out with the Gibbs sampler, an iterative algorithm that constructs a Markov chain and permits empirical estimation of posterior distributions (3).
In the present study, we used a Bayesian model for estimation of the sensitivities and specificities of four tests of multiple populations, which was modified from a model for estimation of the validity of two correlated tests of multiple populations as described elsewhere (2). Our model accounted for the possible effect of conditional dependence between two tests that measured similar biological processes by including sensitivity covariance (Covse) and specificity covariance (Covsp) for bacterial culture and fecal PCR assay and for the two serum assays in the model. The magnitudes of Covse and Covsp are affected by the magnitudes of the test sensitivities and test specificities, respectively, and their limits were defined previously (10).
The prior information about test sensitivities and specificities was provided by a consensus of five experts selected by USDA-APHIS for development of optimal testing strategies for control of JD in cattle (Michael Collins, Ian Gardner, Franklyn Garry, Allen Roussel, and Scott Wells). The experts provided the most likely value and either the lowest or the highest possible value for all parameters in the model with 95% confidence. Uncertainty about the prior information was represented by use of beta (
, ß) distributions, where the values of
and ß determine the shape of the distribution. In this study, the prior beta distributions were assessed by use of software called Betabuster (version 1; downloadable at http://www.epi.ucdavis.edu/diagnostictests/). Elicitations of prior information for all parameters in the models and the corresponding beta distributions are presented in Table 1.
|
| RESULTS |
|---|
|
|
|---|
With samples from noninfected herds, the PCR test specificity varied with the cut point used. At a cut point of 1 SD above the mean negative template control, the test specificity was 48.4% (95% CI = 43.2 to 53.7%). At cut points of 2, 3, 4, 5, and 6 SD above the mean negative template control, the test specificities were 80.4% (95% CI = 76.2 to 84.6%), 94.0% (95% CI = 91.4 to 96.5%), 98.3% (95% CI = 96.9 to 99.6%), 99.1% (95% CI = 98.2 to 100%), and 99.7% (95% CI = 99.1 to 100%). While recognizing that a high cut point leads to a high test specificity at the expense of sensitivity, our objective was a highly specific test, and the 6-SD cut point was used to categorize test results in further data analyses. In comparison to antibody detection tests (Table 2), the fecal PCR assay specificity was higher (99.7%) than that of one of the currently available serum ELISAs (A) and similar to those of the other serum ELISA (B) and the milk ELISA (E).
|
|
|
From Bayesian analyses, the estimated sensitivity of the fecal PCR assay (29%) was much lower than the estimated sensitivity of bacterial culture by CENT (75%) and similar to those of ELISA A (26%) and ELISA B (27%). The probabilities that the sensitivity of the fecal PCR assay was greater than the sensitivities of CENT, ELISA A, and ELISA B were 0, 0.74, and 0.49, respectively, which indicated that the sensitivity of the PCR assay was significantly lower than the sensitivity of CENT but not significantly different from the sensitivities of ELISA A and ELISA B. The estimated specificity of the fecal PCR assay was 99.3%, compared to the estimated specificities of CENT (99.8%), ELISA A (94.9%), and ELISA B (98.0%). The probabilities that the specificity of the fecal PCR assay was greater than the specificities of CENT, ELISA A, and ELISA B were 0.04, 1.0, and 0.99, respectively, which indicated that the specificity of the PCR assay was significantly lower than the specificity of CENT and significantly higher than the specificity of either ELISA A or ELISA B.
| DISCUSSION |
|---|
|
|
|---|
One of our objectives was to estimate the sensitivity of the fecal PCR assay for detection of M. avium subsp. paratuberculosis. Our estimate from classical methods indicated an overall relative sensitivity of 23%, which was slightly lower than that of serum ELISAs. Because of the lack of a perfect gold standard test to identify all infected cattle, we also estimated the sensitivity of the fecal PCR test by using a Bayesian approach. From Bayesian analyses, the true sensitivity estimate of the PCR assay was 29% and not significantly different from that of serum ELISAs.
The sensitivities of all of the assays evaluated in this study for detection of M. avium subsp. paratuberculosis-infected cattle are dependent on the stage of infection. Cattle that shed large numbers of M. avium subsp. paratuberculosis organisms in their feces are more likely to have positive test results. Our results agree with another study using serum ELISA B (18) that showed an overall ELISA sensitivity of 45% ± 5% and a specificity of 99% ± 1%. The sensitivity of the test in that study was highest for cows with clinical paratuberculosis (87% ± 8%) and was lowest for cattle with subclinical, light fecal shedding (15% ± 7%). A study evaluating serum ELISA A with the same sera (8) similarly estimated the sensitivity varying from 15% in light shedders to 88% in cattle with clinical signs. The specificity of this test was 97% overall across several groups of cattle presumed to be uninfected.
A further goal of this study was to estimate the sensitivity of the fecal PCR assay to detect cattle shedding M. avium subsp. paratuberculosis in their feces (relative sensitivity), especially in cattle with heavy fecal shedding. Identification of cattle with heavy fecal shedding is important because these cattle are the most infectious in terms of risk of transmission to susceptible cattle. While fecal shedding is the primary route of transmission of infection, cows shedding high numbers of M. avium subsp. paratuberculosis bacteria in their feces are more likely than light fecal shedders to transmit infection to calves transplacentally (19) and through milk and colostrum (16). Cattle in this study were present in the study herds at various stages of infection, including heavy fecal shedders (Table 3). For this study, we used the mean bacterial culture score across the three culture methods to create M. avium subsp. paratuberculosis concentration categories for test evaluations, since different bacterial culture methods have different sensitivities.
On the basis of results from this study, the fecal TaqMan PCR assay can be used as a quick test for detection of subclinically infected cattle with heavy fecal shedding (76% relative sensitivity), those cattle at highest risk of transmitting infection to susceptible cattle. It is not an effective assay for detection of other, subclinically infected, cattle shedding fewer M. avium subsp. paratuberculosis bacteria (4% relative sensitivity). While lack of sensitivity in light-to-moderate shedders is a limitation of this PCR assay, detection of heavy fecal shedders is more critical for herd control. A proportion of light fecal shedders may be passing M. avium subsp. paratuberculosis bacteria directly through the gastrointestinal tract after oral ingestion (e.g., passive fecal shedders). Identification of light fecal shedders, especially in heavily infected high-prevalence herds, can be problematic to herd managers, as actively infected cattle and passive shedders cannot currently be differentiated. The fecal PCR assay is more sensitive for detection of high-risk cattle (heavy fecal shedders) than the milk ELISAs and at least as sensitive for detection of these cattle as serum ELISAs while less sensitive for light fecal shedders.
Veterinary practitioners must always interpret test results for paratuberculosis in light of the estimated within-herd prevalence of M. avium subsp. paratuberculosis infection, as well as the performance characteristics of the diagnostic test. Predictive values of positive and negative tests provide interpretive context and vary depending upon the within-herd prevalence of infection, as well as the sensitivity and specificity of the diagnostic tests used. Cost-benefit is another important aspect of diagnostic test utility and must be considered before incorporation into disease control programs. Currently, the cost per fecal PCR assay is approximately 80% of the cost of the traditional HEY fecal culture and more than three times that of serum ELISAs. At the decision level, the critical issues to consider are tradeoffs in validity and cost among available tests. The fecal PCR assay is a faster and cheaper replacement for fecal culture and, although less sensitive, is good at detection of heavy fecal shedders. In comparison to ELISAs, the fecal PCR assay has higher sensitivity than the milk ELISA for detection of heavy fecal shedders (and lower sensitivity for detection of light fecal shedders) at a much higher cost.
In summary, these results demonstrate that the sensitivity of a novel high-throughput molecular diagnostic test for paratuberculosis is less than that of fecal culture and an overall sensitivity similar to those of ELISAs while less sensitive than ELISAs for detection of light-to-moderate fecal shedders. It is particularly effective for rapid detection of cattle with heavy fecal shedding. Rapid identification of cattle actively shedding high numbers of M. avium subsp. paratuberculosis bacteria in their feces can allow removal of highest-risk cattle from the herd or segregation of the susceptible cattle, thereby avoiding further environmental contamination.
| ACKNOWLEDGMENTS |
|---|
| FOOTNOTES |
|---|
Published ahead of print on 23 August 2006. ![]()
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Antimicrob. Agents Chemother. | Clin. Microbiol. Rev. | Infect. Immun. |
|---|---|---|
| J. Clin. Microbiol. | J. Virol. | ALL ASM JOURNALS |