INTRODUCTION

Top Abstract Introduction Materials & Methods Results Discussion References

The indirect enzyme-linked immunosorbent assay (IELISA) for detection of antibody to Brucella abortus was introduced in 1976 (1). The reasons for using IELISAs were, firstly, to replace conventional serological tests (3) that in many ways did not perform well and frequently required a panel of tests for diagnosis and, secondly, to introduce an assay which could be standardized, quality controlled, and automated. A large number of IELISAs have been described in the literature (13), but in spite of the numerous modifications, the specificities of these assays were less than expected. The reason for this is partly because antibody resulting from B. abortus 19 vaccination or from exposure to cross-reacting antigens is detected by this procedure.

To increase specificity, competitive ELISAs (CELISAs) were developed (4-6, 8). By selection of a suitable monoclonal antibody to compete with antibody present in test serum, reactivity resulting from the vaccine or cross-reacting antigens could be virtually eliminated. Two of these assays were developed and validated largely in circumstances where brucellosis had been eradicated (Canada) with sera from animals in which B. abortus infection was confirmed by culture as reference sera. It was therefore necessary to field test these assays in areas with brucellosis and vaccination programs. For these purposes, four laboratories in Latin America were selected. These laboratories were selected based on the incidence of brucellosis in each area. Chile had a relatively low incidence, while higher incidences were found in Costa Rica, Colombia, and Argentina.

This communication describes the results obtained with two IELISAs and two CELISAs compared to those from the diagnostic serological tests in use in each laboratory.

	MATERIALS AND METHODS

Top Abstract Introduction Materials & Methods Results Discussion References

Test samples. Samples were defined on the basis of their serological reactions on both the rose bengal agglutination test (RBT) and the complement fixation test (CFT) by the official criteria for positive results as determined by each country for the CFT.

Control sera. Control sera were supplied by the Animal Diseases Research Institute (ADRI) for one IELISA and both CELISAs from ADRI. These consisted of a strong positive control serum from a cow from which B. abortus had been isolated, a weakly positive control for the IELISA that was from a cow inoculated with B. abortus 19 and negative on the CELISA, and a negative control from a pool of cattle with no history of B. abortus infection. Separate controls were supplied by the International Atomic Energy Agency (IAEA) for the IAEA IELISA kit.

Test procedures. The RBT antigen was prepared by Rhone-Merieux, and the assay was performed as described in the National Animal Diseases Laboratory diagnostic reagents manual (11).

Data handling and statistical analysis. The data for each country was compiled in a database and divided into negative or positive results according to serological reactions on both the RBT and the CFT.

	RESULTS

Top Abstract Introduction Materials & Methods Results Discussion References

The numbers of samples used in this study are presented in Table 2. The samples were divided into three populations. The negative population was defined as those primarily from regions that had no history of B. abortus infection and having negative reactions on both the RBT and the CFT. The positive population was defined as those samples from infected herds which had positive reactions on both the RBT and the CFT. The vaccinated population was defined as those animals that had been vaccinated with B. abortus 19 according to the regulations in each country. The exception to this was Argentina, where vaccination is routinely practiced, and it was difficult to collect samples that were defined as being from unexposed cattle. Consequently, the data for the negative category and that for the vaccinated category for Argentina were combined into the negative category.

The data presented in Table 3 is defined in two ways. The data for the positive population from Argentina, Chile, Colombia, and Costa Rica and the combined data are percent positives (%P), or the number of positives found for each ELISA relative to the positive RBT and CFT reactions from cattle in infected herds. The Canadian data is actual sensitivity, since the results were derived from animals from which B. abortus had been isolated. The highest %P, of 100%, for both IELISAs from Colombia and for the IELISA-ADRI from Costa Rica indicate that it is comparable to the actual sensitivity achieved by the IELISA-ADRI in the Canadian study. Data for the IELISA-IAEA and the CELISA-OC for Canada was not part of the original Canadian study and consequently is not available.

The data presented in Table 4 for the negative population from Argentina, Chile, Colombia, and Costa Rica and the combined data are similarily defined as percent negatives (%N), or the number of negatives found for each ELISA relative to the negative RBT and CFT reactions primarily from regions with no history of B. abortus infection. The Canadian data is actual specificity, since the results were derived from cattle in Canada. Canada has been free of B. abortus infection in cattle since 1982. 

The highest %N, of 99.82%, for both IELISAs from Colombia is comparable to the actual specificity of 99.40% achieved by the IELISA-ADRI in the Canadian study. Data for the IELISA-IAEA and the CELISA-OC for Canada was not part of the original Canadian study and consequently is not available.

The %N of the IELISAs and CELISAs relative to the RBT and CFT for the vaccinated population are presented in Table 5. The %N for each country and that for the combined data of Chile, Colombia, and Costa Rica are compared to each other and to the Canadian data. The largest difference in percentage was between the IELISA-ADRI and the CELISA-sLPS in the Canadian study. This was 41.4%. The difference between the IELISA-ADRI and the CELISA-sLPS for the data from Chile was 21.2%. In all cases, the %N for the vaccinated population was greater for the CELISA-sLPS than for the IELISA-ADRI or the IELISA-IAEA, although for the IELISA-IAEA the differences were smaller. Similarly, the %N of the CELISA-OC was greater than that of the IELISA-ADRI in all cases. However, the %N of the IELISA-IAEA was greater than that of the CELISA-OC for Chile and for the data from Costa Rica with calf vaccination. The maximum difference was 2.4%.

Cutoff values for each ELISA by country are presented in Table 6. The IELISA data is expressed as %P. The CELISA data is expressed as percent inhibition (%I). For example, the cutoff value for the IELISA-ADRI for Argentina is 67%P. Samples greater than or equal to 67%P are positive and samples less than 67%P are negative on this IELISA. The lowest cutoff value for the IELISA-ADRI was 16%P. The highest cutoff value for the IELISA-ADRI was 70%P, a difference of 54%. Similarly, the lowest cutoff value for the IELISA-IAEA was 14%P, and the highest cutoff value was 73%P, a difference of 59%. The difference for the CELISA-OC and the CELISA-sLPS was 17 and 26%, respectively, which indicated that the CELISAs were more specific for the negative population.

Agreements between assays are compared in Table 7. The kappa statistic for each ELISA by country is presented. For example, the kappa indices of Argentina, Chile, Colombia, and Costa Rica for the IELISA-ADRI and the IELISA-IAEA are 0.824, 0.963, 0.994, and 0.850, respectively, indicating good agreement between the IELISA-ADRI and the IELISA-IAEA despite the differences in the cutoff values. Except for Costa Rica, the kappa statistic for all assays indicated good agreement between assays. It is generally accepted that a kappa statistic greater than or equal to 0.8 indicates good agreement between assays. The kappa results for Costa Rica were not much lower than 0.8 and were all greater than 0.5, indicating agreement beyond chance.

The cutoff values of each ELISA by country were determined by a combination ROC analysis and by frequency distributions. The ROC analyses are presented in Fig. 1 to 5, along with the respective areas under the curve (AUC). For example, in Fig. 1a, the optimal cutoff value for the IELISA-ADRI is 67%P. In Fig. 1b, the optimal cutoff value for the IELISA-IAEA is 40%P. In Fig. 1c, the optimal cutoff value for the CELISA-sLPS is 44%I, while an optimal cutoff value for the CELISA-OC in Fig. 1d is 35%I. The frequency distributions are presented in Fig. 6 to 10. The frequency distribution for the IELISA-ADRI in Fig. 6a shows considerable overlap between the selected negative and positive populations. With the cutoff as determined by ROC analysis, it is much easier to identify the false negatives. The same is true of the IELISA-IAEA, CELISA-OC, and CELISA-sLPS presented in Fig. 6b, c, and d. The other frequency distributions for the other countries can be interpreted in a similar fashion.

View larger version (30K): [in this window] [in a new window]   FIG. 1.   ROC curves representing each ELISA for data from Argentina. The cutoff value for each assay is indicated in the upper right corner of each panel. The AUC is indicative of how well the test performed. A value of 1.0 is perfect, and a value below the diagonal line represents reactivity due to chance. (a) IELISA-ADRI, AUC = 0.983. (b) IELISA-IAEA, AUC = 0.983. (c) CELISA-sLPS, AUC = 0.995. (d) CELISA-OC, AUC = 0.991.

View larger version (30K): [in this window] [in a new window]   FIG. 2.   ROC curves representing each ELISA for data from Chile. (a) IELISA-ADRI, AUC = 1.000. (b) IELISA-IAEA, AUC = 0.996. (c) CELISA-sLPS, AUC = 1.000. (d) CELISA-OC, AUC = 1.000. See the Fig. 1 legend for additional explanation of the data.

View larger version (30K): [in this window] [in a new window]   FIG. 3.   ROC curves representing each ELISA for data from Colombia. (a) IELISA-ADRI, AUC = 1.000. (b) IELISA-IAEA, AUC = 1.000. (c) CELISA-sLPS, AUC = 0.994. (d) CELISA-OC, AUC = 0.999. See the Fig. 1 legend for additional explanation of the data.

View larger version (31K): [in this window] [in a new window]   FIG. 4.   ROC curves representing each ELISA for data from Costa Rica. (a) IELISA-ADRI, AUC = 0.992. (b) IELISA-IAEA, AUC = 0.991. (c) CELISA-sLPS, AUC = 0.974. (d) CELISA-OC, AUC = 0.969. See the Fig. 1 legend for additional explanation of the data.

View larger version (30K): [in this window] [in a new window]   FIG. 5.   ROC curves representing each combined ELISA for data from Argentina, Chile, Colombia, and Costa Rica. (a) IELISA-ADRI, AUC = 0.985. (b) IELISA-IAEA, AUC = 0.989. (c) CELISA-sLPS, AUC = 0.995. (d) CELISA-OC, AUC = 0.995. See the Fig. 1 legend for additional explanation of the data.

View larger version (29K): [in this window] [in a new window]   FIG. 6.   Frequency distribution of ELISA data from Argentina. Open bars, serologically negative samples; closed bars, serologically positive samples. The numbers in each class limit are indicated on top of the bars. (a) IELISA-ADRI. (b) IELISA-IAEA. (c) CELISA-OC. (d) CELISA-sLPS.

View larger version (25K): [in this window] [in a new window]   FIG. 7.   Frequency distribution of ELISA data from Chile. (a) IELISA-ADRI. (b) IELISA-IAEA. (c) CELISA-OC. (d) CELISA-sLPS. See the Fig. 6 legend for additional explanation of the data.

View larger version (22K): [in this window] [in a new window]   FIG. 8.   Frequency distribution of ELISA data from Colombia. (a) IELISA-ADRI. (b) IELISA-IAEA. (c) CELISA-OC. (d) CELISA-sLPS. See the Fig. 6 legend for additional explanation of the data.

View larger version (23K): [in this window] [in a new window]   FIG. 9.   Frequency distribution of ELISA data from Costa Rica. (a) IELISA-ADRI. (b) IELISA-IAEA. (c) CELISA-OC. (d) CELISA-sLPS. See the Fig. 6 legend for additional explanation of the data.

View larger version (29K): [in this window] [in a new window]   FIG. 10.   Frequency distribution of combined ELISA data from Argentina, Chile, Colombia, and Costa Rica. (a) IELISA-ADRI. (b) IELISA-IAEA. (c) CELISA-OC. (d) CELISA-sLPS. See the Fig. 6 legend for additional explanation of the data.

	DISCUSSION

Top Abstract Introduction Materials & Methods Results Discussion References

ELISAs have a distinct advantage over conventional serological tests in that they are primary binding assays that do not rely on secondary properties of antibodies such as their ability to agglutinate or to fix complement. Secondly, ELISAs can be tailored to be more specific by using highly purified reagents such as antigens and monoclonal antibodies.

In Canada, which is free of brucellosis in domestic animals, both the IELISA and the CELISA were recently validated (9). Approximately 8,000 samples from cattle with no evidence of B. abortus infection were collected and tested in both the IELISA and the CELISA. Similarly, 692 samples from cattle from which B. abortus was isolated from milk or tissues were also tested. Another 261 samples from cattle that were vaccinated with B. abortus 19 and that contained residual antibodies were tested as well.

Unlike in Canada, conditions in Latin America for validation of assays are different. It is more difficult to define negative and positive sera because diagnosis is based on serological evidence or the isolation of B. abortus from herds rather than from individual cattle. In most countries, areas overlap between regions free of B. abortus and regions that contain infected herds, and strain 19 vaccination is widely practiced. For these reasons and for consistency, the negative population and positive population were defined based on the RBT and the CFT reactions in each country under study. As well, determining the B. abortus 19 vaccination status of cattle is sometimes difficult due to insufficient data being available, including the time of vaccination, the number of times that cattle were vaccinated, and identification of cattle that were vaccinated. The numbers of samples defined as positive, negative, and vaccinated are tabulated in Table 2.

Comparison of %P is summarized in Table 3. The results are not dissimilar from the results obtained in the Canadian study (9). Both the IELISA and the CELISA achieved a sensitivity estimate of 100% in Canada. The results obtained in Latin America were comparable. Percent positive values obtained ranged from 92.10% for the CELISA-OC in Costa Rica to 100% for the CELISA-sLPS in Chile, the IELISA-ADRI in Colombia, the IELISA-IAEA in Colombia, and the IELISA-ADRI in Costa Rica. When the data was combined for all countries (except Canada), the performance of both CELISAs was marginally better than that of the IELISAs (presented in Table 3). The maximum difference between the CELISAs and the IELISAs for the combined data is 1.19%. The CELISA-sLPS at 97.47% detects 11.9 more positive reactions per 1,000 animals than does the IELISA-IAEA at 96.28%.

Comparison of %N is presented in Table 4. The specificity for the IELISA in Canada was 99.40%, while the specificity for the CELISA was 99.90% (9). The results obtained in Latin America were similar. The lowest %N achieved was 93.35% for the CELISA-OC in Costa Rica. The highest %N achieved was 99.82% for the IELISA-ADRI, IELISA-IAEA, and CELISA-OC in Colombia. When the data was combined for all the countries (except Canada), it is obvious that the overall performance of both CELISAs is better than that of the IELISAs presented in Table 4. The maximum difference between the CELISAs and the IELISAs for the combined data is 4.75%. The CELISA-sLPS at 98.32% is more specific than the IELISA-ADRI at 93.57%. Thus, the IELISA-ADRI detected 47.5 more animals per 1,000 animals than did the CELISA-sLPS.

Comparison of the %N for vaccinated cattle is tabulated in Table 5. The results of the Canadian study indicated that the CELISA-sLPS was capable of distinguishing animals that were vaccinated or negative from those that were infected, in the majority of the cases. In the Canadian study, the specificity of the IELISA-ADRI was 56.30% while the specificity for the CELISA-sLPS was 97.70%. Similar results were achieved in Latin America. In Chile, the %N for the IELISA-ADRI was 78.82% while the %N for both CELISAs was 94.44 and 100%. In Colombia, the %N for both IELISAs was 86.76 and 87.57%, respectively. The %N for both CELISAs was 95.50 and 92.25%. The combined data clearly indicates that the %N of the CELISAs as presented in Table 5 is better than that of the IELISAs for distinguishing vaccinal antibody. The maximum difference between the CELISAs and the IELISAs for the combined data is 5.98%. The CELISA-sLPS for the combined data at 96.51% is more specific than the IELISA-ADRI at 90.53%. The CELISA-sLPS misinterprets as positives 59.8 fewer vaccinated animals per 1,000 animals than does the IELISA-ADRI.

Ideally, harmonization of cutoff values should be the same in each country for the IELISAs or for the CELISAs. However, analysis of data indicated that this was not possible. The cutoff values for each country and for the combined data were determined by ROC analysis as presented in Fig. 1 to 5 and tabulated in Table 6. From Table 6, the only assay that had cutoff values approximating the 30% chosen for Canada was the CELISA-sLPS, except for Costa Rica. The frequency distributions presented in Fig. 6 to 10 show the difficulty in choosing an optimal cutoff value for each assay. For instance, most of the frequency distributions for the IELISA have some overlap between the negative and the positive populations. The exceptions to this were the frequency distributions from Colombia. The reason for the binomial distribution is better separation of the negative and positive sera. The sera were from defined areas free from B. abortus infection and from areas with a relatively high prevalence of infection. Despite the differences in how the IELISA-ADRI and the IELISA-IAEA were performed, the distribution patterns were very similar. This became quite evident when the frequency distributions of the combined data for the IELISAs presented in Fig. 10 were examined. The distribution patterns of the CELISAs, although different from those of the IELISAs, were similar to each other, and again the similarity was quite evident from the frequency distribution of the combined data presented in Fig. 10. Choosing a cutoff value solely on the basis of frequency distribution could give erroneous results. The frequency distributions of the CELISAs were marginally better than those of the IELISAs due to less overlap between the selected negative and positive populations. However, obtaining the optimal percentage of positives and percentage of negatives for each assay in each country was best determined by ROC analysis and frequency distributions together to get a clearer picture in each instance.

The ROC curves presented in Fig. 1 to 5 all had AUC greater than 0.95. An AUC of 0.95 indicates that a randomly selected individual animal from a positive population will have a test value greater than that of a randomly selected individual animal from the negative population 95% of the time. The lowest AUC was 0.969 for the CELISA-OC in Costa Rica, while the highest AUC was 1.000 for the IELISA-ADRI, the CELISA-sLPS, the CELISA-OC in Chile, and the IELISA-ADRI and the IELISA-IAEA in Colombia. Both CELISAs for the combined data had an AUC of 0.995, which was approximately 1% better than that of the IELISAs.

Finally, a comparison of agreement between assays was calculated and presented in Table 7. A kappa statistic of 1 indicates perfect agreement between assays. A kappa of 0.5 indicates agreement beyond chance. It is generally accepted that kappa indices greater than or equal to 0.8 indicate good agreement between tests. The best agreement was 0.994 between the IELISAs in Colombia. Again, this is probably due to better separation of the negative and positive populations. The lowest kappa statistic was 0.720 between the CELISAs from Costa Rica, where separation of negative and positive populations was more difficult. The highest kappa for both CELISAs was 0.981 from Chile. Overall, the kappa statistics for all the assays were good, indicating good agreement among all assays.

Generally, the technical performance of the assays was good and the results were similar to results obtained in the Canadian study. However, there are some reasons why the results could be improved. Firstly, a bias was introduced in the study. The selected negative and positive populations were defined according to the RBT and CFT reactions. The RBT can produce false-positive results, which when used to define sera can affect the sensitivity of the assay being validated. Secondly, a better separation of the negative and positive populations would have produced better results. For example, if individual animals with proven infection based on isolation of the organism had been selected, instead of positive animals from infected herds, the sensitivity values should have been higher. Thirdly, the RBT and the CFT both detect antibody resulting from B. abortus 19 vaccination or from exposure to cross-reacting antigens. Therefore, the results are biased against the CELISAs, which eliminate many such reactions.

Based on the combined data, the CELISA-sLPS was the best-performing ELISA. It detected 1.19% more positives in the selected positive population, 4.75% fewer positives in the selected negative population, and 5.98% fewer positives in the selected vaccinated population. The implication of this is important. For example, in a population of 15,000,000 animals with a high incidence of brucellosis the CELISA-sLPS would detect 712,500 fewer false positives and, if vaccination were part of the control program, 897,000 fewer false positives. By using the CELISA-sLPS as the primary screening assay in an eradication and control program, significant savings in repeat testing and elimination of other conventional assays can be realized. In addition, the CELISA-sLPS is less costly in reagents than is conventional assays and has excellent quality control, leading to additional savings.