Skip to main page content

• HOME
• Current Issue
• Archive
• Alerts
• About ASM
• Contact us
• Tech Support
• Journals.ASM.org

This Article Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Reprints and Permissions Copyright Information Books from ASM Press MicrobeWorld Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Naugle, A. L. Articles by Collins, M. T. Search for Related Content PubMed PubMed Citation Articles by Naugle, A. L. Articles by Collins, M. T.

 Previous Article

Clinical and Diagnostic Laboratory Immunology, July 2003, p. 729-730, Vol. 10, No. 4
1071-412X/03/$08.00+0     DOI: 10.1128/CDLI.10.4.729-730.2003

LETTER TO THE EDITOR

Calculation Method for Likelihood Ratios Dictates Interpretation

Top Letter References Letter  Reference

 
Recently, likelihood ratios were published for use as an aid in the clinical interpretation of test results obtained from a commercially available enzyme-linked immunosorbent assay (ELISA) for the detection of antibodies against Mycobacterium avium subsp. paratuberculosis in cattle (2). As described by Dr. M. T. Collins, likelihood ratios can provide diagnostic information for multiple levels of a diagnostic test result when results are reported on a continuous scale rather than as a dichotomous (positive and negative) outcome. Unfortunately, the author did not take advantage of this characteristic of likelihood ratios.

Because the formula used to calculate likelihood ratios presented in this article incorporates sensitivity and specificity estimates for the ELISA at several arbitrary cutoff points, the resulting comparisons remain dichotomous in nature. Further, the interpretation of these likelihood ratios as calculated is incorrect. For example, the likelihood ratio calculated at the 0.10 level indicates that cows with S/P ratios of ≥0.10 are five times more likely to be infected than noninfected. With this approach, all cows with ELISA S/P ratios of ≥0.10 are included in the comparison. Multiple likelihood ratios that allow for comparisons between different levels of S/P ratios were not calculated. Because the ELISA S/P ratios were not stratified into multiple categories, the likelihood ratios presented in Table 1 of Collins's article (2) are overestimates resulting from heterogeneity within the defined groups of interest.

View this table: [in this window] [in a new window]

TABLE 1. Calculation of likelihood ratios using a multilevel approach

Table 1 provides likelihood ratios based on the multiple-level approach proposed by Sackett et al. (3) and calculated from the data presented by Dr. Collins in his Table 1 (2).

The likelihood ratios calculated in this manner differ considerably from those presented by Dr. Collins, and differences in their subsequent interpretations may have significant clinical and economic consequences. For example, cows with ELISA S/P ratios of 0.10 to <0.25 are 0.5 times as likely (i.e., half as likely) to be infected than noninfected, not "5 to 15 times" as likely to be infected, as Dr. Collins indicated in his Table 3 (2). Likelihood ratios demonstrating less-than-eightfold differences between comparison groups have been suggested to provide weak statistical evidence that a test result is indicative of a defined outcome (1).

Additionally, the exclusion of ELISA S/P ratios of <0.00 from this analysis is a matter for concern. ELISA results from 1,097 animals, over one-third of the available results, were not considered. As this report indicates, ELISA S/P ratios of <0.00 may constitute a significant proportion of the results obtained during herd screening. These results could have been incorporated into the analysis had the multiple-level approach for likelihood ratio calculations been utilized.

Likelihood ratios can provide additional diagnostic information for use in the clinical interpretation of the ELISA for M. avium subsp. paratuberculosis in cattle. However, care must be taken in order to interpret these values correctly and to make correct management recommendations based on them. These data do not support the interpretation of ELISA S/P ratios presented by Dr. Collins.

Top Letter References Letter  Reference

 

1
1. Blume, J. 2002. Likelihood methods for measuring statistical evidence. Stat. Med. 21:2563-2599.[CrossRef][Medline]
2
2. Collins, M. T. 2002. Interpretation of a commercial bovine paratuberculosis enzyme-linked immunosorbent assay by using likelihood ratios. Clin. Diagn. Lab. Immunol. 9:1367-1371.[Abstract/Free Full Text]
3
3. Sackett, D., R. Haynes, G. Guyatt, et al. 1991. Clinical epidemiology: a basic science for clinical medicine, 2nd ed., p. 119-139. Lippincott, Williams and Wilkins, Philadelphia, Pa.

						Alecia Larew Naugle William P. Shulaw William J. A. Saville Thomas E. Wittum Department of Veterinary Preventive Medicine College of Veterinary Medicine The Ohio State University Columbus, Ohio Beverly Byrum Animal Disease Diagnostic Laboratory Ohio Department of Agriculture Reynoldsburg, Ohio

Author's Reply

Top Letter References Letter  Reference

 
Naugle et al. are correct in stating that Table 1 of the original paper (1) presents likelihood ratios (LRs) based on dichotomous interpretations of the ELISA results at multiple cutoffs and that this method of interpretation is different from a true multilevel LR analysis. That this method was used is indicated in the table by the term "ELISA S/P cutoff value" in the heading of the first column and by the footnote for "LR" in the last column, which gives the LR formula based on sensitivity and specificity.

LRs based on the same data used for the original article, including ELISA S/P results of <0.00, and calculated by using a multilevel approach are shown in Table 1 presented here.

View this table: [in this window] [in a new window]

TABLE 1. LRs for for accurate diagnosis of M. paratuberculosis in cattle from ELISA S/P ratiosa

The small number of infected cattle in the midrange categories limits the precision of the estimated LR. The negative association of ELISA S/P values with Mycobacterium paratuberculosis infection status in the 0.10 to 0.24 range is a function of as-yet-unexplained differences between U.S. and Dutch cattle.

To improve the precision of LR estimates, in the original analysis (1) I tried to include cattle from the full spectrum of infection (those shedding M. paratuberculosis in feces and those not shedding the bacterium) and as large a population of controls from paratuberculosis-free herds as possible. This necessitated the use of sera from Dutch cattle, since The Netherlands had more readily available certified paratuberculosis-free herds than did the United States. The choice of cases and controls significantly affected the five-level LRs, as shown in Table 2.

View this table: [in this window] [in a new window]

TABLE 2. Effects of different case-control combinations on the accuracy of LRs for cattle diagnosed as positive for M. paratuberculosis by ELISA S/P ratios

These results argue for geographically specific LRs. Paratuberculosis ELISA specificity differences were also reported by Van Maanen et al. (Abstr. 7th Int. Colloq. Paratuberculosis, abstr. 139, 2002). More data are required to evaluate the necessity of geographically specific LRs.

I elected to use an S/P ratio of 0.25 as the cutoff because it is the one recommended by the kit manufacturer to define positive results and because I felt it was more pragmatic to use this cutoff than to alter it. Instead, I tried to divide the S/P values below the cutoff into two categories and those above the cutoff into three categories. With U.S. cattle in certified paratuberculosis-free herds used as the comparison control group, the two right-hand columns in Table 2, which includes five levels of LRs, show that dairy cattle in the S/P range of 0.10 to 0.24 are over five times more likely to be infected with M. paratuberculosis. This justifies the proposed low-cost intervention strategies to control the potential spread of paratuberculosis from animals such as those presented in Table 3 of the original paper (1). Based on the same definitions for cases and controls and Blume's criteria for strength of evidence based on LRs, dairy cattle with ELISA results in the 0.40 to 0.99 range show strong evidence of M. paratuberculosis infection.

While the statistical concerns raised by Naugle et al. are valid, the relationship between ELISA S/P ratios and the likelihood of M. paratuberculosis infection in dairy cattle (and the correlation of ELISA S/P positive results with those of other tests for M. paratuberculosis infection, shown in Table 2 of the original paper [1]) remains apparent, regardless of choice of cases and controls. The purpose of the study was to create a simple system for decision making by dairy producers and veterinary practitioners based on M. paratuberculosis ELISA S/P values that was founded on the principles of LR analysis. Feedback from end users about this system is favorable; it allows for the management of this infectious disease without excessive cost, particularly in comparison to the heretofore recommended "test-and-cull" programs for bovine paratuberculosis using tests with only positive and negative interpretations.

Modeling economic decision analysis will require precise, multilevel LRs based on a larger number of well-characterized cases of bovine paratuberculosis and appropriate controls, and it may be necessary to define them for specific geographic regions. One must also keep in mind the effects of biological variation in host response to exposure or infection with mycobacteria as well as technical variations in assay performance and not become too enamored with statistical precision when describing diagnostic test outcomes.

I thank my Ohio colleagues for pointing out the important difference between the dichotomous LRs I provided in Table 1 of the original paper (1) and the five-level LRs calculated according to the method of Sackett et al. Our exchange of ideas and the resulting expanded data analysis have shed more light on the calculation and use of LRs in bovine paratuberculosis ELISA interpretation.

Top Letter References Letter  Reference

 

1
1. Collins, M. T. 2002. Interpretation of a commercial bovine paratuberculosis enzyme-linked immunosorbent assay by using likelihood ratios. Clin. Diagn. Lab. Immunol. 9:1367-1371.

						M. T. Collins School of Veterinary Medicine University of Wisconsin—Madison 2015 Linden Dr. Madison, WI 53706-1102

---

Clinical and Diagnostic Laboratory Immunology, July 2003, p. 729-730, Vol. 10, No. 4
1071-412X/03/$08.00+0     DOI: 10.1128/CDLI.10.4.729-730.2003
Copyright © 2003, American Society for Microbiology. All Rights Reserved.

This article has been cited by other articles:

• Collins, M. T., Wells, S. J., Petrini, K. R., Collins, J. E., Schultz, R. D., Whitlock, R. H. (2005). Evaluation of Five Antibody Detection Tests for Diagnosis of Bovine Paratuberculosis. CVI 12: 685-692 [Abstract] [Full Text]  

This Article Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Reprints and Permissions Copyright Information Books from ASM Press MicrobeWorld Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Naugle, A. L. Articles by Collins, M. T. Search for Related Content PubMed PubMed Citation Articles by Naugle, A. L. Articles by Collins, M. T.