138: Principles of Epidemiology and Research Design

Advances in medical toxicology are achieved through the scientific method using observations, derived from cases of poisoning and nonpoisoning due to xenobiotic exposures, to generate hypotheses. Subsequent research questions are analyzed with epidemiological investigation, and preliminary studies are examined with methodological scrutiny. Initial analytical techniques are improved, and confirmatory studies are performed. Ultimately, models relating cause to effect are formulated.

To optimize patient care, it is useful to grade the quality of available scientific evidence used to justify treatment recommendations. Decisions about how strongly to recommend a medical action will be based on the careful consideration of the risks of leaving a patient untreated, the potential benefits and harms of treatment, the quality of the guiding evidence, a balanced view of resource utilization, and the values of the person to be treated. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) Working Group has provided a framework for assessing and communicating levels of scientific evidence (Table 138–1).²² An understanding of basic principles of research design and epidemiology is required to interpret published studies and to lay the groundwork for future investigation in toxicology.

Table 138–2 lists the different study formats discussed below.

Observational Design: Descriptive

A staggering array of xenobiotics are able to injure people, necessitating reliance of toxicologists on good descriptive data regarding toxic outcomes. Through 2011, the National Poison Data System (NPDS) of the American Association of Poison Control Centers (AAPCC) has amassed a database of more than 50 million human exposures (Chap. 136). Descriptive case reporting serves a valuable purpose in describing the characteristics of a medical condition or procedure and remains a fundamental tool of epidemiological investigation. A case report is a clinical description of a single patient or procedure in a unique context. Case reports are most useful for hypothesis generation. However, single case reports are not always generalizable, as the reported situation may be atypical. A number of case reports can be grouped on the basis of similarities into a case series. Case series can be used to characterize an illness or syndrome, but without a control group they are severely limited in proving cause and effect. In 1966, a case series of two patients with acute liver necrosis following overdose of acetaminophen (APAP)¹² was accompanied by a case report of liver damage and impaired glucose tolerance after APAP overdose,⁴⁰ which led to further study and the eventual creation of the Rumack-Matthew nomogram (Chap. 35). Similarly, a 1979 case report of “hypertension and cerebral hemorrhage after trimolets ingestion”²⁴ led to subsequent animal studies, experimental human studies, and epidemiologic studies culminating in the decision by the US Food and Drug administration to remove phenylpropanolamine from nonprescription cold remedies and appetite suppressants (Chap. 42). The important role for descriptive data in guiding clinical research, focusing educational efforts, and formulating public policy are often underappreciated.

Cross-sectional studies assess a population for the presence or absence of an exposure and condition simultaneously. Such data often provide estimates of prevalence—the fraction of individuals in a population sharing a characteristic or condition at a point in time. These studies are particularly helpful in public health planning and have been extremely useful in monitoring common environmental exposures, such as childhood lead poisoning, or population-wide drug use, such as occurs with tobacco, marijuana, and alcohol. The US National Health and Nutrition Examination Survey demonstrated that the percentage of children with blood lead concentration greater than 10 μg/dL decreased from 88.2% to 4.4% between 1976 and 1991, with the highest rates of plumbism among African American, low-income, or urban children.⁵

An analysis of secular trend is a study type that compares changes in illness over time or geography to changes in risk factors (ecological). These analyses often lend circumstantial support to a hypothesis; however, because of the ecologic nature of their design, individual data on risk factors are not available to allow exclusion of alternative hypotheses also consistent with the data. A prime example of an analysis of secular trends is the finding that reports of Reye syndrome declined between 1980 and 1985, coincident with a fall in sales of, or physician recommendations of, children’s salicylate products.⁴ This investigation suggested an etiologic role of salicylate in the development of Reye syndrome but could not exclude alternative hypotheses such as a change in viral epidemic patterns.

Observational Design: Analytical

Hypotheses that are generated by theoretical reasoning or anecdotal association require analytical testing. Case-control studies and cohort studies are analytical techniques that use observational data, and each technique has its own advantages and disadvantages (Table 138–2). Case-control studies compare affected, treated, or diseased patients (cases) to nonaffected patients (controls) and evaluate for a difference in prior risk factors or exposures (Fig. 138–1A). Because participants are recruited into the study based on prior presence or absence of a particular outcome, case-control studies are always retrospective in nature. They are especially useful when the outcome being studied is rare, and they enable the investigation of any number of potential etiologies for a single disease.

The hypothesis, derived from multiple case reports and case series, that phenylpropanolamine might increase risk for hemorrhagic stroke was well suited to case-control study. Exposure to ingested phenylpropanolamine, as an ingredient in cold remedies and appetite suppressants, was common; but hemorrhagic stroke is rare among children and young adults. Other putative risk factors such as tobacco use, hypertension history, family history, cocaine use, and contraceptive use were identifiable and could be studied simultaneously. In a case-control analysis of 702 participants with hemorrhagic stroke and 1376 controls, the use of appetite-suppressant doses of phenylpropanolamine were found to be independently associated with the occurrence of hemorrhagic stroke.²³

Cohort studies compare patients with certain risk factors or exposures to those patients without the exposure, and then follow these cohorts to see which participants develop the outcome of interest (Fig. 138–1B). In this respect, they allow the comparison of incidence (the number of new outcomes occurring within a population initially free of disease over a period of time) between populations who share an exposure and populations who do not. They may be retrospective or prospective and enable the study of any number of outcomes from a single exposure. They are particularly well suited to investigations in which the outcome of interest is relatively common. In circumstances when an outcome of interest is very uncommon, such as the case with stroke after phenylpropanolamine use, the large number of study participants required might make a cohort study impractical. A cohort of 981 APAP overdose participants was used retrospectively to investigate whether administration of activated charcoal might be beneficial therapy for APAP poisoning.⁸ Participants were separated on the basis of whether or not they were treated with activated charcoal and were subsequently followed to see if they developed concentrations deemed toxic by the Rumack-Matthew nomogram. Perhaps the most famous cohort study was the Framingham Heart Study in which 5209 residents of Framingham, MA, aged 30 to 62 years, were followed for over 50 years. This study provided a useful tool for studying the incidence of lung cancer, stroke, and cardiovascular disease in those exposed to cigarette smoke and other hazardous xenobiotics.¹³

Experimental Design

Experimental studies are those in which the treatment, risk factor, or exposure of interest can be controlled by the investigator to study differences in outcome between the groups (Fig. 138–2). The prototype is the randomized, blinded, controlled clinical trial. Among epidemiologic study types, these provide the most convincing demonstration of causality. Clinical trials are used to measure the efficacy (the treatment effect within a controlled experimental setting) of treatment regimens and to draw inferences about the effectiveness of a treatment applied to the general population. Sometimes a trial can be designed to study drug treatments that are hampered by nonresponders to therapy, expensive drugs, or poorly tolerated regimens. Such trials are termed noninferiority trials, and operate on the null hypothesis that the new (study) drug is worse than the control (standard) drug. Thus, finding a difference between groups in a noninferiority trial means that the alternative hypothesis can be accepted that the new drug is not worse than the standard treatment.

Unfortunately, interventional studies are the most complex to perform, and several questions must be addressed by investigators before performing a clinical trial (Table 138–3). Human clinical trials have been especially difficult to apply to the practice of toxicology. Table 138–4 lists characteristics of poisoned patients, which hamper attempts at clinical trials. Volunteer studies, using nontoxic xenobiotics or nontoxic doses of toxic xenobiotics, are often used to circumvent many of the problems in controlling human poisoning studies; but it is typically difficult to apply results from these studies to the actual context of toxic overdoses. In an experimental, human volunteer study, activated charcoal reduced absorption of ampicillin by 57%.³⁹ Taken out of this artificial setting, a trial of single-dose oral activated charcoal was unable to prove benefit to outcome among 1479 heterogenous participants presenting to an emergency department (ED) for possible poisoning.²⁹ Neither study was able to answer whether activated charcoal reduces morbidity from ingestion of dangerous xenobiotics if given while the xenobiotic is still in the stomach and amenable to adsorption.

As toxicologists strive to find evidence for, or against, the traditions of clinical practice, several important clinical trials have been published. Among them are many important examples and lessons in epidemiologic study design. One trial attempted to evaluate whether or not corticosteroids might be beneficial in preventing esophageal strictures secondary to circumferential caustic injury of the esophagus.³ Because of the inherent difficulty in recruiting eligible patients from a single institution, only a small sample of 60 patients with esophageal injury were recruited over an 18-year period. Another study randomized hyperbaric oxygen therapy versus sham (placebo) therapy, among 152 victims of carbon monoxide poisoning, to investigate its effect on the development of neurocognitive injury.⁴² Certain concerns with the methodology and analyses of clinical studies are examined later in this chapter to illustrate epidemiologic concepts, and must be carefully considered when trying to apply the results of any clinical trial into the patient care setting.

The objective of analytical studies is to define and quantify the degree of statistical dependence between an exposure and an outcome. Such associations are ideally represented by the relative risk of developing an outcome if exposed in comparison to being unexposed. Thus, the relative risk can be defined as the incidence of outcome in exposed individuals compared to the incidence of outcome in unexposed individuals. The relative risk can be calculated directly from cohort or interventional studies. However, in a case-control study, an investigator chooses the numbers of cases and controls to be studied, so true incidence data are not obtained. In case-control studies an odds ratio can be calculated, and the odds ratio will provide an estimate for relative risk in situations in which the outcome is rare, such as when the outcome occurs in fewer than 10% of exposed individuals. Figure 138–3 demonstrates the calculation of relative risk or odds ratio from analytic studies.

A relative risk of 1.0 signifies that an outcome is equally likely to occur whether an individual either is exposed or is not exposed and implies that no association exists between the exposure and the outcome. A relative risk approaching 0 suggests that an exposure is a marker of protection regarding the outcome, and a relative risk approaching infinity suggests the exposure predicts a tendency toward the outcome. Among men and women using phenylpropanolamine appetite suppressants, the odds ratio for development of hemorrhagic stroke was 15.9 which suggests a strong association.²³

One of the essential features of research in medical toxicology is integration of the available data, whether clinical or experimental, into logical assumptions about how the overall system functions. Systems that are studied in medical toxicology research typically involve either elucidation of how a toxic insult influences normal physiology, or the mechanism and degree to which an antidote mitigates toxicity. Prior to undertaking a study, medical toxicology investigators may have no way of knowing whether their hypothesis is correct or applies at all. Thus, predictions made based on a priori hypotheses may be tested by performing experiments or by sampling observed data in an organized fashion. If the results are consistent with the predictions, then the hypothesis is retained. If they are not, then the prior hypothesis is rejected and a new hypothesis is formulated. This process is the basis of the scientific method and is also known as hypothesis testing.

Determination of whether the observed data in a toxicology investigation allow us to retain or refute our a priori hypothesis depends on our ability to gauge the certainty that the data were arrived at by chance. When observing a set of data, investigators can never be 100% certain that what they observed actually happened; it is possible that confounding factors biased what was observed, and it is possible that the observation was made simply by chance. To deal with this problem of interpretation, researchers use statistical methods to evaluate whether their data were arrived at by chance. The presence of an association between two factors in any given study has a number of possible explanations (Table 138–5).

To differentiate between differences due to chance and potentially real differences between comparison groups, researchers can use a philosophical trick known as the null hypothesis. The null hypothesis involves beginning experiments with the assumption that there is no difference between groups being compared. Thus, any significant deviation from observed and expected results under the null hypothesis allows the investigator to reject the null hypothesis and adopt the alternative hypothesis. The alternative hypothesis to the null is that the groups being compared are significantly different from one another (ie, not the same).

To begin, one must assign a probability percentage or P value to the likelihood that their observed results are really the case assuming the null hypothesis to be true. With some exceptions, the standard P value deemed to represent ability to reject the null hypothesis has historically been set to 5% (or P<.05) in medical toxicology epidemiologic science. This value is also termed the alpha (α), the significance level, the type 1 error, and can also be thought of in layman terms as the chance of falsely finding a difference between groups when one does not exist.

Because analytic studies involve only a sample of the total population, they contain two types of inherent error. Type 1 error, also referred to as an alpha (α) error, is the likelihood that an investigator may conclude that an association exists when none truly does. Type 2 error, or a beta (β) error, is the possibility that an investigator will be unable to find an association when one is really present. The most commonly reported measures of type 1 error in published toxicology studies are the P value and the confidence interval (CI). Statistical significance has customarily, but not necessarily, been defined as having less than a one in 20 chance of conducting a false-positive study. Therefore, a type 1 error of less than 5%, which corresponds to a P value of less than 0.05, is usually deemed “statistically significant.” In some cases investigators must use a significance level (P value) even lower than 0.05 when testing multiple hypotheses at once, for example, in genetic marker studies examining multiple genes across an entire genome. Such a correction is termed the Bonferroni correction, and is applied as the P value divided by the number of variables being tested at once: P= (significance level)/(# concurrent variables).³⁷ The result is a smaller (ie, more stringent) corrected significance level for studies that test multiple hypotheses at once. For example, if a study protocol uses a genetic “chip” assay to evaluate 50 markers at once for an association with a predetermined disease in a population, the Bonferroni correction would require significance at (0.05)/(50) = 0.001 in order to deem any association to be statistically significant.

Perhaps a more informative description of the significance of an association is provided through the CI. The CI not only provides a test of statistical significance, it also offers information pertaining to the degree (and possible range) of differences observed. In an unbiased study, the 95% CI provides a range between which, if the study could be repeated an infinite number of times, the observed point estimate would fall between the CI 95% of the time. For example, one study reported that no toddlers ingesting one or two calcium channel blocker tablets became seriously ill, but a subsequent analysis of the CI around this small set of data demonstrated that the true incidence could be as high as 18%.³² A CI around a relative risk or odds ratio is not statistically significant if it includes 1.0, and the narrower the CI the more precise the estimate of the magnitude of effect.

The likelihood that a study will find a difference if one truly exists is termed statistical power and relates to the likelihood of a false-negative study (type 2 error). Power is usually artificially set by an investigator before a study is performed and is typically set at 80% or 90% to practically limit the number of study participants needed. Table 138–6 lists considerations applicable to choice of sample size. The sample size of a study is determined by the frequency of the exposure and outcome within the study population, the strength of association deemed clinically relevant, and the amount of error deemed acceptable in the study. Because power is often set relatively low, it is difficult to state that an association does not exist. It is more appropriate to state that a study was unable to reject the null hypothesis to find an association.

The finding of a low P value indicates a statistically high level of confidence that a difference between study groups exists but offers no indication that the difference is clinically important. The interpretation of statistical versus clinical significance is often facilitated through calculation of CIs. Small actual differences between two groups can become statistically significant if large numbers of participants are studied. Likewise, impressive associations of cause and effect can seem trivial if few participants are in a study. The clinical significance of an association is left to the judgment of the individual interpreting a study. Ideally, a working definition of clinical significance is developed before a study is performed.

The calculation of a P value or CI does nothing to assess the adequacy of study design. These measures are used to quantify the influence of random error, or chance, on research findings. Clinical research involving patients is particularly susceptible to bias, which can be defined as systematic error in the collection or interpretation of data. Because such error can lead to an inappropriate estimate of the association between an exposure and an outcome, careful evaluation of potential biases affecting a clinical study is of paramount importance.

Selection bias refers to error introduced into a study by the manner in which participants are selected for inclusion in the study. This type of bias is most problematic for retrospective studies in which exposures and outcomes have both occurred at the time of participant recruitment. Selection bias may be introduced into a prospective clinical study if the study fails to enroll potential participants, or if potential participants refuse to participate, on a systematic basis. Selection bias may even influence the results of clinical trials. In a 1995 trial that found no difference in outcome between acutely poisoned patients treated with gastric emptying and patients from whom gastric emptying was withheld, all patients presenting to the ED after acute overdose were enrolled.³⁵ Because most patients with poisoning exposure are likely to do well with minimal support, selection of patients on this basis might be expected to bias this study to find no effect. Reasoning suggests that the patients most likely to benefit from gastric emptying are those with life-threatening toxic ingestion presenting within the first hour after overdose. Indeed, subgroup review of the results of this paper suggests clinical benefit within this group of patients, but without conclusive power.

Information bias refers to error introduced into a study as a result of systematic differences in the quality of data obtained between exposed and unexposed groups, or between those with and without the outcome of interest. Several distinct types of information bias may exist. Affected and nonaffected individuals may have differential memories regarding exposures, so recall bias is a concern in retrospective studies. The potential for recall bias may be cited as criticism of retrospective case-control studies of the association between phenylpropanolamine and hemorrhagic stroke, in which patients and families were asked to recollect their phenylpropanolamine use history. Stroke victims and their families might be more vigorous in their recall of exposures than control participants. Similarly, interviewer bias may occur if study personnel differ in how they solicit, record, or interpret information as a result of knowledge of participant status regarding exposures or outcomes.

Prospective studies may be troubled by loss to follow-up, especially if participants are lost from the study for reasons relating to either exposure or outcome such as when participants withdraw from a study because they are feeling better, or are “lost” because they die. Misclassification bias occurs when investigators incorrectly categorize participants with respect to exposure or outcome. In a retrospective study of 378 children regarding the predictability of caustic esophageal injury from clinical signs and symptoms, it was found that 11 of 80 asymptomatic children had significant burns.¹⁴ There is a possibility that these “asymptomatic” children were misclassified because of lack of rigorous written documentation of symptoms or signs within the medical charts. In addition, studies that use “cause of death” as an outcome may be vulnerable to misclassification as well. In a large study comparing 414 poison center (PC) deaths with 7050 poisonings in the corresponding Vital Statistics database, the medical examiner and a medical toxicologist adjudication panel concurred on “cause of death” in only 66%, which the authors interpreted as only fair (ie, less than good) agreement.²⁶ Thus, when investigators define “cause of death” as their study outcome, their results may be biased depending on which specialists are used to provide the “cause of death” interpretation.

Bias is best minimized through careful study design. It is important to precisely define the study question and the population at risk and to carefully define rigorous inclusion and exclusion criteria. The outcome should also be defined precisely. During data acquisition the best way to reduce bias may be to keep study personnel gathering exposure data blinded to outcome, and vice versa. Often, it may also be advisable to keep study participants unaware of their status within a study to the extent that it is ethical (thus, “double-blinded”—neither investigators nor participants are aware of the participants’ status within a study). Use of placebos or “sham treatments” is a way to facilitate blinding. One of the strongest criticisms of a 1995 trial of HBO for the prevention of delayed neurologic syndromes after CO ­poisoning³⁶ has been the failure to blind patients and investigators to the treatment in question,³¹ a flaw that was corrected in a follow-up study published in 2002.⁴² It is inevitable that some degree of potential bias will be present in any clinical study. Such bias should be reviewed in the analysis, and estimations of its magnitude and direction (bias toward or away from rejection of the null hypothesis) should be considered.

Unlike selection and information biases, which are errors introduced into studies primarily by the investigators or participants, confounding is a special type of problem that may occur within a study as a result of interrelationships between the exposure of interest and another exposure. Confounding is a bias wherein an observed association is not a product of cause and effect but instead results from linking of the exposure of interest to another associated exposure. Studies pertaining to adverse effects of drugs of abuse are especially prone to confounding by variables such as concomitant caffeine use, alcohol use, tobacco use, nutritional deficiency, and/or psychiatric illness. Analytic studies may restrict characteristics of enrolled participants or match participant characteristics between comparison groups in an effort to reduce confounding. Accordingly, it has been suggested that future studies on delayed neuropsychiatric manifestations following CO poisoning control for potential confounding from depression and cyanide exposure.²⁷ During data analysis, confounding can often be controlled through stratification of data into subgroups or through ­multivariate analysis techniques.

Randomization is of central importance in clinical trials. It prevents selection bias and insures against unintended bias. It is also an important method to assure that unsuspected confounding factors are equally distributed between treatment groups within interventional studies. The four common types of randomization include (1) simple, (2) block, (3) stratified, and (4) unequal randomization. The simple method is equivalent to tossing a coin for each participant who enters a trial, such as heads = active, tails = placebo. Generally, a random number generator is used for this type of randomization. Block randomization is often used to guarantee balance in numbers during a clinical trial. The basic idea of block randomization is to divide potential patients into m blocks of size 2n, randomize each block such that n patients are allocated to A and n to B, and then to choose the blocks randomly. Less common is stratified randomization to prevent imbalance in prognostic factors that may confound estimating treatment effect. Stratified randomization achieves balance within important subgroups. For example, using block randomization separately for diabetics and nondiabetics in a cardiovascular trial. And finally, unequal randomization may be used when two or more treatments under evaluation have a cost difference such that it may be more economically efficient to randomize fewer patients to the expensive treatment and more to the cheaper one.

In order to improve the reporting of clinical trials, and to improve the recognition and interpretation of biases within them, guidelines referred to as the Consensus Standards of Reporting Trials (CONSORT) have been adopted by many medical journals.³⁰ The CONSORT guidelines provide a checklist to allow authors to systematically and uniformly report data and limitations. When interpreting published studies, it is also important to consider the potential for publication bias. Publication bias refers to the ­tendency for researchers, editors, and pharmaceutical companies to handle the reporting of studies with positive results differently from those with negative or inconclusive results.³⁸ Many journals now require that researchers register all planned clinical trials into a registry as a prerequisite for subsequent publication.

The NPDS database of the AAPCC is an ambitious effort to catalog and describe the epidemiology of poisoning in the United States and Canada. These data serve to help identify new poisoning epidemics, focus prevention and education efforts, guide demographic and economic poisoning analyses, and guide implementation of public health policies. It is a desirable goal to use this database in defining the scope of toxicity for particular xenobiotics and as a clinical research tool. In this regard, it is important to understand the biases inherent in the current database.

It has been suggested that selection bias might exist within PC data if poisoning is unrecognized as a cause of illness or if a caregiver has no questions pertaining to the management of a recognized poisoning.²² Indeed, a survey of 170 emergency physicians in Utah found that 53% admitted to using a PC for symptomatic acute overdoses, and only 10% contacted PC for the purposes of reporting cases to the national database.¹⁰ Such selection might result in a bias of PC data toward more severe cases. On the other end of the spectrum, two large investigations have found selection bias in PC data suggesting that fatal poisonings may be severely underrepresented.^19,26 It is interesting to note that in a 2004 report of the Institute of Medicine,²¹ the estimated range of annual fatal poisonings in the United States was from approximately 1000, derived from AAPCC data, to more than 30,000, derived from other databases. A study of potential spectrum bias in PC ­utilization found that one ED reported 95% of cyclic antidepressant overdoses, 33% of venomous snakebites, and only 3% of inhalation exposures.¹⁷ Further complicating the interpretation of PC data are the findings that such data may also be biased regarding geographic distribution of callers,² age,³⁴ ethnicity,^2,11,34,41 and socioeconomic status.⁴¹

Knowledge of information bias within NPDS data is less well characterized. Phone interviews of callers, many under duress, are certain to be subject to recall and interviewer bias. A comparison of rural hospital chart data to the NPDS database demonstrated deficiencies in PC reporting and in clinical information transfer to the NPDS database.²⁰ Loss to follow-up remains a problem for many PCs, and misclassification of poisonings by health care professionals inadequately trained in medical toxicology remains too common. The clinical conundrum of the unwitnessed ingestion frequently becomes an issue in PC-derived studies designed to create triage policies. Some children having never ingested a xenobiotic of concern may be misclassified as an exposure and may be improperly analyzed.²⁵

Another potential weakness of PC data involves potential misclassification of substances considered by regional consultants and caregivers. Postmortem toxicology can be a surrogate marker for the accuracy of exposure information reported to PCs in fatal cases. Previously, a putative analysis was undertaken at the New Jersey PC to characterize the discordance between PC consultation and postmortem toxicology.¹⁶ The researchers found that in 41 of the 206 (19.9%) fatal cases receiving poison center consultation, substances were found at the time of postmortem examination that were not considered in the poison center consultation. This study highlights the potential for discordance between exposures considered and those confirmed by forensic toxicologists. The reasons for discordance may include a lack of thorough history taking or a cognitive bias to the substances initially reported.

Despite the large volume of descriptive poisoning data available, it has proven difficult to derive valid, clinically useful conclusions from either the NPDS database or from published case reports.⁶ One suggested means through which to minimize information bias in descriptive toxicology is through the use of improved data collection charts.⁷ One large data collection effort underway to address this particular issue is the Toxicology Investigators’ Consortium (ToxIC), supported by the American College of Medical Toxicology, a prospectively collected data registry of patients seen by medical toxicologists at myriad hospitals and medical centers across the United States. Other researchers have found it useful in clinical studies to transform PC data collection from a passive to an active process through the use of specific research instruments.²⁸ Further efforts are required to reduce and to quantify the impact of selection, interviewer, recall, misclassification, and information biases within PC data to optimize the value of this important resource.

As was illustrated in Table 138–5, association of an exposure to an illness does not necessarily equate to cause and effect. In assessing causation, it must be determined if bias is present in the selection or measurement of exposure or outcome. If a study is unbiased, then the role of chance in the occurrence of the observed association must be explored. If an association is unbiased, unlikely to result from random error, and is not subject to confounding, then assumptions regarding to causation can be derived. Table 138–7 provides a list of evidentiary criteria, first proposed by Bradford Hill in 1965,¹⁸ that are often used to support causation.

In medical toxicology it is virtually impossible to prove causal relationships beyond any doubt. The goal is to build empiric evidence so that associations can be confirmed or refuted with conviction. However, many toxicologists deem clinical trials indicating a lack of benefit from gastric emptying, or indicating a therapeutic benefit of HBO therapy for CO intoxication, unconvincing because of the degree of bias present in all relevant published clinical trials. To address this issue, some consensus works (from position papers to formal consensus studies) attempt to provide guidance to clinical toxicologists regarding interventions where equipoise remains despite published clinical trials. For example, the Appraisal of Guidelines for Research and Evaluation (AGREE) Instrument is widely used in consensus guidelines. The instrument in its present form has been updated as “AGREE II” and is composed of 23 items organized into six quality domains (scope, stakeholders, rigor, clarity, applicability, independence).¹

As discussed previously, researchers must devise an appropriate statistical test to determine whether or not their data were arrived at by chance, a process known as inferential statistics. Once the significance level has been chosen, the inferential statistical test of choice depends on the type of data that are being collected. It should be noted that using the wrong statistical test for a given set of data will invalidate the results of a study; thus, choosing the correct statistical test is of paramount importance for research study design, as well as for toxicologists who try to interpret published medical studies.

Once the significance level has been set, then the data must be classified by type. Generally, data are either: (1) categoric/nominal (≥ 2 categories of data with no intrinsic ordering, eg, presence/absence of medical comorbidities), (2) continuous/interval (data along a scale where each value is equidistant from one another, eg, dollar values), or (3) ordinal/ranked (≥ 2 categories of data with clear intrinsic ordering, eg, grade in high school). The method used to compare groups of data thus depends on whether data is categorical, continuous, or ordinal. Most commonly, categorical data are compared in terms of ratios/percentages (Chi-squared test) and continuous data are compared in terms of means (student’s t-test) or medians (Mann-Whitney U test). As a general rule of thumb, an ideal analysis involves comparing interval/ratio data between groups due to the fact that this data contains more information than any of the other forms of data. Finally, the number and relatedness of the independent variables for analysis must be determined. Most commonly, two unrelated samples are compared to assess for associations. However, there are times when groups are paired or related, such as in clinical trials with before/after data in the same participants. Additionally, it is possible to use more advanced techniques to compare three or more independent variables at once (Table 138–8).

Inferential statistical tests are generally classified into parametric versus nonparametric techniques. This distinction is based on the fact that statistical tests make varying assumptions with regard to the population parameters that characterize the distributions for which the test is employed. Parametric tests generally make more stringent assumptions upon the data to produce valid results, while nonparametric tests make few if any assumptions about the data parameters and can thus be thought of as a “backup” test if the parametric assumptions are not met. For example, the t-test assumes that the two groups being compared are independent of each other, and that the dependent variable is continuous and normally distributed along a bell-shaped curve (also known as a Gaussian distribution). Similarly, the Chi-squared test assumes independent observation from a random selection of a given population with a large enough sample size such that any cell in the resultant 2 × 2 table is greater than five. A violation of one or more of these basic assumptions renders the test results invalid and mandates application of the nonparametric versions of these tests. A summary of common parametric and nonparametric tests based on data classifications is illustrated in Table 138–8.

In clinical practice it is often useful to have a test, which may be a laboratory result or clinical paradigm, to help arrive at a diagnosis or predict an outcome. For instance, historical questionnaires, capillary blood lead concentrations, and venous blood lead concentrations might all be used to identify children at risk of neurocognitive injury from plumbism.⁹ However, each of these approaches is likely to have certain disadvantages in terms of effort, cost, discomfort, and/or accuracy. Targeting lead evaluation and therapy in children on the basis of exposure history is expected to be easy and inexpensive, but may not identify some children with significant poisoning; thus, the test may be susceptible to being falsely negative. Capillary blood testing is more costly and uncomfortable and may be susceptible to false-positive test results because of environmental lead dust present on fingertips. The possibility of false-positive or false-negative results must be considered with any diagnostic test (Fig. 138–4).

The utility of diagnostic testing is often described in terms of sensitivity, specificity, predictive value of a positive test (PPV), and predictive value of a negative test (NPV). A cross-sectional design is often used to study diagnostic tests, as we seek to determine the prevalence of positive tests among the diseased (sensitivity), and the prevalence of negative tests among the healthy (specificity). A perfect test would be highly sensitive and specific, but this is seldom possible. A highly sensitive test (eg, D–dimer to assess for pulmonary embolism) is often used in screening programs because they rarely lead to false-negative diagnoses. Specific tests (eg, cardiac troponin to assess for acute coronary syndrome) are typically used to “rule-in” a diagnosis, as they rarely yield false-positive results. Whereas sensitivity and specificity are inherent properties of a diagnostic test applied to a given population, the probability of disease—based on the results of a test—is highly dependent on the prevalence of disease within the population being tested. The PPV is the probability of having disease in a patient with a positive test; the NPV is the probability of not having disease when the test result is negative. Numerous studies have tried to examine the utility of vomiting, leukocytosis, hyperglycemia, total iron-binding capacity, and radiographic findings in predicting toxicity after acute iron overdose. In a retrospective assessment of 40 adults with oral iron overdose, vomiting was found to predict a serum iron concentration above 300 μg/dL with a sensitivity rate of 84%, specificity of 50%, NPV of 44%, and PPV of 87%.³³ This suggested that the presence of vomiting should raise concern for iron toxicity but that the lack of vomiting was not particularly reassuring. Figure 138–4 illustrates the calculation of the sensitivity and specificity rates, as well as the PPV and NPV. It is important to remember that these calculations, too, are subject to bias and are best presented with CIs.

Galen, an influential physician from the second century, remarked of his clinical trial, “All who drink of this remedy recover in a short time, except those whom it does not help, who all die. Therefore, it is obvious that it fails only in incurable cases.” Unfortunately, error in contemporary clinical investigation of poisoning tends to be more insidious than the error in logic in Galen’s conclusion, and skillful scrutiny of published research remains an important endeavor.

• Medical toxicology has embraced the vision of incorporating “evidence-based, or literature-based, medicine” into practice.

• Randomized clinical trials, although a noble goal, are rare and have proven difficult to perform within the discipline.

• As toxicologists move beyond descriptive data reporting, there remains great potential for scientific advancement in the field of toxicology via observational, hypothesis-testing, clinical research.

• Clinical investigators are charged with the imperative to perform studies based on sound epidemiologic principles.

• All studies, by nature of population sampling, are at the mercy of chance, but such random error can be quantified using statistical techniques.

• Systematic error (bias) can be limited, but not entirely excluded, through careful study design.

• Clinicians interpreting published toxicologic research need to thoroughly evaluate a study’s research objectives, design, data acquisition, analysis, and conclusions before applying the results to patient care (Table 138–9).

• Future epidemiological investigation should allow more valid conclusions to be drawn regarding the associations between exposures and outcomes, or regarding the value of treatments for poisonings, discussed in the preceding chapters of this text.

References