Evidentiary Difficulties with Quantitative Risk Assessments Ven R. Walker* In the field of quantitative risk assessment, there is a constant flow of new data, new developments in risk assessment methodol- ogy, and new substantive scientific theories. In the midst of this flux of information and interpretation of information, parties to private lawsuits seek adjudication of scientific issues "once and for all." This article discusses certain evidentiary issues that can arise when quantitative risk assessments that are prepared in the context of an administrative program are employed as evidence in civil litigation. There are many types of issues that can arise when a document prepared by an administrative agency, such as a quantitative risk assessment, is transplanted from the context in which it was pre- pared to the quite different context of civil litigation. An adminis- trative document is prepared for a specific administrative purpose, within a specific administrative program, and consonant with the agency's substantive and methodological policies. Such documents often presuppose a body of scientific expertise within the issuing agency and presume such expertise within the relevant regulated community: an expertise necessary not only in the preparation of the document, but also in its interpretation and application. The document is prepared in reliance upon set pro- cedures for internal agency review, public participation and com- ment, and judicial review of final agency action. The substantive standards by which the administrative action will be reviewed are also well-established from the structure and history of the agency, and from the general standards of administrative law. The context of civil litigation in which an agency's document might be used may be very different from the administrative con- text in which the document was prepared. The isolated tort law- suit, for example, usually presents for decision not a prospective Dr. Walker, Ph.D., J.D., is Associate Professor at Hofstra University School of Law. This article is based on a presentation at a symposium on risk assessment, sponsored by the Environmental Law Committee of the Association of the Bar of the City of New York on February 4, 1988. 469 COLUMBIA JOURNAL OF ENVIRONMENTAL LAw [Vol. 14:469 program for implementation, but a retrospective controversy to be adjudicated.' Moreover, the policy objectives of the tortsys- tem are generally not those of the administrative-agency that pre- pared the document. In addition, neither the court nor the jury has scientific expertise, and the tort system relies principally upon the adversarial interests of the parties to ensure that the appropri- ate decisionmakers are adequately educated concerning the rele- vant scientific issues. The procedures under which the litigation proceeds at the trial level are very different from those governing the administrative action, and the burden of proof in a tort suit is the "preponderance of the evidence" standard, not the "arbitrary and capricious" standard under which an agency's final action, based on its studies and documents, ultimately would be reviewed if challenged. This article does not discuss the entire range of substantive or procedural problems that can arise when a document such as a regulatory risk assessment, which originates in an administrative setting, is transplanted into civil litigation. Rather, the article dis- cusses the relatively narrow, but increasingly significant, question of what evidentiary difficulties can arise when a quantitative risk assessment prepared by a governmental agency is offered as evi- dence in civil litigation. I. QUANTITATIVE RISK ASSESSMENTS AS EVIDENTIARY DOCUMENTS A. Considering Risk Assessments as Documents Throughout this article, the phrase "risk assessment" will be used to mean a quantitative risk assessment of the kind commonly prepared by the United States Environmental Protection Agency (EPA) for toxic agents such as arsenic, radioactive materials such as radon, or synthetic compounds such as chlordane. The phrase "risk assessment" will be used to refer to the whole corpus of relevant documentation concerning an environmental agent, however meager or extensive that documentation might be. In some cases, the risk assessment might be merely several pages in length, but more typically it will be a very extensive document, or will consist of a series of drafts, final documents and supplements generated over many years. The illustrations used throughout 1. Such unusual cases as the Agent Orange litigation or Dalkon Shield litigation come closer to adjudicating the merits of prospective, administrative programs. 470 Evidentiary Difficulties this article will be drawn from the set of documents prepared over several years by EPA on methylene chloride (also referred to as dichloromethane, or DCM). 2 As a secondary and derivative use, the phrase "risk assess- ment" refers not only to the reports or documents themselves, and to their contents, but also to the research and regulatory pro- cess resulting in those reports. In this latter sense, the various EPA documents concerning methylene chloride could be said to be products of a single, complex risk assessment, which in some important respect might be still ongoing within EPA. But for pur- poses of the evidentiary analysis in this article, this reference to the process itself will be a derivative meaning for several reasons. First, in discussing the evidentiary difficulties that might arise in civil litigation, it is important to focus explicitly upon the publicly available documents and upon their contents. Given the com- plexity of these risk assessments and of the evidentiary issues in- volved, it is very likely that the evidentiary issues will be raised and resolved upon written motions, not upon oral motions at trial. Thus, it is critical to identify arguments based upon docu- ments that can be attached as exhibits to the written motions. Second, it will probably be the case, at least with respect to EPA and many other federal agencies, that no witness from the Agency can be subpoenaed to testify concerning the risk assessment doc- uments or the underlying administrative investigation. Under current EPA regulations,3 no EPA employee may testify or pro- duce documents in a private civil court proceeding in connection with the employee's official relationship with EPA, or concerning information acquired in the course of performance of his or her official duties, unless authorized to do so by EPA's General Coun- sel. The objectives of the regulation are to ensure that employ- ees' official time is used only for official purposes, to ensure that public funds are not used for private purposes, and to maintain the impartiality of EPA with respect to private litigants.4 There- fore, if a private party seeks to depose an EPA employee in order 2. Risk assessments on other compounds could have been used as well, but it is not the purpose of this article to survey the various risk assessments currently available from EPA, and it is much more useful here to illustrate how the evidentiary difficulties to be discussed can all arise within the context of the documentation concerning a single environmental agent. 3. 40 C.F.R. §§ 2.401-2.406 (1987). 4. EPA, Public Information; Testimony by Employees and Production of Documents in Civil Legal Proceedings: Final Rule. 50 Fed. Reg. 32.386 (1985). 1989] 471 COLUMBIA JOURNAL OF ENVIRONMENTAL LAw [Vol. 14:469 to obtain an explanatioii of the contents of an EPA risk assess- ment, the likelihood is that EPA will refuse to allow the employee to provide the testimony. EPA's policy in such cases has been to allow an employee to testify in his or her official capacity only if EPA concludes that it is clearly in the Agency's interest to do so.5 One practical result of this agency policy is that parties are un- likely to have their particular questions about the risk assessment answered authoritatively, and they will have to do the best they can with the publicly available documents. For example, the party opposing the introduction of the risk assessment documents as evidence, or opposing their being read to the jury by an expert witness or by opposing counsel, has to base its objecting argu- ments primarily upon the risk assessment documents themselves, and upon other publicly available EPA documents. B. Elements of a Risk Assessment It has become standard practice to recognize four major ele- ments of a risk assessment, which are becoming more readily identifiable in risk assessment documents as agencies increasingly conform to this standard practice.6 An influential report by the National Academy of Sciences7 (hereinafter NAS) identified these four major elements or steps as:8 Hazard Identification: the process of determining whether ex- posure to an agent can cause an increase in the incidence of a health condition (cancer, birth defect, etc.); Dose-Response Assessment: the process of characterizing the relationship between the dose of an agent administered or re- ceived and the incidence of an adverse health effect in exposed populations, and estimating the incidence of the effect as a function of human exposure to the agent; Exposure Assessment: the process of measuring or estimating the intensity, frequency and duration of human exposures to an agent; Risk Characterization: the process of estimating the incidence of a health effect under the various conditions of human expo- sure described in the exposure assessment. 5. See 40 C.F.R. § 2.403 (1987). 6. See.e.g., EPA, RISK ASSESSMENT AND MANAGEMENT: FRAMEWORK FOR DECISION MAK- ING, 13-14 (Dec. 1984) (EPA Document No. 600/9-85-002). 7. NATIONAL RESEARCH COUNCIL, RISK ASSESSMENT IN THE FEDERAL GOVERNMENT: MAN- AGING THE PROCESS (1983) (hereinafter NAS REPORT]. 8. Id. at 19-28. These steps will be referred to in this article as "NAS elements" of a risk assessment. 472 Evidentiary Difficulties While these four elements provide a useful and familiar frame- work for analyzing the contents of a risk assessment, such a pro- file is useful principally from the perspective of the issuing regulatory agencies, and less so from the perspective of civil liti- gation. It is usually more fruitful in civil litigation to conceptual- ize the risk assessment from the standpoint of the substantive issues presented for decision in the particular lawsuit, and the types of evidentiary difficulties that will need to be resolved. The analysis of evidentiary difficulties should proceed in three fundamental steps: i. Identify with precision those issues in the lawsuit to which statements in the risk assessment documents might be relevant as evidence; ii. Identify the precise statements in the risk assessment that appear to be relevant to each substantive issue; and then iii. Analyze the evidentiary difficulties expected to be associ- ated with the particular risk assessment statements. These basic steps provide the lifelines of relevance for the eviden- tiary analysis. It is particularly important that such lifelines be ar- ticulated and maintained in the case of risk assessments, because it is easy to become lost in the minutiae of risk assessment docu- ments, and to overlook or lose sight of the tethering relationship of relevance upon which many evidentiary problems rest. The probative value of the various statements in the risk assessment is often an important factor, but one easily forgotten. Table 1 lists a few examples of substantive issues that can arise in tort litigation concerning cancer, together with the NAS ele- ment of the risk assessment in which relevant assertions might be found.9 Even these few examples illustrate why it is unfortunate, when performing an evidentiary analysis, to think of the risk as- sessment as a single, monolithic document. It is more useful to conceptualize the risk assessment as a very large collection of par- ticular assertions and statements, which are relevant in varying degrees to particular issues presented in the lawsuit. 9. Examples of recent cases posing such factual issues as increased risk of cancer or fear of cancer include: Hagerty v. L & L Marine Services, Inc., 788 F.2d 315 (5th Cir. 1986), on reconsideration, 797 F.2d 256 (1986); Stites v. Sundstrand Heat Transfer, Inc., 660 F. Supp. 1516 (W.D.Mich. 1987); Ayers v. Township ofJackson, 106 N.J. 557, 525 A.2d 287 (N.J. 1987). 1989] 473 COLUMBIA JOURNAL OF ENVIRONMENTAL LAW [Vol. 14:469 TABLE I Substantive Issues Risk Assessment Element Product defect Hazard Identification Cancerphobia Fear of cancer Property damage Punitive damages Increased risk of cancer Dose-Response Assessment Need for medical surveillance Background levels of agent Exposure Assessment Alternative sources of exposure Differential nature of plaintiff's exposure The NAS elements can be useful in trying to understand the risk assessment from the standpoint of the administrative agency, and they may help the litigator to locate assertions relevant to particular substantive issues to be resolved in the litigation. Once those relevant statements within the risk assessment have been located, however, it is more useful for the evidentiary analysis it- self to depart from the functional categories proposed by the NAS. For purposes of the evidentiary analysis, the statements within the risk assessment can be more fruitfully grouped along the following lines: Types of Statement from an Evidentiary Standpoint: 1. Descriptive Statements a. Observational Statements b. Judgmental Statements c. Speculative Statements 2. Statements of Policy 3. Misleading Statements The discussion that follows will explain the meaning of these cat- egories and discuss the evidentiary issues associated with each of these kinds of statements. II. EVIDENTIARY DIFFICULTIES WITH DESCRIPTIVE STATEMENTS Throughout each NAS element of a risk assessment, whether in hazard identification, dose-response assessment, exposure assess- ment or risk characterization, there may be assertions that appear to be descriptive of the world of nature: statements that are thought to be either true or false (whether or not we are in a posi- 474 Evidentiary Difficulties tion to determine their truth value), and subject at least in princi- ple to proof or disproof. Such statements offer the promise of being useful to the parties and the court in resolving factual issues concerning injury and causation. The litigating parties often re- sort to risk assessments to obtain the descriptive, scientific infor- mation that would be practically impossible for private parties to develop independently. With respect to such descriptive state- ments, the evidentiary analysis proceeds by sorting them into a spectrum that ranges from "observational" descriptive statements to "speculative" descriptive statements.'0 A. Observational Statements An "observational statement" is a descriptive statement that is a report of a personal perception,II and whose truth or falsehood can be determined in a relatively noncontroversial manner, with a minimal degree of expert judgment. A common example in risk assessments is a description that references the report of a spe- cific primary study (e.g., a toxicological study) by means of a defi- nite description, and simply relates the reported protocols, data or conclusions of that study in narrative form.12 The most com- mon reporting format is a declarative sentence containing a par- enthetical citation identifying the primary study. An example from an EPA risk assessment is: 10. We need not resolve, at this point, the question of whether statements that appear to be descriptive are sometimes in fact statements of policy or value statements, and not really descriptive statements at all. All that is needed in order for the category of "de- scriptive statements" to be useful is that it is possible to identify, with some degree of consensus, a substantial portion of the risk assessment statements that are appropriately thought to be descriptive. I1. Cf FED. R. EVID. 602 (fact witness may not testify in absence of personal knowledge). 12. If observational statements were considered in logical steps, the most basic state- ment would employ a demonstrative pronoun to reference the primary study, would quote directly from the study, and would report an immediate perception while reading the study. Such statements of the risk assessment author would have the form: "This study reports that ' . . . )." But demonstrative references, while sometimes encountered in testi- mony from fact witnesses, are not likely to be encountered in a risk assessment. Risk as- sessment authors usually employ definite descriptions to reference the primary study, and simply report the primary study's protocols, data, or conclusions without direct or indirect quotation from the study. The lack of quotation signals, however, can mislead nonexpert readers into thinking that the risk assessment author is relating facts from personal knowl- edge, when in reality the author is merely reporting what he or she read in the primary study. 1989] 475 COLUMBIA JOURNAL OF ENVIRONMENTAL LAW [Vol. 14:469 [T]he average number of benign mammary tumors per tumor- bearing female rat [in the cited study by Dow Chemical Com- pany] increased from 1.7 in the control rats, . . . to 3.0 in those exposed to 3,500 ppm.' 3 Another example is: Hazelton Laboratories (1982) studied the carcinogenicity of DCM in a chronic two year drinking water study in Fischer 344 rats, using the protocol as described under longer-term expo- sure. . . The authors stated that DCM did not induce carcinoge- nicity under the conditions of the study.'4 Such statements are ''observational" because they are based primarily on some observational or perceptual act of the assess- ment author, and their truth can be checked by reference to the cited primary study, a task requiring little or no scientific exper- tise. Such statements usually supply the informational foundation upon which most reputable experts in the field will agree, unless the primary study itself comes under scrutiny, which will be con- sidered below. Otherwise, professional judgment or opinion is at a minimum, and an experienced litigating attorney would be in- clined to stipulate whether the risk assessment accurately reports what the primary study asserts. A statement in the risk assessment paraphrasing the results of the primary study, but in the risk assessment author's own words, moves the description a step away from being merely observa- tional. Paraphrasing statements are intended by the risk assess- ment author merely to reformulate the results reported from the primary study, but such statements may require more expertise to evaluate their accuracy. Paraphrases of the primary study, if care- fully performed, add no information or opinion beyond that con- tained in the underlying report, and present no additional evidentiary problems. If the author of the risk assessment, however, describes the re- sults of the primary study in terms that depart significantly from those used in the primary study, or if the risk assessment author begins to reinterpret the study or evaluate its significance, then there is a departure from mere reporting. At that point, either individual expert judgment or administrative policy enters in, and 13. EPA, HEALTH AssESSMENT DocuMENT FOR DICHLOROMETHANE (METHYLENE CHLO- RIDE), 5-51 (Feb. 1985) (EPA Document No. EPA/600/8-82-004F) (Final Report) [herein- after EPA HEALTH ASSESSMENT]. 14. EPA, HEALTH ADVISORY FOR DICHLOROMETHANE, 6 (March 1987) (EPA Document No. PB87-235578) [hereinafter EPA HEALTH ADVISORY]. 476 Evidentiary Difficulties the resulting statement is no longer at the "observational" end of the descriptive spectrum, but rather in one of the other categories of description to be discussed below. Considered from the standpoint of the rules of evidence, obser- vational statements in a risk assessment might raise questions of accuracy or credibility to be resolved by the trier of fact, but they should survive most evidentiary objections provided hearsay ob- jections can be overcome.'5 Hearsay issues can arise concerning either the observational statement in the risk assessment itself, or the statement in the primary study that is being reported in the risk assessment. 1. Risk Assessment Statements as Hearsay Risk assessment documents, as noted earlier,'6 might them- selves be offered as evidence, without the presence of an expert witness to use or explain them. In such a case, a hearsay objec- tion might arise to the introduction of the document or to its be- ing read to the jury, on the grounds that it is being offered to prove the truth of the assertions in the document, but without opportunity to crossexamine the risk assessment author.'7 In the case of a risk assessment prepared by EPA, however, the simple hearsay objection can probably be overcome by reference to Fed- eral Rule of Evidence 803(8)(C), which provides a hearsay excep- tion for records, reports or data compilations of public agencies that set forth factual findings resulting from an investigation made pursuant to authority granted by law, if used in a civil case. 18 15. Hearsay is "a statement, other than one made by the declarant while testifying at the trial or hearing, offered in evidence to prove the truth of the matter asserted." FED. R. EvID. 801(c). 16. See supra text accompanying notes 3-6. 17. See FED. R. EVID. 802. 18. Courts have broadly interpreted the phrase "factual findings", and the Supreme Court recently held that "portions of investigatory reports otherwise admissible under Rule 803(8)(C) are not inadmissible merely because they state a conclusion or opinion." Beech Aircraft Corp. v. Rainey, 57 U.S.L.W. 4043, 4046 (Dec. 12, 1988). See, e.g., Ellis v. International Playtex, Inc., 745 F.2d 292, 301 (4th Cir. 1984) (tentative conclusions and statistical findings of epidemiological studies admissible); Kehm v. Procter & Gamble Mfg. Co., 724 F.2d 613, 618 (8th Cir. 1983) (epidemiological studies containing statistical anal- yses admissible); Baker v. Elcona Homes Corp., 588 F.2d 551, 556-59 (6th Cir. 1978), cert. denied, 441 U.S. 933 (1979); United States v. A. T. & T. Co., 498 F. Supp. 353, 359-64 (D.D.C. 1980) ("factual findings" includes simple descriptions of past acts or events, back- ground matters culled from other public documents, projections or analyses based on agency's expertise, and formal findings reached on basis of facts found in course of investi- 4771989]1 COLUMBIA JOURNAL OF ENVIRONMENTAL LAW [Vol. 14:469 In circumstances where an expert witness is presenting testi- mony and a risk assessment is used as part of the basis for the expert's opinion, then hearsay objections might be overcome for that reason.9 If the risk assessment is information "of a type rea- sonably relied upon by experts in the particular field in forming opinions or inferences upon the subject," then the risk assess- ment itself need not be admissible in evidence.20 The expert may incorporate information from the risk assessment into his or her testimony as part of the basis for his expert opinions.21 2. Problems with Hearsay Within Hearsay Even if the observational statement in the risk assessment sur- vives a hearsay objection directed against it, the hearsay objection might be pressed against the assertion in the primary study report that is being quoted or paraphrased. In all likelihood, the pri- mary study data or reported results are being reported in sum- mary fashion by the risk assessment author because the author takes them as true, and will use them in conducting the risk as- sessment analysis.22 The question then becomes whether the quoted material from the primary study should be excluded as hearsay.23 This presents the potentially more difficult problem of gation, but excludes conjectures about future and agency conclusions about party's failure to meet burden of proof). 19. See FED. R. EvID. 703. 20. Id Rule 702 refers to "facts or data in the particular case upon which an expert bases an opinion or inference," but the advisory committee's note to Rule 702 states that "an expert on the stand may give a dissertation or exposition of scientific or other princi- ples relevant to the case." State evidence law not adopting a modification of the Federal Rules of Evidence may also follow this same approach. For example, the strict rule in New York that expert testi- mony must be based on material in evidence has largely been abandoned, and such testi- mony is not rendered inadmissible because it is partly based on the hearsay reports of others, provided that such data are of the type reasonably relied upon by experts in the field in forming opinions or inferences upon the subject. Borden v. Brady, 92 A.D.2d 983, 461 N.Y.S.2d 497 (N.Y.App.Div. 1983). 21. See FED. R. EVID. 705 (anticipating testimony about bases for expert opinions). 22. In instances where the risk assessment author is merely performing a service by summarizing the available scientific literature, without then relying upon the studies so reported in conducting the risk assessment, evidentiary objections would follow those dis- cussed in the text, with the additional possibility that questions may arise concerning the relevance of the primary study. Cf FED. R. EvID. 401, 402 (evidence inadmissible as irrele- vant if it has no tendency to make the existence of facts of consequence to the determina- tion of the action more probable or less probable). 23. Cf FED. R. EVID. 805: "Hearsay included within hearsay is not excluded under the hearsay rule if each part of the combined statements conforms with an exception to the hearsay rule provided in these rules." One court has noted: 478 Evidentiary Difficulties determining the reliability of the primary study reported and uti- lized in the risk assessment. However, the evidentiary problems presented are no different than those that attend trying to intro- duce the primary study itself. It is usually the case that the primary study being relied upon is available to the risk assessment author only in the form reported in the open scientific literature. The primary study's "raw data" are usually not available to the risk assessment author, any more than to the general public, and have never been audited by the agency for their reliability. In such a case, however, the hearsay exception for "learned treatises" may be relevant.24 If this hear- say exception is applicable, the relevant statements may be read into evidence by an expert witness, but may not be received as exhibits, out of concern that the published material itself may be misunderstood and misapplied without expert assistance and supervision.25 A less frequent, but not uncommon, situation is when the un- derlying primary study has not been published, but has been con- ducted by the agency itself or by one of its contractors. In such a case, the primary data are available to the agency and probably have been reviewed by an appropriate agency office (e.g., a pro- ject officer responsible for oversight of the study), even if a review of the primary data has not been conducted by the risk assess- ment author himself. Federal Rule of Evidence 803(8)(C), which allows a hearsay exception for those records, reports or data com- pilations of public agencies setting forth factual findings,26 would seem clearly to apply in the case where the underlying primary [alithough the multiple hearsay problem has been mentioned with regard to Rule 803(8)(C) [citation omitted], it has generally been held that the author of the report or decision is not necessarily required to have first-hand knowledge of the facts upon which his findings are based[,J and concluded that "the multiple hearsay issue is reducible to one of the trustworthiness of the factual findings." United States v. A. T. & T. Co., 498 F. Supp. 353, 364 (D.D.C. 1980). On this approach. the discussion in this section of this article is relevant to the trustworthiness condition under Rule 803(8)(C), although not to the multiple hearsay is- sue as such. However, not all courts have so dismissed the multiple hearsay problem. See Baker v. Elcona Homes Corp., 588 F.2d 551, 559 (6th Cir. 1978), cert. denied, 441 U.S. 933 (1979) (statement of driver included in police accident report not admissible under Rule 803(8)) since neither observation nor factual finding of police officer, although driver's statement found not to be hearsay under Rule 801). 24. FED. R. EvID. 803(18). 25. See FED. R. EvID. 803(18) advisory committee's note. 26. See supra text accompanying note 20. 47919891 COLUMBIA JOURNAL OF ENVIRONMENTAL LAW [Vol. 14:469 study was conducted by a governmental agency pursuant to its statutory mandate.27 A third and more difficult situation, however, is presented when the primary study being relied upon in the risk assessment is neither published in the open literature nor the work of a govern- mental agency. In an increasing number of risk assessments, data reported and relied upon may have been obtained directly from private parties, such as data in support of a pesticide regis- tration under Section 3 of the Federal Insecticide, Fungicide and Rodenticide Act,28 or data contained in a premanufacture notifi- cation under Section 5 of the Toxic Substances Control Act.29 Perhaps this is precisely the kind of situation in which hearsay ob- jections are appropriate, since there would appear to be no in- dependent check upon the reliability of the underlying study.30 It might be possible, depending upon the circumstances sur- rounding the particular risk assessment, to convince the court to find an "other exception" to the hearsay exclusion rule. Federal Rule of Evidence 803(24) allows such case-by-case exceptions if "circumstantial guarantees of trustworthiness" can be found that are "equivalent" to those supporting the specifically covered ex- ceptions of Rule 803. In the context of such a determination, sev- eral factors relevant to risk assessments are worth noting. First, studies submitted by private parties to agencies may have been audited by the agency. In those cases, it would seem reasonable to accept the agency audit as an independent check on the relia- 27. The agency conducting the underlying primary study would not have to be the same agency issuing the risk assessment. Rule 803(8)(C) may be read to cover results of studies by state agencies, Kehm v. Procter & Gamble Mfg. Co., 724 F.2d 613, 617-20 (8th Cir. 1983), as well as the documents of foreign offices or agencies, In rejapanese Elec. Products Antitrust Litig., 723 F.2d 238, 271-74 (3d Cir. 1983), revd on other grounds. 475 U.S. 574 (1986). 28. 7 U.S.C. § 136a (1982). 29. 15 U.S.C. § 2604 (1982 & Supp. IV 1986). 30. If the company that conducted the study is an adverse party to the civil lawsuit, then the data and conclusions of the study might be admissible as not being hearsay because they constitute an "admission by party-opponent" under Rule 801(d)(2). Consideration should also be given in appropriate circumstances to the hearsay excep- tion for "records of [a] regularly conducted activity" or "business records," FED. R. EVID. 803(6), which allows exception for memoranda, reports, records or data compilations of "acts, events, conditions, opinions or diagnoses" made at or near the time by a person with knowledge, "if kept in the course of a regularly conducted business activity, and if it was the regular practice of that business activity to make the . . . report, . . . unless the source of information or the method or circumstances of preparation indicate lack of trustworthiness." 480 Evidentiary Difficulties bility of at least the primary study's data. Indeed, the existence of such an audit may well make those data more reliable than the data of many studies published in the scientific literature. Second, even if the study data have not been audited, the court might consider whether the study had been submitted to the agency in accordance with law. The submitter may be subject to criminal sanctions for misrepresentations,3 ' and this would be a relevant factor to consider in determining the study's reliability.32 Third, the study may have been designed according to guide- lines or regulations issued by the agency for the appropriate de- sign of such a study. In such a case, the existence of the guidelines or regulations would add to the reliability of the study design. If, however, the study design departs from usual or ac- cepted protocols in unexplained ways, then this could weigh against the reliability of the study, especially if the study was per- formed by a party in order to obtain some commercial benefit on the basis of the study results.33 Moreover, civil or criminal sanc- tions against misrepresentations in submissions might not pro- vide a guarantee of the appropriateness of the design of the study, which may be inadequate but not misrepresented in the report. Fourth, the very fact that the expert agency decided to rely upon the results of the study is a factor to be considered in decid- ing whether to find a hearsay exception. It might be the case that the agency has had a course of dealings with the reporting party that inclines the agency to view studies conducted by that party as reliable. Corporations often value their reputations for reliable data and careful study controls, and agency officials often develop informal opinions concerning the quality of the studies conducted by particular companies. If the agency conducting the risk assess- ment probably has a basis upon which to judge, at least in a cir- 31. Specific statutes may contain civil or criminal sanctions for false filings, or see generally 18 U.S.C. § 1001 (1982) (criminal sanctions for false, fictitious or fraudulent filings). 32. Cf FED. R. EvID. 804(b)(3) (statement not excluded by hearsay rule if declarant unavailable as witness and statement at time of its making "so far tended to subject the declarant to civil or criminal liability . . . that a reasonable person in the declarant's posi- tion would not have made the statement unless believing it to be true"). 33. Suspicions of unreliability would be lessened to the extent that the study was per- formed by a party that specializes in conducting such scientific studies, which has a reputa- tion for reliability to uphold, and which is independent from the party seeking to rely upon the results of the study. 1989] 481 COLUMBIA JOURNAL OF ENVIRONMENTAL LAw [Vol. 14:469 cumstantial way, the reliability of the study submitted, then the fact of the agency's informed reliance upon the study would be a factor to consider in determining the study's acceptability from the standpoint of hearsay.34 The evidentiary difficulties associated with independent asser- tions later incorporated into risk assessments should not be un- derestimated. The unfortunate fact is that regulatory agencies, due to resource constraints, often rely heavily upon other sources of information in order to perform their risk assessments. How- ever, the commonly accepted format for reporting such incorpo- rated information-i.e., declaratory form, without direct or indirect quotation, and citation by parenthetical naming of au- thors - is easily misleading to nonexperts, to whom it may ap- pear that the agency itself vouches for the truth of what is being asserted.35 In fact, much of what is reported in risk assessments in the form of observational statements is drawn from other sources, without independent confirmation of accuracy, and it is left to the litigating attorneys, their expert witnesses and the court to divine the evidentiary significance for each particular lawsuit. B. Judgmental Statements In the middle of the spectrum of descriptive statements are judgmental statements, those agency opinions whose truth or falsehood is determinable only by evaluating the significant ex- pert judgment used in making them. An example from an EPA risk assessment is: The major sources of exposure to DCM are from contaminated water. Air and food are only a minor sources [sic] (U. S. EPA, 1980c).36 Such a statement reflects EPA's expert judgment, presumably based upon the monitoring data and other evidence available to the Agency. A second example might be the following: 34. This is somewhat analogous to the reasoning behind Fed. R. Evid. 703, which pro- vides that facts or data used by an expert witness to base an opinion or inference need not be admissible in evidence if they are "of a type reasonably relied upon by experts in the particular field." Note, however, that in the situation under consideration in the text, it is assumed that the agency conducting the risk assessment will not provide an expert em- ployee to testify concerning the risk assessment or its bases. 35. The generic problem of "misleading" statements is discussed infra at Section IV. 36. EPA HEALTH ADVISORY, supra note 14, at 3. 482 Evidentiary Difficulties There are no human data on the metabolism of DCM to carbon dioxide. However, based on animal data and on uptake data, it is likely that this pathway is functional in humans (DiVincenzo and Kaplan, 1981a).37 Of course, to the extent that such a statement is really an observa- tional statement, merely quoting or paraphrasing the opinions of the authors of a primary study, then the evidentiary problems are those discussed in Section II.A, supra. What is of interest here are the truly judgmental statements: those in which the author and agency reach an opinion by relying upon their own expert judgment. In the case of judgmental statements, the probability of their being true is a function of the adequacy of the expert judgment involved. This dependence upon expert judgment makes rele- vant the experience, qualifications and expertise of the expert making the judgment, and requires a determination whether his or her judgment in such an area is reliable.3 8 Yet the hearsay problems become most acute when the expert is not present to have his or her expert qualifications, reasoning, and judgmental inferences subjected to cross-examination. Nevertheless, Federal Rule of Evidence 803(8)(C) provides an exception expressly for "factual findings," as well as for "data compilations," and most federal courts have interpreted the phrase "factual findings" widely enough to include many expert opinions.3 9 Thus, while the evidentiary considerations are potentially serious ones, the evi- dentiary objections to judgmental statements are not peculiar to risk assessments. Once issues of relevance and hearsay have been resolved,40 the admissibility of such judgmental statements or agency opinions ordinarily presents no unique evidentiary issues. The determina- tion of the credibility of such statements is then ordinarily a mat- 37. EPA, ADDENDUM TO THE HEALTH ASSESSMENT DOCUMENT FOR DICHLOROMETHANE (METHYLENE CHLORIDE): UPDATED CARCINOGENICTIY ASSESSMENT OF DICHLOROMETHANE (METHYLENE CHLORIDE), 67 (Sept. 1985) (EPA document no. EPA/600/8-82/004FF) (Fi- nal Report) [hereinafter EPA UPDATED CARCINOGENICIrY ASSESSMENT]. 38. See FED. R. EVID. 803(8) advisory committee's note (factors which may be of assist- ance in passing upon admissibility of evaluative reports include the timeliness of the inves- tigation, special skill or experience of the official, whether a hearing was held, and possible motivation problems). 39. See supra note 18. 40. See supra text accompanying notes 10 and 16-35. 1989] 483 COLUMBIA JOURNAL OF ENVIRONMENTAL Lhw [Vol. 14:469 ter for the trier of fact (e.g., the jury).4 1 Although such statements go beyond merely observational description, they are precisely the type of opinion statements for which the court and the parties turn to experts. The opinions of such experts are essential for educating and informing the decisions of the fact-finder. There is one additional concern, however, which might in a particular case form the basis for an evidentiary objection. An essential aspect of any expert witness' opinion is his or her testi- mony as to the probability that the opinion is true. If an expert witness were testifying along the lines of either example quoted above, he or she would normally testify directly or be cross-ex- amined on the degree of probability that the opinion correctly describes what actually occurs in nature. It would be a normal part of an expert's testimony to assert that it is more likely than not that the expert's opinion is true. It is rare, however, that a risk assessment document will contain a probability estimate for particular judgmental statements occurring in the document.4 2 The language of a particular judgmental statement might be am- biguous as to its probability of being true (e.g., ". . .it is reason- able to conclude that. . .") or unacceptably weak (e.g., ". . . it is consistent with these data to conclude that . . ."). Therefore, this problem may rise to the level of an evidentiary objection in situa- tions where the judgmental statement is central to the case.4 3 Finally, it is not suggested here that judgmental statements are readily identifiable in risk assessments. In many cases, it will be extremely difficult to sort descriptive statements into the catego- ries of observational statements or judgmental statements. Most statements encountered in risk assessments are merely unquali- fied assertions, and the reader is left to determine whether the statement is a report that could be easily verified or an opinion whose basis should be explored by expert testimony. Moreover, in scientific statements there often occur terms that appear to be simple descriptors, but whose use is a matter of theoretical con- 41. It is outside the scope of this article to discuss the difficulties in educating juries concerning the credibility of such judgmental statements. It is likewise outside that scope to discuss the adequacy of the jury trial system for dealing effectively with such issues. 42. No agency expert is likely to be produced by the agency to testify on this issue, see supra text accompanying notes 4-6. so the risk assessment document will probably not be clarified by any authoritative witness. 43. The evidentiary objection could be framed in terms of prejudice, Fed. R. Evid. 403, discussed in Section V, infra. or of asking the jury to speculate, discussed in Section II.C. infra. 484 Evidentiary Difficulties troversy within the scientific community." In many cases, the classification of a descriptive statement as being either observa- tional or judgmental can only be done with expert assistance. C. Speculative Statements Speculative statements are at the opposite extreme of the spec- trum from observational statements. Speculative statements are descriptive of nature, and are meaningfully considered to be either true or false, but their truth value is largely unknown, even by experts.4 5 The following is an example from EPA's methylene chloride risk assessment: Using the above value for q 1*, the lifetime incremental cancer risk [for humans from drinking methylene chloride in water] is estimated at 7.5 X 10-8 (ug/L)-'. 4 6 Several reasons for thinking that such a risk estimate is specula- tive will be discussed in the next section, addressing policy decisions. The "speculative" character of such an example cannot be eas- ily determined, since the speculative nature of risk assessment statements is rarely obvious. That is why they so frequently es- 44. For a discussion of evidentiary problems with novel scientific tests in criminal cases, see Giannelli, The Admissibility of Novel Scientific Evidence: Frye v. United States a Half-Century Later, 80 COLUM. L. REV. 1197 (1980). At least one court has held, however, that evidence that falls within the Rule 803(8)(C) exception has already met the Frye standard, because evidence gathered and presented by a public agency may be presumed to reflect method- ologies accepted by the scientific community. Ellis v. International Playtex, Inc., 745 F.2d 292, 303-04 (4th Cir. 1984). 45. See BLACK'S LAw DICTIONARY 273 (rev. 5th ed. 1979), defining "conjecture" as: [a]n explanation consistent with but not deducible as a reasonable inference from known facts or conditions. In popular use, synonymous with "guess." 46. EPA UPDATED CARCINOGENICITY AsSESSMENT, supra note 37, at 105. The variable ql* is the 95% upper confidence limit for the linear coefficient of the linearized multistage low-dose extrapolation model, see id. at 75-83, estimated for DCM by using the liver tu- mors in female mice (the sex with the higher risk estimate). Id. at 105. The EPA HEALTH ADVISORY states: The estimated excess cancer risk associated with lifetime exposure to drinking water containing DCM at 1,750 ug/L is approximately 3.7 X 10- [3.7 cases of cancer per 10,000 people]. EPA HEALTH ADVISORY, supra note 14, at 9. The document also suggests the speculative nature of this estimate by adding: This estimate represents the upper 95% confidence limit from extrapolations pre- pared by EPA's Carcinogen Assessment Group using the linearized, multistage model. The actual risk is unlikely to exceed this value, but there is considerable uncertainty as to the accuracy of risks calculated by this methodology. Id. 1989) 485 COLUMBIA JOURNAL OF ENVIRONMENTAL LAw [Vol. 14:469 cape notice as a significant evidentiary category distinct from judgmental statements. Yet it is important to try to identify them, and distinguish them from truly judgmental statements. What is distinctive about speculative statements is that even those experts asserting them will admit that they do not have an informed judg- ment as to the real probability that the statement is true.4 7 From an evidentiary standpoint, speculation about facts and speculative opinions are objectionable.48 The expert may not speculate or guess in a civil lawsuit,49 and an expert should not be allowed to testify concerning causation when he could do so only by speculating as to a necessary condition for causation.50 When the precise cause of an injury is left to conjecture and may be reasonably attributed to a condition for which no liability attaches as to one for which it does, then the plaintiff is not entitled to recover and the evidence should not be submitted to the jury.5 ' There are several varieties of such speculative statements. One important variety in risk assessments is the kind of statement that is made on the basis of mere assumptions about circumstances or 47. It would seem that the danger of a speculative description occurring in a risk assess- ment increases as a risk assessment author abandons technical terminology and attempts to express his or her conclusions in terms readily understandable to the nonexper. Thus, the danger of speculation is greatest precisely when the statement under consideration appears to be the most useful from the standpoint of communicating risk estimates to a jury. For example, if the statement quoted from the EPA UPDATED CARCINOGENICITY ASSESS- MENT, see supra text accompanying note 46, were interpreted as really a narrow, technical statement about ql*, merely reporting the numerical results of abstract mathematical mod- eling, then it becomes in effect a mere observational statement. If, on the other hand, it is interpreted as one would normally interpret it, as an attempt to estimate real incremental risk from exposure, then the risk assessor's effort to tell us something useful and impor- tant about the real world may result in speculation. 48. See, e.g., In re "Agent Orange" Prod. Liab. Litig., 611 F. Supp. 1223, 1248-50 (E.D.N.Y. 1985), aff'd, 818 F.2d 187 (1987). cert. denied, 108 S.Ct. 2898 (1988) (discussing cases excluding expert opinion as not based on data reasonably relied upon in the field where expert's conclusions are speculative or unfounded in fact). See also W. KEETON, D. DOBBS, R. KEETON & D. OWEN, PROSSER AND KEETON ON THE LAW OF TORTs 242 (5th ed. 1984): As long as the conclusion is a matter of mere speculation or conjecture, . . . it becomes the duty of the court to direct the jury that the burden of proof has not been sus- tained. [footnote omitted] 49. E.g., Aetna Casualty & Sur. Co. v. Barile, 86 A.D.2d 362, 364, 450 N.Y.S.2d 10, 12 (N.Y. App.Div. 1982). 50. See Vodery v. Niagara Mohawk Power Corp., 47 A.D.2d 983, 984, 367 N.Y.S.2d 129, 132 (N.Y. App.Div. 1972). 51. See Smith v. Squire Homes, Inc., 38 A.D.2d 879, 880, 329 N.Y.S.2d 243, 245 (N.Y. App.Div. 1972) 486 Evidentiary Difficulties actual conditions in the world. Such assumptions are often made because the risk assessment author needs to assume some value for a variable in order to make a computation (e.g., number of hours of exposure per day, average concentration of a compound per liter of drinking water). Of course, such assumptions can be incorporated as qualifiers into the concluding statement itself (e.g., "assuming four hours of exposure per day"), and the extent of speculation in the statement is reduced accordingly.52 Specula- tions based upon assumptions can also be made less speculative by supplying the missing information-for example, by supplying sufficient, representative empirical data on exposure times. Often, however, speculative statements are inherently specula- tive, at least in the practical sense that the information or data that is missing cannot be supplied in any practical way. There is another variety of speculative statement that is criti- cally important in risk assessments: speculative statements that are based not upon an isolated assumption about missing infor- mation or data, but rather upon a more systematic policy decision about how to proceed with risk assessment at some fundamental juncture, or in the face of nearly total ignorance. Risk assessment techniques are being developed in large part to provide a rational basis for regulatory decisionmaking in the absence of data.53 If policy decisions have been made in the absence of either data or any scientific theory probably reflective of nature, then descrip- tive statements based on those policy decisions are no less specu- lative than the underlying policy decision. In judicial review of agency decisionmaking where the standard of review is the "arbitrary and capricious" standard,54 final agency actions based upon policy decisions having some reason- able basis may survive review. But it is quite another matter to determine whether descriptive statements based upon agency policy decisions are "more likely than not" to be true. If even the expert agency conducting the risk assessment were to acknowl- edge that it has no basis for concluding that a given risk ratio is 52. Of course, a statement's relevance and probative value in a specific lawsuit might become more questionable by this explicit listing of assumptions. The degree of relevance and the probative value should not really change, however, but at most become more obvious, because the assumption provides part of the basis for the statement whether or not there is an explicit incorporation of the assumption into the statement. 53. See, e.g., NAS REPORT, supra note 7, at 11-13. 54. E.g, Small Refiner Lead Phase-Down Task Force v. EPA, 705 F.2d 506, 520-21 (D.C. Cir. 1983). 1989) 487 COLUMBIA JOURNAL OF ENVIRONMENTAL LAW [Vol. 14:469 more likely than not a good estimate of the real risk, then it would seem reasonable to consider any estimates of real risk as speculative.55 A serious problem for the opponent of a risk assessment, how- ever, is how to demonstrate to a judge that certain apparently de- scriptive statements in the risk assessment are in fact speculative, and therefore objectionable, while others are merely judgmental and of the kind reserved for the trier of fact. This problem of proof may become less severe to the extent that one can find within the risk assessment documents clear statements of the kind to be discussed next: assertions of policy decisions. III. DIFFICULTIES WITH ASSERTIONS OF POLICY DECISIONS Assertions of policy decisions differ from descriptive statements in an essential way: they are not meant to be descriptive state- ments (true or false) about nature or the actual state of things in the world, but rather they express or dictate decisional rules about how the risk assessor is to proceed. Descriptive statements are thought to be either true or false, and an evaluation of their truth-value is relevant to their acceptability as evidence. Policy decisions, by contrast, are not meaningfully either "true" or "false." They are appropriately evaluated upon some other set of criteria, such as administrative convenience, prudential concerns, consistency with statutory authority, or whether they meet some test of reasonableness.5 6 The following is an example of a statement asserting a policy decision: In animal studies it is assumed, unless evidence exists to the contrary, that if a carcinogenic response occurs at the dose levels used in the study, then responses will also occur at all lower doses. . 57 This decision rule tells the risk assessor (and the risk assessor in turn is alerting the reader) that he or she should proceed to extra- 55. For a discussion of policy decisions relating to risk estimates, see infra text accompa- nying notes 56-65. 56. Of course, there are descriptive statements that merely assert what policy decisions have been made by an agency, and those statements are either true or false, depending upon whether the agency really has made such a policy decision. Such descriptive state- ments are to be distinguished from the type of assertion being discussed in this section, which is a statement that really operates as a decision rule, not as a mere description of agency policies. 57. EPA HEALTH AsSESSMENT, supra note 13, at 5-90. 488 Evidentiary Difficulties polate a low-dose response curve from high-dose data. It is usu- ally the case that animal bioassays are designed only with relatively high doses given to the test animals, and animal re- sponses to lower doses are not empirically studied. The risk as- sessment author, however, is frequently confronted with the need to evaluate possible responses in the low-dose range. This deci- sion rule tells the risk assessor that if a carcinogenic response oc- curs in the high-dose range, then he or she should proceed as though a positive response would also occur at any arbitrarily low dose level. This example was chosen because it suggests in relatively clear language that an "assumption" is being made. Assertions of pol- icy decisions are not always so clear, and they can be stated in such a way that a nonexpert is unable to determine that a policy decision, not an accepted scientific proposition, is being an- nounced. Even in the case of this example, one might wonder whether the "assumption" is a calculated, scientific judgment, or a policy decision designed to guide regulatory risk assessments in the absence of low-dose data. In this particular case, however, EPA has been helpful in documenting the extent to which this statement announces a policy decision with little or no scientific basis: The CAG's [EPA Carcinogen Assessment Group's] position is that, given the limited data available from these animal bioas- says, especially at the high-dose levels required for testing, al- most nothing is known about the true shape of the dose- response curve at low environmental levels .. .8 A statement such as this one helps make it clear that the policy decision stated earlier was not based on scientific information. EPA's scientists are not concluding that, more probably than not, the low-dose extrapolations being made in the risk assessment ac- curately represent the true low-dose response for methylene chloride. Elsewhere EPA suggests that the particular low-dose extra- polation technique used by the agency was chosen, at least in part, to achieve "consistency" in estimating a conservative "upper bound" for potential risk: No current understanding of the biological mechanism of carci- nogenesis is able to predict which of these [low-dose extrapola- 58. Id. at 5-98. 1989] 489 COL.UMBIA JOURNAL OF ENVIRONMENTAL LAW [Vol. 14:469 tion] models is more accurate than another. . . . In the interest of consistency of approach and in providing an upper bound on the potential cancer risk, the Agency has recommended use of the linearized multistage approach.9 Consistency in achieving conservative estimates of risk is a policy objective, which EPA has sought to achieve by this decision rule. It should be added that there is nothing surprising about the need for policy decisions. Risk assessment is a quantitative technique, and when data are absent some method must be followed for sup- plying the numbers needed to proceed with the quantitative as- sessment.6 0 The difficulties arise in the context of civil litigation, when policy decisions are not recognized and treated as such. As- sertions of policy decisions are not descriptive statements. More- over, any descriptive statements that depend for their truth-value on policy decisions such as low-dose extrapolation should be con- sidered speculative.6 ' Another example of a policy decision that may give rise to spec- ulative descriptive statements is the decision rule for deriving human doses presumed to be equivalent to doses given to test animals. EPA has decided to use a conversion factor based on relative surface area between the species.62 But EPA also states: 59. EPA HEALTH ADVISORY, supra note 14, at 10. EPA has recently reopened discussion on low-dose extrapolation techniques. EPA, Intent to Review Guidelines for Carcinogen Risk Assessment, 53 Fed. Reg. 32,656. 32,657 (1988). 60. The NAS REPORT recognized that "[wihen scientific uncertainty is encountered in the risk assessment process, inferential bridges are needed to allow the process to con- tinue." NAS REPORT, supra note 7, at 28. The NAS REPORT also pointed out that "policy considerations inevitably affect, and perhaps determine, some of the choices among the inference options." Id. at 33. For a general discussion of this problem, see id. at 28-38. EPA has elaborated upon this theme: The National Research Council (NRC,1983 [the NAS REPORT]) pointed out that there are many questions encountered in the risk assessment process that are unan- swerable given current scientific knowledge. To bridge the uncertainty that exists in these areas where there is no scientific consensus, inferences must be made to en- sure that progress continues in the assessment process. EPA, Guidelines for Carcinogen Risk Assessment. 51 Fed. Reg. 33,992, 33,994 (1986). 61. It is not appropriate here to explore conceptual difficulties in reconciling how an agency's final action based upon such a policy decision can be nonarbitrary in the context of administrative law, while descriptive statements based on the policy decision should be barred from evidence in civil litigation as speculative. That is a problem of administrative law that digresses from the issues being discussed here. 62. EPA, Guidelines for Carcinogen Risk Assessment, 51 Fed. Reg. 33,992, 33,998 (1986); EPA UPDATED CARCINOGENICHiy ASSESSMENT. supra note 37, at 101-03, 105. 490 Evidentiary Difficulties [Tihe concept of equivalent doses for humans compared to ani- mals on a mg/surface area basis is virtually without experimen- tal verification regarding carcinogenic response.6 3 Thus, any estimation of what human doses are physiologically equivalent to animal doses, if such an estimate were derived using EPA's policy decisions, should be considered a speculative conclusion. One should not be misled by the above examples into thinking that policy decisions are often clearly identified as such in risk as- sessments, or that there are usually clear statements documenting a lack of scientific basis for decisions whenever such a basis is in fact lacking. Rather, the examples above are highly unusual in the risk assessment literature precisely because the relevant clarifying statements are also available. It is as likely, or perhaps more likely, that a policy decision will go unreported or be stated merely in a descriptive form, and that the risk assessment will contain no statement discussing the underlying scientific basis (or lack thereof) for the decision rule being employed. Thus, it is usually very difficult for litigating attorneys to demonstrate on motions precisely which risk assessment statements are specula- tive descriptions, or are policy decisions, and which are judgmen- tal descriptions.64 It is not that agencies seek to conceal such information: there is no administrative interest that must be protected by concealment, 63. EPA HEALTH AssESSMENrr, supra note 13, at 5-97. EPA's policy decision on the ex- trapolation of equivalent doses has been stated as follows: In the absence of comparative toxicological, physiological, metabolic, and pharmaco- logical data for a given suspect carcinogen, the Agency takes the position that the extrapolation on the basis of surface area is considered to be appropriate because certain pharmacological effects commonly scale according to surface area. (Dedrick, 1973; Freireich et al., 1966; Pinkel, 1958) EPA, supra note 62 at 33,998. EPA acknowledges that dose-scaling based on surface area is not a universal approach. EPA, supra note 59 at 32,657. 64. Policy decisions themselves can be thought of as occurring in several varieties, which mirror the kinds of descriptive statements that can be generated by using those policy decisions. Some policy decisions merely routinize an expert opinion held by the agency, such as a policy decision that certain judgmental conclusions should be drawn consistently from the available data (e.g., an EPA decision that five years of meteorological data probably constitutes a representative sample for purposes of air dispersion model- ing). Other policy decisions, such as those discussed here, direct that certain assumptions or calculations should be made in the absence of confirming data. Thus, policy decisions might provide part of the basis for judgmental statements, as well as for speculative state- ments. The point to be stressed, however, is that the mere presence or operation of a policy decision does not "convert" an otherwise speculative statement into a judgmental one. 1989] 491 COLUMBIA JOURNAL OF ENVIRONMENTAL LAW [Vol. 14:469 and agencies have and recognize a duty not to issue misleading information to the public. Rather, the volume of risk assessment work to be done is so great, the documents reporting risk assess- ments are already so voluminous, and the technical terminology and jargon of the risk assessment "community" is so efficient and elliptical, that risk assessment authors often simply fail to see the need for additional explanation in their documents. Moreover, the practice and art of communicating risk assessments has re- ceived increasing attention over the last several years, with the result that risk assessments written five or six years ago may well contain less explanation or discussion for the nonexpert than those written more recently.6 5 This is not to suggest, however, that the attorney involved in evaluating a typical risk assessment from the standpoint of suita- bility as evidence in civil litigation faces anything other than a for- midable task of sorting out speculation from scientific judgment. As formidable as the task typically is, however, it is essential that it be done. In view of the extensive gaps in the toxicological and exposure data bases for most environmental agents, and the im- portant unknowns in human physiology and pharmacology, risk assessments often proceed, by necessity, upon numerous policy decisions that bridge ignorance to arrive at a basis for regulatory actions. Unless the extent and importance of such policy deci- sions are made clear in a particular civil case, however, there is substantial danger of an unwarranted decision on the issue of civil liability. In addition to the issue of speculation, moreover, the issue of prejudicial effect also arises, and will be considered next. IV. DIFFICULTIES WITH MISLEADING ExPRESSIONS The discussion thus far has centered upon the evidentiary problems posed by irrelevant statements, hearsay, and speculative assertions. There is yet another basis, however, upon which in- troduction of a particular risk assessment might be objectionable: 65. For example, note the statement of William D. Ruckelshaus, then Administrator of EPA, that "we must search for ways to describe risk in terms that the average citizen can comprehend." Ruckelshaus, Science, Risk, and Public Policy, 221 SCIENCE 1026, 1028 (1983). In recent years, scholarly attention has turned to the problems of adequately communicat- ing risk assessments. Cf, e.g., Slovic. Informing and Educating the Public About Risk. 6 RISK ANALYSIS 403 (1986) ; Keeney & von Winterfeldt. Improving Risk Communication. 6 RISK ANALYSIS 417 (1986). 492 Evidentiary Difficulties the presence of misleading or prejudicial statements.66 State- ments that contain misleading expressions or phrases may be de- scriptive statements or statements of policy decisions, discussed supra, but the misleading nature of the statement should be con- sidered, for the sake of clarity, as a separate evidentiary issue. Federal Rule of Evidence 403 provides that even relevant or otherwise admissible evidence may be excluded "if its probative value is substantially outweighed by the danger of . .. misleading the jury." Some misleading expressions or phrases subtly change the meaning of commonly understood terms or terms that have a particular meaning in civil litigation. Other misleading expres- sions have technical meanings that are too elliptical for a nonex- pert, as when important qualifiers are dropped to achieve an efficient "short-hand." What these expressions have in common is an ability, perhaps a tendency, to confuse and mislead a nonexpert.67 One important example of a potentially misleading expression found in EPA's risk assessments is the phrase "probable human carcinogen." As commonly understood, the adjective "probable" means "likely to be true," "supported by evidence strong enough to establish presumption but not proof,"68 or "based on or arises from adequate fairly convincing though not absolutely conclusive intrinsic or extrinsic evidence or support."69 In the context of civil litigation, the meaning employed is generally expressed as "more likely than not to be true," supported by "a preponder- ance of the evidence,"70 or "having more evidence for than against."7' In a particular lawsuit, a key issue might be whether it is more likely than not that a particular environmental agent is 66. It is important to note that if the basis for finding a hearsay exception for a risk assessment were Fed. R. Evid. 803(18) ("learned treatises"), instead of Fed. R. Evid. 803(8) ("public records and reports"), then the risk assessment documents themselves could not be received as exhibits, and an expert witness would be needed to explain and apply the risk assessment material. See supra text accompanying notes 25-26. This aid to avoid misunderstanding is not required under Fed. R. Evid. 803(8). 67. Indeed, it is possible that even expert witnesses, versed in scientific fields such as pathology or toxicology, miss important nuances in terminology that has acquired a "regu- latory" meaning different from that assumed by the expert. 68. WEBSTER'S NINTH NEW COLLEGIATE DICTIONARY 937 (1988). 69. WEBSTER's THIRD NEW INTERNATIONAL DICTIONARY 1806 (1986). 70. See MCCORMICK ON EVIDENCE 339, at 957 (3d ed. 1984): The most acceptable meaning to be given to the expression, proof by a preponder- ance, seems to be proof which leads the jury to find the existence of the contested fact is more probable than its nonexistence [footnote omitted]. 71. BLACK's LAW DICTONARY 1081 (rev. 5th ed. 1979). 4931989] COLUMBIA JOURNAL OF ENVIRONMENTAL LAw [Vol. 14:469 capable of causing cancer in humans: whether, on balance, the preponderance of the evidence indicates that humans can, under any circumstances, develop cancer as a causal result of exposure to the agent. With respect to such an issue, EPA's determination that the en- vironmental agent in question is a "probable human carcinogen" would appear to be clearly relevant. The question that arises is whether EPA's nomenclature is misleading under the circum- stances. EPA uses the label "probable human carcinogen" to identify a category designated as carcinogen Group B2. 7 2 There is good reason to suspect, however, that when EPA classifies a com- pound as being in Group B2, the agency is not necessarily conclud- ing that more evidence points toward human carcinogenic action than away from such action. For example, in the case of compounds for which the only evi- dence of carcinogenic action in any species is the causing of tu- mors only in the livers of mice, EPA notes that [flor a number of reasons, there are widely diverging scientific views . . . about the validity of mouse liver tumors as an indica- tion of potential carcinogenicity in humans when such tumors occur in strains with high spontaneous background incidence and when they constitute the only tumor response to an agent.73 Yet EPA usually classifies such compounds as being in Group B2,74 even though there may be limited epidemiological data showing no increased incidence of cancer in humans, and there may be toxicological studies in species other than mice (e.g., rats, hamsters, dogs, monkeys) that show no carcinogenic action. It seems highly unlikely that EPA is asserting, by so classifying such a compound, that the body of evidence, taken as a whole, makes it "more likely than not" that the compound causes cancer in humans.75 If EPA does not intend to assert this, however, it can 72. EPA, supra note 62 at 34,000. 73. EPA, Id. at 33,995. 74. See EPA, Id. at 33,995, 34,000. 34002. 75. Being forced to reason circumstantially on such an important point is one unfortu- nate result of not being able, in the context of civil litigation, to ask agency officials such questions directly, through interrogatories or in depositions. See supra text accompanying notes 4-6. 494 Evidentiary Difficulties be very misleading in a litigation context to tell a jury that EPA considers the compound a "probable human carcinogen."76 A second example of a potentially misleading expression used in hazard identification is the expression "sufficient evidence from animal studies." This phrase is also, like "probable human carcinogen," a label for a "weight of evidence" category, and the criteria for classifying an environmental agent into this category are reasonably well-defined."7 But such classifications are often made on the basis of significant policy decisions -such as the de- cision rule to count both benign and malignant tumors "unless the benign tumors are not considered to have the potential to progress to the associated malignancies of the same histogenic origin."78 This policy decision has been made in the face of, and presumably with due regulatory regard for, the fact that there is considerable scientific debate over the, significance for human car- cinogenic potential of benign tumor formation in animals.79 But it might be very misleading in a particular lawsuit simply to relate to a jury that EPA considers the animal data for a particular environ- mental agent to be "sufficient" evidence for. carcinogenicity in humans, if by that the jury might understand that the animal data demonstrate, with "sufficient" clarity and strength under the bur- den of proof of civil litigation, that the agent causes cancer in humans. Examples of potentially misleading expressions can also be found in dose-response discussions of quantitative risk assess- ments. The "maximum likelihood estimate"s0 would seem to be 76. EPA has recently invited public comment on the use of the adjective "probable" to classify carcinogens. EPA. supra note 59 at 32,657. 77. See EPA. supra note 62 at 33,999. 78. EPA, supra note 62 at note 3. 79. See, e.g., NAS REPORT, supra note 7, at 34. See aLso Office of Science and Technology Policy, Chemical Carcinogens: A Review of the Science and Its Associated Principles, Feb- ruary 1985, 50 Fed. Reg. 10.371, 10,416-19 (1985); EPA. supra note 59 at 32,657. EPA describes the OSTP final report as -[a] summary of the current state of knowledge in the field of carcinogenesis and a statement of broad scientific principles of carcinogen risk assessment." and states that the OTSP report "forms an important basis for these [EPA Carcinogen] Guidelines." supra note 62 at 32,993. 80. The "maximum likelihood estimate" ("ILE") is defined as a function of ql, the linear coefficient in the multistage dose-response model employed by EPA to extrapolate low-dose risk estimates from high-dose bioassay data. See EPA UPDATED CARCINOGENICITY AsSESSMENT, supra note 37, at 74-75, 83; See also Occupational Safety and Health Adminis- tration, Proposed Rule on Air Contaminants, 53 Fed. Reg. 20,960, 21,190-91 (1988). Thus, the "maximum likelihood estimate" of risk generated for such compounds as methy- lene chloride, see, e.g., EPA UPDATED CARCINOGENICITY ASSESSMENT, supra note 37. at 6. 71- 4951989] COLUMBIA JOURNAL OF ENVIRONMENTAL LAW [Vol. 14:469 the prediction most likely to be true-the estimate of risk or pre- dicted incidence of cancer with the "maximum" likelihood to be true. In fact, however, EPA considers it impossible to tell whether the real risk equals the "plausible upper limit" for risk, the "maximum likelihood estimate," zero, or any specific risk ra- tio.' Presumably, the real risk is as likely to be zero as to be the "maximum likelihood estimate." If this is the case, referring to any particular risk ratio (e.g., I case per 1,000 persons) simply as the "maximum likelihood estimate" can be very misleading. An additional example of a potentially misleading expression can be drawn from the area of risk characterization. Even in cases where the real risk of causing cancer in humans may well be zero, one may still find EPA calculating what the agency calls an "esti- mated excess cancer risk" (e.g., I case per 10,000 people).8 2 EPA may have sufficient administrative reasons for making such a cal- culation (e.g., as a means of setting priorities in targeting pollu- tants for regulation). Use of such a phrase, however, might mislead the jury (and the judge) into thinking that there emerges from all of EPA's quantitative and technical analysis a simply ex- pressed "bottom line" conclusion that is predictive of real risk and readily understandable, even by a nonexpert. But the last thing that many cancer risk assessments, properly understood, are capable of producing is a simple, unqualified risk ratio as a conclusion. Such potentially misleading expressions allow the layers upon layers of uncertainty in the risk analysis to be left behind. 83, is a highly technical term, whose use in the absence of sufficient qualifiers and explana- tory materials could be highly misleading. 81. EPA states: The true value of the risk is unknown and may be as low as zero. The range of risks, defined by the upper limit given by the chosen model and the lower limit which may be as low as zero, should be explicitly stated. An established procedure does not yet exist for making "most likely" or "best" estimates of risk within the range of uncer- tainty defined by the upper and lower limit estimates. EPA. supra note 62 at 33,998. 82. See, e.g.,supra note 46. EPA often estimates excess cancer risk in "unit" terms: Under an assumption of low-dose linearity, the unit cancer risk is the excess lifetime risk due to a continuous constant lifetime exposure of one unit of carcinogen concen- tration. Typical exposure units include ppm or ppb in food or water, mg/kg-day by ingestion, or ppm or ug/m' in air. EPA, supm note 62 at 33.998. 496 Evidentiary Difficulties V. OTHER EVIDENTIARY PROBLEMS The above discussions focused on statements found in risk as- sessment documents: statements that are hearsay, speculative in nature, expressive of policy decisions, or constructed with mis- leading expressions. In addition to the evidentiary problems al- ready discussed, further evidentiary difficulties arise when the probative value of risk assessment statements in a particular law- suit is substantially outweighed by the danger of unfair prejudice, the danger of confusion of the issues, or considerations of undue delay or waste of time. Under Federal Rule of Evidence 403, introduction of a risk as- sessment into evidence may be objectionable when its probative value is substantially outweighed by the danger of unfair preju- dice. By "unfair prejudice" is meant an undue tendency to sug- gest a decision on an improper basis-commonly, but not necessarily, an emotional one.8 3 For example, if sufficient facts are not in evidence in a particular case, the reading of a risk as- sessment might be barred as inviting unwarranted speculation by the jury about the facts of the case.84 Testimony about a risk as- sessment might be barred if it would convey a false aura of scien- tific infallibility. 85 Or a risk assessment document (as well as testimony about its contents) might be excluded if its aura of offi- cial trustworthiness would not be commensurate with its actual reliability.8 6 A risk assessment might also be objectionable if its probative value is outweighed by the danger of confusion of the issues. For example, in the case of a particular risk assessment, it might be virtually impossible to determine where science ends and policy begins. It is possible that such a risk assessment would confuse, rather than clarify, such issues as the relevance of animal bioassay results to conclusions about human carcinogenicity. 83. FED. R. Evio. 403 advisory committee's note. 84. See Ellis v. International Playtex, Inc., 745 F.2d 292, 305-06 (4th Cir. 1984) (read- ing of published scientific article barred). 85. See In re "Agent Orange" Prod. Liab. Litig., 611 F. Supp. 1223, 1256 (E.D.N.Y. 1985), af'd, 818 F.2d 187 (1987), cert. denied, 108 S.Ct. 2898 (1988) (false aura of scientific infallibility coupled with speculation and unfounded assumptions in doctors' testimony potentially misleading to jurors). 86. See City of New York v. Pullman Inc., 662 F.2d 910, 915 (2d Cir. 1981), cert. denied, 454 U.S. 1164 (1982) (excluding "a so-called government report" which was incomplete and based largely on hearsay). 1989] 497 498 COLUMBIA JOURNAL OF ENVIRONMENTAL LAw (Vol. 14:469 The probative value of a risk assessment might also be out- weighed by considerations of undue delay or waste of time. If a particular risk assessment has, for example, a minimal amount of descriptive, factual material (e.g., in hazard identification) and the quantitative dose-response assessment is only marginally relevant to the issues to be decided in the lawsuit, then it is possible that extensive testimony by experts about the methodology employed by the agency to derive the quantitative risk estimate would con- stitute an unwarranted digression.87 Another evidentiary difficulty under Rule 403 might arise if the procedural context in which the risk assessment was generated undermines its reliability or trustworthiness. A particular risk as- sessment might be inadmissible as untrustworthy if sufficient neg- ative factors are present, including:"" timeliness of the study or report,9 whether a hearing was held or a notice-and-comment procedure was utilized,90 the skill or experience of the official who conducted the study,9 ' possible motivation problems,92 and other such factors affecting reliability. In addition, staff memoranda, proposed draft reports, and "interim" reports that were never adopted by the agency itself might not be sufficiently reliable so as to outweigh their aura of authoritativeness.93 87. See, e.g., In re "Agent Orange" Prod. Liab. Litig., 611 F. Supp. 1223, 1256 (E.D.N.Y. 1985), af'd. 818 F.2d 187, cert. denied, 108 S.Ct. 2898 (1988) (establishing low probative value of doctors' testimony at trial would entail unwarranted expenditure of time and effort). 88. See FED. R. EvID. 803(8) advisory committee's note. 89. For example, the risk assessment might be rather old, or completed before impor- tant new data were available. 90. For example, a risk assessment performed solely by regulatory staff in an expedited or summary fashion in order to be used in an emergency regulatory proceeding might be entitled to less deference than one created in a less adversarial atmosphere, allowing ade- quate time for preparation and full public participation. 91. It is unfortunate that it is not always possible to determine which agency employees or contractors authored which portions of a risk assessment, and what the qualifications or credentials of those individuals are. These factors are certainly relevant to assessing the reliability of the numerous judgmental opinions contained in a risk assessment. 92. For example, whether the risk assessment was prepared by an agency's staff toxicol- ogists at the request of private parties, who were known by the staff to be then engaged in private litigation. 93. Fed. R. Evid. 803(8)(C) establishes a hearsay exception in civil actions for reports of public agencies presenting "factual findings ... unless the sources of information or other circumstances indicate lack of trustworthiness." Staff memoranda, proposed draft reports, and unadopted interim reports might not constitute "factual findings" within this rule. United States v. American Tel. and Tel. Co., 498 F. Supp. 353 (D.D.C. 1980); City of New York v. Pullman Inc., 662 F.2d 910 (2d Cir. 1981), cert. denied, 454 U.S. 1164 (1982). Evidentiary Difficulties Finally, if the party seeking introduction of the risk assessment is relying in part upon an expert witness who intends to base his or her opinion testimony on a risk assessment, it may be that the risk assessment methodology itself lies outside the expert witness' field of expertise with the result that the expert is unable to ex- plain key elements of the risk assessment methodology. If a pathologist, for example, is not familiar with risk assessment methodology, it is reasonable to doubt whether the pathologist sufficiently appreciates the assumptions, policy decisions, techni- cal terminology, and jargon encountered in the risk assessment.94 Regulatory risk assessment has become so technical and special- ized that courts should regard the area as a separate area of expertise. Moreover, to the extent that the judge finds a particular risk assessment to be potentially misleading and confusing, the ab- sence of a qualified expert witness to explain the risk assessment to the jury should weigh heavily in a decision about excluding the risk assessment from evidence. VI. CONCLUSION Risk assessment, as currently practiced by agencies such as EPA, has evolved into a very technical procedure, with its own objectives, assumptions, techniques, and terminology. The one conclusion that can safely be drawn is that the average nonexpert cannot comprehend most risk assessment documents without a substantial danger of misinterpretation. It is important, there- fore, for parties and courts, in the context of civil litigation, to scrutinize a risk assessment closely from the standpoint of the standard evidentiary considerations, but with particular attention to the peculiar issues posed by risk assessments. The general an- alytical methodology presented in this article-focusing upon general relevancy and hearsay considerations, speculative de- sciptive statements, assertions of policy decisions, potentially misleading expressions, and contextual considerations of reliabil- ity-should assist parties and judges in recognizing and correctly deciding the peculiar evidentiary issues that arise. 94. For example. when an expert witness is qualified in a particular field, the opinions asked of the witness are required to be within that field. See Hileman v. Schmitt's Garage, Inc., 58 A.D.2d 1029, 1030. 397 N.Y.S.2d 501. 503 (N Y.App.Div. 1977). 1989] 499