Risk Policy Report, February 7, 2006
EPA SCIENTISTS CRITICIZE OMB'S CALL FOR ALTERNATIVE RISK MODELS
[Rachel's introduction: The Office of Management and Budget (OMB) has proposed new rules for risk assessments conducted by federal agencies. Scientists within the U.S. Environmental Protection Agency are not enthusiastic.]
EPA staff scientists are strongly criticizing recent calls by the White House Office of Management & Budget (OMB) to evaluate alternative "models" for how chemicals affect health, saying the mandate may force the agency to accept "junk science" and unjustifiably lower environmental standards.
The criticism could lead to a rift with EPA science chief George Gray over crafting agency comments on the draft guidance, which are due to be submitted to OMB later this year.
Gray said in a recent interview that considering alternative models submitted to the agency in support of risk assessments about chemicals' biological impact will help improve decisions by detailing the uncertainty surrounding those estimates.
Gray has made expanding uncertainty analysis in agency risk reviews a key element of his emerging agenda (Risk Policy Report, Jan. 31, p1).
One senior EPA science manager reportedly said at a Jan. 23 meeting of the agency's high-level Science Policy Council (SPC) that the bulletin is "dreadful" and reflects a "1980s view of risk assessment" that does not reflect currently accepted practices, according to sources who attended the meeting.
OMB's recently proposed risk guidance calls on agencies to consider all "plausible" models of the biochemical impacts of a contaminant at low levels of exposure. "Where a risk can be plausibly characterized by alternative models, the difference between the results of the alternative models is model uncertainty ...When model uncertainty is substantial, the central or expected estimate may be a weighted average of the results from alternative models," according to the draft guidance.
But some EPA scientists and observers say this provision will result in "model-shopping" among industry scientists. "This may open the door to everyone coming in with their own model." Simply because they can write a computer program to "compute results they prefer, doesn't mean that a model is valid," according to one science policy expert. "Accepting all 'plausible' models sets a pretty low bar. These models should require strong biological support before they are considered," an agency source says.
Agency critics also say the approach is flawed because the strength of the biological support should determine which models the agency accepts for inclusion in chemical risk reviews. They also say it is hard to find chemicals for which there is sufficient biological information to conduct fully informed modeling exercises mandated by OMB. "There are only a few dioxins, arsenics and mercury's out there with substantial enough databases," according to another agency source.
But administration officials are strongly defending their approach. During a Jan. 19 interview, then-OMB regulatory chief John Graham said "if there's no evidence for looking at an alternative model, then there's no value in considering it. Statistical backing is evidence, although if the support is biological as well as statistical, then that's stronger than statistical evidence alone."
And, during a Jan. 25 interview with Risk Policy Report, Gray agreed with EPA critics that biological evidence is important, but argued that multiple models are essential to understanding the uncertainty of chemicals' impacts. "I don't think anyone would suggest that statistical goodness-of-fit would determine [the quality of a model]. It's biological evidence that counts." But Gray said that examining different models with "biological plausibility" is a useful way to understand the uncertainty that surrounds chemical potency estimates.
Gray also said during his interview, "Models are just ways for us to implement different biological theories about what may be going on. What I think is important is when we don't know about different biological pathways, considering multiple models is probably a good idea."
Asked whether the call in the OMB bulletin would open the agency to alternatives to the precautionary "linear model" currently used for evaluating carcinogens, Gray said, "The linear model has certain biological principles it is based on and what you can do is look at how well the data fit that model. Other models have different biological data that support them. That process helps you think about the data and the model helps you quantify the uncertainty around that."
But industry analysts say EPA and academic scientists also manipulate statistics to buttress their arguments about the health impacts of environmental contaminants without strong biological support in agency risk-based drinking water and air standards. Industry observers say the linear model does not describe the impact of many potentially carcinogenic chemicals at low doses and point to the agency's 1998 decision to use the linear model for chloroform, which was rejected both by agency science advisers and in a March 2000 U.S. Court of Appeals for the District of Columbia Circuit decision as untenable (Risk Policy Report, April 18, 2000, p5).
But EPA risk assessors say the OMB may recreate the confusion sparked by open calls for a variety of risk models accepted by EPA programs in the 1970s and 1980s. At the time, industry and other outside experts could chose among the linear, "the one-hit, multi-hit, Weibull, log- probit and other models. Eventually EPA and other federal agencies realized that interested parties would 'model shop' and use the model that gave them the answers they preferred," according to the science policy observer.
This resulted in the agency formally adopting the linear model to provide some comparability across risk estimates in the 1986 cancer risk guidelines and a set of interagency principles the White House Office of Science and Technology Policy adopted.
But industry officials say risk modeling has grown more sophisticated since that time and companies know convincing and detailed data on a chemical's biochemical impacts are now required. "With the publication of the 2005 updated cancer guidelines, companies know that significant biological evidence is required to overcome protective assumptions like the linear model EPA uses," according to one industry source.
Agency staff and environmentalists say they are concerned key protections will be eroded if the bulletin is finalized in its current form.
"This [OMB guidance] is worrisome because models can be constructed to give any kind of low-dose answer you want, and the practical alternatives" are limited, an EPA source says. And an environmental scientist says, "This is an assault on protective assumptions with far-reaching consequences." A science policy observer agrees, saying, "Concerns about an assault on the linear model are accurate and well founded."
But conservative science policy analysts also argue that academic and EPA scientists can rely too heavily on statistically-backed but biologically deficient arguments to support their views.
In a Jan. 17 critique of EPA air pollution standards for particulate matter, American Enterprise Institute visiting fellow Joel Schwartz says, "There are literally millions of plausible statistical models relating to air pollution health outcomes, and no objective way to choose among them. Under these circumstances, researchers have a tendency to select those models that give the largest or most statistically significant effects."