2/16/2012 SEWER COMMISSION Minutes

Home Browse Search

Massachusetts Estuary Project(MEP) Linked Watershed EnrbaynrentModel Peer Review The large number of parameters and their varied impact makes it difficult to provide a quantitative estimate of a single degree of uncertainty. Rather,for complicated and dynamic systems such as these, there will never be a single correct answer and any prediction,no matter how refined,will necessarily carry some degree of uncertainty. Uncertainty arises from the fact that model parameters are not known exactly(parametric uncertainty),that input data is not fully known(input uncertainty)and that processes are inherently more complex than can ever be completely captured by models(i.e.,intrinsic or structural uncertainty). In addition,natural processes are variable in tune and space, and there can be consistent changes in conditions over time,contributing to non-stationarity in statistical estimates of parameters, 'forcing conditions and internal adjustments of watershed and responding water bodies(Milly et al. 2008). The issue of non-stationarity reinforces the need for adaptive management, involving continued monitoring and modeling to test restoration expectations and appropriately refine management activities. As discussed above in Section 5.2.1,U.S. EPA(2009)recommends sensitivity analysis as the principal evaluation tool for characterizing the most and least important sources of uncertainty in environmental models. Perhaps the best place to get a sense for the degree of uncertainty in the MEP linked model is the MEP sensitivity study(Howes et al.2002). Table III-7 of the sensitivity study shows the extent to which varying selected model parameters changed the predicted nitrogen concentration in Great Pond,the test case for sensitivity evaluation. Relatively drastic variation(±50%)in the watershed loads fiom septic sources and lawn fertilizers produced comparatively modest changes in the predicted nitrogen concentrations (on the order of± 10 to 20%). The Panel's evaluation of the parameter values used in the loading calculations indicates that the loads are uncertain to a degree less than 50% and thus that the predicted nitrogen concentrations are uncertain to a degree less than 20%. One aspect of the MEP reports that we recommend for improvement is in providing some sense of the uncertainty in the study results. The Panel believes that it would assist those attempting to develop nitrogen control measures if they were provided a realistic sense of the degree to which model results are uncertain and therefore the degree to which flexibility should be included in any action plans. To this end, each of the MEP studies should include a sensitivity analysis of key components and links in the nitrogen loading and transport chain. The goal would be to drive these with realistic estimates of the uncertainty in model inputs and processes in order to determine how those components affect the bottom- line uncertainty in the nitrogen concentrations forecast for the receiving bays and estuaries.The sensitivity analysis included in the MEP sensitivity study(Howes et al. 2002)is a good model,but rather than a one-tune exercise, a similar approach should be included in the report for each estuary. Such a sensitivity study is needed for each estuary because sensitivity to the different model parameters varies substantially within the different estuary systems. The Panel also recommends that the list of parameters evaluated in the sensitivity study for each estuary be expanded from that in Howes et al. (2002)to also include the concentration of nitrogen at the ocean boundary since this is known to be a sensitive parameter. The Panel considered more rigorous and exhaustive alternatives to assessing model uncertainty: for example,Monte Carlo simulation to derive a probability distribution for predicted nitrogen concentrations. In the end,we concluded that such an effort was inconsistent with the scope and purpose of the MEP program. The completion of sensitivity analyses constitutes,we believe,a reasonable compromise between the need for information to understand the level of accuracy in the MEP results and the potentially considerable effort necessary to derive that information. Moreover,sensitivity analyses address the uncertainty in the overall MEP approach to predicting receiving-water nitrogen concentrations. It implicitly considers,for example,model-based linkages using the groundwater, watershed loading,hydrodynamic,salinity and water quality models. Despite the value of the recornrnended sensitivity analyses,they fall short of assessing the uncertainty in the response of receiving waters to changes in nitrogen concentrations-for example,the state of eelgrass. Unfortunately,the state of science is such that at the present time,prediction uncertainties for data-based December 30,2011 _ �� _ 29 —�