Final Water Report Author: Errors of Draft ‘Fixed’

Print Share on LinkedIn Comments More
Image courtesy of the USDA Natural Resources Conservation Service.

Image courtesy of the USDA Natural Resources Conservation Service.

Around the same time Ozzy Osborne stopped by San Antonio City Hall on Thursday, Roel Lopez also was there amid the commotion. The director of Texas A&M University’s Institute of Renewable Natural Resources hadn’t heard of the retired rock star’s pending visit. Lopez was on an entirely unrelated mission, and one attracting fewer cameras.

Lopez delivered an edited and revised edition of a much discussed water report commissioned by the City Council more than on year ago to inform the SA Tomorrow planning initiative and give city officials a second opinion on growth and water supply projections provided by SAWS.

Lopez called the final report an “unusual” document in a Friday morning interview, saying the final version of the Institute’s controversial Water Policy Analysis (download here) includes the misinterpreted data, and inaccurate mathematical assumptions found in a draft report submitted by the project’s original lead author, Calvin Finch. Now retired, Finch continues to defend the report he submitted.

That draft has gone through a rigorous scientific peer review. Five experts in water and water-project related fields, including hydrology, engineering, and resource management, took hard looks at the methodology and recommendations of the report. (Readers of the 187-page report will find their comments highlighted in yellow boxes.)

These anonymous researchers independently concluded – without consulting with the City, the report’s sponsor, or SAWS – that there were substantial flaws in how data was interpreted and presented, Lopez said. Lopez invited the Rivard Report and the Express-News to the institute’s Northside office for a briefing Friday to help explain how the final report differs from the draft. He will make a similar presentation to City Council on Nov. 12 during A Session, which starts at 9 a.m.

Director of IRNR Raul Lopez gives a statement to city staff. Photo by Scott Ball.

Director of IRNR Raul Lopez explains the peer review process for the water report to the Committee. Photo by Scott Ball.

Data interpretation from both Finch as well as the scientific review panel is included. The report, in a sense, sacrifices its potential for clarity in the interest of transparency. It’s heavy with asterisks and side notes. Probably not exactly the report Councilmember Ron Nirenberg (D8) had in mind when he initiated the report more than a year ago, but now that it’s done, City Council can vote on SAWS’ proposed rate increase and rate structure.

“What we essentially have now is two reports (in one) from highly credible scientists and water experts,” Nirenberg said Thursday. “Both are intended to be non-prescriptive … and will provide good context for this council and future councils when making important decisions.”

Lopez and a team of scientists have been working almost nonstop over the past few weeks to finalize the report, which analyzes 24 water planning issues and 12 water sources/projects for San Antonio, in time for proper community and City leadership review in the run up to the Nov. 19 vote.

Whether the contents of report will affect the outcome remains to be seen. The rate structure seems widely supported by most members of Council. They’ve been hesitant to comment either way about the rate increase, though Mayor Ivy Taylor did signal her support of both during a recent forum.

“(Original assumptions of the draft were) left intact for transparency and comparison,” Lopez said. Because a draft report was leaked in September, a priority of the institution became describing the process behind the revision and proposed views so that they couldn’t be accused of nefariously “deleting” a point of view, he said. “You decide. There it is, they’re both out there. … (The) panel’s uncertainty analysis was offered to both illustrate how a good scorecard is important, to illustrate the biases, and to offer an alternative (set) to look at.”

The most notable opposing views in the report are the risk/uncertainty assessments that gives the much-debated Vista Ridge water pipeline two different ratings. The draft gives the water source a “high risk” rating while the final gives it a “medium uncertainty” rating. In this case, Lopez noted, even the term “risk” was used inappropriately, when what was being described was the definition of uncertainty.

Draft report's risk analysis summary.

Draft report’s risk analysis summary included in the final report.

“Metrics used in the assessment did not measure factors correctly, evenly, overly favored or penalized projects, and in some cases were not uniformly applied; metrics included factors that were not relevant to the study; disparities between risk and uncertainty; and subjectivity in study design and conclusions,” states the review panel in the report.

Lopez explained that – among other mistakes – Finch used metrics such as “distance from San Antonio,” “treatment required,” costs of the project, and the financial status of Abengoa for its analysis of Vista Ridge project, but similar metrics were not used across the 11 other projects that the report analyzed, biasing the results of the risk scale.

Final report's uncertainty ratings.

Final report’s uncertainty ratings.

There also were discrepancies found by the panel in the grade assessment of water issues, he said.

“(Finch was) comparing apples to oranges,” Lopez said.

Had Finch’s draft not been leaked to the media, he said, the conclusions from it would not have made it into the final report. It makes for a complicated read, but easier than the draft that Finch submitted in October – which was only slightly different from the leaked July version.

Despite the additions of the review panel, the report was edited down from 235 to 187 pages by reorganizing and consolidating information and deleting redundancies.

“That had to be done because it was very difficult to read,” Lopez said. “It’s more than cosmetics. We didn’t just rearrange things and put it in a place that’s easier, it’s much deeper than that. …The values and assumptions were reviewed and validated and a lot of the changes are beyond what’s in the yellow box or in the appendix. There were analysis that were redone and incorporated because they were incorrect.”

Would the report have gone through a scientific peer review process if the draft had not been leaked? Lopez said he and Finch had considered it as they began work on the project, but that he wasn’t sure how it would have been handled.

 

Calvin Finch, former San Antonio Water System conservation director and Water Policy Study draft author. Photo by Edmond Ortiz.

For Finch, the answer is an emphatic “No.”

Calvin Finch, former San Antonio Water System conservation director, speaks in a panel discussion about the Vista Ridge project at UTSA. Photo by Edmond Ortiz.

Calvin Finch, former San Antonio Water System conservation director and Water Policy Study draft author, speaks in a panel discussion about the Vista Ridge project at UTSA in October. Photo by Edmond Ortiz.

“This is not the usual peer review,” he told the Rivard Report on Friday afternoon. “It’s (typically) completed for the benefit of the author and they get to include and respond to it as (they see fit).”

In this case, Finch and the three other authors on his team are no longer named on the report. Instead, authorship is attributed institutionally to the Institute of Renewable Natural Resources’ Texas Water Resources Institute.

He took issue with the anonymity of the review panel as well.

“If it’s published, their names should be available,” he said. “How can they can offer their opinions without being responsible for their opinions?”

Finch read the final report Friday morning and maintains that the methodologies and conclusions of his report – what the institute and others are calling a “draft” – are fair and accurate and the new report isn’t as true to his version as Lopez says it is. According to Finch, the institute has weakened some of the language used in his assessment of Vista Ridge and the issue of water loss in SAWS’ systems.

“They changed the text to be more sympathetic to some issues,” Finch said. “And this report contends that Vista Ridge is the (has the) same risk factor numbers as Edwards Aquifer … it will be interesting to hear them defend that.”

As for how he conducted his risk analysis, “not every project has the same characteristics, you don’t toss out one risk factor just because other projects don’t have it. The idea isn’t to have apples to apples, it’s to have parts of the project identified so that people can get the full picture.”

Describing the “full picture” is what the report was intended to do – give the community and City leadership the information needed to make policy decisions.

“The bottom line is our charge was to develop a tool that could be used by the city’s decision makers for doing just that.” Lopez said. “We’re in the business of facts.”

Finch said he will likely attend the Nov. 12 presentation to City Council, but he hasn’t decided if he’ll sign up to speak.

 

*Top image: Courtesy of the USDA Natural Resources Conservation Service.

Related Stories:

Highly Anticipated Water Report Released

Water Forum VI: Vista Ridge, Conservation, Rate Increase

Vista Ridge Deal Dominates UTSA Water Symposium

Vista Ridge Project To Dominate Two Water Forums

City Council to Delay Votes on SAWS Rate Increase, Annexation

Controversial Water Report to Get Peer Review

6 thoughts on “Final Water Report Author: Errors of Draft ‘Fixed’

  1. Councilman Nirenberg’s intentions have been thwarted by the circus surrounding the reports. There is obviously conflict between Lopez and Finch. There are now two differing views on the issue-which do you take to make a decision? I believe their is value in having another entity review the study-an entity without political ties that affect the outcome.

  2. There is growing concern now from all camps, on not only this project, but HOW the project has been pushed on an entire region.

    I certainly cannot speak for Councilman Nirenberg (or any other council member), since I live in the “unwilling donor” area (Bastrop). I believe he has raised concerns about the regional impact as well.

    Don’t do this to us, San Antonio. Folks can watch our 17 minute Vista Ridge video on our home page. We can back up all claims made there.

    We will also be delivering thousands of petitions from the “donor” communities on Tuesday at noon on the steps of City Hall.

    Linda Curtis, League of Independent Voters (www.StopWaterGrab.org)

    PS Thanks for your continued coverage. We read you in the hinterlands as this project has brought us together — one good thing created by this destructive project.

  3. You will notice that the list of potential water supply options is devoid of the Zero Net Water concept, under which water supply would be focused on building-scale rainwater harvesting — living largely on the water falling on you. An introduction to the concept is at http://waterblogue.com/2014/01/21/zero-net-water/. I’ve also made two presentations on the concept in San Antonio and would be happy to share them with anyone who is interested. This is a sustainable water concept, rather than a raid on “water colonies” like is envisioned in the Vista Ridge project, which from all reports is not a sustainable strategy, just puts the day of reckoning off a generation or so. Which is a “risk” no one seems to want to consider. They say they “need” that water to serve the population growth that will surely occur, but at the end of their 50-year planning period, won’t that population still be there, still needing a water supply for another 50 years and so on beyond that? And if you’ve depleted that supply to grow that population, then what? Will you have run society into a box canyon, simply have engaged in a slow-motion rearrangement of the deck chairs as the ship went down? But our institutions seem unwilling — or is it incapable? — of considering long-term sustainability as a major factor in such a decision. Rather it’s, the water is there, we should use it, even use it up, and then worry about what to do next. Essentially saying, “f*** the grandkids, we want what we want now!” If these Aggies wanted to actually render a credible report that considers ALL options fairly, well then they’ve failed, because they’ve left the sustainable water option out altogether.

  4. Does he mean the report was “fixed” like I had my cats and my dog “fixed”? Sure, I’ll accept that.

    Otherwise, he’s got some explaining to do. Read the final report, do a text search for the phrase “It is suggested”. Then search for the phrase “research suggests”.

    The repeat that exercise in the draft report.

    “It is suggested that the moon is made of green cheese. Research suggests, however, that it is actually ricotta.”

    Those two sentences carry the same level of rigor as the significantly revised section on the rate structure. No citations, no attributions. And the SRP inserted them or modified them to read that way. They are not carryovers from the draft.

    I’m no fan of Dr. Finch’s work, particularly in the Conservation section. No, you do not cite co-workers PowerPoint slides as a good source, particularly a debunked, dated PowerPoint slide (footnote #3, if you’re interested). But the SRP took inadequate work and made it worse.

  5. A primitive culture wanted to choose a new field to plant crops, so they asked the elders to choose. A group of elders chose to use sticks and stones to select the field. If they liked a field, they gave it a stick, since a stick symbolized growth. If they didn’t, they gave it a stone, since stones made it hard to plow. They totaled up the sticks and stones and told the villagers where they thought it was risky or not.

    A different group of elders said “No, you must use only edible fruit.” In their system, they used an apple to represent a beneficial condition, and did not give an apple if there was no beneficial condition. These elders totaled up the apples, and made a recommendation to the villagers. Certainty of good crops in fields with lots of apples was “high” where certainty with fewer apples was “low”.

    That’s about the level of difference in the uncertainty analysis vs. risk analysis. I was really hoping for something technical sounding from the SRP like “mean value” and “standard deviation”.

    Or maybe Finch just didn’t use apples, that’s why his risk analysis was flawed.

    Instead, for the SRP, it was “Water projects were then ranked (highest to lowest) with the upper 1/3 ranked projects assigned a label of “high”, middle 1/3 “medium”, and lower 1/3 being “low” for comparative purposes.” Choosing the top third without regard to the previous scoring criteria is literally “arbitrary” def: 1.based on random choice or personal whim, rather than any reason or system.

    For a simple illustrative analysis of how this is rating is so flawed, under the SRP, the Edwards Aquifer is “less certain” as a water project than Vista Ridge. Not only a little less certain, more than twice as uncertain. Let’s see, the current source of water for the city, the primary one, the one from which we get the VAST majority of our water today and projected in the future is TWICE as uncertain as one that doesn’t exist yet? Okay, I guess, if you say so…

    Please write your Council Person and tell them to stop wasting money on that risky Edwards Aquifer thingy. I want my more-certain Vista Ridge!

    P.S. How is the Twin Oaks ASR that gets all its water from the Edwards three times less uncertain than the Edwards? If you can’t get water out of the Edwards for some reason, what happens to the ASR? Does it magically fill up? Mr. SRP, can you ‘splain, please?

Leave a Reply

Your email address will not be published. Required fields are marked *