The Pac-12 presidents and chancellors’ unanimous decision last month to postpone all conference sports competition until at least Jan. 1 came after they saw a presentation that included erroneous statistics that overstated the prevalence of COVID-19 in several of the conference’s communities during the first week of August.
The most glaring incorrect metric listed the seven-day average positivity rate for tests in Los Angeles County as 19% — more than three times the 5.49% average listed by the L.A. County Department of Public Health.
The Pac-12 says it pulled the data from COVID Act Now, the L.A. County Department of Public Health and California Department of Public Health on Aug. 8. Those three sources all deny ever listing the inaccurate statistic and an L.A. County Department of Public Health official said the seven-day average positivity rate for tests in Los Angeles County has never been 19%.
A member of the conference’s medical advisory group says it still would have recommended delaying the season, even without that statistic.
“The data presented by the Pac-12 COVID-19 Medical Advisory Committee to the CEO Group was accurately sourced at the time,” Dr. Kim Harmon, associate team physician at the University of Washington, said Thursday in a statement via the Pac-12 to The Oregonian/OregonLive. “To the extent that there were subsequent updates to the reported data by COVID Act Now, state or local dashboards, they would not have changed the overarching recommendation of the Medical Advisory Committee to the CEO Group.”
A review of the statistics featured in the 13-slide presentation by the Pac-12′s Student-Athlete Health and Well-Being Initiative and COVID-19 Advisory Committee to the conference’s presidents and chancellors, obtained by The Oregonian/OregonLive via a public records request, reveals numerous discrepancies in data for the seven-day moving average of positive tests within the counties of Pac-12 schools. The seven-day average is one of the two primary metrics used by the conference’s medical experts regarding the prevalence level of the virus. The other was new infections per 100,000 residents within the county of each member school.
The conference used those metrics to determine how frequently athletes should be tested at each of its 12 schools, per the Harvard Global Health Institute’s recommendations. Based on the data, Arizona, Arizona State, UCLA, USC and Utah were recommended to test daily. Cal, Colorado, Washington and Washington State were recommended to test every other day and Oregon, Oregon State and Stanford were recommended to test every three days.
The Oregonian/OregonLive found all the data for the new infections per 100,000 residents aligned with data listed by COVID Act Now.
But the newsroom found significant discrepancies between what was presented to presidents for at least four of 11 counties of the conference’s schools and published data from COVID Act Now, county and state health departments, though some were explainable. That led to corresponding overstated recommendations for testing frequency at UCLA, USC, Washington, Washington State, Colorado and possibly Arizona.
The origins for the erroneous data in Los Angeles is in dispute and the origins for the seven-day positivity rates for Boulder County Colorado and Pima County Arizona are unclear as COVID Act Now doesn’t list that data for those counties, their county health department websites don’t list that metric and the Colorado Department of Public Health and Environment and Arizona Health Department of Health Services data are not in line with the numbers on the chart in the presentation. Data discrepancies for King and Whitman counties in Washington are explainable due to data recording changes in that state in early August.
The Pac-12 cited the lack of availability of daily testing among its primary reasons for postponing sports on Aug. 11 and Pac-12 commissioner Larry Scott reiterated that point as recently as Wednesday night during an interview on ESPN. The Pac-12 has since secured daily rapid antigen testing capability thanks to its deal with Quidel Corporation that was announced Sept. 3 and testing machines are expected to be in place at each of the conference’s athletic departments by the end of the month.
Scott, via a Pac-12 spokesman, did not respond to a request for comment regarding the erroneous data.
On Aug. 11, University of Oregon President Michael Schill, who serves as chair of the Pac-12 CEO Group, said that the group would “constantly be reassessing the data,” that they would “be looking at facts, not just opinions” and were unanimous in deciding to postpone sports “because we all recognize this was the morally correct thing to do and we did it.”
“I can’t speak for the other presidents, but my vote to postpone fall competition was based on the advice of medical experts at the time, my understanding of public health regulations in Oregon and California, and a lack of sufficient testing resources,” Schill said in a statement Thursday to The Oregonian/OregonLive. “It was not based (on) one single piece of data.”
UCLA and USC, both in Los Angeles County, were listed in the presentation to the Pac-12′s presidents and chancellors as having 22.3 daily new cases per 100,000 population and a 19% seven-day average positivity rate as of Aug. 8, the date listed in the presentation and confirmed by a Pac-12 spokesman as when the data was compiled. COVID Act Now lists the daily new cases per 100,000 population in L.A. as 22.3 for Aug. 7 and 22.9 for Aug. 8, with a 5.5% seven-day average positive test rate for both those days, revised down from 6.5% as of Sept. 8.
The COVID-19 surveillance dashboard on the L.A. County Department of Public Health’s website lists a total of 13,048 positive tests out of 237,711 tests (5.49%) between Aug. 1-7.
The 22.3 daily new cases per 100,000 population and a 19% positivity rate correlated to recommended daily testing at USC and UCLA, but a 5.5% positive rate would have resulted in recommended testing every other day.
According to L.A. County Department of Public Health’s data, the seven-day average positivity rate has never reached 19% during the nearly six months of data amid the pandemic for both total tests or persons tested.
Officials from COVID Act Now, the L.A. County Department of Public Health and California Department of Public Health say they were not the sources of the erroneous statistic.
“As far as I can see, Covid Act Now never showed a seven-day average test positivity rate for Los Angeles County, CA of 19%,” said Michael Lehenbauer, a software engineer at COVID Act Now who reviewed the outlet’s data for The Oregonian/OregonLive. “In August the highest rate I see was less than 10%.”
An official from the L.A. County Department of Public Health told The Oregonian/OregonLive the county’s data “has not listed a seven-day average positivity rate on Aug. 7 or Aug. 8 as 19%” and that “the seven-day average positivity rate in Los Angeles County has never been 19%.”
The California Department of Public Health said it “has not been able to identify any CDPH report of Los Angeles County test positivity in the range of 19% during Aug. 7 or 8.”
It’s unclear whether USC President Carol Folt or UCLA Chancellor Gene Block were aware of the apparent statistical errors during the Aug. 11 presentation or raised any objections to it. There was no immediate response to requests for comment from Folt and Block via spokespeople at USC and UCLA, respectively.
The Pac-12′s numbers were incorrect in the presentation for the University of Washington’s King County (5.1%) and Washington State’s Whitman County (6.2%) because of a change in how the state reports cases. The Washington State Department of Health adjusted its methodology for calculating the positivity rate across the state on Aug. 12 to use the total number of tests completed rather than the total number of unique individuals ever tested, which “does not fully reflect the actual testing volume or the current test positive rate, since only the first negative result for each person is included,” according to a release from the department.
As a result, the state’s COVID-19 testing dashboards did not update and showed preliminary data through Aug. 10 using the old methodology. An official with the Washington State Department of Health told The Oregonian/OregonLive the incorrect data could have been listed by the state in early August due to a data system failure, which required data to be reentered manually.
King County is listed as having a 5.1% seven-day average of positive tests in the Pac-12′s presentation, but that figure is 2.9% on King County’s COVID-19 daily outbreak summary data dashboard as well as the Washington State Department of Health’s COVID-19 data dashboard for testing in King County. The incorrect 5.1% positivity rate, combined with the 6.8 new daily cases per 100,000 population metric as of Aug. 8 resulted in recommended testing every other day rather than every three days at 2.9%.
“Based on our current data for Aug. 7, the positivity rate would have penciled out to 4.3% under the previously used method, and 2.9% under the new one,” an official from Public Health Seattle — King County told The Oregonian/OregonLive. “I’m not sure of the source of the 5.1% figure, but it’s important to note that a number of widely used data aggregators, such as COVID Act Now and The New York Times, have generally not matched very well up with the authoritative numbers collected by the county and state DOH.
“It’s difficult to re-create what data and method were available at earlier points in time — the number for a given date can fluctuate as preliminary or incomplete data are fleshed out. That means we cannot definitively say the number for that date was never reported as greater than or equal to 5.1%, but it seems likely that we were not the source of that particular data point.”
COVID Act Now does not list the seven-day average positivity rate for each county in Washington, but it did list 6.2% for Aug. 3 at the time the Pac-12 pulled the data.
“Washington actually changed their test reporting in early August and for 2-3 weeks did not provide updated data,” Lehenbauer, the COVID Act Now software engineer said. “And so for those weeks (including likely on 8/7) the CAN website showed a chart something like (chart below), with the 8/3 rate of 6.2% being the latest available value.”
For Arizona State and Washington State, the Pac-12′s presentation notes the seven-day average positivity rate numbers are state-level information because county data was unavailable. The 15.4% positivity rate listed for ASU (Maricopa County) aligns with Arizona’s overall state positivity rate on Aug. 7 (15.5%), per COVID Act Now, but the 6.2% statewide rate for WSU (Whitman County) was since revised down to 4.8% by COVID Act Now and 4.7% on the Washington State Department of Health’s data dashboard.
Like UW, WSU was recommended to have testing every other day. But had the newer 4.8% positivity rate figure been used, it would have resulted in recommended testing every three days.
Testing data in Whitman County is limited due to the area’s low population and skews to extremes. For example, as of Wednesday there have been 1,091 cases in Whitman County with a positivity rate from Sept. 8-13 of 19%, according to the county’s public health website.
The Pac-12′s presentation lists the seven-day average positivity rates for Colorado’s Boulder County and Arizona’s Pima County as 6.6% and 10.7%, respectively, as of Aug. 8. However, it’s unclear where those figures originated because COVID Act Now does not list seven-day average positivity rates and county and state health department data does not align with those figures.
Boulder County’s five-day rolling average of positive PCR tests during the first week of August hovers around 2%, according to the county’s website data, while the statewide seven-day moving average on Aug. 7 was 4%, according to the Colorado Dept. of Public Health and Environment website. Either of those levels, rather than the listed 6.6%, would lead to recommended testing every three days for Colorado rather than ever other day.
Pima County’s website refers to positivity statistics from the Arizona Department of Health Services, which has a data dashboard for lab testing listing 563 positive tests out of 11,080 tests reported electronically (4.77%) in Pima County for the week ending Aug. 9.
The seven-day average positivity rates for Cal (6%) and Utah (10%) listed in the Pac-12 presentation are higher than the 5.5% found on the Alameda County COVID-19 testing dashboard and 9.6% listed on the Salt Lake County Health Department website from Aug. 1-7, but those discrepancies are not statistically significant, can be attributed to data lags and revisions and wouldn’t have resulted in the recommended frequency of testing every other day at Cal or daily at Utah.
All of the data listed for Arizona State, Oregon, Oregon State and Stanford are still in line with what’s listed by COVID Act Now for each school’s respective county and the Santa Clara County COVID-19 testing dashboard.
The Pac-12′s decision to postpone fall sports will cost more than $600 million, based on at least $50 million in projected losses at each school. Oregon athletics is budgeting for a loss of $56 million to $81 million, depending whether there is an eight-game spring football season with or without fans, no football season or no Pac-12 sports at all in 2020-21.
The decision to postpone sports has already led to furloughs, layoffs and pay cuts at Pac-12 athletic departments, the conference office and the Pac-12 Networks, along with player transfers, opt-outs and NFL draft declarations. The move was based primarily on the prevalence of COVID-19 in the conference’s communities, the need for greater and faster testing capacity and concerns about myocarditis, a rare inflammation of the heart muscle that can be brought on by the virus, as well as other viral infections.
However, that decision could be revisited as soon as Friday, when the Pac-12′s presidents and chancellors will hold their next meeting and will go over schedules for football and basketball season, among other topics.
On Wednesday, California Gov. Gavin Newsom and Oregon Gov. Kate Brown agreed to lift restrictions in their respective states to allow Pac-12 teams to return to play, pending approval of written plans and protocols from the conference. Also Wednesday, USC athletic director Mike Bohn and UCLA athletic director Martin Jarmond also got the L.A. County Department of Public Health to permit those schools to begin contact sports practices, according to The (San Jose) Mercury News.
Earlier Thursday, Benton County gave Oregon State the go-ahead to begin contact practices and games.