new report shows widespread air quality data manipulation

Last fall, I wrote about Steven Andrews' report demonstrating data biasing in Beijing's air quality reporting.

The China Environment Forum has just published a second peer-reviewed report written by Mr. Andrews, this one detailing on a much larger scale the data manipulation present in air quality reporting across all of China. Mr. Andrews' overall conclusion is as follows:

Publicizing the API and where cities rank in terms of air quality keeps the public informed of air quality and potential health threats. However, misleading data presentation and revised laws have prevented the API system from accurately communicating air quality problems to the public.

Mr. Andrews' report focuses on several points, including:

1) SEPA's loosening of ambient air quality standards in 2000 artificially inflated the number of cities in compliance:

In 2006, the annual average NO2 concentration in Beijing was 66μg/m3 and in Guangzhou it was 67μg/m3 (BJEPB, 2007; GZEPB, 2007). Under the 1996 standards, Beijing and Guangzhou would have exceeded the annual average NO2 standard in 2006 by 65 percent and 67 percent, respectively. Under the revised standards, both were in compliance (SEPA, 2000).

(Note: link to old standards here; link to new standards here; discussion and comparison to international standards here.)

2) The assignment of 100 as the cut-off point for a "Blue Sky Day," coupled with rising pressure for cities to meet increasing numbers of annual Blue Sky Days, has encouraged the "bumping" of API data just above 100 to just below. Mr. Andrews writes:

Although the establishment of “Blue Sky” targets and well-publicized tallies of the number of days meeting the national standard has resulted in an easily understood metric for air quality, it strongly appears that pollution levels near this boundary are being manipulated in many major cities.

This was one of his core findings in his previous report on Beijing alone. In this report, Mr. Andrews expands the analysis to many more Chinese cities. The table below shows 30 cities which reported above 90% of all API values within the range 96-105 as 100 or below in a given year. (Statistically, one would expect around 50% of data points in this range to be on either side of 100.)3) The moving of monitoring stations within cities has artificially inflated air quality:

Although there has been a reported 10.8 percent decrease in Beijing’s annual average NO2 level between 1998 and 2006, the two stations in traffic areas have reported annual average NOx concentrations 100 percent higher than the non-traffic stations (BJEPB, 1998). This indicates that all the reported decrease in NO2 concentrations in Beijing from 1998-2006 may be due to the changing locations of monitoring stations.

4) Although not one of Mr. Andrews' key conclusions, one of his smaller but fascinating findings is that, apparently, there was a mistake in the English-language version of MEP's website regarding how to calculate API. This is something I never realized, but has apparently wreaked some havoc in international data analyses of air quality in China:

Although the calculation methodologies to go from API values to pollutant concentrations are straightforward, an error in the sample calculation on the MEP website has lead to misunderstandings of the true severity of pollution levels—inaccuracies that have been replicated in several leading reports on air pollution in China.

It seems that MEP has since removed the English explanation of API calculation, so I'm not sure what this error was; I'll keep digging and see if I can find out more.

Thoughts

Similar to Mr. Andrews' September 2008 report, this report is a scathing indictment and well-supported criticism of MEP's air quality data quality and transparency. It highlights a number of issues that MEP - as well as city and provincial-level EPBs - should ideally work quickly to resolve in order to regain international trust and credibility.