AQS Memos - Questions and Answers Related to the AMP-255 Report

Questions and Answers Related to the AMP-255 Report

In 2009 the National Air Data Group (NADG) revised the AMP-255 report based on QA regulations that were revised in 2006 and more recently with the change in the Pb NAAQS in 2008. Monitoring organizations had commented that the older version of the AMP-255 had too many tables and was cumbersome to review. NADG revised to AMP-255 into fewer tables and also added summary report in PDF format.

So far, comments on the new report are positive but there are a few features that have caused some concern in the new version that will be discussed below. A few have already been rectified. The issues will be discussed in a question and answer format.

Q1: The semi-annual flow rate audits (40 CFR Part 58 App A Section 3.2.4) for particulate matter require 2 audits per year. I’ve done at least 2; sometimes I do more. Why does the AMP-255 report indicate that I have not met the criteria?

Answer- The regulation states “Every 6 months audit the flow rate of the PM₁₀ PM_10-2.5 and PM_2.5 particulate analyzers.”

For the “Criteria Met” column to be a “Y”, two criteria must be met:

1. There must be enough audits for the site based on the sites’ begin and end date. If the site operates > 7 months then 2 are required. If the site operates <3 months an audit is not required. If a site operates > 3months but < 7 months, 1 is required.

2. The audits must be conducted within a 5 to 7 month period. Since the audits are required every 6 months they should not be conducted too close to each other or too far apart. EPA set the criteria in a AMP-255 that audits conducted within 5 to 7 months would be a reasonable achievement of the requirements. In the case above, even though two audits were accomplished, the audits were either less than 5 months apart or greater than 7 months apart. In Example 2 below (bounded in red) even though 3 audits were conducted (more than the requirement) the audits conducted between Q2/Q3 and Q3/Q4 were less than 5 months apart and the audits conducted between Q2 and Q4 were greater than 7 months apart.

Q2: Is the failure to meet the 5-7 month interval criteria cause for data invalidation or will EPA not certify data not meeting this criteria?

Answer- Data should not be invalidated and EPA will certify the data as long as two audits are implemented (based on begin/end date) for sites that are, at a minimum, audited in 2 different quarters. It is possible to perform audits every quarter and still fail the 5-7 month rule. The AMP-255 will be modified by 2012 to have the criteria considered met if there are flow rate audits in 4 quarters, even if the 5-7 month rule is not met.

Q3: What’s going on with the gaseous pollutant annual performance evaluations (PEs)? I submitted data in audit concentration level fields 1, 2 and 3 and the report did not have them in the same levels on the AMP-255 Report?

Answer- The report will “bin” the audit concentration data, based on the “actual” concentration, in the PE audit levels provided in the PE table in 40 CFR Part 58 Appendix A.

Q4: Not only did I submit my data in three different levels but the report only shows data in two levels. What happened?

Answer- Based on binning process discussed above, if the actual concentration of two audits, considered as separate audit levels by the monitoring organization, fall into the same audit level in the PE table in 40 CFR Part 58 Appendix A, data from the two audits concentrations will be combined. Example 3 illustrates this (site bounded in red). In fact, in this case two separate audits were conducted in Q1 and Q3 but not at 3 separate levels.

Q5: Based on issues related to questions 3 and 4 why is the criteria considered not met?

Answer: The AMP-255 looks for the reporting of three consecutive levels of audits. So if you have three separate levels but they are not consecutive (based on PE audit level concentrations in the Table in Appendix A) or the audit concentrations reported can not be “binned” into three consecutive levels (as illustrated in example above), the AMP-255 will indicate a “N” in the criteria met column. Notice that the “OBS/Q” columns for Q1 and Q3 that the data does indicate that 3 audits were reported even though only two audit levels of data are indicated. This lets the evaluator know that 3 audit levels were attempted.

On November 10, 2010 OAQPS expanded the five performance evaluation audits to 10 levels^.

This guidance still requires PEs at three levels but we no longer require consecutive levels. The AMP-255 will be revised by the start of calendar year 2012 so the current AMP-255 in 2011 will continue to indicate an “N” in the criteria met column if it does not “see” 3 consecutive levels.

Q6: Will not having data in three separate levels be cause for invalidation or will EPA not certify data not meeting this criteria?

Answer: Data should not be invalidated and EPA will certify the data as long as one audit is implemented in the year and there is a 3 in the “OBS/Q” column for one quarter. However, monitoring organizations should strive to audit 3 audit concentration levels as required in Appendix A. The new 10 levels will make it easier to find 3 distinct audit levels.

Q7: Appendix A also indicates that audits levels should be chosen that match the concentrations of our routine data. Also we are changing the operational ranges of our NCore monitors to lower ranges. In the case of CO we may be operating lower than the third audit level in the PE Table in Appendix A. What do we do when we try to pick good audit levels that don’t match the PE Table in Appendix A?

Answer: The new 10 level audit ranges should help in the selection of the appropriate ranges for the sites in a network. If the new audit levels still do not help, then it may be appropriate to report three audits from two levels (as is the case with the example above).

Q8: Flow rate verifications are not required to be reported. Why are we seeing results especially when we are not reporting them?

Answer: Flow rate verifications are only required reporting for PM10 automated instruments.

For PM10 automated instruments, AMP255 will always show if the reporting requirements have been met, and the results. Flow rate verifications for these monitors are mandatory reporting.

For the other parameters (TSP, PM_{10 manual,} PM_10-2.5 and PM_2.5 particulate samplers/analyzers), reporting is not mandatory:

• if you do not report data from these monitors, then AMP255 will not show any results.

• if you do report flow rate verifications for some or all of these monitors, the AMP255 will include results only for the monitors for which you reported flow rate verification data. The other monitors without flow rate verification data will not be included in the AMP255 report results.

Some monitoring organizations are reporting the flow rate verifications. EPA encourages this and we wanted to provide the assessments for these verifications in the AMP-255 report. NADG has revised the AMP-255 report and will only report values for PQAOs that are reporting flow rate verifications and will only report the data for those sites that provided at least one flow rate verification. So for any PQAO that does not report any flow rate verifications, you will not see a report. For a PQAO that has, for example, 30 PM₁₀ sites, but report flow rate verification for only 10 sites, the AMP-255 will report flow rate verification results for those 10 sites. Since the verifications (with the exception of PM₁₀ automated instruments) are not required, OAQPS will not review non-required flow rate verification data during certification.

Q9: My ozone season begins May 1; to be ready for the start of the season, I bring my network up piecemeal throughout the month of April, and often conduct my first biweekly precision check before my ozone season starts. I also run a few of my ozone monitors year round. How can I make AMP-255 handle this properly?

Answer: There may be a few solutions. The AMP-255 was developed to follow the CFR requirements as closely as possible. The report was constructed to evaluate, at a minimum, that the completeness requirements for the one point precision checks for ozone were being met during the required ozone season. So the default option for the AMP-255 is to use the ozone season. Example 4 shows two reports for the same PQAO. The top half of the report is generated using the ozone season and the “# Obs” column represents the number of 1-point checks implemented during the ozone season.

If you regularly monitor for ozone all year, despite having an ozone season less than 12 months long, then you may override the default setting on the AMP-255 by clicking on the “Report Options” tab and changing the “Restrict to Monitoring Season” from the default of “yes” to “no”. This will then change the begin and end date to the full year (based on the latest sample period in the monitor record for that site) and the AMP-255 report will show the “# Obs” for all the 1point QC checks conducted in that year. However, the AMP-255 runs all the sites in the PQAO the same way unless you separate reports for those running all year from those running in just the required ozone season. An example of this difference is “Site Example A” and “Site Example B”. It appears that the Site in Example A (blue lines) was operating all year. In the ozone season it was required to perform 10 precision checks and it performed 23 and for the year it was to perform 26 checks and it performed 52. In the case of Site B (red lines) it does not appear that the site was operating the whole year since it had the same number of precision checks for the ozone season (22) as it had for the year.

Fortunately, this PQAO was doing more checks than required. If it was performing them as required (10), the Site B completeness for the year would appear to be 38% (10/26*100).

There is one additional solution to this and that is the use of “SAMPLE PERIOD” parameter in the monitor record. Each year, the monitoring organization can enter in the correct begin and end date for the “SAMPLE PERIOD” for each ozone monitor. Then, one would select the full year reporting function and the AMP-255 will calculate the appropriate precision statistics for each monitor. In order to ensure that the AMP-255 performs correctly, the monitoring organization would have to enter the correct “SAMPLE PERIOD” for the monitor each year.

Q10: I started a new ozone monitor Dec. 5, after the end of my ozone season. Why does AMP255 show a negative number of biweekly single point checks required?

Answer: For the two sites in Example 4 with begin dates of Dec 05 this is an error. It has been reported and repaired.

Q11: In the upper right hand corner of the report there is an area that says “APP A? : Yes” What does this mean?

Answer: This area (circled in red in Example 4) identifies whether the sites in the report are required to follow the Appendix A criteria and is derived on the assignment of "MONITOR TYPE" for the monitor. In the past, EPA has questioned monitoring organizations about data quality results being reported only to find out that the monitors/sites in question were not being implemented or used for NAAQS purposes and not required to meet the Appendix A criteria. On 3/31/09, OAQPS posted a technical guidance document titled “Use of "Non-Regulatory"

Monitor Type Code in AQS to Identify Criteria Pollutant Monitors That Do Not, or Are Not Intending to Provide Data for Regulatory Purposes”  The document provides guidance on using the “non-regulatory” "MONITOR TYPE" for those monitors not needing to meet the Appendix A criteria and therefore not being used in NAAQS decisions. The AMP-255 does not combined data from regulatory and non regulatory monitors. PQAOs that have monitors with a “nonregulatory” "MONITOR TYPE" will have a second table produced and in the upper right hand corner one would see “APP A? : No” to identify this difference.