Data Sources

There is a variety of public and private sources of environmental data, including these federal data sources by water body type:

• Lakes and reservoirs
  • National Eutrophication Survey (NES)
  • National Surface Water Survey (NSWS)
  • U.S. Army Corps of Engineers
  • U.S. Department of the Interior, Bureau of Reclamation
  • National surveys such as the EPA Eastern Lakes Survey
  • Clean Lakes Program
  • Regional limnological studies
• Rivers and streams
  • U.S. Geological Survey (USGS) Hydrologic Benchmark Network (HBN) and National Stream Quality Accounting Network (NASQAN)
  • USGS National Water-Quality Assessment Program (NAWQA)
  • USGS Water, Energy, and Biogeochemical Budgets (WEBB)
  • U.S. Department of Agriculture Agricultural Research Service (ARS)
  • U.S. Forest Service
• Estuaries and coastal waters
  • Ocean Data Evaluation System (ODES)
  • Chesapeake Bay Program
  • National Estuarine Programs
  • NOAA
  • National Estuarine Research Reserve System

Data Acquisition

Data acquisition involves accessing and documenting the variable data of interest. Data can be retrieved from a variety of data storage formats, including Excel, CSV, and DB files, that might be downloaded from website or FTP sites or otherwise provided. Because data acquisition should be a repeatable and transparent process, it is important that you document each aspect of the process. Try to avoid manual transcription because of the potential for errors to be introduced into the data set. The important aspects to document include the data source (e.g., URL, agency providing the data, version), the download or access date, and the download procedure. For replicability and QA, you should maintain a copy of the raw, unaltered downloaded data. The raw data also can be important in troubleshooting processing errors introduced during analysis and in maintaining version control. Dynamic data sets can be updated by data owners at any time. Preserve field names and transfer them to a data dictionary that describes each variable from the raw data. Maintain records of collection and laboratory analytical methods, QC plans in place, location and time of collections, and so forth. Document the sources of original data and any alterations made (e.g., file merges, new fields, deleted observations). You can facilitate the assembly process by using a file folder system and naming convention that allow users to easily track and identify data sources and versions. Set up read-only folders for original versions of data files. Always work on copies, not the originals. Budget time to clean up and review final data sets before starting the analysis. Every data processing effort should include a formal process for reviewing data sources to identify and document issues with the data that might cause inclusion or exclusion of data points or data sets.

Data Review

Evaluating older historical data sets is a common problem because data quality often is unknown. Knowledge of data quality also is problematic for long-term data repositories such as STORET and long-term state databases, where objectives, methods, and investigators might have changed many times over the years. The most reliable data tend to be those collected by a single agency using the same protocol for a limited number of years. Selecting data from particular agencies with known consistent sampling and analytical methods will reduce variability caused by unknown quality problems. Requesting data review for QA from the collecting agency will reduce uncertainty about data quality. Supporting documentation also should be examined to determine the consistency of sampling and analysis protocols. The amount of documentation associated with a particular source can vary widely. Documentation of the source, including metadata documented in project reports, validation reports, and any database information, should be maintained along with the data. Research into the origin and documentation of a data source might be necessary to properly evaluate the data source. Potential sources for this documentation might include the website for the agency or group that collected the data, published reports, research articles, and personal communication with the original researcher or monitoring group staff. Consider this series of general questions when evaluating the quality of any data source and the applicability of the data to the current project:

1. What was the original purpose for which the data were collected?
2. What kind of data are they, and when and how were they collected?
3. What cleaning and/or recording procedures have been applied to the data?

When evaluating a water quality data source, also consider the following questions, which are more specific to water quality data:

1. Were the data generated under an approved QA project plan or other documented sampling procedure?
2. If multiple data sets are being combined, were the data sets generated using comparable sampling or analytical methods?
3. Were the analytical methods (detection limits [DLs]) sensitive enough to meet project needs?
4. Is the sampling method indicated (e.g., grab, composite, calculated)?
5. Was the sampling effort representative of the waterbodies of interest in a random way, or could bias have been introduced by targeted sampling?
6. Are the data qualified? Are sampling and laboratory qualification codes or comments included? Are the qualification codes defined?
7. Is sufficient metadata available about variables like sampling station location, date, time, depth, rainfall, or other confounding variables?

Be sure to document the rationale for including or excluding data from an analysis to ensure the transparency and defensibility of the decision-making process. Factors to consider when evaluating a data set include the accuracy, precision, representativeness, and comparability of the data set. Depending on the data use, different quality evaluation criteria might be needed to establish the best balance between data quality and usefulness of the data. An indication of the quality of the data used in the analysis should be included in project documentation and applied across all sources. Also, examine the data, looking for records that indicate that sample integrity was maintained, approved sample collection and analysis methods were used, established QC measures were in place for the laboratory conducting analyses, and instruments were calibrated and verified for performance. Documentation of evaluation criteria for each parameter is especially important when using data across multiple sources. Data might have been recorded using analytical methods under separate synonymous names or incorrectly entered when data were first added to the database. Review of recorded data and analytical methods recorded by knowledgeable personnel is necessary to correct these problems. To compare data collected under different sampling programs or by different agencies, sampling protocols, and analytical methods must demonstrate comparable data. The most efficient way to produce comparable data is to use sampling designs and analytical methods that are widely used and accepted such as Standard Methods for the Examination of Water and Wastewater (Rice et al. 2012) and EPA methods manuals. As stated earlier, if the impact of combining data is unknown or cannot be determined, then it might be appropriate to perform separate analyses on each source of data. The results of the separate analyses can then inform criteria development in a post-hoc analysis. To the extent practical, use multimetric indices (MMIs) of biological condition that have been regionally calibrated based on broader data sets of known quality. Using MMIs previously established by local, state, or regional agencies requires that the same or similar methods be used for field sampling and laboratory processing for other streams and sites being evaluated relative to best management practices or other issues. This approach of using accepted protocols and calibrated metrics and indices, coupled with sufficient and appropriate QC checks, improves defensibility of assessment results and increases confidence in natural resource management decision-making.

Data Cleanup

Before any further data processing occurs, consider initial data reduction to save time and effort. For successful data reduction, you must have a clear idea of the analysis to be performed and a clear definition of the sample unit to be used in the analysis. Data reduction may be an iterative process; a conservative approach of including more data may be more efficient than having to add data back into your analysis at later stages. Include these steps in reducing data:

1. Select the long-term time period for analysis.
2. Select an index period.
3. Select relevant variables of interest.
4. Identify the quality of analytical methods.
5. Identify the quality of the data recorded.
6. Estimate values for analysis (i.e., mean, median, minimum, maximum) based on the reduction selected

Reducing water quality data by screening nontarget data involves interpreting certain fields to select the data of interest to the analysis. Examples include:

• • Sample media (e.g., water, soil, groundwater).
  • Sample type (e.g., routine samples versus duplicate or QC samples such as spike samples, field replicates, or laboratory replicates). Checking routine values against duplicate values can be an effective QC check, but verify that duplicate values are not included in the data set used for analysis.
  • Sampling type or location (e.g., effluent, ambient, stormwater, baseflow, pipes, finished water, or process water, are sometimes available). Sampling focused on effluent outfalls or pristine waters could bias a sampling effort. Sampling type or location can be an important indicator of sampling bias or spatial bias inherent in the data set resulting from opportunistic sampling rather than random sampling.
  • Waterbody type (e.g., stream/river, lake/reservoir, estuary, ocean, wetland, canal, stormwater). You can use this field to further refine sampling data to the data of interest.

Data cleaning includes data checking and error detection, data validation, and error correction. This section describes typical efforts to standardize a data set created by combining multiple data sets and to screen a data set in preparation of analysis.

Standardization

Frequently, parameter names, comment fields, and result values are inconsistent across data sources or even within a single source. To combine data while retaining the original data, it is good practice to create user-specified fields to represent common parameters, standardized comments, and comparable values. Creating these fields enables you to correct errors and perform transformation while retaining the original data in separate fields as well as the capability to go back to the original data if necessary. Maintaining documentation of data transformation and error correction is especially important when the processes are being performed by anyone other than the primary data collector. Creating user-specified fields provides an opportunity to convert units to like units, standardize parameter names, interpret comment fields, convert nondetect values, or institute other data transformations. For instance, a user-specified data qualifier field might be used to flag or exclude blank samples or samples with nonnumeric characters in the value field. Figure 3 provides an example of how user-specified fields might be used to convert field names and units and to interpret comment fields. Another important application of user-specified fields is creating a column that documents the original source and the row identifier of the original source when merging data, so that if systemic issues are found in a source, they can be resolved and cleaned more effectively.

Figure 3. Example of user-specified fields.

Descriptive fields such as temporal indicators (e.g., date, year, time), sample depth, latitude/longitude, and units often are included in varying formats. Common fields and transformations that you should consider are described:

• Temporal fields: Ensures that all date and time fields are in the same format (e.g., DD-MM-YYYY, YYYY-MM-DD). It is recommended that you use military time and account for time zones. It might be helpful to have one field with “Date” and separate fields for “Year,” “Month,” “Day,” and “Time.” If a measurement of diurnal fluctuations is not needed in a parameter, averaging data by day might remove some inconsistencies resulting from data without time information or with slightly different times resulting from different labs processing the data or from data entry error. Searching for dates outside the range of interest or outside reasonable date or time values (e.g., month 12, day 31, year < 1900, time 24) can be a helpful screening tool. Having a sampling date is a reasonable minimum requirement for data.
• Depth fields: Depth should generally be a numeric field. Sometimes a surface or bottom indicator is included as well as a numeric depth field (e.g., S, B). It can be helpful, especially in lakes and estuaries, to add a separate text depth column for profile data that indicate surface, depth, or bottom measurements for some parameters (e.g., DO). Depth units should be standardized to a consistent format (e.g., feet or meters). Having a sampling depth (numeric field or indicator) is a reasonable minimum requirement for lake and estuary data.
• Latitude/Longitude fields: Ensures that latitude and longitude are reported in a consistent format. Latitude and longitude units are most often reported in degrees, minutes, and seconds (DMS) (e.g., 39°59’56.055”N, 102°3’5.452”W), decimal degrees (DD) (e.g., 39.999012, -102.052062), or sometimes UTM coordinates (e.g., 17T 630084 4833438). These examples all are roughly from the same point on the border of Kansas, Nebraska, and Colorado. To convert from DMS to DD, use the following formula:
   
  (degrees) + (minutes/60) + (seconds/3600) = decimal degrees

  If values are missing, consider digitizing from GIS or geocoding from an address if one is provided. One of the most frequent errors is omitting the negative sign (-) in decimal degree coordinates from the southern or eastern hemispheres. If all the records are from North America, all the longitude values should include a negative sign. Consider spatial accuracy: With today’s standards, be wary of decimal degree data with less than six digits of precision accuracy or seconds reported with less than two digits of precision (although for larger waterbodies, less precision might be acceptable). A typical minimum data requirement for station-level data is that the station must have a latitude and longitude measurement as well as the reported datum. Look for extreme values: Latitude should never be outside the range of 90 to -90 degrees; longitude, 180 to -180.

• Units: Units should be standardized by parameter and among parameters. Check for systematic incorrect reporting of units when converting all values for a parameter to one unit of measurement. Note that laboratories often report results on a weight-per-weight basis, such as parts per million (ppm) or parts per billion (ppb). In water samples, 1 ppm is essentially equivalent to 1 milligram per liter (mg/L) and 1 ppb is equivalent to 1 microgram per liter (µg/L) unless concentrations are very high (> 7,000 mg/L). In addition, µg/L and milligrams per cubic meter (mg/m³) can be considered identical in water samples in most cases. Outliers for a parameter might be an indication that data are reported in varying units. While it is a common practice not to include units as a data field in a database (i.e., the units are inherently known to the data generator or are documented in metadata), it is important to ensure that all units are identified.

Water Quality Characteristic Nomenclature

The naming conventions for water quality monitoring parameters often are inconsistent across different sources or even within the same source. Each separate analytical method should yield a unique variable (e.g., five ways of measuring TP results in five unique variables). Data generated using different analytical methods should not be combined in data analyses because methods differ in accuracy, precision, and DLs. Data analyses should concentrate on a single analytical method for each parameter of interest. Data generated from one method might be too limited, making it important to select the most frequently used analytical methods in the database. Data generated using the same analytical methods might not always be obvious because of synonymous names or analytical methods. Consistency in taxonomic conventions and indicator measurements also is important for biological variables and MMI comparisons. Review of recorded data and analytical methods by knowledgeable personnel is important to ensure that no problems occur with data sets developed from a particular database. One way to compare and combine data without applying a consistent naming convention is to add another user-specified name column to combine parameters under a unique name. Metadata from the data source, other fields within the data, and best professional judgment can be used to inform the consolidation of characteristic names into fewer unique parameters. Consult a chemist or the data owner if you are uncertain about what is being measured. If you still have questions about what parameter actually was being monitored, consider contacting the original monitoring group for clarification. Document the process of cross-walking the original parameter names to standardized parameter names. Using a lookup table to translate the fields often is an easy way to consolidate names and document the process in a transparent way. Keep in mind that categorizing data can lead to a loss of data and a potential reduction in quality. Storing data with the original resolution and exporting data as categorized for a particular use is good practice. Many monitoring groups also have parameter codes associated with each characteristic name. These codes provide a more detailed description of the parameter such as sampling method, analytical method, typical units, and sampling media. For example, USGS and the U.S. Environmental Protection Agency (EPA) have parameter codes associated with each parameter name (USGS NWIS EXIT; EPA Legacy STORET EXIT). In addition, EPA maintains a Substance Registry System (SRS) that provides a common basis for the identification of chemicals and biological organisms. The SRS pulls information from several registries, including Chemical Abstracts Service (CAS) number, taxonomic serial number (TSN), and EPA internal tracking number (EPA ID number). A CAS number or CAS registry number (CASRN) is a unique number assigned by CAS to chemical substances, and additional information about each substance is maintained along with the CAS number. A TSN is a unique numeric identifier used to identify biological species. Routine biological monitoring programs in the United States typically collect data on benthic macroinvertebrates, fish, and algae; some programs include macrophytes, seagrasses, corals, or reptiles and amphibians. Biological data often are reported with varying levels of processing—from raw species observations and operational taxonomic unit abundances to algal biomass (e.g., AFDM or chlorophyll a) and biological metrics (e.g., algal, invertebrate, fish). Metadata are vital to documenting the collection method, sample identification process, index calculation method, and other attributes of the data. All biological sample data in a table should be based on the same collection method, processing, subsampling, and taxonomy. To avoid confusion, use a completely separate table for data collected using different methods.

Biological Data Nomenclature

Naming conventions for species identified in biological assessment data often follow different formats. This section presents an overview of the species identification process and naming conventions. Depending on the organism, several options for unique identification might be most applicable.

• The Integrated Taxonomic Information System (ITIS) EXIT uses the TSN to provide a list of unique biological species, with more information about each species available.
• American Fisheries Society Fish Identification guidance: Common and Scientific Names of Fishes of the United States, Canada, and Mexico (Nelson et al. 2004).
• The Academy of Natural Sciences of Drexel University has a series of online databases and collections Exit for species identification.
• Taxon names: EPA’s SRS and USGS maintain acceptable lists of taxon names that can be used as a basis for unique identification:
  • EPA SRS
  • Catalogue of Life Exit
  • Global Biodiversity Information Facility (GBIF) Name Parser Exit

A unique taxon name should be identified, but often identification is made down to only the family or genus, not all the way to the species level. Increasingly, genetic signature data such as DNA barcodes might be included. Depending on the use of the data, identification to the genus level or higher might be acceptable for metric calculations or other uses. Genus-level taxonomy is the principal hierarchical level used by most routine biological monitoring programs for benthic macroinvertebrates, although occasionally the family-level is used. Be aware that, in some cases, unresolved taxonomic issues might be present such as data with no parent family and daughter genus data in the same data set. Taxonomic classifications are opinions, and differing opinions by different taxonomic experts can lead to the same organism being classified differently. Databases should have the flexibility to include alternative names to capture disagreement in taxonomic opinion as well as different accepted names (Chapman 2005a). Some databases store taxon names in one field, which can lead to more errors. It is recommended that taxon fields be divided into separate fields (e.g., genus, species, infraspecific name, certainty) to improve QA and organizational capabilities. Plants especially have several levels below species that are frequently used—subspecies, variety, subvariety, forma, subforma, cultivars, and hybrids. Misspellings are common in scientific names. Finding spelling errors using one of the sources mentioned above should be straightforward at the family or genus level but might be difficult at the species level. Similarity algorithms can be used to compare and determine the degree of similarity between scientific names to identify misspelling errors. Table 1 provides some typical examples of misspellings for just one species (Karenia brevis, or red tide) (e.g., Dinophycae, Karania, brev); lack of consistency in data entry (e.g., brevis?, sp. brevis, aff. brevis); and changes in naming convention (e.g., K. brevis was formerly called G. breve or P. brevis).

Table 1. Examples of differences and errors in taxon naming convention

Screening

Data screening is an important processing step in preparing a data set for analysis. Screening can include identifying and accounting for duplicate or missing data, especially large or small values (outliers) and data re-marked with qualifier text or codes for duality control concerns as well as some other considerations specific to biological data.

Duplication and Missing Data

Duplication frequently occurs in water quality data sets. It might be the result of obtaining the same data from different sources or simply data entry error. Duplication can occur within a single data source or among different data sources. Merging two data sets sometimes creates new inconsistencies and duplication. Because duplication can skew and bias data, instances should be flagged and screened from the analysis as much as possible. Some samples might resemble duplicate entries but actually have different depths, times, or other distinguishing features. If the only fields that are different are descriptive fields, such as comment fields, that might indicate duplication. The organization identifier and sampling name can be good indicators that duplication has occurred, but also look for duplicate values in the data over the same time frame. For example, several identical numerical values on the same day might indicate duplicate data. Sorting the date chronologically and looking for duplicate sample results is one way to begin to identify duplication. MS Excel has features you can use to identify and highlight duplicate values in a field; when the data are sorted chronologically, Excel can identify potential duplicates. Duplicate records should be flagged using a user-specified field but generally not deleted. Simply deleting duplicate data, rather than flagging and excluding the data, creates a potential for error and data loss that is difficult to identify. Blank fields should be checked to determine whether the value was intentionally left blank or is the result of an error. It is good practice to document the reason for a blank value in a separate field (such as “not applicable”, “not relevant”, or “unknown”), but reasons are not always well documented. A user-specified data qualifier field might be used to flag or exclude blank samples. The data should be 100 percent error-checked for transposition errors and typos. Screen the data for inappropriate nonnumeric characters. The reality of any monitoring program is that samples will be missed, equipment will fail or be overwhelmed, droughts and floods will occur, and sample analysis limitations will be exposed, resulting in missing values (USEPA 2016c, p. 7-20). Data missing randomly generally has a minimal effect on estimated relationships, but data missing in a predictable pattern can bias them and should be considered further. For example, data often can be missing as the result of systematic issues encountered during data collection such as instrument DLs. You can statistically test for a particular variable to determine whether it is missing at random by splitting the data set into one subset in which the variable has a value and a second subset in which a value for the variable is missing. Then, you can examine other measured variables for significant differences across the two subsets. If data are missing at random, then samples with missing data can simply be excluded from the data set. If data are not missing at random, then analysts should consider the effect of the missing data. When a data set includes a large number of missing values, it might be necessary to consult a statistician about the most appropriate method either to develop a complete data set or to apply appropriate modeling approaches that can use incomplete data.

Outliers

Outliers are especially large or small sample values relative to the distribution of the rest of the data. Outliers are of concern because they might misrepresent the distribution and can have a strong bias on average values and nutrient-biological response relationships. Potential outliers can be initially identified using exploratory data analysis methods such as summary statistics, histograms, normal-quantile plots, and scatterplots . Plotting the data in a time series plot by date or stratifying by season or flow conditions can reveal patterns in extreme values that might be related to an actual event or measurement or recording error. Several statistical tests to confirm that values are true outliers also are available that include Dixon’s Extreme Value Test, the Discordance Test, Rosner’s Test, and Walsh’s Test (refer to USEPA 2006 for guidance on performing these tests). Several software programs for identifying outliers of geographic coordinates are available that include spOUTLIER and Diva-GIS (Chapman 2005b). Ultimately, the decision to correct an outlier, remove the value from the data set, or use the data point should be based on best professional judgment and have a scientific basis supported by visual analysis and/or a statistical test. Both failure to identify true outliers and removing false outliers will result in distortion of the statistical analyses of the data. Removal of outliers should focus on cases in which the cause can be identified as measurement or equipment error, laboratory error, or recording error (USEPA 2016c). Environmental data often contain legitimate extreme values, so correct or remove outliers with caution. Initial screening of data often is a necessary step before outlier analysis is performed. Sometimes data entry software requires that analysts add an entry, so dummy values such as ‘0’, ’-99’, ‘Null’, or ‘NoData’ are entered. Analysts should evaluate and address zero or negative values for most variables unless there is a legitimate reason for those variables to have a zero value (e.g., temperature). A table of minimum and maximum values can be used to identify and potentially screen out negative or extreme values as a first-step screening tool. Note that the screening values differ by parameter; depending on the sampling, different ranges of values should be expected for different parameters. Environmental data often contain legitimate extreme values, so correct or remove outliers with caution.

Data Qualifiers

Data qualifiers—codes or notations that provide additional qualifying information about the measurement value being reported—should be included in a water quality data set. A data set that does not include data qualifiers could be a red flag about the quality of the data. Contact the monitoring group for clarification if possible. Qualifiers can take a variety of forms depending on the data set, including a system of qualifier codes or flags, entries in comment fields (sampling and/or laboratory), and fields for type of sample (e.g., duplicate, spiked, replicate, routine). Analysts will need to interpret data qualifiers to be able to use them to screen out data with quality concerns. It is not recommended that samples be deleted at this stage, but flagged for exclusion from the analysis after all applicable data qualifiers have been interpreted. If samples are excluded from analysis, reasons for exclusion should be documented. Creating an internal, user-specified qualifier field can be useful for documenting choices and combining multiple data qualifier fields into one field for decision-making. While there are several commonly used sets of data qualifiers, there is neither a universal set that applies to all types of analyses nor a universal specification for their use. Table 2 provides some examples of data qualifiers. Data qualifiers can provide descriptive information about a sample without calling data quality into question (Group 1), provide an indication of how censored data were reported (Group 2), or call into question the quality of the data (Group 3). Samples with qualifier codes in Group 2 will need to be further processed (refer to Censored Data for further guidance). Qualifiers such as those in Group 3 could possibly lead to sample screening. Check the metadata or other documentation associated with a data set for the key and description of each qualifier code specific to the data set.

Table 2. Examples of data qualifiers (codes and descriptions)

Sources: Florida DEP IWR; USGS NWIS EXIT; USGS Upper Midwest Environmental Sciences Center EXIT; ANERR—Alabama NERR data.

Biological Data-Specific Considerations

Some screening considerations are unique to biological or habitat data. All calculated metrics should be correct for the appropriate bioregion or biological class; scored values should be provided. Seasonal community changes such as differences in taxa reproduction, growth, and emergence phenologies can affect the comparability of calculated metrics. Observations should be plotted by month to determine whether significant seasonal bias is present in the sampling data. If the data are simply metrics, they can be reported in the same table, using different codes for the same metric collected two different ways (e.g., EPTRichness_HesterDendy, EPTRichness_Sweep). The data should be evaluated for collection method compatibility and excluded if methods are not found to be compatible. Alternatively, the data is evaluated independently and considered as multiple lines of evidence for the purposes of nutrient criteria development. Similar to water quality data, biological assessment data often include a series of data qualifier codes or comment fields that further describe the data. These fields might include life stage codes, organism condition codes, missed level reason codes, or other general notes. The degree of certainty of the taxonomic identification should be included. Many of the QC techniques described in previous sections also apply to biological data (e.g., duplicate screening, outliers, source documentation, spatial data analyses). An example of a QC technique with biological data is described in the Virginia Ecological Limits of Hydrologic Alteration (ELOHA) report (USEPA 2012b). Multiple sources of fish and benthos data were compiled for analysis for this project, and one QC check was to plot sample abundance across data sources. Figure 4 shows sample abundance by data source for benthos. It was determined that the USGS NAWQA benthos results were reported as densities by taxon rather than sample counts as with the other sources. The NAWQA density values included multipliers to account for subsampling and area sampled, but the multipliers were not reported along with the data set. The study analysts contacted USGS and received the multipliers, which allowed the analysts to convert the data back to counts per sample to make the data comparable to the other data sources.

Figure 4. Benthos sample abundance by data source from the VA ELOHA project with the NAWQA data source (left) and without the NAWQA data source (right) (USEPA 2012a).

Censored Data

Data reported as not detected or below a DL should be used but accounted for statistically (Bolks et al. 2014).

Censored data are data that are measured to be less than or greater than a DL or quantitation limit (QL). DLs, such as a method detection limit (MDL) or instrument detection limit, are the lowest concentration that can be distinguished from a zero value (i.e., the parameter is present, but it cannot be quantified). QLs, such as the practical quantitation limit (PQL), represent the lowest concentration that can be not only detected, but also quantified with a degree of precision. Laboratories often set and report a PQL associated with each testing method performed at that laboratory; thus PQLs vary from laboratory to laboratory. MDLs also vary based on the method of collection, storage, and analysis of a particular parameter. Each combination of parameter, analytical method, and laboratory should have its own DL and QL reported along with each censored data point. When data are described as nondetect or left-censored (ND, U, <), the actual value of the data lies somewhere between zero and the DL. Interval-censored values lie between the DL and QL. Right-censored (>) data report values greater than a high-end QL, such as too numerous to count in bacteriological analyses, flow exceeding flow gauge limits during floods, or Secchi depth measurements of visible on bottom. Table 3 provides descriptions of the types of censored data.

Table 3. Common descriptions or terms associated with different types of censored data

Source: Bolks et al. 2014. Censored data are of particular concern with water quality data because of varying treatment and reporting of nondetected values. It is common for nondetects to be reported with DLs omitted, DL types undefined, zero entered as the reported value, a value less than the DL reported, or “less than a value” (e.g., < 0.05) entered as the reported value. Varying approaches to handling censored data will need to be standardized in any analysis using multiple data sets. There are a variety of methods for processing censored data, and approaches vary depending on the data and purpose of the analysis. Documentation of how censored data are processed is a crucial part of any analysis. Extensive research in water resources as well as in other fields of science such as survival analysis (e.g., how long does a cancer patient live after treatment) has been conducted into numerous techniques of accounting for censored data. One deficiency over the last 20 years has been the lack of readily available tools for widespread use, making many of the tools out of reach for general use. Efforts to improve the availability of these tools is continuing. With improved tool access, past methods for accommodating censored observations can be avoided. The most notable method is simple substitution, which involves the replacement of censored observations with zero, ½DL, or DL. Although simple substitution is commonly used (and even recommended) in some state and federal government reports as well as in some refereed journal articles, there is no real theoretical justification for the procedure. Substitution can perform poorly compared to other more statistically robust procedures, especially when censored data represent a high proportion of the entire data set. More egregiously, some reports have simply deleted observations less than the DL. In the past, some researchers have recommended simply reporting the actual measured concentrations even if they are below the DL. This approach has not become popular as laboratories are reluctant to implement such a practice, although Porter et al. (1988) suggested that an estimate of the observation error could be reported to better qualify the measurement. While simple substitution might be convenient for initial exploratory analyses using spreadsheet tools, more robust procedures are available and are recommended. If substitution of numerical values for nondetects is believed to have a significant effect on the results of an analysis, a sensitivity analysis can be performed. This method can be performed by substituting all zeros or the full DL or QL for nondetects, instead of the current substitution method, and repeating the results to determine whether the analysis is sensitive to the selected substitution method. In addition to substitution or replacement, other more statistically robust imputation procedures are available for processing censored data. These techniques are especially applicable in cases in which more than 10–15 percent of the data are nondetects and a sensitivity analysis shows the data to be sensitive to the substitution method. Cohen’s method provides estimates of the mean and standard deviation that account for nondetected data, but it does not provide replacement values for each nondetect value (USEPA 2006). In addition, the statistical software package R offers several methods for estimating summary statistics from censored data such as maximum likelihood estimation, robust regression on order statistics (robust ROS), and the Kaplan-Meier method. A test of proportions might be appropriate in cases in which more than 50 percent of the data are below the DL but at least 10 percent are detected. Rather than reporting mean values, a percentile of the data greater than the DL could be reported (e.g., hypothesis test of the 75th percentile or simply 95 percent of the TN data are below a threshold).