\documentclass[11pt]{article}
\usepackage{geometry, times}                % See geometry.pdf to learn the layout options. There are lots.
\geometry{letterpaper}                   % ... or a4paper or a5paper or ... 
%\geometry{landscape}                % Activate for for rotated page geometry
\usepackage[parfill]{parskip}    % Activate to begin paragraphs with an empty line rather than an indent
\usepackage{fullpage, graphicx, amssymb, epstopdf, hyperref}
\hypersetup{
  colorlinks,
  linkcolor=blue,
  urlcolor=blue
}
\renewcommand{\UrlBreaks}{\do\&\do\=\do\?\do\-\do\/\do\.}
\usepackage{float}
\usepackage{verbatim}
\usepackage{pdfpages}

\DeclareGraphicsRule{.tif}{png}{.png}{`convert #1 `dirname #1`/`basename #1 .tif`.png}

\SweaveOpts{keep.source=TRUE}
%\VignetteIndexEntry{Using seawaveQ}
\usepackage[utf8]{inputenc}

\title{Vignette for seawaveQ---An R Package Providing a Model and Utilities for Analyzing Trends in Chemical Concentrations in Streams with a Seasonal Wave (seawave) and Adjustment for Streamflow (Q) and Other Ancillary Variables}
\date{\today}                                % Activate to display a given date or no date

\begin{document}
\maketitle
\tableofcontents

\section{Introduction}

This R package, \textbf{seawaveQ}, is designed for fitting a parametric regression model for assessing variability and trends in pesticide concentration in streams and was developed by Vecchia and others (2008), and subsequently refined and referred to as the ``SEAWAVE-Q'' model in several trend analyses (Ryberg and others, 2010; Sullivan and others, 2009; Vecchia and others, 2009). In these publications, ``SEAWAVE-Q'' stands for seasonal wave (SEAWAVE) with adjustment for streamflow (Q). The model was developed to ``handle a number of difficulties often found in pesticide data, such as strong seasonality in response to use patterns, high numbers of concentrations below laboratory reporting levels (RLs), complex relations between streamflow and concentration, and intermittent or changing sampling frequencies (both inter-annually and intra-annually)'' (Vecchia and others, 2008). This R package provides a standardized methodology for fitting the model and makes the trend analysis method widely available for use by others. In addition, several enhancements to the model have been included as well as utility functions for working with chemical concentration data. The enhancements and utilities include procedures for preparing and summarizing input data; flexibility to include other explanatory variables besides streamflow; graphical methods for assessing model fit; and plotting routines that may be used for pesticide and other chemical concentration data. A flow chart showing how the various function in the package work together is shown in figure 1 of the U.S. Geological Survey Open-File Report documenting this package (Ryberg and York, 2020). 

The statistical methodology for the model is described in Vecchia and others (2008) and in the U.S. Geological Survey Open-File Report documenting this package (Ryberg and York, 2020). Users new to this model should read both of those documents before applying the model to their own data. An important part of the model and the output shown below is the seasonal wave. The seasonal wave is a periodic (period of 1 year) solution to a differential equation (Vecchia and others, 2008) that has a pulse input function, a seasonal shift that determines the time at which the seasonal wave reaches its maximum, and a model half-life (see appendix 3. Visualizations of the Seasonal Wave; Ryberg and York, 2020).

\section{Input Data}

The model needs two types of input data. The first is the water-quality sample data including dates, the concentration data, and qualification codes, indicating which values are censored (less than a laboratory reporting level). The second type of data is the continuous ancillary data used in the model, such as streamflow anomalies (Ryberg and Vecchia, 2012).  These ancillary data also are used to produce a continuous estimate of pesticide concentration.  Examples of the necessary format of these two datasets are provided and documented in the package. The following code shows how to access the example data.
\vspace{5 mm}

<<results=verbatim,echo=TRUE>>=
options(width = 65)
# load necessary packages, assuming they have already been installed on the system
library(waterData)
library(survival)
library(seawaveQ)
# load example data that comes with the package
data(swData)
# show first few rows of water-quality data for Missouri River at Omaha, Nebr.
head(qwMoRivOmaha)
# get a description of the data including definitions of the columns
# by viewing the help documentation
?qwMoRivOmaha
@

Optionally, a function has been provided to plot concentration data. This function produces a scatter plot that indicates the censored, less than, values. The functions is \textit{cenScatPlot} and an example of its use follows. Previous versions of the software included an option to generate box plots properly representing regression on order statistics using an implementation of a regression on order statistics designed for multiply-censored analytical-chemistry data (Helsel, 2005). This relied on the R package \textbf{NADA} (Lee, 2017); however, long-term maintenance of this package has been an issue (oral commun., R Core Team, 2020) and the functionality was removed. Users may investigate the availability of \textbf{NADA} and use it separately to produce box plots.
\vspace{5 mm}

\begin{figure}[H]
\centering
%\setkeys{Gin}{width=1.45\textwidth}
<<fig=TRUE,width=7,height=4.5,echo=TRUE>>=
# scatter plot showing quantified, estimated, and censored  values
cenScatPlot(qwMoRivOmaha, pname = "04035")
@
\end{figure}

\begin{figure}[H]
\centering
%\setkeys{Gin}{width=1.45\textwidth}
<<fig=TRUE,width=7,height=4.5,echo=TRUE>>=
# scatter plot with many additional plotting arguments
# these options provide a plot closer to the plotting standards
# of the U.S. Geological Survey, however, these plots may not 
# meet all U.S. Geological Survey publication requirements
par(las = 1, tcl = 0.5)
cenScatPlot(qwMoRivOmaha, pname = "04035",
            site = "06610000 Missouri River at Omaha, Nebraska",
            ylabel = "Simazine concentration, in micrograms per liter",
            legcex = 0.7, qwcols = c("R", "P"), ylim = c(0,0.1), yaxs = "i", 
            cex.lab = 0.9, cex.axis = 0.9, xaxs = "i", xaxt = "n",
            xlim = c(as.Date("1996-01-01"), as.Date("2004-01-01")))
axdates <- c("1996-01-01", "1998-01-01", "2000-01-01", "2002-01-01", 
             "2004-01-01")
axis(1, as.Date(axdates), labels = c("1996", "1998", "2000", "2002", "2004"), 
     cex.axis = 0.9)
@
\end{figure}

The second data set needed is the one containing the continuous ancillary data for building the model that describes pesticide concentrations.  
\vspace{5 mm}

<<results=verbatim,echo=TRUE>>=
data(swData)
# show last few rows of water-quality data for Missouri River at Omaha, Nebr.
tail(cqwMoRivOmaha)
# get a description of the data including definitions of the columns
# by viewing the help documentation
?cqwMoRivOmaha
@
\vspace{5 mm}

In this case, the continuous ancillary data includes daily streamflow (dflow) and daily sediment concentration (dsed), as well as the 30-day and 1-day streamflow (flowa30 and flowa1) and sediment (seda30 and seda1) anomalies. [The anomalies were calculated using the \textbf{waterData} package for R (Ryberg and Vecchia, 2012).]

In order to build a model using one or more of these ancillary variables as explanatory variables for pesticide concentration, the continuous ancillary variables need to be associated with the water-quality samples. The function \textit{combineData} will combine water-quality sample data and continuous (daily) ancillary variables and drop unnecessary columns. One needs to specify the water-quality sample data, the continuous ancillary data, and the columns representing the station identifier (staid), the sample date, the qualification code (<, E) columns, and the concentration columns as shown in the following code.  See Oblinger Childress and others (1999) for an explanation of the qualification codes used by the U.S. Geological Survey.
\vspace{5 mm}

<<results=verbatim,echo=TRUE>>=
data(swData)
MoRivOmaha <- combineData(qwdat = qwMoRivOmaha, cqwdat = cqwMoRivOmaha,
qwcols = c("staid", "dates", "R", "P"))
# view combined data set
head(MoRivOmaha)
@

\section{Fitting the SEAWAVE-Q Model}

One can now fit the model using the data explored and combined in the previous code examples. The following code fits three different  models (with differing continuous ancillary variables) for two pesticides in the data set. The pesticides are 04035, simazine; 04037, prometon; and 04041, cyanazine. See the help documentation for further information about the function arguments shown and additional arguments.
\vspace{5 mm}

<<results=verbatim,echo=TRUE>>=
data(swData)

# associate continuous water-quality data with each sample
# combineData does this for you
modMoRivOmaha <- combineData(qwdat=qwMoRivOmaha, cqwdat=cqwMoRivOmaha)

# then fit model(s)
myfitLinearTrend <- fitswavecav(cdat = modMoRivOmaha, cavdat = cqwMoRivOmaha, 
                                tanm = "myfitLinearTrend", 
                                pnames = c("04035", "04037", "04041"), 
                                yrstart = 1995, yrend = 2003, tndbeg = 1995, 
                                tndend = 2003, iwcav = c("flowa30", "flowa1"), 
                                dcol = "dates", qwcols = c("R", "P"), mclass = 1,
                                plotfile = FALSE)
@

\section{Model Output}
The model fitting process finds the best pulse input function and model half-life for the concentration data and uses survival regression to fit a regression model. Three types of output are provided: 1) a list, the first element being a data frame with information about the model and its parameters, the second element being the survival regression summary, the third element the observed concentration (censored and uncensored), the fourth element the concentrations predicted by the model, the fifth element the summary statistics for the predicted concentrations; and the sixth element provides trend results expressed as trends in percent or trends in concentration units;  2) text files showing a summary of the survival regression results, like the second element of the list, but with additional measures of model quality and information about the R session; and 3) a pdf file of plots showing the model, trend, and diagnostic plots. The data frame results for the three models for simazine and cyanazine are shown below.
\vspace{5 mm}

<<results=verbatim,echo=TRUE>>=
# get the first element of the list for each model/constituent combination
# the data frame with information about each model/constituent combination
myfitLinearTrend[[1]]

# get the second element of the list for each model/constituent combination
# the survival regression summary for each model/constituent combination
myfitLinearTrend[[2]]

# get the first few lines of the third element of the list
head(myfitLinearTrend[[3]])

# get the first few lines of the fourth element of the list
head(myfitLinearTrend[[4]])

# get the summary of predicted concentrations
myfitLinearTrend[[5]]

# get summary of trend results
myfitLinearTrend[[6]]
@
\vspace{5 mm}
The first element of the list, the data frame, contains information about each model including the pesticide analyzed; the model class (a class 1 model incorporates a linear trend, a class 2 model incorporates restricted cubic splines); the choice of model or pulse input function, an integer 1 through 14; the model half-life in months, an integer, 1 to 4 months; the decimal season of maximum concentration; the scale factor from the survreg.object; the log-likelihood for the model; the coefficient for the model intercept; the coefficient for the seasonal wave; the coefficient for the trend component of the model; 0 or more values representing coefficients for the continuous ancillary variables; the standard error for the intercept; the standard error for the seasonal wave; the standard error for the trend; and 0 or more columns representing standard errors for the continuous ancillary variables.

The second element of the list is provided so that users could extract the attributes of the survival regression summary programmatically (rather than viewing them in the text file) and create their own summaries or plots of the results.  The third, fourth, fifth, and sixth elements of the list are provided for user-generated plots and further user analysis (see R help for more details).

<<results=verbatim,echo=TRUE>>=

attributes(myfitLinearTrend[[2]][[1]])
myfitLinearTrend[[2]][[1]]$n
myfitLinearTrend[[2]][[1]]$table
@
% \vspace{5 mm}
% 
% The text file for the first of the pesticides modeled above is inserted here as an example. Users may run the model fitting code themselves and view the resulting text files for all three pesticide models. The results for all three are too long to include in this vignette.
% 
% \vspace{5 mm}

% \verbatiminput{myfitLinearTrend_survregCall.txt}%

\vspace{5 mm}

The plots are written to a pdf file if the argument plotfiles in the fitswavecav funcntion is set to TRUE. The user must explicitly set the argument to TRUE to give the user permission to write the files.
\vspace{5 mm}

The plotting position used for representing censored values in the model plots (produced by the internal function \textit{seawaveQPlots} that is further described in the package help documentation) is an important consideration for interpreting model fit. Plotting values obtained by using the censoring limit, or something smaller such as one-half of the censoring limit, produce plots that are difficult to interpret if there are a large number of censored values. Therefore, to make the plots more representative of diagnostic plots used for standard (non-censored) regression, a method for substituting randomized residuals in place of censored residuals was used. If a log-transformed concentration is censored at a particular limit, $logC < L$, then the residual for that concentration is censored as well, $logC - fitted(logC) < L - fitted(logC) = rescen)$. In that case, a randomized residual was generated from a conditional normal distribution, as shown in the following R code:
\begin{verbatim}
	resran  <-  scl * qnorm(runif(1) * pnorm(rescen / scl))
\end{verbatim}
where \textit{scl} is the scale parameter from the survival regression model, \textit{pnorm} is the R function for computing cumulative normal probabilities, \textit{runif} is the R function for generating a random variable from the uniform distribution, and \textit{qnorm} is the R function for computing quantiles of the normal distribution. Under the assumption that the model residuals are uncorrelated, normally distributed random variables with mean zero and standard deviation \textit{scl}, the randomized residuals generated in this manner are an unbiased sample of the true (but unknown) residuals for the censored data. This is an application of the probability integral transform (Mood and others, 1974) to generate random variables from continuous distributions. The plotting position using a censored concentration is $fitted(logC) + resran$. Note that each time a new model fit is performed, a new set of randomized residuals is generated and thus the plotting positions for censored values can change.

\section{Restricted Cubic Splines Option}
The original SEAWAVE-Q model assumes a linear trend over a given trend period. Linearity is a common, useful assumption in trend analysis, but becomes problematic as trend periods become longer because of the potential for abrupt, or gradual, non-linear changes. Examples include cancellation of some or all of the uses of a particular pesticide, phase-outs of a pesticide, or changes in usage. As additional data are collected, longer trend periods may be calculated and a strict linear trend may be inappropriate for many site-pesticide combinations. Restricted cubic splines (transformations of the time variable) were incorporated into the SEAWAVE-Q model to allow flexibility over time. The range of values of the time variable is split up, with “knots” defining the end of one segment and the start of the next. Separate curves are then fit to each segment (Harrell, 2010, 2016). Overall, the splines are defined so that the resulting fitted curve is smooth and continuous, passing through each knot. Other studies have shown that 3 to 5 knots typically are sufficient for most models (Korn and Graubard, 1999). For small sample sizes (< 30, a sample size insufficient for a pesticide trend model), 3 knots can be used; 4 knots are sufficient for most models and represent a compromise between overfitting and model flexibility (Harrell, 2010; Croxford, 2016). If the sample size is large (>100, as is often the case for pesticide trend modeling) or the relation changes quickly over the trend period, 5 to 7 knots could be used, although 5 are usually sufficient (Stone, 1986; Croxford, 2016).

\section{Fitting the SEAWAVE-Q Model with Restricted Cubic Splines}

The following code fits a model with restricted cubic splines instead of a linear trend for one pesticide in the dataset. The pesticide is 04035, simazine. See the help documentation for further information about the function arguments shown and additional arguments.
\vspace{5 mm}

<<results=verbatim,echo=TRUE>>=
data(swData)

# associate continuous water-quality data with each sample
# combineData does this for you
modMoRivOmaha <- combineData(qwdat = qwMoRivOmaha, cqwdat = cqwMoRivOmaha)

# then fit model
myfitRCS <- fitswavecav(cdat = modMoRivOmaha, cavdat = cqwMoRivOmaha, 
                        tanm = "myfitRCS", pnames = c("04035"), yrstart = 1995, 
                        yrend = 2003, tndbeg = 1995, tndend = 2003, 
                        iwcav = c("flowa30", "flowa1"), dcol = "dates", 
                        qwcols = c("R", "P"), mclass = 2, numk = 4, 
                        plotfile = FALSE)
@

\section{Model Output}

<<results=verbatim,echo=TRUE>>=
# get the first element of the list for each model/constituent combination
# the data frame with information about each model/constituent combination
myfitRCS[[1]]

# get the second element of the list for each model/constituent combination
# the survival regression summary for each model/constituent combination
myfitRCS[[2]]

# get the first few lines of the third element of the list
head(myfitRCS[[3]])

# get the first few lines of the fourth element of the list
head(myfitRCS[[4]])

# get the summary of predicted concentrations
myfitRCS[[5]]

# get summary of trend results
myfitRCS[[6]]
@


<<results=verbatim,echo=TRUE>>=
attributes(myfitRCS[[2]][[1]])
myfitRCS[[2]][[1]]$n
myfitRCS[[2]][[1]]$table
@
\vspace{5 mm}

% The text output file is inserted here as an example.
% 
% \vspace{5 mm}
% 
% % \verbatiminput{myfitRCS_survregCall.txt}%
% 
% \vspace{5 mm}

\section{Bootstrapping for Significance with Restricted Cubic Splines}
The trend component of a model using restricted cubic splines is more complex than with previous implementations of the model. Because the model now as curvilinear components, the trend can be expressed a change in concentration from the beginning of the curvilinear trend line to the end of the curvilinear trend line (see Ryberg and York, 2020, for more details). In some cases, where the addition of restricted cubic splines has little change to the model, this trend may be the same as that of the linear model; however, sometimes the trend can be dramatically different because restricted cubic splines can respond to abrupt or gradual changes in use or regulation that cannot be modeled linearly. In addition, the determination of statistical significance is different for the model with restricted cubic splines. In order to determine if the trends in pesticide concentration calculated using restricted cubic splines are substantially different from a pattern produced by chance alone, we added functionality to perform a bootstrap analysis. Bootstrapping creates a distribution by resampling the dataset. Normal bootstrapping eliminates all trends and produces trend-free data, while block bootstrapping preservers any autocorrelation detected in the sample if the sample is serially dependent (\"On\"oz and Bayazit, 2012). To ensure that block bootstrapping preserves autocorrelation, blocks are often overlapped and are wrapped around from the end to the start of the series (Canty, 2002). The bootstrap analysis in seawaveQ uses a variation of block bootstrapping and an attained significance level similar to the method used in Hirsch and Ryberg (2012). However, Hirsch and Ryberg used a random length block with block length geometrically distributed with a mean length of 20 years to replicate hydrologic conditions that might have climatic persistence of variable length. In our case, we did not want to randomize the seasonal patterns in concentration trends, because we would then be testing the significance of the seasonal wave in the model rather than the trend. Therefore, our block units are entire years. The significance calculation is based on an attained level of significance method described in Hirsch and Ryberg (2012). For each bootstrap replicate, the resampled time series of pesticide concentrations is regressed against the variables in the SEAWAVE-Q model. For iteration \textit{j}, \textit{mj} is the estimate of the trend. The \textit{mj}s form a distribution of possible trend values across the bootstrap resampled dataset. The test statistic is \textit{M}, is the estimated trend with the observations from the original data (data in their original order). The null hypothesis is that the expected value of M is zero (no trend). Using 1,000 iterations of the bootstrap, we compute the attained, two-sided significance level. The p-value is the fraction of the iterations in which |\textit{mj}| is greater than or equal to |\textit{M}|. If few of the trends from the randomized data (the \textit{mj}s) are larger than the original trend value, the trend is statistically significant, depending on the analyst’s choice of significance level. The seawaveQ results report the attained \textit{p}-value so that one may choose to compare it to any significance level.

The bootstrap process is too time consuming to include in an R vignette. However, the function call would be something like the following:

<<results=verbatim,echo=TRUE>>=
# myfitRCSTrend <- fitswavecav(cdat = modMoRivOmaha, cavdat = cqwMoRivOmaha,
#                             tanm = "myfitRCSTrend", 
#                             pnames = c("04035", "04037", "04041"), 
#                             yrstart = 1995, yrend = 2003, tndbeg = 1995, 
#                             tndend = 2003, iwcav = c("flowa30", "flowa1"), 
#                             dcol = "dates", qwcols = c("R", "P"), mclass = 2, 
#                             numk = 4, bootRCS = TRUE, nboot = 1000, plotfile=FALSE)
@


\section{References Cited}
Canty, A.J., 2002, Resampling methods in R—The boot package: R News, v. 2, no. 3, p. 2--7.

Croxford, R., 2016, Restricted cubic spline regression—A brief introduction: SAS Institute Inc., Paper 5621. 

Harrell, F.E., Jr., 2010, Regression modeling strategies---With applications to linear models, logistic regression, and survival analysis: New York, Springer-Verlag, 568 p.

Harrell, F.E., Jr., 2016, rms: Regression modeling strategies; R package version 5.0-1: accessed August 14, 2018, at \url{https://CRAN.R-project.org/package=rms}.

Helsel, D.R., 2005, Nondetects and data analysis: New York, John Wiley and Sons, 250 p.

Hirsch, R.M., and Ryberg, K.R., 2012, Has the magnitude of floods across the USA changed with CO2 levels?: Hydrological Sciences Journal, v. 57, no. 1, p. 1--9, \url{https://doi.org/10.1080/02626667.2011.621895}.

Korn, E.L., and Graubard, B.I., 1999, Appendix C---Restricted cubic regression splines \textit{in} Korn, E.L, and Graubard, B.I., Analysis of health surveys: New York, John Wiley and Sons, p. 345--346, \url{https://doi.org/10.1002/9781118032619}.

Lee, Lopaka, 2017, NADA---Nondetects and data analysis for environmental data: R package version 1.6-1, \url{https://CRAN.R-project.org/package=NADA}.

Mood, A.M., Graybill, F.A., and Boes, D.C., 1974, Introduction to the theory of statistics (3d ed.): New York, McGraw-Hill, Inc., 564 p.

Oblinger Childress, C.J., Foreman,W.T., Connor, B.F., and Maloney, T.J., 1999, New reporting procedures based on long-term method detection levels and some considerations for interpretations of water-quality data provided by the U.S. Geological Survey: U.S. Geological Survey Open-File Report 99--193, 19 p. [Also available at \url{http://water.usgs.gov/owq/OFR_99-193/index.html}.]

\"On\"oz, B., and Bayazit, M., 2012, Block bootstrap for Mann–Kendall trend test of serially depend data: Hydrological Processes v. 26, no. 23, p. 3552--3560, \url{https://doi.org/10.1002/hyp.8438}.

Ryberg, K.R. and Vecchia, A.V., 2012, waterData---An R package for retrieval, analysis, and anomaly calculation of daily hydrologic time series data, version 1.0: U.S. Geological Survey Open-File Report 2012--1168; 8 p., accessed March 1, 2013, at \url{http://pubs.usgs.gov/of/2012/1168/}.

Ryberg, K.R. and York, B.C., 2020, seawaveQ---An R package providing a model and utilities for analyzing trends in chemical concentrations in streams with a seasonal wave (seawave) and adjustment for streamflow (Q) and other ancillary variables: U.S. Geological Survey Open-File Report 2020--1082, 25 p., with 4 appendixes, \url{https://doi.org/10.3133/ofr20201082}. 

Ryberg, K.R., Vecchia, A.V., Martin, J.D., and Gilliom, R.J., 2010, Trends in pesticide concentrations in urban streams in the United States, 1992--2008: U.S. Geological Survey Scientific Investigations Report 2010--5139; 101 p.,  accessed May 1, 2012, at \url{http://pubs.usgs.gov/sir/2010/5139/}.

Stone, C.J., 1986, Comment—Generalized additive models: Statistical Science, v. 1, no. 3, p. 312–314.

Sullivan, D.J., Vecchia, A.V., Lorenz, D.L., Gilliom, R.J., and Martin, J.D., 2009, Trends in pesticide concentrations in corn-belt streams, 1996--2006: U.S. Geological Survey Scientific Investigations Report 2009--5132; 75 p.,  accessed May 1, 2012, at \url{http://pubs.usgs.gov/sir/2009/5132/}.

Vecchia, A.V., Gilliom, R.J., Sullivan, D.J., Lorenz, D.L., and Martin, J.D., 2009, Trends in concentrations and use of agricultural herbicides for Corn Belt rivers, 1996--2006:  Environmental Science and Technology, v. 43; p. 9096--9102, accessed May 1, 2012, at \url{http://water.usgs.gov/nawqa/pubs/es902122j.pdf}.

Vecchia, A.V., Martin, J.D., and Gilliom, R.J., 2008, Modeling variability and trends in pesticide concentrations in streams: Journal of the American Water Resources Association, v. 44, no. 5, p. 1308--1324, accessed May 1, 2012, at \url{http://dx.doi.org/10.1111/j.1752-1688.2008.00225.x}.

\end{document}  
