diff --git a/inst/doc/dataRetrieval.pdf b/inst/doc/dataRetrieval.pdf
index 1def98857083c27b2beccf76cabc2af7c6078832..026dc6f66de0b73c9f481b3b743544343f6cd96d 100644
Binary files a/inst/doc/dataRetrieval.pdf and b/inst/doc/dataRetrieval.pdf differ
diff --git a/vignettes/dataRetrieval-concordance.tex b/vignettes/dataRetrieval-concordance.tex
index 4e0d91d301a16c0d0b2a53fc95bfe3c5efe616cf..ea0c92ef7dd656a679fea8379d113340557f4342 100644
--- a/vignettes/dataRetrieval-concordance.tex
+++ b/vignettes/dataRetrieval-concordance.tex
@@ -1,9 +1,9 @@
\Sconcordance{concordance:dataRetrieval.tex:dataRetrieval.Rnw:%
-1 49 1 50 0 1 6 11 1 1 5 46 1 113 0 7 1 42 0 21 1 10 %
-0 16 1 9 0 21 1 5 0 6 1 8 0 14 1 42 0 17 1 4 0 15 1 6 %
-0 16 1 10 0 5 1 8 0 20 1 4 0 18 1 3 0 21 1 10 0 20 1 %
-4 0 4 1 18 0 29 1 8 0 10 1 10 0 14 1 21 0 19 1 5 0 19 %
-1 5 0 17 1 8 0 14 1 15 0 16 1 5 0 9 1 5 0 62 1 6 0 11 %
-1 1 4 36 1 5 0 24 1 5 0 20 1 38 0 13 1 10 0 79 1 5 0 %
-5 1 13 0 5 1 5 0 11 1 5 0 7 1 5 0 16 1 51 0 15 1 49 0 %
-7 1 32 0 24 1 19 0 8 1 5 0 56 1}
+1 49 1 50 0 1 6 11 1 1 5 42 1 4 0 27 1 10 0 16 1 9 0 %
+21 1 5 0 6 1 8 0 24 1 4 0 15 1 6 0 16 1 10 0 5 1 8 0 %
+20 1 4 0 18 1 3 0 21 1 10 0 20 1 4 0 4 1 18 0 29 1 8 %
+0 10 1 10 0 14 1 21 0 19 1 5 0 19 1 5 0 17 1 8 0 14 1 %
+15 0 16 1 5 0 9 1 5 0 62 1 6 0 11 1 1 4 36 1 5 0 24 1 %
+5 0 20 1 38 0 13 1 10 0 101 1 5 0 5 1 14 0 5 1 5 0 11 %
+1 5 0 7 1 5 0 16 1 51 0 15 1 49 0 7 1 32 0 24 1 19 0 %
+8 1 5 0 56 1}
diff --git a/vignettes/dataRetrieval.Rnw b/vignettes/dataRetrieval.Rnw
index 7e5ab3318dde9a3a50b62ddfbb2ec421a5700d4b..81967b613b859e66630bb6988f630e416815ea04 100644
--- a/vignettes/dataRetrieval.Rnw
+++ b/vignettes/dataRetrieval.Rnw
@@ -81,53 +81,48 @@ knit_hooks$set(crop = hook_pdfcrop)
The dataRetrieval package was created to simplify the process of getting hydrologic data in the R enviornment. It has been specifically designed to work seamlessly with the EGRET R package: Exploration and Graphics for RivEr Trends (EGRET). See: \url{https://github.com/USGS-R/EGRET/wiki} for information on EGRET. EGRET is designed to provide analysis of water quality data sets using the WRTDS method of data analysis (WRTDS is Weighted Regressions on Time, Discharge and Season) as well as analysis of discharge trends using robust time-series smoothing techniques. Both of these capabilities provide both tabular and graphical analyses of long-term data sets.
-The dataRetrieval package is designed to retrieve many of the major data types of USGS hydrologic data that are available on the web, but also allows users to make use of other data that they supply from spreadsheets. Section 2 provides examples of how one can obtain raw data from USGS sources on the web and ingest them into data frames within the R environment. The functionality described in section 2 is for general use and is not tailored for the specific uses of the EGRET package. The functionality described in section 3 is tailored specifically to obtaining input from the web and structuring them specifically for use in the EGRET package. The functionality described in section 4 is for converting hydrologic data from user-supplied spreadsheets and structuring them specifically for use in the EGRET package.
+The dataRetrieval package is designed to retrieve many of the major data types of United States Geological Survey (USGS) hydrologic data that are available on the web, but also allows users to make use of other data that they supply from spreadsheets. Section 2 provides examples of how one can obtain raw data from USGS sources on the web and ingest them into data frames within the R environment. The functionality described in section 2 is for general use and is not tailored for the specific uses of the EGRET package. The functionality described in section 3 is tailored specifically to obtaining input from the web and structuring them for use in the EGRET package. The functionality described in section 4 is for converting hydrologic data from user-supplied spreadsheets and structuring them specifically for use in the EGRET package.
For information on getting started in R and installing the package, see Appendix (\ref{sec:appendix1}): Getting Started.
-
-%------------------------------------------------------------
-\section{General USGS Web Retrievals}
-%------------------------------------------------------------
-In this section, we will run through 5 examples, documenting how to get raw data from the web. This includes site information (\ref{sec:usgsSite}), measured parameter information (\ref{sec:usgsParams}), historical daily values(\ref{sec:usgsDaily}), real-time (unit) values (\ref{sec:usgsRT}), and water quality data (\ref{sec:usgsWQP}) or (\ref{sec:usgsSTORET}). We will use the Choptank River near Greensboro, MD as an example. The site-ID for this gage station is 01491000. Daily discharge measurements are available as far back as 1948. Additionally, forms of nitrate have been measured dating back to 1964. The functions/examples in this section are for raw data retrieval. This may or may not be the easiest data to work with. In the next section, we will use functions that retrieve and process the data in a dataframe that may prove more friendly for R analysis.
-
-\newpage
-Quick workflow example:
-<<workflow, echo=TRUE,eval=TRUE>>=
+Quick workflow for major dataRetrieval functions:
+<<workflow, echo=TRUE,eval=FALSE>>=
library(dataRetrieval)
# Site ID for Choptank River near Greensboro, MD
siteNumber <- "01491000"
ChoptankInfo <- getSiteFileData(siteNumber)
parameterCd <- "00060"
+
+#Raw data:
rawDailyData <- retrieveNWISData(siteNumber,parameterCd,
"1980-01-01","2010-01-01")
-head(rawDailyData)
+# Data compiled for EGRET analysis
Daily <- getDVData(siteNumber,parameterCd,
"1980-01-01","2010-01-01")
-head(Daily)
-ChoptankInfo <- getSiteFileData(siteNumber)
-colnames(ChoptankInfo)
-ChoptankAvail <- getDataAvailability(siteNumber)
-head(ChoptankAvail)
+# Sample data Nitrate:
+parameterCd <- "00618"
+Sample <- getSampleData(siteNumber,parameterCd,
+ "1980-01-01","2010-01-01")
+
+# Metadata on site and nitrate:
+INFO <- getMetaData(siteNumber,parameterCd)
-parameterCd <- "00618" # Nitrate
-parameterINFO <- getParameterInfo(parameterCd)
-colnames(parameterINFO)
+# Merge discharge and nitrate data to one dataframe:
+Sample <- mergeReport()
@
-\newpage
-<<workflow2, echo=TRUE,eval=TRUE>>=
-ChoptankAvail <- getDataAvailability(siteNumber)
-head(ChoptankAvail)
-@
+%------------------------------------------------------------
+\section{General USGS Web Retrievals}
+%------------------------------------------------------------
+In this section, we will run through 5 examples, which document how to get raw data from the web. This includes site information (\ref{sec:usgsSite}), measured parameter information (\ref{sec:usgsParams}), historical daily values(\ref{sec:usgsDaily}), real-time (unit) values (\ref{sec:usgsRT}), and water quality data (\ref{sec:usgsWQP}) or (\ref{sec:usgsSTORET}). We will use the Choptank River near Greensboro, MD as an example. The site-ID for this gage station is 01491000. Daily discharge measurements are available as far back as 1948. Additionally, forms of nitrate have been measured dating back to 1964. The functions/examples in this section are for raw data retrieval. In the next section, we will use functions that retrieve and process the data in a dataframe that may prove more friendly for R analysis, and specifically tailored to EGRET analysis.
%------------------------------------------------------------
\subsection{Introduction}
%------------------------------------------------------------
-The United States Geological Survey organizes their hydrological data in standard structure. Streamgages are located throughout the United States, and each streamgage has a unique ID. Often (but not always), these ID's are 8 digits. The first step to finding data is discoving this 8-digit ID. There are many ways to do this, one is the National Water Information System: Mapper \url{http://maps.waterdata.usgs.gov/mapper/index.html}.
+The USGS organizes their hydrological data in a standard structure. Streamgages are located throughout the United States, and each streamgage has a unique ID. Often (but not always), these ID's are 8 digits. The first step to finding data is discoving this 8-digit ID. There are many ways to do this, one is the National Water Information System: Mapper \url{http://maps.waterdata.usgs.gov/mapper/index.html}.
Once the site-ID is known, the next required input for USGS data retrievals is the 'parameter code'. This is a 5-digit code that specifies what measured paramater is being requested. A complete list of possible USGS parameter codes can be found at \url{http://go.usa.gov/bVDz}.
@@ -194,22 +189,15 @@ Site information is obtained from \url{http://waterservices.usgs.gov/rest/Site-T
\subsubsection{getDataAvailability}
\label{sec:usgsDataAvailability}
%------------------------------------------------------------
-To find out the available data at a particular USGS site, including measured parameters, period of record, and number of samples (count), use the getDataAvailability function:
-
-<<getSiteAvailability, echo=TRUE>>=
-# Continuing from the previous example:
-ChoptankAvailableData <- getDataAvailability(siteNumber)
-head(ChoptankAvailableData)
-@
-
-There is an additional argument to the getDataAvailability called longNames, which defaults to FALSE. Setting longNames to TRUE will cause the function to make a web service call for each parameter and return expanded information on that parameter. Currently, this is a very slow process because each parameter code makes a unique web service call. If the site does not have many measured parameters, setting longNames to TRUE is reasonable.
-
-It is also possible to only request parameter information for a subset of variables. In the following example, we retrieve just the daily mean parameter information from the Choptank data availability dataframe (excluding all unit value and water quality values).
+To find out the available data at a particular USGS site, including measured parameters, period of record, and number of samples (count), use the getDataAvailability function. It is also possible to only request parameter information for a subset of variables. In the following example, we retrieve just the daily mean parameter information from the Choptank data availability dataframe (excluding all unit value and water quality values).
<<getSiteExtended, echo=TRUE>>=
# Continuing from the previous example:
# This pulls out just the daily data:
+
+ChoptankAvailableData <- getDataAvailability(siteNumber)
+
ChoptankDailyData <- subset(ChoptankAvailableData,
"dv" == service)
@@ -712,7 +700,29 @@ Data Type & EGRET Functions & Other Retrieval Functions \\
\end{minipage}
\end{table}
+\begin{table}[!ht]
+\begin{minipage}{\linewidth}
+\begin{center}
+\caption{dataRetrieval miscellaneous functions}
+\begin{tabular}{ll}
+ \hline
+Function Name & Description \\
+ \hline
+ compressData & Converts value/qualifier into ConcLow, ConcHigh, Uncen\\
+ getRDB1Data & Retrieves and converts RDB data to dataframe\\
+ getWaterML1Data & Retrieves and converts WaterML1 data to dataframe\\
+ getWaterML2Data & Retrieves and converts WaterML2 data to dataframe\\
+ mergeReport & Merges flow data from the daily record into the sample record\\
+ populateDateColumns & Generates Julian, Month, Day, DecYear, and MonthSeq columns\\
+ removeDuplicates & Removes duplicated rows\\
+ renameColumns & Renames columns from raw data retrievals\\
+ \hline
+\end{tabular}
+\end{center}
+\end{minipage}
+\end{table}
+\clearpage
\appendix
%------------------------------------------------------------
diff --git a/vignettes/dataRetrieval.log b/vignettes/dataRetrieval.log
index 4acb9c778fa3c2977883cd63ef49e1a647ef02bb..9ccdae4e23e9083bb200d2abb316d9216b5d667e 100644
--- a/vignettes/dataRetrieval.log
+++ b/vignettes/dataRetrieval.log
@@ -1,4 +1,4 @@
-This is pdfTeX, Version 3.1415926-2.3-1.40.12 (MiKTeX 2.9) (preloaded format=pdflatex 2012.1.6) 28 OCT 2013 16:27
+This is pdfTeX, Version 3.1415926-2.3-1.40.12 (MiKTeX 2.9) (preloaded format=pdflatex 2012.1.6) 30 OCT 2013 11:04
entering extended mode
**dataRetrieval.tex
(D:\LADData\RCode\dataRetrieval\vignettes\dataRetrieval.tex
@@ -512,280 +512,126 @@ Overfull \vbox (21.68121pt too high) has occurred while \output is active []
[2]
-Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-
-
-[3]
-Package color Info: Redefining color shadecolor on input line 138.
+Package color Info: Redefining color shadecolor on input line 131.
LaTeX Font Info: Font shape `OT1/pcr/bx/n' in size <10.95> not available
-(Font) Font shape `OT1/pcr/b/n' tried instead on input line 140.
+(Font) Font shape `OT1/pcr/b/n' tried instead on input line 133.
LaTeX Font Info: Font shape `OT1/pcr/m/it' in size <10.95> not available
-(Font) Font shape `OT1/pcr/m/sl' tried instead on input line 141.
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 259--259
-[] \OT1/pcr/m/n
-/10.95 parameter_nm[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 259--259
-[]\OT1/pcr/m/n/10.95 1 Location in cross section, distance from right bank loo
-king upstream, feet[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 259--259
-[]\OT1/pcr/m/n/10.95 2
- Stream width, feet[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 259--259
-[]\OT1/pcr/m/n/10.95 3 Sampl
-e accounting number[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 259--259
-[]\OT1/pcr/m/n/10.95 4 Location in cross section, distance from left bank looki
-ng downstream, feet[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 259--259
-[]\OT1/pcr/m/n/10.95 5 Temperature, wat
-er, degrees Celsius[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 259--259
-[]\OT1/pcr/m/n/10.95 6 Temperature, wat
-er, degrees Celsius[]
- []
-
-
-Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-
-
-[4]
-Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-
-
-[5]
-Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-
-
-[6]
-Package color Info: Redefining color shadecolor on input line 277.
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 318--318
-[] \OT1/pcr/m/n
-/10.95 parameter_nm[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 318--318
-[]\OT1/pcr/m/n/10.95 1 Location in cross section, distance from right bank loo
-king upstream, feet[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 318--318
-[]\OT1/pcr/m/n/10.95 2
- Stream width, feet[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 318--318
-[]\OT1/pcr/m/n/10.95 3 Sampl
-e accounting number[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 318--318
-[]\OT1/pcr/m/n/10.95 4 Location in cross section, distance from left bank looki
-ng downstream, feet[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 318--318
-[]\OT1/pcr/m/n/10.95 5 Temperature, wat
-er, degrees Celsius[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 318--318
-[]\OT1/pcr/m/n/10.95 6 Temperature, wat
-er, degrees Celsius[]
- []
-
+(Font) Font shape `OT1/pcr/m/sl' tried instead on input line 134.
LaTeX Font Info: Font shape `OT1/ptm/bx/n' in size <12> not available
-(Font) Font shape `OT1/ptm/b/n' tried instead on input line 324.
+(Font) Font shape `OT1/ptm/b/n' tried instead on input line 168.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[7]
-Overfull \hbox (22.25568pt too wide) in paragraph at lines 356--357
+[3]
+Overfull \hbox (22.25568pt too wide) in paragraph at lines 200--201
[][]$\OT1/pcr/m/n/10.95 http : / / nwis . waterdata . usgs . gov / nwis / help
/ ?read _ file = stat&format = table$[]
[]
-Package color Info: Redefining color shadecolor on input line 394.
+Package color Info: Redefining color shadecolor on input line 238.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[8]
-Package color Info: Redefining color shadecolor on input line 408.
+[4]
+Package color Info: Redefining color shadecolor on input line 252.
-Overfull \hbox (43.87282pt too wide) in paragraph at lines 418--419
+Overfull \hbox (43.87282pt too wide) in paragraph at lines 262--263
[]\OT1/ptm/m/n/10.95 Site in-for-ma-tion is ob-tained from []$\OT1/pcr/m/n/10.9
5 http : / / waterservices . usgs . gov / rest / Site-[]Test-[]Tool .$
[]
-Package color Info: Redefining color shadecolor on input line 428.
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 470--470
-[] \OT1/pcr/m/n
-/10.95 parameter_nm[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 470--470
-[]\OT1/pcr/m/n/10.95 1 Location in cross section, distance from right bank loo
-king upstream, feet[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 470--470
-[]\OT1/pcr/m/n/10.95 2
- Stream width, feet[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 470--470
-[]\OT1/pcr/m/n/10.95 3 Sampl
-e accounting number[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 470--470
-[]\OT1/pcr/m/n/10.95 4 Location in cross section, distance from left bank looki
-ng downstream, feet[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 470--470
-[]\OT1/pcr/m/n/10.95 5 Temperature, wat
-er, degrees Celsius[]
- []
-
-
-Overfull \hbox (57.81561pt too wide) in paragraph at lines 470--470
-[]\OT1/pcr/m/n/10.95 6 Temperature, wat
-er, degrees Celsius[]
- []
-
-
-Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-
-
-[9]
-Package color Info: Redefining color shadecolor on input line 481.
+Package color Info: Redefining color shadecolor on input line 273.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[10]
-Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-
-
-[11]
-Package color Info: Redefining color shadecolor on input line 527.
-Package color Info: Redefining color shadecolor on input line 545.
+[5]
+Package color Info: Redefining color shadecolor on input line 322.
+Package color Info: Redefining color shadecolor on input line 340.
-Overfull \hbox (30.64148pt too wide) in paragraph at lines 555--556
+Overfull \hbox (30.64148pt too wide) in paragraph at lines 350--351
[]\OT1/ptm/m/n/10.95 Parameter in-for-ma-tion is ob-tained from []$\OT1/pcr/m/n
/10.95 http : / / nwis . waterdata . usgs . gov / nwis / pmcodes/$[]
[]
-Package color Info: Redefining color shadecolor on input line 566.
+Package color Info: Redefining color shadecolor on input line 361.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[12]
-Package color Info: Redefining color shadecolor on input line 585.
-Package color Info: Redefining color shadecolor on input line 606.
+[6]
+Package color Info: Redefining color shadecolor on input line 380.
+Package color Info: Redefining color shadecolor on input line 401.
+
+Overfull \vbox (21.68121pt too high) has occurred while \output is active []
+[7]
+
LaTeX Warning: No positions in optional float specifier.
- Default added (so using `tbp') on input line 623.
+ Default added (so using `tbp') on input line 418.
-<figure/getNWIStemperaturePlot.pdf, id=236, 505.89pt x 505.89pt>
+<figure/getNWIStemperaturePlot.pdf, id=219, 505.89pt x 505.89pt>
File: figure/getNWIStemperaturePlot.pdf Graphic file (type pdf)
<use figure/getNWIStemperaturePlot.pdf>
-Package pdftex.def Info: figure/getNWIStemperaturePlot.pdf used on input line 6
-25.
+Package pdftex.def Info: figure/getNWIStemperaturePlot.pdf used on input line 4
+20.
(pdftex.def) Requested size: 448.07928pt x 448.07928pt.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[13]
-Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-
-
-[14 <D:/LADData/RCode/dataRetrieval/vignettes/figure/getNWIStemperaturePlot.pdf
->]
-Package color Info: Redefining color shadecolor on input line 644.
-Package color Info: Redefining color shadecolor on input line 657.
+[8 <D:/LADData/RCode/dataRetrieval/vignettes/figure/getNWIStemperaturePlot.pdf>
+]
+Package color Info: Redefining color shadecolor on input line 439.
+Package color Info: Redefining color shadecolor on input line 452.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[15]
-Package color Info: Redefining color shadecolor on input line 693.
-Package color Info: Redefining color shadecolor on input line 717.
+[9]
+Package color Info: Redefining color shadecolor on input line 488.
+Package color Info: Redefining color shadecolor on input line 512.
LaTeX Warning: No positions in optional float specifier.
- Default added (so using `tbp') on input line 726.
+ Default added (so using `tbp') on input line 521.
-<figure/getQWtemperaturePlot.pdf, id=264, 505.89pt x 505.89pt>
+<figure/getQWtemperaturePlot.pdf, id=239, 505.89pt x 505.89pt>
File: figure/getQWtemperaturePlot.pdf Graphic file (type pdf)
<use figure/getQWtemperaturePlot.pdf>
-Package pdftex.def Info: figure/getQWtemperaturePlot.pdf used on input line 728
+Package pdftex.def Info: figure/getQWtemperaturePlot.pdf used on input line 523
.
(pdftex.def) Requested size: 448.07378pt x 448.07928pt.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[16]
+[10]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[17 <D:/LADData/RCode/dataRetrieval/vignettes/figure/getQWtemperaturePlot.pdf>]
+[11 <D:/LADData/RCode/dataRetrieval/vignettes/figure/getQWtemperaturePlot.pdf>]
-Overfull \hbox (50.793pt too wide) in paragraph at lines 741--742
+Overfull \hbox (50.793pt too wide) in paragraph at lines 536--537
\OT1/ptm/m/n/10.95 There are ad-di-tional data sets avail-able on the Wa-ter Qu
al-ity Data Por-tal ([]$\OT1/pcr/m/n/10.95 http : / / www . waterqualitydata .$
[]
-Package color Info: Redefining color shadecolor on input line 744.
-LaTeX Font Info: Try loading font information for TS1+pcr on input line 746.
+Package color Info: Redefining color shadecolor on input line 539.
+LaTeX Font Info: Try loading font information for TS1+pcr on input line 541.
("C:\Program Files (x86)\MiKTeX 2.9\tex\latex\psnfss\ts1pcr.fd"
File: ts1pcr.fd 2001/06/04 font definitions for TS1/pcr.
)
-Package color Info: Redefining color shadecolor on input line 779.
+Package color Info: Redefining color shadecolor on input line 574.
-Overfull \hbox (5.25568pt too wide) in paragraph at lines 788--788
+Overfull \hbox (5.25568pt too wide) in paragraph at lines 583--583
[][]\OT1/pcr/m/n/10.95 url_uv[] []<-[] []\OT1/pcr/b/n/10.95 constructNWISURL[][
]\OT1/pcr/m/n/10.95 (siteNumber,[][]"00060"[][],startDate,endDate,[][]\TS1/pcr/
m/n/10.95 '\OT1/pcr/m/n/10.95 uv\TS1/pcr/m/n/10.95 '[][]\OT1/pcr/m/n/10.95 )[][
@@ -796,57 +642,56 @@ m/n/10.95 '\OT1/pcr/m/n/10.95 uv\TS1/pcr/m/n/10.95 '[][]\OT1/pcr/m/n/10.95 )[][
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[18]
-Package color Info: Redefining color shadecolor on input line 809.
-Package color Info: Redefining color shadecolor on input line 828.
+[12]
+Package color Info: Redefining color shadecolor on input line 604.
+Package color Info: Redefining color shadecolor on input line 623.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[19]
-Package color Info: Redefining color shadecolor on input line 883.
-Package color Info: Redefining color shadecolor on input line 896.
+[13]
+Package color Info: Redefining color shadecolor on input line 678.
+Package color Info: Redefining color shadecolor on input line 691.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[20]
+[14]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[21]
+[15]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[22]
+[16]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[23]
-Package color Info: Redefining color shadecolor on input line 1017.
-LaTeX Font Info: Try loading font information for OMS+pcr on input line 1022
-.
+[17]
+Package color Info: Redefining color shadecolor on input line 812.
+LaTeX Font Info: Try loading font information for OMS+pcr on input line 817.
+
("C:\Program Files (x86)\MiKTeX 2.9\tex\latex\psnfss\omspcr.fd"
File: omspcr.fd
)
LaTeX Font Info: Font shape `OMS/pcr/m/n' in size <10.95> not available
-(Font) Font shape `OMS/cmsy/m/n' tried instead on input line 1022.
-
-Package color Info: Redefining color shadecolor on input line 1046.
+(Font) Font shape `OMS/cmsy/m/n' tried instead on input line 817.
+Package color Info: Redefining color shadecolor on input line 841.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[24]
-Package color Info: Redefining color shadecolor on input line 1066.
+[18]
+Package color Info: Redefining color shadecolor on input line 861.
-Overfull \hbox (44.67563pt too wide) in paragraph at lines 1091--1091
+Overfull \hbox (44.67563pt too wide) in paragraph at lines 886--886
[] \OT1/pcr/m/n/10.95 First day of the discharge record is 2000-01-01 and last
day is 2013-01-01[]
[]
-Overfull \hbox (44.67563pt too wide) in paragraph at lines 1091--1091
+Overfull \hbox (44.67563pt too wide) in paragraph at lines 886--886
[] \OT1/pcr/m/n/10.95 The first sample is from 2000-01-04 and the last sample i
s from 2012-12-18[]
[]
@@ -855,150 +700,150 @@ s from 2012-12-18[]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[25]
-Package color Info: Redefining color shadecolor on input line 1123.
+[19]
+Package color Info: Redefining color shadecolor on input line 918.
LaTeX Warning: No positions in optional float specifier.
- Default added (so using `tbp') on input line 1129.
+ Default added (so using `tbp') on input line 924.
-<figure/egretEx.pdf, id=325, 505.89pt x 505.89pt>
+<figure/egretEx.pdf, id=300, 505.89pt x 505.89pt>
File: figure/egretEx.pdf Graphic file (type pdf)
<use figure/egretEx.pdf>
-Package pdftex.def Info: figure/egretEx.pdf used on input line 1131.
+Package pdftex.def Info: figure/egretEx.pdf used on input line 926.
(pdftex.def) Requested size: 448.07378pt x 448.07928pt.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[26]
+[20]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[27 <D:/LADData/RCode/dataRetrieval/vignettes/figure/egretEx.pdf>]
+[21 <D:/LADData/RCode/dataRetrieval/vignettes/figure/egretEx.pdf>]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[28
+[22
]
-Package color Info: Redefining color shadecolor on input line 1214.
-Package color Info: Redefining color shadecolor on input line 1224.
+Overfull \vbox (21.68121pt too high) has occurred while \output is active []
+
+
+[23]
+Package color Info: Redefining color shadecolor on input line 1031.
+Package color Info: Redefining color shadecolor on input line 1041.
+Package color Info: Redefining color shadecolor on input line 1060.
+Package color Info: Redefining color shadecolor on input line 1075.
+
+Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-Overfull \hbox (51.24562pt too wide) in paragraph at lines 1235--1235
-[] \OT1/pcr/m/n/10.95 Sample1 <- localSample[!duplicated(localSample[c("DecYea
-r","ConcHigh")]),][]
- []
-Package color Info: Redefining color shadecolor on input line 1242.
-Package color Info: Redefining color shadecolor on input line 1257.
+[24
-Overfull \hbox (157.60596pt too wide) in paragraph at lines 1266--1267
+]
+Overfull \hbox (157.60596pt too wide) in paragraph at lines 1084--1085
\OT1/ptm/m/n/10.95 C:/Users/userA/Documents/R/win-library/2.15/dataRetrieval, a
nd the de-fault for a Mac: /Users/userA/Library/R/2.15/library/dataRetrieval.
[]
-Package color Info: Redefining color shadecolor on input line 1270.
-
-Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-
+Package color Info: Redefining color shadecolor on input line 1088.
-[29]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[30]
+[25]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[31]
+[26]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[32
+[27
]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[33]
+[28]
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[34
+[29
]
-Package color Info: Redefining color shadecolor on input line 1457.
-Package color Info: Redefining color shadecolor on input line 1495.
+Package color Info: Redefining color shadecolor on input line 1275.
+Package color Info: Redefining color shadecolor on input line 1313.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[35
+[30
]
-Overfull \hbox (11.82567pt too wide) in paragraph at lines 1513--1513
+Overfull \hbox (11.82567pt too wide) in paragraph at lines 1331--1331
[]\OT1/pcr/m/n/10.95 Suspended sediment concentration (SSC) 1980-10-01 1991-09-
30 3651 mg/l[]
[]
-<table1.png, id=381, 554.07pt x 125.71968pt>
+<table1.png, id=361, 554.07pt x 125.71968pt>
File: table1.png Graphic file (type png)
<use table1.png>
-Package pdftex.def Info: table1.png used on input line 1532.
+Package pdftex.def Info: table1.png used on input line 1350.
(pdftex.def) Requested size: 554.06865pt x 125.71936pt.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[36 <D:/LADData/RCode/dataRetrieval/vignettes/table1.png>]
-Underfull \hbox (badness 1983) in paragraph at lines 1548--1549
+[31 <D:/LADData/RCode/dataRetrieval/vignettes/table1.png>]
+Underfull \hbox (badness 1983) in paragraph at lines 1366--1367
[]\OT1/ptm/m/n/10.95 Hirsch, R. M., Moyer, D. L. and Arch-field, S. A. (2010),
Weighted Re-gres-sions on
[]
-Underfull \hbox (badness 1221) in paragraph at lines 1548--1549
+Underfull \hbox (badness 1221) in paragraph at lines 1366--1367
\OT1/ptm/m/n/10.95 Time, Dis-charge, and Sea-son (WRTDS), with an Ap-pli-ca-tio
n to Chesa-peake Bay River
[]
-Underfull \hbox (badness 2443) in paragraph at lines 1548--1549
+Underfull \hbox (badness 2443) in paragraph at lines 1366--1367
\OT1/ptm/m/n/10.95 In-puts. JAWRA Jour-nal of the Amer-i-can Wa-ter Re-sources
As-so-ci-a-tion, 46: 857-880.
[]
-Underfull \hbox (badness 3690) in paragraph at lines 1548--1549
+Underfull \hbox (badness 3690) in paragraph at lines 1366--1367
\OT1/ptm/m/n/10.95 doi: 10.1111/j.1752-1688.2010.00482.x []$\OT1/pcr/m/n/10.95
http : / / onlinelibrary . wiley . com / doi /$
[]
-Package atveryend Info: Empty hook `BeforeClearDocument' on input line 1555.
+Package atveryend Info: Empty hook `BeforeClearDocument' on input line 1373.
Overfull \vbox (21.68121pt too high) has occurred while \output is active []
-[37
+[32
]
-Package atveryend Info: Empty hook `AfterLastShipout' on input line 1555.
+Package atveryend Info: Empty hook `AfterLastShipout' on input line 1373.
(D:\LADData\RCode\dataRetrieval\vignettes\dataRetrieval.aux)
-Package atveryend Info: Executing hook `AtVeryEndDocument' on input line 1555.
-Package atveryend Info: Executing hook `AtEndAfterFileList' on input line 1555.
+Package atveryend Info: Executing hook `AtVeryEndDocument' on input line 1373.
+Package atveryend Info: Executing hook `AtEndAfterFileList' on input line 1373.
Package rerunfilecheck Info: File `dataRetrieval.out' has not changed.
(rerunfilecheck) Checksum: 30AE4E6A4E0A9704D03232DABCF58698;2061.
)
Here is how much of TeX's memory you used:
- 8264 strings out of 494045
- 123333 string characters out of 3145961
- 228280 words of memory out of 3000000
+ 8260 strings out of 494045
+ 123314 string characters out of 3145961
+ 215275 words of memory out of 3000000
11326 multiletter control sequences out of 15000+200000
30981 words of font info for 68 fonts, out of 3000000 for 9000
715 hyphenation exceptions out of 8191
- 44i,15n,42p,912b,449s stack positions out of 5000i,500n,10000p,200000b,50000s
+ 44i,15n,42p,905b,449s stack positions out of 5000i,500n,10000p,200000b,50000s
{C:/Program Files (x86)/MiKTeX 2.9/fonts/enc/dvips/fontname/8r.enc}<C:/Progra
m Files (x86)/MiKTeX 2.9/fonts/type1/public/amsfonts/cm/cmmi10.pfb><C:/Program
Files (x86)/MiKTeX 2.9/fonts/type1/public/amsfonts/cm/cmsy10.pfb><C:/Program Fi
@@ -1008,9 +853,9 @@ les (x86)/MiKTeX 2.9/fonts/type1/urw/courier/ucrb8a.pfb><C:/Program Files (x86)
/type1/urw/times/utmb8a.pfb><C:/Program Files (x86)/MiKTeX 2.9/fonts/type1/urw/
times/utmr8a.pfb><C:/Program Files (x86)/MiKTeX 2.9/fonts/type1/urw/times/utmri
8a.pfb>
-Output written on dataRetrieval.pdf (37 pages, 297127 bytes).
+Output written on dataRetrieval.pdf (32 pages, 290736 bytes).
PDF statistics:
- 465 PDF objects out of 1000 (max. 8388607)
- 97 named destinations out of 1000 (max. 500000)
+ 442 PDF objects out of 1000 (max. 8388607)
+ 93 named destinations out of 1000 (max. 500000)
261 words of extra memory for PDF output out of 10000 (max. 10000000)
diff --git a/vignettes/dataRetrieval.pdf b/vignettes/dataRetrieval.pdf
index 53bb4ac34aeb114a9960c7c3ab88d14834993f6e..026dc6f66de0b73c9f481b3b743544343f6cd96d 100644
Binary files a/vignettes/dataRetrieval.pdf and b/vignettes/dataRetrieval.pdf differ
diff --git a/vignettes/dataRetrieval.synctex.gz b/vignettes/dataRetrieval.synctex.gz
index ad1c132034b02d18b3718135c829518311cba83a..090a1de6a3cc0a8f6566d13529c07bd78a70db7f 100644
Binary files a/vignettes/dataRetrieval.synctex.gz and b/vignettes/dataRetrieval.synctex.gz differ
diff --git a/vignettes/dataRetrieval.tex b/vignettes/dataRetrieval.tex
index 709a75ac99d19439dc6ca258bdc5abd5daede792..c32dd89954a9187db03f5e1ea4e1f90a14dd9417 100644
--- a/vignettes/dataRetrieval.tex
+++ b/vignettes/dataRetrieval.tex
@@ -122,18 +122,11 @@
The dataRetrieval package was created to simplify the process of getting hydrologic data in the R enviornment. It has been specifically designed to work seamlessly with the EGRET R package: Exploration and Graphics for RivEr Trends (EGRET). See: \url{https://github.com/USGS-R/EGRET/wiki} for information on EGRET. EGRET is designed to provide analysis of water quality data sets using the WRTDS method of data analysis (WRTDS is Weighted Regressions on Time, Discharge and Season) as well as analysis of discharge trends using robust time-series smoothing techniques. Both of these capabilities provide both tabular and graphical analyses of long-term data sets.
-The dataRetrieval package is designed to retrieve many of the major data types of USGS hydrologic data that are available on the web, but also allows users to make use of other data that they supply from spreadsheets. Section 2 provides examples of how one can obtain raw data from USGS sources on the web and ingest them into data frames within the R environment. The functionality described in section 2 is for general use and is not tailored for the specific uses of the EGRET package. The functionality described in section 3 is tailored specifically to obtaining input from the web and structuring them specifically for use in the EGRET package. The functionality described in section 4 is for converting hydrologic data from user-supplied spreadsheets and structuring them specifically for use in the EGRET package.
+The dataRetrieval package is designed to retrieve many of the major data types of United States Geological Survey (USGS) hydrologic data that are available on the web, but also allows users to make use of other data that they supply from spreadsheets. Section 2 provides examples of how one can obtain raw data from USGS sources on the web and ingest them into data frames within the R environment. The functionality described in section 2 is for general use and is not tailored for the specific uses of the EGRET package. The functionality described in section 3 is tailored specifically to obtaining input from the web and structuring them for use in the EGRET package. The functionality described in section 4 is for converting hydrologic data from user-supplied spreadsheets and structuring them specifically for use in the EGRET package.
For information on getting started in R and installing the package, see Appendix (\ref{sec:appendix1}): Getting Started.
-
-%------------------------------------------------------------
-\section{General USGS Web Retrievals}
-%------------------------------------------------------------
-In this section, we will run through 5 examples, documenting how to get raw data from the web. This includes site information (\ref{sec:usgsSite}), measured parameter information (\ref{sec:usgsParams}), historical daily values(\ref{sec:usgsDaily}), real-time (unit) values (\ref{sec:usgsRT}), and water quality data (\ref{sec:usgsWQP}) or (\ref{sec:usgsSTORET}). We will use the Choptank River near Greensboro, MD as an example. The site-ID for this gage station is 01491000. Daily discharge measurements are available as far back as 1948. Additionally, forms of nitrate have been measured dating back to 1964. The functions/examples in this section are for raw data retrieval. This may or may not be the easiest data to work with. In the next section, we will use functions that retrieve and process the data in a dataframe that may prove more friendly for R analysis.
-
-\newpage
-Quick workflow example:
+Quick workflow for major dataRetrieval functions:
\begin{knitrout}
\definecolor{shadecolor}{rgb}{0.969, 0.969, 0.969}\color{fgcolor}\begin{kframe}
\begin{alltt}
@@ -142,188 +135,39 @@ Quick workflow example:
\hlstd{siteNumber} \hlkwb{<-} \hlstr{"01491000"}
\hlstd{ChoptankInfo} \hlkwb{<-} \hlkwd{getSiteFileData}\hlstd{(siteNumber)}
\hlstd{parameterCd} \hlkwb{<-} \hlstr{"00060"}
+
+\hlcom{#Raw data:}
\hlstd{rawDailyData} \hlkwb{<-} \hlkwd{retrieveNWISData}\hlstd{(siteNumber,parameterCd,}
\hlstr{"1980-01-01"}\hlstd{,}\hlstr{"2010-01-01"}\hlstd{)}
-\hlkwd{head}\hlstd{(rawDailyData)}
-\end{alltt}
-\begin{verbatim}
- agency_cd site_no datetime X02_00060_00003
-1 USGS 01491000 1980-01-01 100
-2 USGS 01491000 1980-01-02 96
-3 USGS 01491000 1980-01-03 92
-4 USGS 01491000 1980-01-04 89
-5 USGS 01491000 1980-01-05 91
-6 USGS 01491000 1980-01-06 100
- X02_00060_00003_cd
-1 A
-2 A
-3 A
-4 A
-5 A
-6 A
-\end{verbatim}
-\begin{alltt}
+\hlcom{# Data compiled for EGRET analysis}
\hlstd{Daily} \hlkwb{<-} \hlkwd{getDVData}\hlstd{(siteNumber,parameterCd,}
\hlstr{"1980-01-01"}\hlstd{,}\hlstr{"2010-01-01"}\hlstd{)}
-\end{alltt}
-\begin{verbatim}
-There are 10959 data points, and 10959 days.
-\end{verbatim}
-\begin{alltt}
-\hlkwd{head}\hlstd{(Daily)}
-\end{alltt}
-\begin{verbatim}
- Date Q Julian Month Day DecYear MonthSeq
-1 1980-01-01 2.832 47481 1 1 1980 1561
-2 1980-01-02 2.718 47482 1 2 1980 1561
-3 1980-01-03 2.605 47483 1 3 1980 1561
-4 1980-01-04 2.520 47484 1 4 1980 1561
-5 1980-01-05 2.577 47485 1 5 1980 1561
-6 1980-01-06 2.832 47486 1 6 1980 1561
- Qualifier i LogQ Q7 Q30
-1 A 1 1.0409 NA NA
-2 A 2 1.0000 NA NA
-3 A 3 0.9575 NA NA
-4 A 4 0.9243 NA NA
-5 A 5 0.9466 NA NA
-6 A 6 1.0409 NA NA
-\end{verbatim}
-\begin{alltt}
-\hlstd{ChoptankInfo} \hlkwb{<-} \hlkwd{getSiteFileData}\hlstd{(siteNumber)}
-\hlkwd{colnames}\hlstd{(ChoptankInfo)}
-\end{alltt}
-\begin{verbatim}
- [1] "agency.cd" "site.no"
- [3] "station.nm" "site.tp.cd"
- [5] "lat.va" "long.va"
- [7] "dec.lat.va" "dec.long.va"
- [9] "coord.meth.cd" "coord.acy.cd"
-[11] "coord.datum.cd" "dec.coord.datum.cd"
-[13] "district.cd" "state.cd"
-[15] "county.cd" "country.cd"
-[17] "land.net.ds" "map.nm"
-[19] "map.scale.fc" "alt.va"
-[21] "alt.meth.cd" "alt.acy.va"
-[23] "alt.datum.cd" "huc.cd"
-[25] "basin.cd" "topo.cd"
-[27] "instruments.cd" "construction.dt"
-[29] "inventory.dt" "drain.area.va"
-[31] "contrib.drain.area.va" "tz.cd"
-[33] "local.time.fg" "reliability.cd"
-[35] "gw.file.cd" "nat.aqfr.cd"
-[37] "aqfr.cd" "aqfr.type.cd"
-[39] "well.depth.va" "hole.depth.va"
-[41] "depth.src.cd" "project.no"
-[43] "queryTime"
-\end{verbatim}
-\begin{alltt}
-\hlstd{ChoptankAvail} \hlkwb{<-} \hlkwd{getDataAvailability}\hlstd{(siteNumber)}
-\hlkwd{head}\hlstd{(ChoptankAvail)}
-\end{alltt}
-\begin{verbatim}
- parameter_cd statCd startDate endDate count service
-1 00001 1974-11-04 1984-08-01 109 qw
-2 00004 2013-03-27 2013-03-27 1 qw
-3 00008 1972-10-24 1973-12-26 12 qw
-4 00009 1974-03-22 1974-03-22 1 qw
-5 00010 00001 1988-10-01 2012-05-09 894 dv
-6 00010 00002 2010-10-01 2012-05-09 529 dv
- parameter_group_nm
-1 Information
-2 Physical
-3 Information
-4 Information
-5 Physical
-6 Physical
- parameter_nm
-1 Location in cross section, distance from right bank looking upstream, feet
-2 Stream width, feet
-3 Sample accounting number
-4 Location in cross section, distance from left bank looking downstream, feet
-5 Temperature, water, degrees Celsius
-6 Temperature, water, degrees Celsius
- casrn srsname
-1
-2 Instream features, est. stream width
-3
-4
-5 Temperature, water
-6 Temperature, water
- parameter_units
-1 ft
-2 ft
-3 nu
-4 ft
-5 deg C
-6 deg C
-\end{verbatim}
-\begin{alltt}
-\hlstd{parameterCd} \hlkwb{<-} \hlstr{"00618"} \hlcom{# Nitrate}
-\hlstd{parameterINFO} \hlkwb{<-} \hlkwd{getParameterInfo}\hlstd{(parameterCd)}
-\hlkwd{colnames}\hlstd{(parameterINFO)}
-\end{alltt}
-\begin{verbatim}
-[1] "parameter_cd" "parameter_group_nm"
-[3] "parameter_nm" "casrn"
-[5] "srsname" "parameter_units"
-\end{verbatim}
-\end{kframe}
-\end{knitrout}
+\hlcom{# Sample data Nitrate:}
+\hlstd{parameterCd} \hlkwb{<-} \hlstr{"00618"}
+\hlstd{Sample} \hlkwb{<-} \hlkwd{getSampleData}\hlstd{(siteNumber,parameterCd,}
+ \hlstr{"1980-01-01"}\hlstd{,}\hlstr{"2010-01-01"}\hlstd{)}
-\newpage
+\hlcom{# Metadata on site and nitrate:}
+\hlstd{INFO} \hlkwb{<-} \hlkwd{getMetaData}\hlstd{(siteNumber,parameterCd)}
-\begin{knitrout}
-\definecolor{shadecolor}{rgb}{0.969, 0.969, 0.969}\color{fgcolor}\begin{kframe}
-\begin{alltt}
-\hlstd{ChoptankAvail} \hlkwb{<-} \hlkwd{getDataAvailability}\hlstd{(siteNumber)}
-\hlkwd{head}\hlstd{(ChoptankAvail)}
+\hlcom{# Merge discharge and nitrate data to one dataframe:}
+\hlstd{Sample} \hlkwb{<-} \hlkwd{mergeReport}\hlstd{()}
\end{alltt}
-\begin{verbatim}
- parameter_cd statCd startDate endDate count service
-1 00001 1974-11-04 1984-08-01 109 qw
-2 00004 2013-03-27 2013-03-27 1 qw
-3 00008 1972-10-24 1973-12-26 12 qw
-4 00009 1974-03-22 1974-03-22 1 qw
-5 00010 00001 1988-10-01 2012-05-09 894 dv
-6 00010 00002 2010-10-01 2012-05-09 529 dv
- parameter_group_nm
-1 Information
-2 Physical
-3 Information
-4 Information
-5 Physical
-6 Physical
- parameter_nm
-1 Location in cross section, distance from right bank looking upstream, feet
-2 Stream width, feet
-3 Sample accounting number
-4 Location in cross section, distance from left bank looking downstream, feet
-5 Temperature, water, degrees Celsius
-6 Temperature, water, degrees Celsius
- casrn srsname
-1
-2 Instream features, est. stream width
-3
-4
-5 Temperature, water
-6 Temperature, water
- parameter_units
-1 ft
-2 ft
-3 nu
-4 ft
-5 deg C
-6 deg C
-\end{verbatim}
\end{kframe}
\end{knitrout}
+
+%------------------------------------------------------------
+\section{General USGS Web Retrievals}
+%------------------------------------------------------------
+In this section, we will run through 5 examples, which document how to get raw data from the web. This includes site information (\ref{sec:usgsSite}), measured parameter information (\ref{sec:usgsParams}), historical daily values(\ref{sec:usgsDaily}), real-time (unit) values (\ref{sec:usgsRT}), and water quality data (\ref{sec:usgsWQP}) or (\ref{sec:usgsSTORET}). We will use the Choptank River near Greensboro, MD as an example. The site-ID for this gage station is 01491000. Daily discharge measurements are available as far back as 1948. Additionally, forms of nitrate have been measured dating back to 1964. The functions/examples in this section are for raw data retrieval. In the next section, we will use functions that retrieve and process the data in a dataframe that may prove more friendly for R analysis, and specifically tailored to EGRET analysis.
+
%------------------------------------------------------------
\subsection{Introduction}
%------------------------------------------------------------
-The United States Geological Survey organizes their hydrological data in standard structure. Streamgages are located throughout the United States, and each streamgage has a unique ID. Often (but not always), these ID's are 8 digits. The first step to finding data is discoving this 8-digit ID. There are many ways to do this, one is the National Water Information System: Mapper \url{http://maps.waterdata.usgs.gov/mapper/index.html}.
+The USGS organizes their hydrological data in a standard structure. Streamgages are located throughout the United States, and each streamgage has a unique ID. Often (but not always), these ID's are 8 digits. The first step to finding data is discoving this 8-digit ID. There are many ways to do this, one is the National Water Information System: Mapper \url{http://maps.waterdata.usgs.gov/mapper/index.html}.
Once the site-ID is known, the next required input for USGS data retrievals is the 'parameter code'. This is a 5-digit code that specifies what measured paramater is being requested. A complete list of possible USGS parameter codes can be found at \url{http://go.usa.gov/bVDz}.
@@ -331,7 +175,7 @@ Not every station will measure all parameters. A short list of commonly measured
% latex table generated in R 3.0.2 by xtable 1.7-1 package
-% Mon Oct 28 16:27:10 2013
+% Wed Oct 30 11:04:35 2013
\begin{table}[ht]
\centering
\begin{tabular}{rll}
@@ -357,7 +201,7 @@ For real-time data, the parameter code and site ID will suffice. For most varia
Some common stat codes are shown in Table \ref{tab:stat}.
% latex table generated in R 3.0.2 by xtable 1.7-1 package
-% Mon Oct 28 16:27:10 2013
+% Wed Oct 30 11:04:35 2013
\begin{table}[ht]
\centering
\begin{tabular}{rll}
@@ -422,59 +266,7 @@ Site information is obtained from \url{http://waterservices.usgs.gov/rest/Site-T
\subsubsection{getDataAvailability}
\label{sec:usgsDataAvailability}
%------------------------------------------------------------
-To find out the available data at a particular USGS site, including measured parameters, period of record, and number of samples (count), use the getDataAvailability function:
-
-\begin{knitrout}
-\definecolor{shadecolor}{rgb}{0.969, 0.969, 0.969}\color{fgcolor}\begin{kframe}
-\begin{alltt}
-\hlcom{# Continuing from the previous example:}
-\hlstd{ChoptankAvailableData} \hlkwb{<-} \hlkwd{getDataAvailability}\hlstd{(siteNumber)}
-\hlkwd{head}\hlstd{(ChoptankAvailableData)}
-\end{alltt}
-\begin{verbatim}
- parameter_cd statCd startDate endDate count service
-1 00001 1974-11-04 1984-08-01 109 qw
-2 00004 2013-03-27 2013-03-27 1 qw
-3 00008 1972-10-24 1973-12-26 12 qw
-4 00009 1974-03-22 1974-03-22 1 qw
-5 00010 00001 1988-10-01 2012-05-09 894 dv
-6 00010 00002 2010-10-01 2012-05-09 529 dv
- parameter_group_nm
-1 Information
-2 Physical
-3 Information
-4 Information
-5 Physical
-6 Physical
- parameter_nm
-1 Location in cross section, distance from right bank looking upstream, feet
-2 Stream width, feet
-3 Sample accounting number
-4 Location in cross section, distance from left bank looking downstream, feet
-5 Temperature, water, degrees Celsius
-6 Temperature, water, degrees Celsius
- casrn srsname
-1
-2 Instream features, est. stream width
-3
-4
-5 Temperature, water
-6 Temperature, water
- parameter_units
-1 ft
-2 ft
-3 nu
-4 ft
-5 deg C
-6 deg C
-\end{verbatim}
-\end{kframe}
-\end{knitrout}
-
-
-There is an additional argument to the getDataAvailability called longNames, which defaults to FALSE. Setting longNames to TRUE will cause the function to make a web service call for each parameter and return expanded information on that parameter. Currently, this is a very slow process because each parameter code makes a unique web service call. If the site does not have many measured parameters, setting longNames to TRUE is reasonable.
-
-It is also possible to only request parameter information for a subset of variables. In the following example, we retrieve just the daily mean parameter information from the Choptank data availability dataframe (excluding all unit value and water quality values).
+To find out the available data at a particular USGS site, including measured parameters, period of record, and number of samples (count), use the getDataAvailability function. It is also possible to only request parameter information for a subset of variables. In the following example, we retrieve just the daily mean parameter information from the Choptank data availability dataframe (excluding all unit value and water quality values).
\begin{knitrout}
@@ -482,6 +274,9 @@ It is also possible to only request parameter information for a subset of variab
\begin{alltt}
\hlcom{# Continuing from the previous example:}
\hlcom{# This pulls out just the daily data:}
+
+\hlstd{ChoptankAvailableData} \hlkwb{<-} \hlkwd{getDataAvailability}\hlstd{(siteNumber)}
+
\hlstd{ChoptankDailyData} \hlkwb{<-} \hlkwd{subset}\hlstd{(ChoptankAvailableData,}
\hlstr{"dv"} \hlopt{==} \hlstd{service)}
@@ -494,7 +289,7 @@ It is also possible to only request parameter information for a subset of variab
% latex table generated in R 3.0.2 by xtable 1.7-1 package
-% Mon Oct 28 16:27:11 2013
+% Wed Oct 30 11:04:36 2013
\begin{table}[ht]
\centering
\begin{tabular}{rlllll}
@@ -502,7 +297,7 @@ It is also possible to only request parameter information for a subset of variab
& shortName & Start & End & Count & Units \\
\hline
1 & Temperature, water & 2010-10-01 & 2012-05-09 & 529 & deg C \\
- 2 & Stream flow, mean. daily & 1948-01-01 & 2013-10-27 & 24042 & ft3/s \\
+ 2 & Stream flow, mean. daily & 1948-01-01 & 2013-10-29 & 24044 & ft3/s \\
3 & Specific conductance & 2010-10-01 & 2012-05-09 & 527 & uS/cm @25C \\
4 & Suspended sediment concentration (SSC) & 1980-10-01 & 1991-09-30 & 3651 & mg/l \\
5 & Suspended sediment discharge & 1980-10-01 & 1991-09-30 & 3652 & tons/day \\
@@ -705,8 +500,8 @@ To get USGS water quality data from water samples collected at the streamgage (a
\hlkwd{names}\hlstd{(dissolvedNitrateSimple)}
\end{alltt}
\begin{verbatim}
-[1] "dateTime" "qualifier.00618" "value.00618"
-[4] "qualifier.71851" "value.71851"
+[1] "dateTime" "qualifier.71851" "value.71851"
+[4] "qualifier.00618" "value.00618"
\end{verbatim}
\end{kframe}
\end{knitrout}
@@ -843,7 +638,7 @@ There are 4750 data points, and 4750 days.
Details of the Daily dataframe are listed below:
% latex table generated in R 3.0.2 by xtable 1.7-1 package
-% Mon Oct 28 16:27:21 2013
+% Wed Oct 30 11:04:43 2013
\begin{table}[ht]
\centering
\begin{tabular}{rllll}
@@ -952,7 +747,7 @@ As an example to understand how the dataRetrieval package handles a more complex
\begin{center}
% latex table generated in R 3.0.2 by xtable 1.7-1 package
-% Mon Oct 28 16:27:22 2013
+% Wed Oct 30 11:04:44 2013
\begin{table}[ht]
\centering
\begin{tabular}{rllrlrlr}
@@ -1192,7 +987,29 @@ Data Type & EGRET Functions & Other Retrieval Functions \\
\end{minipage}
\end{table}
+\begin{table}[!ht]
+\begin{minipage}{\linewidth}
+\begin{center}
+\caption{dataRetrieval miscellaneous functions}
+\begin{tabular}{ll}
+ \hline
+Function Name & Description \\
+ \hline
+ compressData & Converts value/qualifier into ConcLow, ConcHigh, Uncen\\
+ getRDB1Data & Retrieves and converts RDB data to dataframe\\
+ getWaterML1Data & Retrieves and converts WaterML1 data to dataframe\\
+ getWaterML2Data & Retrieves and converts WaterML2 data to dataframe\\
+ mergeReport & Merges flow data from the daily record into the sample record\\
+ populateDateColumns & Generates Julian, Month, Day, DecYear, and MonthSeq columns\\
+ removeDuplicates & Removes duplicated rows\\
+ renameColumns & Renames columns from raw data retrievals\\
+ \hline
+\end{tabular}
+\end{center}
+\end{minipage}
+\end{table}
+\clearpage
\appendix
%------------------------------------------------------------
@@ -1226,10 +1043,11 @@ To see the raw code for a particular code, type the name of the function:
\hlstd{removeDuplicates}
\end{alltt}
\begin{verbatim}
-function(localSample=Sample) {
- Sample1 <- localSample[!duplicated(localSample[c("DecYear","ConcHigh")]),]
-
- return(Sample1)
+function (localSample = Sample)
+{
+ Sample1 <- localSample[!duplicated(localSample[c("DecYear",
+ "ConcHigh")]), ]
+ return(Sample1)
}
<environment: namespace:dataRetrieval>
\end{verbatim}
@@ -1286,7 +1104,7 @@ After installing the package, you need to open the library each time you re-star
%------------------------------------------------------------
% latex table generated in R 3.0.2 by xtable 1.7-1 package
-% Mon Oct 28 16:27:25 2013
+% Wed Oct 30 11:04:46 2013
\begin{table}[ht]
\centering
\begin{tabular}{rl}
@@ -1352,7 +1170,7 @@ After installing the package, you need to open the library each time you re-star
There are 62 columns returned from the water quality portal.
% latex table generated in R 3.0.2 by xtable 1.7-1 package
-% Mon Oct 28 16:27:25 2013
+% Wed Oct 30 11:04:46 2013
\begin{table}[ht]
\centering
\begin{tabular}{rl}
@@ -1409,7 +1227,7 @@ There are 62 columns returned from the water quality portal.
\FloatBarrier
% latex table generated in R 3.0.2 by xtable 1.7-1 package
-% Mon Oct 28 16:27:25 2013
+% Wed Oct 30 11:04:46 2013
\begin{table}[ht]
\centering
\begin{tabular}{rl}
@@ -1479,7 +1297,7 @@ There are a few steps that are required in order to create a table in a Microsof
5 Suspended sediment discharge 1980-10-01
End Count Units
1 2012-05-09 529 deg C
-2 2013-10-27 24042 ft3/s
+2 2013-10-29 24044 ft3/s
3 2012-05-09 527 uS/cm @25C
4 1991-09-30 3651 mg/l
5 1991-09-30 3652 tons/day
diff --git a/vignettes/dataRetrieval.toc b/vignettes/dataRetrieval.toc
index 274fbcf6fdcc0aafb85ad13d8d2de65bdab11c45..df271abb2e8bbdce35c6550f19d8917e528dbe43 100644
--- a/vignettes/dataRetrieval.toc
+++ b/vignettes/dataRetrieval.toc
@@ -1,31 +1,31 @@
\select@language {american}
\contentsline {section}{\numberline {1}Introduction to dataRetrieval}{2}{section.1}
\contentsline {section}{\numberline {2}General USGS Web Retrievals}{3}{section.2}
-\contentsline {subsection}{\numberline {2.1}Introduction}{8}{subsection.2.1}
-\contentsline {subsection}{\numberline {2.2}Site Information}{9}{subsection.2.2}
-\contentsline {subsubsection}{\numberline {2.2.1}getSiteFileData}{9}{subsubsection.2.2.1}
-\contentsline {subsubsection}{\numberline {2.2.2}getDataAvailability}{9}{subsubsection.2.2.2}
-\contentsline {subsection}{\numberline {2.3}Parameter Information}{12}{subsection.2.3}
-\contentsline {subsection}{\numberline {2.4}Daily Values}{12}{subsection.2.4}
-\contentsline {subsection}{\numberline {2.5}Unit Values}{15}{subsection.2.5}
-\contentsline {subsection}{\numberline {2.6}Water Quality Values}{16}{subsection.2.6}
-\contentsline {subsection}{\numberline {2.7}STORET Water Quality Retrievals}{18}{subsection.2.7}
-\contentsline {subsection}{\numberline {2.8}URL Construction}{18}{subsection.2.8}
-\contentsline {section}{\numberline {3}Data Retrievals Structured For Use In The EGRET Package}{19}{section.3}
-\contentsline {subsection}{\numberline {3.1}INFO Data}{19}{subsection.3.1}
-\contentsline {subsection}{\numberline {3.2}Daily Data}{19}{subsection.3.2}
-\contentsline {subsection}{\numberline {3.3}Sample Data}{20}{subsection.3.3}
-\contentsline {subsection}{\numberline {3.4}Censored Values: Summation Explanation}{22}{subsection.3.4}
-\contentsline {subsection}{\numberline {3.5}User-Generated Data Files}{23}{subsection.3.5}
-\contentsline {subsubsection}{\numberline {3.5.1}getDailyDataFromFile}{23}{subsubsection.3.5.1}
-\contentsline {subsubsection}{\numberline {3.5.2}getSampleDataFromFile}{24}{subsubsection.3.5.2}
-\contentsline {subsection}{\numberline {3.6}Merge Report}{25}{subsection.3.6}
-\contentsline {subsection}{\numberline {3.7}EGRET Plots}{26}{subsection.3.7}
-\contentsline {section}{\numberline {4}Summary}{28}{section.4}
-\contentsline {section}{\numberline {A}Getting Started in R}{28}{appendix.A}
-\contentsline {subsection}{\numberline {A.1}New to R?}{29}{subsection.A.1}
-\contentsline {subsection}{\numberline {A.2}R User: Installing dataRetrieval}{29}{subsection.A.2}
-\contentsline {section}{\numberline {B}Columns Names}{30}{appendix.B}
-\contentsline {subsection}{\numberline {B.1}INFO dataframe}{30}{subsection.B.1}
-\contentsline {subsection}{\numberline {B.2}Water Quality Portal}{32}{subsection.B.2}
-\contentsline {section}{\numberline {C}Creating tables in Microsoft from R}{35}{appendix.C}
+\contentsline {subsection}{\numberline {2.1}Introduction}{4}{subsection.2.1}
+\contentsline {subsection}{\numberline {2.2}Site Information}{5}{subsection.2.2}
+\contentsline {subsubsection}{\numberline {2.2.1}getSiteFileData}{5}{subsubsection.2.2.1}
+\contentsline {subsubsection}{\numberline {2.2.2}getDataAvailability}{5}{subsubsection.2.2.2}
+\contentsline {subsection}{\numberline {2.3}Parameter Information}{6}{subsection.2.3}
+\contentsline {subsection}{\numberline {2.4}Daily Values}{6}{subsection.2.4}
+\contentsline {subsection}{\numberline {2.5}Unit Values}{9}{subsection.2.5}
+\contentsline {subsection}{\numberline {2.6}Water Quality Values}{10}{subsection.2.6}
+\contentsline {subsection}{\numberline {2.7}STORET Water Quality Retrievals}{12}{subsection.2.7}
+\contentsline {subsection}{\numberline {2.8}URL Construction}{12}{subsection.2.8}
+\contentsline {section}{\numberline {3}Data Retrievals Structured For Use In The EGRET Package}{13}{section.3}
+\contentsline {subsection}{\numberline {3.1}INFO Data}{13}{subsection.3.1}
+\contentsline {subsection}{\numberline {3.2}Daily Data}{13}{subsection.3.2}
+\contentsline {subsection}{\numberline {3.3}Sample Data}{14}{subsection.3.3}
+\contentsline {subsection}{\numberline {3.4}Censored Values: Summation Explanation}{16}{subsection.3.4}
+\contentsline {subsection}{\numberline {3.5}User-Generated Data Files}{17}{subsection.3.5}
+\contentsline {subsubsection}{\numberline {3.5.1}getDailyDataFromFile}{17}{subsubsection.3.5.1}
+\contentsline {subsubsection}{\numberline {3.5.2}getSampleDataFromFile}{18}{subsubsection.3.5.2}
+\contentsline {subsection}{\numberline {3.6}Merge Report}{19}{subsection.3.6}
+\contentsline {subsection}{\numberline {3.7}EGRET Plots}{20}{subsection.3.7}
+\contentsline {section}{\numberline {4}Summary}{22}{section.4}
+\contentsline {section}{\numberline {A}Getting Started in R}{24}{appendix.A}
+\contentsline {subsection}{\numberline {A.1}New to R?}{24}{subsection.A.1}
+\contentsline {subsection}{\numberline {A.2}R User: Installing dataRetrieval}{24}{subsection.A.2}
+\contentsline {section}{\numberline {B}Columns Names}{25}{appendix.B}
+\contentsline {subsection}{\numberline {B.1}INFO dataframe}{25}{subsection.B.1}
+\contentsline {subsection}{\numberline {B.2}Water Quality Portal}{27}{subsection.B.2}
+\contentsline {section}{\numberline {C}Creating tables in Microsoft from R}{30}{appendix.C}
diff --git a/vignettes/figure/egretEx.pdf b/vignettes/figure/egretEx.pdf
index caa10ccace269b25cbaefb74fe0a4cf9ab83427e..a286d810b2587cb6d161c63d79d02b07f6a28e75 100644
Binary files a/vignettes/figure/egretEx.pdf and b/vignettes/figure/egretEx.pdf differ
diff --git a/vignettes/figure/getNWIStemperaturePlot.pdf b/vignettes/figure/getNWIStemperaturePlot.pdf
index 7638a399f4604d17cf3e04a70ab85da1d300acb6..12f1c279fe581977c1f0de15430105ab8a0f8d20 100644
Binary files a/vignettes/figure/getNWIStemperaturePlot.pdf and b/vignettes/figure/getNWIStemperaturePlot.pdf differ
diff --git a/vignettes/figure/getQWtemperaturePlot.pdf b/vignettes/figure/getQWtemperaturePlot.pdf
index 868ab54872ebe7fd8d25fda94962c9d4e24d76bb..a6906d34dc4b819179d8dc9e555b66cc83bac244 100644
Binary files a/vignettes/figure/getQWtemperaturePlot.pdf and b/vignettes/figure/getQWtemperaturePlot.pdf differ