Skip to content
Snippets Groups Projects
Commit d2605ffa authored by Laura A DeCicco's avatar Laura A DeCicco
Browse files

Latest version of vignette.

parent 482fce72
No related branches found
No related tags found
No related merge requests found
No preview for this file type
\Sconcordance{concordance:dataRetrieval.tex:dataRetrieval.Rnw:%
1 80 1 1 4 1 10 16 0 1 2 5 1 1 10 15 0 1 2 5 1 1 2 1 0 1 3 1 0 1 1 3 0 %
1 2 1 1 1 2 7 0 1 2 5 1 1 3 2 0 1 1 3 0 1 2 1 1 1 2 7 0 1 2 10 1 1 3 2 %
0 3 1 1 2 3 0 1 2 1 1 1 2 10 0 1 2 3 1 1 3 2 0 4 1 1 3 4 0 1 2 3 1 2 6 %
4 0 1 1 1 4 3 0 2 1 3 0 1 2 3 1 1 -5 1 9 13 1 1 3 2 0 2 1 1 2 1 0 1 3 5 %
0 2 2 10 0 1 2 2 1 1 6 7 0 1 2 3 1 1 -5 1 9 10 1 1 3 2 0 1 2 1 0 2 1 1 %
2 3 0 1 2 2 1 1 2 1 0 1 1 12 0 1 2 3 1 1 6 7 0 1 2 3 1 1 -5 1 9 21 1 1 %
2 1 0 3 1 1 2 7 0 2 1 1 2 11 0 1 1 19 0 1 2 31 1 1 2 1 0 3 1 12 0 1 2 %
16 1 1 2 1 0 3 1 19 0 1 2 21 1 1 3 5 0 1 2 2 1 1 4 6 0 1 2 2 1 1 4 6 0 %
1 2 3 1 1 2 4 0 1 2 6 1 1 2 1 0 1 1 3 0 1 2 1 1 1 2 4 0 1 2 5 1 1 2 1 0 %
1 2 3 1 4 0 1 3 3 1 1 2 4 0 1 3 50 0 1 2 4 1 1 2 1 0 1 1 12 0 1 2 3 1 1 %
2 1 0 1 1 47 0 1 2 3 1 1 2 1 0 3 1 1 3 1 0 1 1 15 0 1 2 3 1 1 2 1 0 1 1 %
1 2 1 0 1 3 2 0 1 1 12 0 1 2 3 1 1 2 1 0 2 1 7 0 1 1 18 0 1 2 3 1 1 2 1 %
0 2 1 68 0 1 2 3 1 1 2 1 0 1 1 17 0 1 2 3 1 1 2 1 0 1 1 18 0 1 2 20 1}
1 77 1 1 7 1 2 1 10 16 0 1 2 5 1 1 10 15 0 1 2 5 1 1 2 1 0 1 2 1 0 1 1 %
3 0 1 2 1 1 1 7 17 0 1 2 2 1 1 2 7 0 1 2 5 1 1 3 2 0 2 1 7 0 1 2 1 1 1 %
2 7 0 1 2 8 1 1 3 2 0 3 1 1 2 3 0 1 2 1 1 1 2 10 0 1 2 4 1 1 3 2 0 4 1 %
1 3 4 0 1 2 4 1 1 6 4 0 1 1 1 4 3 0 2 1 3 0 1 2 3 1 1 -5 1 9 11 1 1 2 1 %
0 2 1 1 2 1 0 1 4 6 0 2 2 10 0 1 2 3 1 1 5 7 0 1 2 3 1 1 -5 1 9 10 1 1 %
2 1 0 1 2 1 0 2 1 1 3 4 0 1 2 3 1 1 5 37 0 1 2 3 1 1 3 2 0 1 1 7 0 1 2 %
3 1 1 6 8 0 1 2 2 1 1 -4 1 8 14 1 1 2 1 0 3 1 1 2 1 1 29 0 1 2 4 1 1 2 %
1 0 1 1 28 0 1 2 2 1 1 2 4 0 1 2 1 5 27 0 1 2 3 1 1 2 12 0 1 1 19 0 1 2 %
31 1 1 2 1 0 3 1 12 0 1 2 15 1 1 2 1 0 3 1 19 0 1 2 19 1 1 3 5 0 1 2 2 %
1 1 4 6 0 1 2 2 1 1 4 6 0 1 2 3 1 1 2 4 0 1 2 6 1 1 2 1 0 1 1 3 0 1 2 1 %
1 1 2 4 0 1 2 22 1}
No preview for this file type
No preview for this file type
No preview for this file type
......@@ -257,7 +257,7 @@ A simple plotting example is shown in Figure 2:
<<label=getNWISUnit, echo=TRUE>>=
with(dischargeToday, plot(
datetime, X02_00060,
ylab="Discharge [cfs]"
ylab="Discharge [cfs]",xlab=""
))
@
\newpage
......@@ -289,13 +289,25 @@ dissolvedNitrate <- getRawQWData(siteNumber, parameterCd,
startDate, endDate)
@
There is a large amount of data returned for each observation. The available data can be viewed in Appendix 2: getRawQWData. To get a simplified dataframe that contains only datetime, value, and qualifier, use the function getQWData:
There is a large amount of data returned for each observation. The column names are listed below:
<<label=colNamesQW, echo=FALSE,results=tex>>=
infoDF <- data.frame(ColumnNames=names(dissolvedNitrate[1:31]),ColumnNames=names(dissolvedNitrate[32:62]))
data.table <- xtable(infoDF,
caption="Column names in dissolvedNitrate")
print(data.table, caption.placement="top",floating="FALSE",latex.environments=NULL)
@
\\*
To get a simplified dataframe that contains only datetime, value, and qualifier, use the function getQWData:
<<label=getQWData, echo=TRUE>>=
dissolvedNitrateSimple <- getQWData(siteNumber, parameterCd,
startDate, endDate)
names(dissolvedNitrateSimple)
@
Note that in this dataframe, datatime is only imported as Dates (no times are included), and the qualifier is either blank or \verb@"<"@ signifying a censored value.
Note that in this dataframe, datatime is imported as Dates (no times are included), and the qualifier is either blank or \verb@"<"@ signifying a censored value.
An example of plotting the above data (Figure 3):
......@@ -321,19 +333,12 @@ with(dissolvedNitrateSimple, plot(
%------------------------------------------------------------
\section{Polished Data: USGS Web Retrieval Examples}
%------------------------------------------------------------
Rather than using raw data as retrieved by the web, the dataRetrieval package also includes functions that return the data in a structure that has been designed to work with the EGRET R package (\url{https://github.com/USGS-R/EGRET/wiki}). In general, these dataframes may be much more 'R-friendly' than the raw data, and will contain additional information that allows for efficient data analysis.
Rather than using the raw data as retrieved by the web, the dataRetrieval package also includes functions that return the data in a structure that has been designed to work with the EGRET R package (\url{https://github.com/USGS-R/EGRET/wiki}). In general, these dataframes may be much more 'R-friendly' than the raw data, and will contain additional date information that allows for efficient data analysis.
In this section, we use 3 dataRetrieval functions to get sufficient data to perform an EGRET analysis. We will continue analyzing the Choptank River. We will need essentially the same data that was retrieved in the previous section, but we will get the daily discharge values in a dataframe called Daily, the nitrate sample data in a dataframe called Sample, and the data about the station and parameters in a dataframe called INFO. These are the dataframes that were exclusively designed to work with the EGRET R package, however can be very useful for all hydrologic studies.
The funtion to obtain the daily values (discharge in this case) is getDVData. It requires the inputs siteNumber, ParameterCd, StartDate, EndDate, interactive, and convert. Most of these arguments are described in the previous section, however 'convert' is a new argument, it's default is TRUE, and it tells the program to convert the values from cfs to cms. If you don't want this conversion, set convert=FALSE in the function call.
The function to obtain sample data from the water quality portal is getSampleData. The arguments for this function are also siteNumber, ParameterCd, StartDate, EndDate, interactive. These are the same inputs as getRawQWData or getQWData as described in the previous section.
The function to obtain "metadata", data about the gage station and measured parameters is getMetaData. This function essentially combines getSiteFileData and getParameterInfo, producing one dataframe called INFO.
The structure of each dataframe can be seen in Appendix 2.
<<firstExample>>=
siteNumber <- "01491000"
parameterCd <- "00631" # Nitrate
......@@ -341,9 +346,35 @@ startDate <- "1964-01-01"
endDate <- "2013-01-01"
Daily <- getDVData(siteNumber, "00060", startDate, endDate,interactive=FALSE)
summary(Daily)
@
Date is a column with dates stored in an R Date type. Julian is an integer number of days since January 1, 1850. Month and Day are integers from the start of each year. DecYear is the decimal year. MonthSeq is the (integer) number of months since 1850. Qualifier is the code from NWIS as mentioned in the raw data section. i is a count of observations. LogQ is the natural logarithm of Q. The code will shift the discharge values to 0.001 times the mean if there are zero values detected in order to perform the logarithm. Columns Q7 and Q30 are 7 and 30 day running averages.
The function to obtain sample data from the water quality portal is getSampleData. The arguments for this function are also siteNumber, ParameterCd, StartDate, EndDate, interactive. These are the same inputs as getRawQWData or getQWData as described in the previous section.
<<secondExample>>=
Sample <-getSampleData(siteNumber,parameterCd,startDate, endDate,interactive=FALSE)
summary(Sample)
@
The function to obtain "metadata", data about the gage station and measured parameters is getMetaData. This function essentially combines getSiteFileData and getParameterInfo, producing one dataframe called INFO.
<<ThirdExample>>=
INFO <-getMetaData(siteNumber,parameterCd, interactive=FALSE)
@
<<label=colNamesQW, echo=FALSE,results=tex>>=
infoDF <- data.frame(ColumnNames=names(INFO[1:21]),ColumnNames=names(c(INFO[22:41],"")))
data.table <- xtable(infoDF,
caption="Column names in the INFO dataframe")
print(data.table, caption.placement="top",floating="FALSE",latex.environments=NULL)
@
\\*
<<forthExample>>=
Sample <- mergeReport()
head(Sample)
@
......
No preview for this file type
No preview for this file type
No preview for this file type
This diff is collapsed.
\select@language {american}
\contentsline {section}{\numberline {1}Introduction to dataRetrieval}{2}{section.1}
\contentsline {section}{\numberline {2}USGS Web Retrieval Examples}{2}{section.2}
\contentsline {subsection}{\numberline {2.1}USGS Web Retrieval Introduction}{3}{subsection.2.1}
\contentsline {subsection}{\numberline {2.1}USGS Web Retrieval Introduction}{2}{subsection.2.1}
\contentsline {subsection}{\numberline {2.2}USGS Site Information Retrievals}{3}{subsection.2.2}
\contentsline {subsection}{\numberline {2.3}USGS Parameter Information Retrievals}{4}{subsection.2.3}
\contentsline {subsection}{\numberline {2.4}USGS Daily Value Retrievals}{5}{subsection.2.4}
\contentsline {subsection}{\numberline {2.5}USGS Unit Value Retrievals}{7}{subsection.2.5}
\contentsline {subsection}{\numberline {2.6}USGS Water Quality Retrievals}{9}{subsection.2.6}
\contentsline {section}{\numberline {3}Polished Data: USGS Web Retrieval Examples}{11}{section.3}
\contentsline {section}{\numberline {4}Retrieving User Generated Data Files}{14}{section.4}
\contentsline {subsection}{\numberline {4.1}getDailyDataFromFile}{14}{subsection.4.1}
\contentsline {subsection}{\numberline {4.2}getSampleDataFromFile}{15}{subsection.4.2}
\contentsline {section}{\numberline {5}Appendix 1: Getting Started}{17}{section.5}
\contentsline {subsection}{\numberline {5.1}New to R?}{17}{subsection.5.1}
\contentsline {subsection}{\numberline {5.2}R User: Installing dataRetrieval from downloaded binary}{17}{subsection.5.2}
\contentsline {subsection}{\numberline {5.3}R Developers: Installing dataRetrieval from gitHub}{18}{subsection.5.3}
\contentsline {section}{\numberline {6}Appendix 2: Dataframe column names and data types}{19}{section.6}
\contentsline {subsection}{\numberline {6.1}getSiteFileData}{19}{subsection.6.1}
\contentsline {subsection}{\numberline {6.2}getParameterInfo}{20}{subsection.6.2}
\contentsline {subsection}{\numberline {6.3}getMetaData}{20}{subsection.6.3}
\contentsline {subsection}{\numberline {6.4}retrieveNWISData}{21}{subsection.6.4}
\contentsline {subsection}{\numberline {6.5}retrieveUnitNWISData}{22}{subsection.6.5}
\contentsline {subsection}{\numberline {6.6}getDVData}{22}{subsection.6.6}
\contentsline {subsection}{\numberline {6.7}getRawQWData}{23}{subsection.6.7}
\contentsline {subsection}{\numberline {6.8}getQWData}{25}{subsection.6.8}
\contentsline {subsection}{\numberline {6.9}getSampleData}{25}{subsection.6.9}
\contentsline {section}{\numberline {4}Retrieving User-Generated Data Files}{16}{section.4}
\contentsline {subsection}{\numberline {4.1}getDailyDataFromFile}{16}{subsection.4.1}
\contentsline {subsection}{\numberline {4.2}getSampleDataFromFile}{17}{subsection.4.2}
\contentsline {section}{\numberline {5}Appendix 1: Getting Started}{19}{section.5}
\contentsline {subsection}{\numberline {5.1}New to R?}{19}{subsection.5.1}
\contentsline {subsection}{\numberline {5.2}R User: Installing dataRetrieval from downloaded binary}{19}{subsection.5.2}
\contentsline {subsection}{\numberline {5.3}R Developers: Installing dataRetrieval from gitHub}{20}{subsection.5.3}
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment