How do I retrieve USGS internal data?
EDITED AS OF dataRetrieval
2.4.0 and greater:
Within a USGS firewall, it is possible to retrieve data flagged as 'internal' by using the setAccess
function:
library(dataRetrieval)
setAccess('internal')
data <- readNWISuv("373015122071000","00095")
Original message: Within a USGS firewall, it is possible to retrieve data flagged as 'internal' using "Access=3". Using the url attribute, that can be done as follows:
library(dataRetrieval)
url <- "http://nwis.waterdata.usgs.gov/ca/nwis/uv?cb_00095=on&cb_00095=on&format=rdb&site_no=373015122071000&period=&begin_date=2013-03-06&end_date=2015-07-07&Access=3"
DS <- importRDB1(url, asDateTime=TRUE)
findData <- whatNWISdata("373015122071000")
nrow(findData)
[1] 26
urlToFind <- attr(findData, "url")
urlToFind <- paste0(urlToFind, "&Access=3")
findData2 <- importRDB1(urlToFind)
nrow(findData2)
[1] 31
Depending on the function you are initially trying to retrieve data, you'll want to check if the import is RDB (tab delimited) or WaterML1 and replicate what was happening in the original convenience function. This can be done by either inspecting the function:
readNWISdv
function (siteNumber,parameterCd,startDate="",endDate="",statCd="00003"){
url <- constructNWISURL(siteNumber,parameterCd,startDate,endDate,"dv",statCd=statCd)
data <- importWaterML1(url, asDateTime=FALSE)
if(nrow(data)>0){
data$dateTime <- as.Date(data$dateTime)
data$tz_cd <- NULL
names(data)[names(data) == "dateTime"] <- "Date"
}
return (data)
}
<environment: namespace:dataRetrieval>
So, for the readNWISdv
function, add the "&Access=3" to the url, call importWaterML1
, and convert the dateTime to a Date:
siteNumber <- '04085427'
startDate <- '2012-01-01'
endDate <- '2012-06-30'
pCode <- '00060'
rawDailyQ <- readNWISdv(siteNumber,pCode, startDate, endDate)
newURL <- attr(rawDailyQ, "url")
newURL <- paste0(newURL,"&Access=3")
newData <- importWaterML1(newURL, asDateTime=FALSE)
newData$dateTime <- as.Date(newData$dateTime)
newData$tz_cd <- NULL
names(newData)[names(newData) == "dateTime"] <- "Date"
Alternatively, the readNWISdata
could be used. For the example above:
siteNumber <- '04085427'
startDate <- '2012-01-01'
endDate <- '2012-06-30'
pCode <- '00060'
readNWISdata(service = "dv", sites=siteNumber, parameterCd=pCode,
startDate=startDate, endDate=endDate, Access=3)