Maintenance scheduled for Thursday, October 22nd at 15:00 MDT. Expected downtime <1 hour.

Commit f19b605c authored by Blodgett, David L.'s avatar Blodgett, David L.

fixes #59

parent aabef776
This diff is collapsed.
......@@ -232,7 +232,7 @@
<li>
<code>data_frames</code> is a list containing one <code>data.frame</code> for each variable read from the NetCDF file.</li>
<li>
<code>global_attributes</code> contains standard glabal attributes found in the file. All of the variables that have one element per timeseries variable, are named the same as the NetCDF variable names so they can be accessed by name as shown below.</li>
<code>global_attributes</code> contains standard global attributes found in the file. All of the variables that have one element per timeseries variable, are named the same as the NetCDF variable names so they can be accessed by name as shown below.</li>
</ol>
</div>
......
......@@ -81,7 +81,7 @@
<p><code>ncdfgeom</code> reads and writes geometry data (points lines and polygons), attributes of geometries, and time series associated with the geometries in a standards-compliant way.</p>
<p>It implements the NetCDF-CF Spatial Geometries specification and the timeSeries feature type of the <a href="http://cfconventions.org/cf-conventions/cf-conventions.html#discrete-sampling-geometries">Discrete Sampling Geometry</a> NetCDF-CF specification.</p>
<p><strong>Visit the <a href="http://usgs-r.github.io/ncdfgeom/dev/articles/ncdfgeom.html"><code>pkgdown</code> site</a> for a complete overview of the package.</strong></p>
<p><strong>Visit the <a href="http://usgs-r.github.io/ncdfgeom/articles/ncdfgeom.html"><code>pkgdown</code> site</a> for a complete overview of the package.</strong></p>
<p>Given that this package is fairly new and in active development, please test it out and consider <a href="https://github.com/USGS-R/ncdfgeom/issues">submitting issues and/or contributions!</a></p>
<div id="installation" class="section level2">
<h2 class="hasAnchor">
......
......@@ -32,7 +32,7 @@
<meta property="og:title" content="Read NetCDF-CF spatial geometries — read_geometry" />
<meta property="og:description" content="Attemps to convert a NetCDF-CF DSG Simple Geometry file into a sf data.frame." />
<meta property="og:description" content="Attempts to convert a NetCDF-CF DSG Simple Geometry file into a sf data.frame." />
<meta name="twitter:card" content="summary" />
......@@ -89,7 +89,7 @@
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="../articles/geometry.html">Reading and Writing Geometry</a>
<a href="../articles/geometry.html">Reading and Writing NetCDF-CF Geometry</a>
</li>
<li>
<a href="../articles/timeseries.html">Reading and Writing Timeseries</a>
......@@ -119,7 +119,7 @@
<div class="ref-description">
<p>Attemps to convert a NetCDF-CF DSG Simple Geometry file into a sf data.frame.</p>
<p>Attempts to convert a NetCDF-CF DSG Simple Geometry file into a sf data.frame.</p>
</div>
......
......@@ -91,7 +91,7 @@ Will also add attributes if provided data has them." />
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="../articles/geometry.html">Reading and Writing Geometry</a>
<a href="../articles/geometry.html">Reading and Writing NetCDF-CF Geometry</a>
</li>
<li>
<a href="../articles/timeseries.html">Reading and Writing Timeseries</a>
......@@ -151,7 +151,7 @@ use package default -- "instance" -- if not supplied.</p></td>
</tr>
<tr>
<th>variables</th>
<td><p><code>character</code> If a an existing netcdf files is provided, this
<td><p><code>character</code> If a an existing netCDF files is provided, this
list of variables that should be related to the geometries.</p></td>
</tr>
</table>
......@@ -170,7 +170,7 @@ list of variables that should be related to the geometries.</p></td>
<span class='no'>hucPolygons_nc</span> <span class='kw'>&lt;-</span> <span class='kw pkg'>ncdfgeom</span><span class='kw ns'>::</span><span class='fu'><a href='https://www.rdocumentation.org/packages/ncdfgeom/topics/write_geometry'>write_geometry</a></span>(<span class='kw'>nc_file</span><span class='kw'>=</span><span class='fu'><a href='https://www.rdocumentation.org/packages/base/topics/tempfile'>tempfile</a></span>(),
<span class='kw'>geom_data</span> <span class='kw'>=</span> <span class='no'>hucPolygons</span>)
<span class='no'>ncdump</span> <span class='kw'>&lt;-</span> <span class='fu'><a href='https://www.rdocumentation.org/packages/base/topics/system'>system</a></span>(<span class='fu'><a href='https://www.rdocumentation.org/packages/base/topics/paste'>paste</a></span>(<span class='st'>"ncdump -h"</span>, <span class='no'>hucPolygons_nc</span>), <span class='kw'>intern</span> <span class='kw'>=</span> <span class='fl'>TRUE</span>)
<span class='fu'><a href='https://www.rdocumentation.org/packages/base/topics/cat'>cat</a></span>(<span class='no'>ncdump</span> ,<span class='kw'>sep</span> <span class='kw'>=</span> <span class='st'>"\n"</span>)</div><div class='output co'>#&gt; netcdf file82433c68c282 {
<span class='fu'><a href='https://www.rdocumentation.org/packages/base/topics/cat'>cat</a></span>(<span class='no'>ncdump</span> ,<span class='kw'>sep</span> <span class='kw'>=</span> <span class='st'>"\n"</span>)</div><div class='output co'>#&gt; netcdf fileee7d62399989 {
#&gt; dimensions:
#&gt; instance = 2 ;
#&gt; char = 38 ;
......
......@@ -92,7 +92,7 @@ timesteps per instance (e.g. geometry or station)." />
</a>
<ul class="dropdown-menu" role="menu">
<li>
<a href="../articles/geometry.html">Reading and Writing Geometry</a>
<a href="../articles/geometry.html">Reading and Writing NetCDF-CF Geometry</a>
</li>
<li>
<a href="../articles/timeseries.html">Reading and Writing Timeseries</a>
......@@ -132,7 +132,8 @@ timesteps per instance (e.g. geometry or station).</p>
<pre class="usage"><span class='fu'>write_timeseries_dsg</span>(<span class='no'>nc_file</span>, <span class='no'>instance_names</span>, <span class='no'>lats</span>, <span class='no'>lons</span>, <span class='no'>times</span>, <span class='no'>data</span>,
<span class='kw'>alts</span> <span class='kw'>=</span> <span class='fl'>NA</span>, <span class='kw'>data_unit</span> <span class='kw'>=</span> <span class='st'>""</span>, <span class='kw'>data_prec</span> <span class='kw'>=</span> <span class='st'>"double"</span>,
<span class='kw'>data_metadata</span> <span class='kw'>=</span> <span class='fu'><a href='https://www.rdocumentation.org/packages/base/topics/list'>list</a></span>(<span class='kw'>name</span> <span class='kw'>=</span> <span class='st'>"data"</span>, <span class='kw'>long_name</span> <span class='kw'>=</span> <span class='st'>"unnamed data"</span>),
<span class='kw'>attributes</span> <span class='kw'>=</span> <span class='fu'><a href='https://www.rdocumentation.org/packages/base/topics/list'>list</a></span>(), <span class='kw'>add_to_existing</span> <span class='kw'>=</span> <span class='fl'>FALSE</span>, <span class='kw'>overwrite</span> <span class='kw'>=</span> <span class='fl'>FALSE</span>)</pre>
<span class='kw'>time_units</span> <span class='kw'>=</span> <span class='st'>"days since 1970-01-01 00:00:00"</span>, <span class='kw'>attributes</span> <span class='kw'>=</span> <span class='fu'><a href='https://www.rdocumentation.org/packages/base/topics/list'>list</a></span>(),
<span class='kw'>add_to_existing</span> <span class='kw'>=</span> <span class='fl'>FALSE</span>, <span class='kw'>overwrite</span> <span class='kw'>=</span> <span class='fl'>FALSE</span>)</pre>
<h2 class="hasAnchor" id="arguments"><a class="anchor" href="#arguments"></a>Arguments</h2>
<table class="ref-arguments">
......@@ -182,6 +183,10 @@ Valid options: 'short' 'integer' 'float' 'double' 'char'.</p></td>
<th>data_metadata</th>
<td><p><code>list</code> A named list of strings: list(name='ShortVarName', long_name='A Long Name')</p></td>
</tr>
<tr>
<th>time_units</th>
<td><p><code>character</code> units string in udunits format to use for time. Defaults to 'days since 1970-01-01 00:00:00'</p></td>
</tr>
<tr>
<th>attributes</th>
<td><p>list An optional list of attributes that will be added at the global level.
......
......@@ -38,8 +38,8 @@ Additional read / write functions to include additional DSG feature types will b
At the time of writing, installation is only available via `devtools` or building the package directly as one would for development purposes.
```
install.packages("devtools")
devtools::install_github("USGS-R/ncdfgeom")
install.packages("remotes")
remotes::install_github("USGS-R/ncdfgeom")
```
When on CRAN, the package will be installed with the typical install.packages method.
......@@ -48,80 +48,42 @@ When on CRAN, the package will be installed with the typical install.packages me
install.packages("ncdfgeom")
```
## Input Data
For this demo, we'll use `sf`, `dplyr`, and `ncdfgeom`.
We will start with two dataframes: **`prcp_data` and `climdiv_poly`.**
Code to download and prep these data is shown below. More info: [doi:10.7289/V5M32STR](https://doi.org/10.7289/V5M32STR)
```{r libs, echo=FALSE, include=FALSE, message=FALSE, warning=FALSE}
library(dplyr)
library(tidyr)
```{r libs, message=FALSE, warning=FALSE}
library(sf)
library(dplyr)
library(ncdfgeom)
```
```{r setup_secret, echo = FALSE}
# Description here: ftp://ftp.ncdc.noaa.gov/pub/data/cirs/climdiv/divisional-readme.txt
prcp_url <- "ftp://ftp.ncdc.noaa.gov/pub/data/cirs/climdiv/climdiv-pcpndv-v1.0.0-20190408"
prcp_file <- "prcp.txt"
if(!file.exists(prcp_file)) {
download.file(url = prcp_url, destfile = prcp_file, quiet = TRUE)
}
division_url <- "ftp://ftp.ncdc.noaa.gov/pub/data/cirs/climdiv/CONUS_CLIMATE_DIVISIONS.shp.zip"
division_file <- "CONUS_CLIMATE_DIVISIONS.shp.zip"
if(!file.exists(division_file)) {
download.file(url = division_url, destfile = division_file, quiet = TRUE)
unzip("CONUS_CLIMATE_DIVISIONS.shp.zip")
}
## Sample Data
suppressWarnings(climdiv_poly <- read_sf("GIS.OFFICIAL_CLIM_DIVISIONS.shp") %>%
select("CLIMDIV", CLIMDIV_NAME = "NAME") %>%
mutate(CLIMDIV = ifelse(nchar(as.character(CLIMDIV)) == 3,
paste0("0",as.character(CLIMDIV)),
as.character(CLIMDIV))) %>%
st_simplify(dTolerance = 0.0125))
We will start with two dataframes: **`prcp_data` and `climdiv_poly`** containing precipitation estimate timeseries and polygon geometries that describe the boundaries of climate divisions respectively. The precipitation data has one time series for each geometry.
month_strings <- c("jan", "feb", "mar", "apr", "may", "jun",
"jul", "aug", "sep", "oct", "nov", "dec")
prcp_data <- read.table(prcp_file, header = FALSE,
colClasses = c("character",
rep("numeric", 12)),
col.names = c("meta", month_strings))
# Here we gather the data into a long format and prep it for ncdfgeom.
prcp_data <- prcp_data %>%
gather(key = "month", value = "precip_inches", -meta) %>%
mutate(climdiv = paste0(substr(meta, 1, 2), substr(meta, 3, 4)),
year = substr(meta, 7, 10),
precip_inches = ifelse(precip_inches < 0, NA, precip_inches)) %>%
mutate(date = as.Date(paste0("01", "-", month, "-", year),
format = "%d-%b-%Y")) %>%
select(-meta, -month, -year) %>%
filter(climdiv %in% climdiv_poly$CLIMDIV) %>%
spread(key = "climdiv", value = "precip_inches") %>%
as_tibble()
Code to download and prep these data is shown at the bottom. More info about the data can be found at: [doi:10.7289/V5M32STR](https://doi.org/10.7289/V5M32STR)
# Now make sure things are in the same order.
climdiv_names <- names(prcp_data)[2:length(names(prcp_data))]
climdiv_row_order <- match(climdiv_names, climdiv_poly$CLIMDIV)
climdiv_poly <- climdiv_poly[climdiv_row_order, ]
```
The data required for this demo has been cached in the `ncdfgeom` package and is available as shown below.
### `prcp_data`
```{r data_shape_2, echo=FALSE}
options(tibble.width = 50, tibble.print_max = 10)
```{r data_shape_2}
prcp_data <- readRDS(system.file("extdata/climdiv-pcpndv.rds", package = "ncdfgeom"))
print(prcp_data, n_extra = 0)
plot(prcp_data$date, prcp_data$`0101`, col = "red",
xlab = "date", ylab = "monthly precip (inches)", main = "Sample Timeseries for 0101-'Northern Valley'")
lines(prcp_data$date, prcp_data$`0101`)
```
### `climdiv_poly`
```{r data_shape, echo=FALSE}
climdiv_poly
```{r data_shape}
climdiv_poly <- read_sf(system.file("extdata/climdiv.gpkg", package = "ncdfgeom"))
print(climdiv_poly)
plot(st_geometry(climdiv_poly), main = "Climate Divisions with 0101-'Northern Valley' Highlighted")
plot(st_geometry(filter(climdiv_poly, CLIMDIV == "0101")), col = "red", add = TRUE)
```
As shown above, we have two `data.frame`s. One has 344 columns and the other 344 rows. These 344 climate divisions will be our "instance" dimension when we write to NetCDF.
## Write Timeseries and Geometry to NetCDF
The NetCDF discrete sampling geometries timeseries standard requires point lat/lon coordinate locations for timeseries data. In the code below, we calculate these values and write the timeseries data to a netcdf file.
```{r write_ts, warning = FALSE}
......@@ -150,27 +112,29 @@ write_timeseries_dsg(nc_file = nc_file,
data_prec = "float",
data_metadata = prcp_meta,
attributes = list(title = "Demonstation of ncdfgeom"),
add_to_existing = FALSE) -> nc_file
add_to_existing = FALSE)
climdiv_poly <- st_sf(st_cast(climdiv_poly, "MULTIPOLYGON"))
write_geometry(nc_file = "climdiv_prcp.nc",
geom_data = climdiv_poly,
variables = "climdiv_prcp_inches") -> nc_file
variables = "climdiv_prcp_inches")
```
Now we have a file with a structure as shown in the `ncdump` output below.
```{r ncdump, echo=FALSE}
```{r ncdump}
ncdump <- system(paste("ncdump -h", nc_file), intern = TRUE)
cat(ncdump ,sep = "\n")
cat(ncdump, sep = "\n")
```
For more information about the polygon and timeseries data structures used here, see the [NetCDF-CF standard.](http://cfconventions.org/cf-conventions/cf-conventions.html)
## Read Data
## Read Timeseries and Geometry from NetCDF
Now that we have all our data in a single file, we can read it back in.
```{r read}
# First read the timeseries.
prcp_data <- read_timeseries_dsg("climdiv_prcp.nc")
# Now read the geometry.
climdiv_poly <- read_geometry("climdiv_prcp.nc")
```
......@@ -186,8 +150,6 @@ prcp_data$global_attributes
names(climdiv_poly)
```
To better understand what is actually in this file, let's visualize the data we just read. Below we join a sum of the precipitation timeseries to the climate division polygons and plot them up.
```{r p_colors_source, echo=FALSE}
# Because we've gotta have pretty colors!
p_colors <- function (n, name = c("precip_colors")) {
......@@ -214,6 +176,8 @@ p_colors <- function (n, name = c("precip_colors")) {
}
```
To better understand what is actually in this file, let's visualize the data we just read. Below we join a sum of the precipitation timeseries to the climate division polygons and plot them up.
```{r plot, fig.height=6, fig.width=8}
climdiv_poly <- climdiv_poly %>%
st_transform(3857) %>% # web mercator
......@@ -241,27 +205,27 @@ plot(prcp["prcp"], lwd = 0.1, pal = p_colors,
```
This is the code used to download and prep the precipitation data.
```{r setup, eval = FALSE}
This is the code used to download and prep the precipitation and spatial data. Provided for reproducibility and is not run here.
```{r setup_secret, eval = FALSE}
# Description here: ftp://ftp.ncdc.noaa.gov/pub/data/cirs/climdiv/divisional-readme.txt
prcp_url <- "ftp://ftp.ncdc.noaa.gov/pub/data/cirs/climdiv/climdiv-pcpndv-v1.0.0-20190408"
prcp_file <- "prcp.txt"
if(!file.exists(prcp_file)) {
download.file(url = prcp_url, destfile = prcp_file, quiet = TRUE)
}
download.file(url = prcp_url, destfile = prcp_file, quiet = TRUE)
division_url <- "ftp://ftp.ncdc.noaa.gov/pub/data/cirs/climdiv/CONUS_CLIMATE_DIVISIONS.shp.zip"
division_file <- "CONUS_CLIMATE_DIVISIONS.shp.zip"
if(!file.exists(division_file)) {
download.file(url = division_url, destfile = division_file, quiet = TRUE)
unzip("CONUS_CLIMATE_DIVISIONS.shp.zip")
}
download.file(url = division_url, destfile = division_file, quiet = TRUE)
unzip("CONUS_CLIMATE_DIVISIONS.shp.zip")
climdiv_poly <- read_sf("GIS.OFFICIAL_CLIM_DIVISIONS.shp") %>%
select("CLIMDIV", CLIMDIV_NAME = "NAME") %>%
mutate(CLIMDIV = ifelse(nchar(as.character(CLIMDIV)) == 3,
paste0("0",as.character(CLIMDIV)),
as.character(CLIMDIV)))
as.character(CLIMDIV))) %>%
st_simplify(dTolerance = 0.0125)
month_strings <- c("jan", "feb", "mar", "apr", "may", "jun",
"jul", "aug", "sep", "oct", "nov", "dec")
......@@ -281,12 +245,21 @@ prcp_data <- prcp_data %>%
format = "%d-%b-%Y")) %>%
select(-meta, -month, -year) %>%
filter(climdiv %in% climdiv_poly$CLIMDIV) %>%
spread(key = "climdiv", value = "precip_inches")
spread(key = "climdiv", value = "precip_inches") %>%
filter(!is.na(`0101`))
as_tibble()
# Now make sure things are in the same order.
climdiv_names <- names(prcp_data)[2:length(names(prcp_data))]
climdiv_row_order <- match(climdiv_names, climdiv_poly$CLIMDIV)
climdiv_poly <- climdiv_poly[climdiv_row_order, ]
sf::write_sf(climdiv_poly, "climdiv.gpkg")
saveRDS(prcp_data, "climdiv-pcpndv.rds")
unlink("GIS*")
unlink("CONUS_CLIMATE_DIVISIONS.shp.zip")
unlink("prcp.txt")
```
Here's the `p_colors` function used in plotting above.
......@@ -315,8 +288,5 @@ p_colors <- function (n, name = c("precip_colors")) {
}
```
```{r cleanup, echo=FALSE}
unlink("GIS*")
unlink("CONUS_CLIMATE_DIVISIONS.shp.zip")
unlink("prcp.txt")
unlink("climdiv_prcp.nc")
```
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment