get_data.Rmd

---
title: "GFv2 Get Data"
output: html_document
---

This notebook pulls data from a number of sources and populates the GFv2 data directory. Any new data requirements should be added as code chunks here. 

Each code chunk should create a path to the file you want to use in a process step, check if that path exists, and put the data there if it does not. All paths are stored in a list that is saved to the `cache` directory. If changes are made to the output of this notebook, they should be checked in.

**If resources from ScienceBase need to be downloaded Rmarkdown document should be run from RStudio so username and password authentication will work**

```{r}
library(sbtools)
library(nhdplusTools)
library(jsonlite)
data_dir <- "data"
out_list <- list("data_dir" = data_dir)
out_file <- file.path("cache", "data_paths.json")
```



```{r}
hu12_points_path <- file.path(data_dir, "hu_points.gpkg")

if(!file.exists(hu12_points_path)) {
  if(is.null(sbtools::current_session()))
    sb <- authenticate_sb()
  
  sbtools::item_file_download("5dbc53d4e4b06957974eddae", names = "hu_points.gpkg", 
                              destinations = hu12_points_path, session = sb)
}

out_list <- c(out_list, list(hu12_points_path = hu12_points_path))
```

```{r}
gagesiii_points_path <- file.path(data_dir, "GAGESIII_gages")

if(!file.exists(gagesiii_points_path)) {
  dir.create(gagesiii_points_path, recursive = TRUE, showWarnings = FALSE)
  
  if(is.null(sbtools::current_session()))
    authenticate_sb()
  
  g3z <- "GAGESIII_gages.zip"
  
  sbtools::item_file_download("5dbc53d4e4b06957974eddae", names = g3z, 
                              destinations = file.path(gagesiii_points_path, g3z))
  
  unzip(file.path(gagesiii_points_path, g3z), exdir = gagesiii_points_path)
  
  rm(g3z)
}

out_list <- c(out_list, list(gagesiii_points_path = gagesiii_points_path))
```

```{r}
nhdplus_dir <- file.path(data_dir, "NHDPlusNationalData")
if(!file.exists(nhdplus_dir)) {
  nhdplus_gdb <- download_nhdplusv2(data_dir)

  nhdplus_path(nhdplus_gdb)

  staged_nhdplus <- stage_national_data()
}

out_list <- c(out_list, list(nhdplus_dir = nhdplus_dir))
```

Don't edit the following chunk.

```{r}
write_json(out_list, path = out_file, pretty = TRUE, auto_unbox = TRUE)
```