diff --git a/README.md b/README.md index 21a9d6eda7dd63ebc1bb94ba7356630b1b55dcf5..c39c248e5e7f2cf95e5eda26fd19bbe17541f28c 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,9 @@ Geomag algorithms includes tools to fetch, process, and output geomag data. ### [Install](./docs/install.md) First time install. Walk through dependencies and other considerations. +Document: [/docs/install.md](./docs/install.md) + + ### [Command Line Usage](./docs/usage.md) Details and examples for proper usage. Get started quickly. @@ -17,13 +20,21 @@ Basic usage: - Use the main script, `geomag.py -h` - In python scripts, `import geomagio` +Document: [/docs/usgage.md](./docs/usage.md) + + ### [Python API](./docs/api.md) API Overview and examples for using the geomag-algorithms project in python. -### [Develop](./docs/develop_install.md) +Document: [/docs/api.md](./docs/api.md) + + +### [Develop](./docs/develop.md) Development dependencies discussed here. Project is built with Grunt and Node and is written primarily in Python 2.7. +Document: [/docs/develop.md](./docs/develop.md) + ## Supported Formats diff --git a/docs/develop.md b/docs/develop.md index 56b824e1b9da3be93a1d72ddb04720db9b0ebb84..2a861bd0a633cde084162d866347df7dbed64162 100644 --- a/docs/develop.md +++ b/docs/develop.md @@ -31,14 +31,12 @@ Begin Developing ### Windows -1. Install the newest release of [Node (http://nodejs.org/download/)][] for - Windows, using the Windows Installer (.msi). - - All of the defaults should be sufficient. +1. Install the newest release of Node + [(http://nodejs.org/download/)](http://nodejs.org/download/) for Windows, + using the Windows Installer (.msi). 1. Close and re-open your terminal so that your new PATH is loaded. -[Node]: http://nodejs.org/download/ - ### Coding Standards @@ -51,27 +49,3 @@ This project adheres to PEP8 standards in most cases: - line continuations should use two indentations (8 spaces). - do not use visual indents. - - -### Add an SSH Key to GitHub - -1. `ssh-keygen -t rsa -b 4096` (in terminal) - -2. Press **Enter** to accept the default save location. - -3. Enter a passphrase that you will remember. - -4. `cat ~/.ssh/id_rsa.pub` - Copy the text block that is displayed. - This is your SSH key. - If you're on Window and can't copy the text from the terminal, go - to `C:\Users\[your username here]\.ssh` and open the `id_rsa.pub` file - with notepad. - -5. In GitHub, click **Edit Your Profile**. - -6. Select **SSH Keys** on the left. - -7. Click **Add SSH key**. Give it a meaningful title. - -8. Copy your SSH Key into the Key, and click **Add key**. diff --git a/docs/install.md b/docs/install.md index acc78ec56d532d76218565d03f0f024842b4c5c1..c3ff9cb0dca1f3a2023257105fb2908520d492cc 100644 --- a/docs/install.md +++ b/docs/install.md @@ -1,15 +1,14 @@ Installation ============ -We recommend using -[virtual environments (http://docs.python-guide.org/en/latest/dev/virtualenvs/)][]. +We recommend using virtual environments: +[(http://docs.python-guide.org/en/latest/dev/virtualenvs/)](http://docs.python-guide.org/en/latest/dev/virtualenvs/) -Check out the [Mac](#mac) and [Windows](#windows) details below for more -specific dependency information. +[Mac](#mac) and [Windows](#windows) specific details are below. 1. Install __geomagio__ - pip install git+https://github.com/usgs/geomag-algorithms.git + pip install git+https://github.com/usgs/geomag-algorithms.git 1. Use __geomagio__ @@ -19,25 +18,25 @@ specific dependency information. The [Command Line Usage](./usage.md) and [Python API](./api.md) pages have more instructions and examples. -[virtual environments]: http://docs.python-guide.org/en/latest/dev/virtualenvs/ - --- ### Mac ### 1. Install `python` (2.7.X) - On OS X, we recommend using Homebrew ( http://brew.sh/ ) + On OS X, we recommend using Homebrew [http://brew.sh/](http://brew.sh/) 1. Install numpy, obspy requires this be installed separately. pip install numpy 1. Install `node`, `git` and `python` (2.7.X). - On OS X, we recommend using [Homebrew (http://brew.sh/)][] - brew install node - brew install git + On OS X, we recommend using Homebrew +``` +brew install node +brew install git +``` 1. Use pip to install `numpy`, `scipy`, `obspy`, and `flake8` @@ -54,30 +53,18 @@ pages have more instructions and examples. 1. Close and re-open your terminal so that your new PATH is loaded. Make sure to navigate back to your `geomag-algorithms` project directory. -[Homebrew]: http://brew.sh/ --- ### Windows ### -1. You will need a terminal tool for Windows. - [Git Bash (http://git-scm.com/download/win)][] was used to test - these steps, but [Cygwin (http://cygwin.com/install.html)][] or another - unix-like editor should work too. - - - Choosing PATH environment: We recommend the __last option__ here to include - Unix tools, but if you don't understand what that entails then use the - _second option_ which still adds Git to your system PATH. - - Besides that, all of the defaults should be sufficient. +1. You will need a terminal tool for Windows. We recommend Git Bash + [(http://git-scm.com/download/win)](http://git-scm.com/download/win), but + another unix-like editor should work too. -1. Install [Anaconda (http://continuum.io/downloads)][], which includes - `numpy`, `scipy` and `flake8`. - - We recommend using _"Just Me"_ for the install to avoid potential admin - issues. +1. Install Anaconda + [(http://continuum.io/downloads)](http://continuum.io/downloads), which + includes `numpy`, `scipy` and `flake8`. 1. Run `python` to verify that Anaconda's version of Python is being used. If it isn't, update your PATH so that Conda's HOME is before any other versions of Python. - -[Git Bash]: http://git-scm.com/download/win -[Cygwin]: http://cygwin.com/install.html -[Anaconda]: http://continuum.io/downloads diff --git a/docs/io.md b/docs/io.md index e2ae9cfea7522b8264736fdcef60c43580d7db8b..cd75237ae154126b211388dc4d4a7b707dbbde4d 100644 --- a/docs/io.md +++ b/docs/io.md @@ -4,105 +4,136 @@ IO Formats #### EDGE `--input-edge HOST PORT` + Specify an IP address or host name where your Edge lives along with a port. + #### Iaga2002 `--input-iaga-file FILENAME` + Specify the name of the file to read from. `--input-iaga-magweb` + Data will be pulled from geomag.usgs.gov/data/magnetometer if it exists. `--input-iaga-stdin` + Use redirection on the command line to send your data in. `--input-iaga-url URL` + Use a file pattern to read from multiple IAGA2002 files. `--output-iaga-file FILENAME` + Specify the name of the file to write to. `--output-iaga-stdout` + Output will be send directly to the command line. `--output-iaga-url URL` + Use a file pattern to write to multiple IAGA2002 files. + #### IMFV283 `--input-imfv283-file FILENAME` + Specify the name of the file to read from. `--input-imfv283-stdin` + Use redirection on the command line to send your data in. `--input-imfv283-url URL` + Use a file pattern to read IMFV283 file. `--input-imfv283-goes` + Reads IMFV283 data from an internet goes server. + #### PCDCP `--input-pcdcp-file FILENAME` + Specify the name of the file to read from. `--input-pcdcp-stdin` + Use redirection on the command line to send your data in. `--input-pcdcp-url URL` + Use a file pattern to read from multiple PCDCP files. `--output-pcdcp-file FILENAME` + Specify the name of the file to write to. `--output-pcdcp-stdout` + Output will be send directly to the command line. `--output-pcdcp-url` + Use a file pattern to write to multiple PCDCP files. ## IO Methods + Several methods exist for retrieving and storing data. #### Edge Server `--input-edge HOST PORT` + Specify an IP address or host name where your Edge lives along with a port. #### Single File -`--input-iaga-file FILENAME` -`--input-imfv283-file FILENAME` -`--input-pcdcp-file FILENAME` -`--output-iaga-file FILENAME` -`--output-pcdcp-file FILENAME` +``` +--input-iaga-file FILENAME +--input-imfv283-file FILENAME +--input-pcdcp-file FILENAME +--output-iaga-file FILENAME +--output-pcdcp-file FILENAME +``` + Specify a single file name for the data to be read from or written to. #### Multiple Files -`--input-iaga-url` -`--input-imfv283-url` -`--input-pcdcp-url` -`--output-iaga-url` -`--output-pcdcp-url` +``` +--input-iaga-url +--input-imfv283-url +--input-pcdcp-url +--output-iaga-url +--output-pcdcp-url +``` + URLs can be used to fetch or store groups of data using pattern matching. In order to use a directory of files on a local machine, just specify "file://" at the beginning of the pattern. Patterns that will be matched with information from the data: - - __%(i)s__ : interval abbreviation (sec, min, hor, etc.) - - __%(interval)s__: interval name (second, minute, hour, etc.) - - __%(julian)s__ : julian day formatted as JJJ - - __%(obs)s__ : lowercase observatory 3-letter code - - __%(OBS)s__ : uppercase observatory 3-letter code - - __%(t)s__ : type abbreviation (v, q, d, etc.) - - __%(type)s__ : type name (variation, quasi-definitive, definitive, etc.) - - __%(year)s__ : year formatted as YYYY - - __%(ymd)s__ : time formatted as YYYYMMDD +| Pattern | Purpose | +| ----------------- | -------------------------------------------------------- | +| __%(i)s__ | interval abbreviation (sec, min, hor, etc.) | +| __%(interval)s__ | interval name (second, minute, hour, etc.) | +| __%(julian)s__ | julian day formatted as JJJ | +| __%(obs)s__ | lowercase observatory 3-letter code | +| __%(OBS)s__ | uppercase observatory 3-letter code | +| __%(t)s__ | type abbreviation (v, q, d, etc.) | +| __%(type)s__ | type name (variation, quasi-definitive, definitive, etc.)| +| __%(year)s__ | year formatted as YYYY | +| __%(ymd)s__ | time formatted as YYYYMMDD | Typical IAGA2002 files are stored as `file://./%(obs)s%(ymd)s%(t)%(i)s.%(i)s` @@ -114,10 +145,12 @@ Example: BOU2013092.min #### Std In and Std Out -`--input-iaga-stdin` -`--input-imfv283-stdin` -`--input-pcdcp-stdin` -`--output-iaga-stdout` -`--output-pcdcp-stdout` +``` +--input-iaga-stdin +--input-imfv283-stdin +--input-pcdcp-stdin +--output-iaga-stdout +--output-pcdcp-stdout +``` For standard in, pass the data in with redirection. diff --git a/docs/usage.md b/docs/usage.md index a0bc321e649eef05c34694469449a2133f452330..ea3ce2ba26e1604356e2930f9bcef17585f936d6 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -10,22 +10,25 @@ input parameters. Some of the key parameters that should be specified are listed here along with examples. They include - - __start time__ - YYYY-MM-DDTHH:MM:SSZ - - __end time__ - YYYY-MM-DDTHH:MM:SSZ - - __observatory code__ - 3-letter observatory code - - __data input source__ - {--input-edge, --input-iaga-file, ...} - - __data output source__ - {--output-pcdcp-url, --output-iaga-file, ...} - - __input data channels__ - {[CHANNEL [CHANNEL ...]]} - - __output data channels__ - {[CHANNEL [CHANNEL ...]]} - - __data type__ - {variation, quasi-definitive, definitive} - - __interval__ - {minute, second, hour, day} +| Parameter | Format | +| -------------------------- | -------------------------------------------- | +| __start time__ | YYYY-MM-DDTHH:MM:SSZ | +| __end time__ | YYYY-MM-DDTHH:MM:SSZ | +| __observatory code__ | 3-letter observatory code | +| __data input source__ | {--input-edge, --input-iaga-file, ...} | +| __data output source__ | {--output-pcdcp-url, --output-iaga-file, ...}| +| __input data channels__ | {[CHANNEL [CHANNEL ...]]} | +| __output data channels__ | {[CHANNEL [CHANNEL ...]]} | +| __data type__ | {variation, quasi-definitive, definitive} | +| __interval__ | {minute, second, hour, day} | + ### Examples ### -To retrieve all _raw_ (variation) _H_, _E_, _Z_ and _F_ _minute_ data from -_Boulder Observatory_ for the entire day of _July 1st 2014_ from an _iaga2002_ -formatted file and output _H_, _E_, _Z_ and _F_ data to a _PCDCP_ formatted -file: +To retrieve all raw (**_variation_**) **_H_**, **_E_**, **_Z_** and **_F_** +**_minute_** data from Boulder Observatory (**_BOU_**) for the entire day of +**_July 1st 2014_** from an **_iaga2002_** formatted file and output +**_H_**, **_E_**, **_Z_** and **_F_** data to a **_pcdcp_** formatted file: geomag.py --type variation --inchannels H E Z F --interval minute \ --observatory BOU \ @@ -35,10 +38,11 @@ file: --outchannels H E Z F \ --output-pcdcp-file BOU2014182.min -To retrieve all _raw_ (variation) _H_, _E_, _Z_ and _F_ _minute_ data from -_Tucson Observatory_ for the entire month of _March 2013_ from _pcdcp_ -formatted files in a "data-pcdcp" directory and output _H_, _E_, _Z_ and _F_ -data to a group of _iaga2002_ formatted files in a "data-iaga" directory: +To retrieve all raw (**_variation_**) **_H_**, **_E_**, **_Z_** and **_F_** +**_minute_** data from Tucson Observatory (**_TUC_**) for the entire month of +**_March 2013_** from a group of **_pcdcp_** formatted files in a "data-pcdcp" +directory and output **_H_**, **_E_**, **_Z_** and **_F_** data to a group of +**_iaga2002_** formatted files in a "data-iaga" directory: geomag.py --type variation --inchannels H E Z F --interval minute \ --observatory TUC \ @@ -48,9 +52,10 @@ data to a group of _iaga2002_ formatted files in a "data-iaga" directory: --output-iaga-url file://data-iaga/./$(obs)s%(Y)s%(j)s.%(i)s \ --outchannels H E Z F -To retrieve all _Dst 4 minute_, and _Dst3 minute_ data from _USGS_ for the -entire day of _Oct 1st 2015_ from _edge server cwbpub.cr.usgs.gov_ output -_Dst 4 minute_, and _Dst 3 minute_ data to an _iaga2002_ formatted file: +To retrieve all **_Dst 4 minute_**, and **_Dst 3 minute_** data from **_USGS_** +for the entire day of **_Oct 1st 2015_** from an **_edge server_** (at +cwbpub.cr.usgs.gov) and output **_Dst 4 minute_**, and **_Dst 3 minute_** +data to an **_iaga2002_** formatted file: geomag.py --type variation --inchannels MGD MSD --interval minute \ --observatory USGS \ @@ -69,11 +74,16 @@ There are flags to specify certain algorithms should be run against the data. #### XYZ #### `--algorithm xyz` + `--xyz-from {geo, mag, obs, obsd}` (default is `obs`) + `--xyz-to {geo, mag, obs, obsd}` (default is `geo`) #### [XYZ Usage](./algorithms/XYZ_usage.md) #### + Rotate data from HEZ (obs) or HDZ (mag) to XYZ (geo) and back. +Document: [/algorithms/XYZ_usage.md](./algorithms/XYZ_usage.md) + Extensive explanation of all input and output methods: [IO Methods](./io.md)