diff --git a/README.md b/README.md
index c218923adeab3457d599284e4d3ba7b10411e761..969d24c6030d3ed5b582610e6b168574a2ad45cf 100644
--- a/README.md
+++ b/README.md
@@ -3,64 +3,42 @@ Geomag Algorithms
Geomag algorithms includes tools to fetch, process, and output geomag data.
-[Documentation](./docs/README.md)
+## Supported Formats ##
+### [Edge](readme_io.md#edge) ###
-Install
--------
+Use an Edge server for data input.
-We recommend using [virtual environments][].
+### [IAGA](readme_io.md#iaga2002)###
-__Windows__: begin with the Windows dependencies in the
-[Dependency install](readme_dependency_install.md#windows).
+Use IAGA2002 formatted files for input and/or output.
-__Mac/Linux__: begin with required dependencies in the
-[Dependency install](readme_dependency_install.md#mac).
+### [PCDCP](readme_io.md#) ###
-1. Install geomagio
+Use PCDCP formatted files for input and/or output.
- pip install git+https://github.com/usgs/geomag-algorithms.git
-1. Use geomagio
+---
+## Supported Algorithms ##
- - Use the main script, `geomag.py -h`.
- - In python scripts, `import geomagio`.
+### [XYZ](./docs/XYZ_usage.md) ###
+Rotate data between coordinate systems. From HEZ or HDZ to XYZ and back.
-[virtual environments]: http://docs.python-guide.org/en/latest/dev/virtualenvs/
-Developing
-----------
+---
+## Getting Started ##
-1. Use npm to install `grunt-cli`
+### [Install](readme_dependency_install.md) ###
+First time install. Walk through dependencies and other considerations.
- npm install -g grunt-cli
+### [Use](readme_usage.md) ###
+Details and examples for proper usage. Get started quickly.
-2. Install `numpy`, `scipy`, `obspy`, and `flake8`
+Basic usage:
-3. Clone this project (or fork and clone your fork)
+ - Use the main script, `geomag.py -h`
+ - In python scripts, `import geomagio`
- git clone https://github.com/usgs/geomag-algorithms.git
-
-4. From root directory of project, install npm dependencies
-
- npm install
-
-5. Run grunt to run unit tests, and watch for changes to python files
-
- grunt
-
-
-Coding Standards
-----------------
-
-This project adheres to PEP8 standards in most cases:
- https://www.python.org/dev/peps/pep-0008
-
-### PEP8 Exceptions
-
-- Hanging/Visual indents (E126, E127, E128, E131)
-
- - line continuations should use two indentations (8 spaces).
- - do not use visual indents.
-
-[Dependency install details for Windows and Mac](readme_dependency_install.md)
+### [Develop](readme_develop_install.md) ###
+Development dependencies discussed here. Project is built with Grunt and Node
+and is written primarily in Python 2.7.
diff --git a/docs/README.md b/docs/README.md
index c445b2db84c45825bedd6d43657a15232cbaa9e6..04a0eb1634834307e8db1b54c6be4a53fbd8b2f5 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -6,10 +6,11 @@ Documents explaining geomag-algorithms
[XYZ Algorithm](./XYZ.md)
------
-Describes the theory behind the XYZ algorithm, as well as some implementation issues and solutions.
+Describes the theory behind the XYZ algorithm, as well as some implementation
+issues and solutions.
[Trace Metadata](./metadata.md)
-----------
Describes the metadata stored in the obspy.core.trace.stats class used by the
-geomag algorithms.
+geomag algorithms.
diff --git a/docs/XYZ.md b/docs/XYZ.md
index df1b9276d687b784bc98682eeff893e618e166a6..5d79fe75956020f0fb9a3b69bd4d5b0060ef63ec 100644
--- a/docs/XYZ.md
+++ b/docs/XYZ.md
@@ -19,8 +19,10 @@ commercial entities. Inverse transformations are also provided.
Historically, the most common coordinate system used to specify measured
geomagnetic fields has been HDZ, where:
-- `H` is the magnitude of the geomagnetic field vector tangential to the Earth's surface;
-- `D` is the declination, or clockwise angle from the vector pointing to the geographic north pole to the H vector;
+- `H` is the magnitude of the geomagnetic field vector tangential to the
+ Earth's surface;
+- `D` is the declination, or clockwise angle from the vector pointing to the
+ geographic north pole to the H vector;
- `Z` is the downward component of the geomagnetic field.
> Note: this library internally refers to the `HDZ` coordinate system as "mag",
@@ -88,9 +90,9 @@ One thing that is not labeled in this figure is the angle d (see [Eq. 4](#eq4)),
which is the difference between declination D, and a declination
baseline (D0, or DECBAS).
-The equations [Eq. 4](#eq4), [Eq. 5](#eq5), [Eq. 6](#eq6) describe how to convert the
-horizontal components of a USGS magnetometer's raw data element into more
-standard H and D components.
+The equations [Eq. 4](#eq4), [Eq. 5](#eq5), [Eq. 6](#eq6) describe how to
+convert the horizontal components of a USGS magnetometer's raw data element
+into more standard H and D components.
- <a name="eq4"></a>Equation 4: `d = arctan(e/h)`
- <a name="eq5"></a>Equation 5: `D = D0 + d`
@@ -104,7 +106,8 @@ To inverse transform from `XY` to `HD`:
...and from `HD` to `he`:
- <a name="eq9"></a>Equation 9: `d = D - D0`
-- <a name="eq10"></a>Equation 10: `h = sqrt(H*H / (1 + tan(d)*tan(d))) = H cos(d)`
+- <a name="eq10"></a>Equation 10:
+ `h = sqrt(H*H / (1 + tan(d)*tan(d))) = H cos(d)`
- <a name="eq11"></a>Equation 11: `e = h * tan(d)`
It is worth noting that there is potential for mathematically undefined results
@@ -170,29 +173,3 @@ before checking data flags. This is not an issue if data flags are NaN
(not-a-number values), but more typical for Geomag data, these are values like
99999, which can lead to seemingly valid, but erroneous values at times when the
raw data were known to be bad.
-
-> Note: this library internally represents data gaps as NaN, and factories convert
-> to this where possible.
-
-
-## Library Notes
-
-This library references the 3 reference frames as `geo` for
-geographic/cartesian, `obs` for observatory and `mag` for magnetic/cylindrical:
-
-- `geo` is XYZ
-- `obs` is heZ
-- `mag` is HDZ
-
-> Note: within this library all channels are uppercase.
-> We use context (ie obs vs. mag vs geo), to differentiate between h,H; e,E; and d,D.
-> This mirrors the various data formats, (ie IAGA2002, etc).
-
-The underlying library provides calculations for both the basic conversions,
-such as get_get_y_from_mag, which is based off of Y = H sin(D), and higher
-level conversions, such as get_geo_from_mag. (Which converts HD to XY).
-These are provided by `geomagio.ChannelConverter`.
-
-Upper libraries only provide higher level conversions, ie get_geo_from_mag.
-This is the level most users should be accessing.
-These are provided by `geomagio.StreamConverter`.
diff --git a/docs/XYZ_usage.md b/docs/XYZ_usage.md
new file mode 100644
index 0000000000000000000000000000000000000000..765b0b2ab32b848ce0edb226b29ae00ae6e6dad5
--- /dev/null
+++ b/docs/XYZ_usage.md
@@ -0,0 +1,63 @@
+
+# XYZ Usage #
+
+`geomag.py --xyz {geo, mag, obs, obsd} {geo, mag, obs, obsd}`
+
+### Reference Frames ###
+
+There are 3 reference frames in this library.
+
+#### Geographic or cartesian ####
+
+ - `geo` is XYZ
+
+ - 'geo': ['X', 'Y', 'Z', 'F']
+
+#### Magnetic or cylindrical ####
+
+ - `mag` is HDZ
+
+ - 'mag': ['H', 'D', 'Z', 'F']
+
+#### Observatory ####
+
+ - `obs` is heZ
+
+ - 'obs': ['H', 'E', 'Z', 'F']
+ - 'obsd': ['H', 'D', 'Z', 'F']
+
+### Example ###
+
+To convert HEZF data in pcdcp files to XYZF for Tucson observatory for all of
+March 2013 output to iaga2002 files:
+
+ geomag.py --xyz obs geo --observatory TUC \
+ --starttime 2013-03-01T00:00:00Z --endtime 2013-03-31T23:59:00Z \
+ --input-pcdcp-url file://data-pcdcp/./%(OBS)s%(year)s%(julian)s.%(i)s \
+ --output-iaga-url file://data-iaga/./$(obs)s%(Y)s%(j)s.%(i)s \
+ --type variation --interval minute
+
+
+---
+### Library Notes ###
+
+> Note: Within this library all channels are uppercase.
+> We use context (ie obs vs. mag vs geo), to differentiate between h,H; e,E;
+> and d,D. This mirrors the various data formats, (ie IAGA2002, etc).
+
+The underlying library provides calculations for both the basic conversions,
+such as get_geo_y_from_mag, which is based off of Y = H sin(D), and higher
+level conversions, such as get_geo_from_mag. (Which converts HD to XY).
+These are provided by `geomagio.ChannelConverter`.
+
+Upper libraries only provide higher level conversions, ie get_geo_from_mag.
+This is the level most users should be accessing.
+These are provided by `geomagio.StreamConverter`.
+
+> Note: this library internally represents data gaps as NaN, and factories
+> convert to this where possible.
+
+---
+### [Algorithm Theoretical Basis for "Geomag XYZ"](XYZ.md) ###
+Describes the theory behind the XYZ algorithm, as well as some implementation
+issues and solutions.
diff --git a/readme_dependency_install.md b/readme_dependency_install.md
index ba8c1913ba558433907c8254f4484ca301244bc3..be0863c6a8b41c2e0466dbfb1d53ff61b440063a 100644
--- a/readme_dependency_install.md
+++ b/readme_dependency_install.md
@@ -1,47 +1,26 @@
-Readme dependencies
+Readme install dependencies
-### Windows ###
-
-#### Install ####
-1. You will need a terminal tool for Windows. We used [Git Bash][] to test these
- steps, but [Cygwin][] or another unix-like editor should work fine too.
-
- - Choosing PATH environment: We recommend the __last option__ here to include
- Unix tools, but if you don't understand what that entails then use the
- _second option_ which still adds Git to your system PATH.
- - Besides that, all of the defaults should be sufficient.
-
-1. Install [Anaconda][], which includes `numpy`, `scipy` and `flake8`.
- - We recommend using "Just Me" for the install to avoid potential admin
- issues.
+## Install ##
-1. Run `python` to verify that Anaconda's version of Python is being used.
- If it isn't, update your PATH so that Conda's HOME is before any other
- versions of Python.
+We recommend using
+[virtual environments (http://docs.python-guide.org/en/latest/dev/virtualenvs/)][].
-1. Head back over to the main [install](README.md#install) and complete the
- instructions there.
+Check out the [Mac](#mac) and [Windows](#windows) details below for more
+specific dependency information.
-#### Develop ####
+1. Install __geomagio__
-These instructions only need to be completed if you plan on developing new
-code for this project.
+ pip install git+https://github.com/usgs/geomag-algorithms.git
-1. Install the newest release of [Node][] for Windows, using the Windows
- Installer (.msi).
- - Use all of the defaults.
+1. Use __geomagio__
-1. Close and re-open your terminal so that your new PATH is loaded.
- Make sure to navigate back to your `geomag-algorithms` project directory.
+ - Use the main script, `geomag.py -h`
+ - In python scripts, `import geomagio`
-1. Install `obspy` from the terminal.
+The [Usage](readme_usage.md) page has more detailed instructions and examples.
- easy_install obspy
+[virtual environments]: http://docs.python-guide.org/en/latest/dev/virtualenvs/
-[Git Bash]: http://git-scm.com/download/win
-[Cygwin]: http://cygwin.com/install.html
-[Node]: http://nodejs.org/download/
-[Anaconda]: http://continuum.io/downloads
---
### Mac ###
@@ -55,7 +34,7 @@ code for this project.
pip install numpy
1. Install `node`, `git` and `python` (2.7.X).
- On OS X, we recommend using [Homebrew][]
+ On OS X, we recommend using [Homebrew (http://brew.sh/)][]
brew install node
brew install git
@@ -75,27 +54,30 @@ code for this project.
1. Close and re-open your terminal so that your new PATH is loaded.
Make sure to navigate back to your `geomag-algorithms` project directory.
-1. Head back over to the main [install](README.md#install) and complete the
- instructions there.
-
[Homebrew]: http://brew.sh/
---
-### Add an SSH Key to GitHub ###
-
-This only needs to be completed if you plan on developing and pushing new code
-to this project.
-
- 1. `ssh-keygen -t rsa -b 2048` (in terminal)
- 2. Press **Enter** to accept the default save location.
- 3. Enter a passphrase that you will remember.
- 4. `cat ~/.ssh/id_rsa.pub`
- Copy the text block that is displayed.
- This is your SSH key.
- If you're on Window and can't copy the text from the terminal, go
- to `C:\Users\[your username here]\.ssh` and open the `id_rsa.pub` file
- with notepad.
- 5. In GitHub, click **Edit Your Profile**.
- 6. Select **SSH Keys** on the left.
- 7. Click **Add SSH key**. Give it a meaningful title.
- 8. Copy your SSH Key into the Key, and click **Add key**.
+### Windows ###
+
+1. You will need a terminal tool for Windows.
+ [Git Bash (http://git-scm.com/download/win)][] was used to test
+ these steps, but [Cygwin (http://cygwin.com/install.html)][] or another
+ unix-like editor should work too.
+
+ - Choosing PATH environment: We recommend the __last option__ here to include
+ Unix tools, but if you don't understand what that entails then use the
+ _second option_ which still adds Git to your system PATH.
+ - Besides that, all of the defaults should be sufficient.
+
+1. Install [Anaconda (http://continuum.io/downloads)][], which includes
+ `numpy`, `scipy` and `flake8`.
+ - We recommend using _"Just Me"_ for the install to avoid potential admin
+ issues.
+
+1. Run `python` to verify that Anaconda's version of Python is being used.
+ If it isn't, update your PATH so that Conda's HOME is before any other
+ versions of Python.
+
+[Git Bash]: http://git-scm.com/download/win
+[Cygwin]: http://cygwin.com/install.html
+[Anaconda]: http://continuum.io/downloads
diff --git a/readme_develop_install.md b/readme_develop_install.md
new file mode 100644
index 0000000000000000000000000000000000000000..1b07b644df752d60aeee2d3fc5da16cba07172d2
--- /dev/null
+++ b/readme_develop_install.md
@@ -0,0 +1,82 @@
+Readme development dependencies
+
+## Develop ##
+
+These instructions only need to be completed if you plan on developing new
+code for this project.
+
+If developing on windows, see the [Windows](#windows) section below.
+
+Begin Developing
+----------------
+
+1. Use npm to install `grunt-cli`
+
+ npm install -g grunt-cli
+
+2. Install `numpy`, `scipy`, `obspy`, and `flake8` if they aren't already
+ installed.
+
+3. Clone this project (or fork and clone your fork)
+
+ git clone https://github.com/usgs/geomag-algorithms.git
+
+4. From root directory of project, install npm dependencies
+
+ npm install
+
+5. Run grunt to run unit tests, and watch for changes to python files
+
+ grunt
+
+
+---
+### Windows ###
+
+1. Install the newest release of [Node (http://nodejs.org/download/)][] for
+ Windows, using the Windows Installer (.msi).
+ - All of the defaults should be sufficient.
+
+1. Close and re-open your terminal so that your new PATH is loaded.
+
+[Node]: http://nodejs.org/download/
+
+
+---
+### Coding Standards ###
+
+This project adheres to PEP8 standards in most cases:
+ https://www.python.org/dev/peps/pep-0008
+
+### PEP8 Exceptions ###
+
+- Hanging/Visual indents (E126, E127, E128, E131)
+
+ - line continuations should use two indentations (8 spaces).
+ - do not use visual indents.
+
+
+---
+### Add an SSH Key to GitHub ###
+
+1. `ssh-keygen -t rsa -b 2048` (in terminal)
+
+2. Press **Enter** to accept the default save location.
+
+3. Enter a passphrase that you will remember.
+
+4. `cat ~/.ssh/id_rsa.pub`
+ Copy the text block that is displayed.
+ This is your SSH key.
+ If you're on Window and can't copy the text from the terminal, go
+ to `C:\Users\[your username here]\.ssh` and open the `id_rsa.pub` file
+ with notepad.
+
+5. In GitHub, click **Edit Your Profile**.
+
+6. Select **SSH Keys** on the left.
+
+7. Click **Add SSH key**. Give it a meaningful title.
+
+8. Copy your SSH Key into the Key, and click **Add key**.
+
diff --git a/readme_io.md b/readme_io.md
new file mode 100644
index 0000000000000000000000000000000000000000..f0b82d7cbe945854f263301140b0658a581b9685
--- /dev/null
+++ b/readme_io.md
@@ -0,0 +1,109 @@
+Readme input output details
+
+## IO Formats ##
+There are currently 2 supported file formats for input and output in addition
+to Edge server input.
+
+#### EDGE ####
+
+`--input-edge HOST PORT`
+Specify an IP address or host name where your Edge lives along with a port.
+
+#### Iaga2002 ####
+
+`--input-iaga-file FILENAME`
+Specify the name of the file to read from.
+
+`--input-iaga-magweb`
+Data will be pulled from geomag.usgs.gov/data/magnetometer if it exists.
+
+`--input-iaga-stdin`
+Use redirection on the command line to send your data in.
+
+`--input-iaga-url`
+Use a file pattern to read from multiple IAGA2002 files.
+
+`--output-iaga-file FILENAME`
+Specify the name of the file to write to.
+
+`--output-iaga-stdout`
+Output will be send directly to the command line.
+
+`--output-iaga-url`
+Use a file pattern to write to multiple IAGA2002 files.
+
+#### PCDCP ####
+
+`--input-pcdcp-file FILENAME`
+Specify the name of the file to read from.
+
+`--input-pcdcp-stdin`
+Use redirection on the command line to send your data in.
+
+`--input-pcdcp-url`
+Use a file pattern to read from multiple PCDCP files.
+
+`--output-pcdcp-file FILENAME`
+Specify the name of the file to write to.
+
+`--output-pcdcp-stdout`
+Output will be send directly to the command line.
+
+`--output-pcdcp-url`
+Use a file pattern to write to multiple PCDCP files.
+
+
+## IO Methods ##
+Several methods exist for retrieving and storing data.
+
+#### Edge Server ####
+
+`--input-edge HOST PORT`
+Specify an IP address or host name where your Edge lives along with a port.
+
+#### Single File ####
+
+`--input-iaga-file FILENAME`
+`--input-pcdcp-file FILENAME`
+`--output-iaga-file FILENAME`
+`--output-pcdcp-file FILENAME`
+Specify a single file name for the data to be read from or written to.
+
+#### Multiple Files ####
+
+`--input-iaga-url`
+`--input-pcdcp-url`
+`--output-iaga-url`
+`--output-pcdcp-url`
+URLs can be used to fetch or store groups of data using pattern matching. In
+order to use a directory of files on a local machine, just specify "file://"
+at the beginning of the pattern.
+
+Patterns that will be matched with information from the data:
+
+ - __%(i)s__ : interval abbreviation (sec, min, hor, etc.)
+ - __%(interval)s__: interval name (second, minute, hour, etc.)
+ - __%(julian)s__ : julian day formatted as JJJ
+ - __%(obs)s__ : lowercase observatory 3-letter code
+ - __%(OBS)s__ : uppercase observatory 3-letter code
+ - __%(t)s__ : type abbreviation (v, q, d, etc.)
+ - __%(type)s__ : type name (variation, quasi-definitive, definitive, etc.)
+ - __%(year)s__ : year formatted as YYYY
+ - __%(ymd)s__ : time formatted as YYYYMMDD
+
+Typical IAGA2002 files are stored as `file://./%(obs)s%(ymd)s%(t)%(i)s.%(i)s`
+
+Example: bou20130402vmin.min
+
+Typical PCDCP files are stored as `file://./%(OBS)s%(Y)s%(j)s.%(i)s`
+
+Example: BOU2013092.min
+
+#### Std In and Std Out ####
+
+`--input-iaga-stdin`
+`--input-pcdcp-stdin`
+`--output-iaga-stdout`
+`--output-pcdcp-stdout`
+
+For standard in, pass the data in with redirection.
diff --git a/readme_usage.md b/readme_usage.md
new file mode 100644
index 0000000000000000000000000000000000000000..7175e3e4c220c7111e476844171f2623f926b0d6
--- /dev/null
+++ b/readme_usage.md
@@ -0,0 +1,67 @@
+Readme usage details
+
+## Usage ##
+
+You can install the project if you haven't already with
+
+ pip install git+https://github.com/usgs/geomag-algorithms.git
+
+Running `geomag.py -h` or `geomag.py --help` will show an extensive list of
+input parameters.
+
+Some of the key parameters that should be specified are listed here along with
+examples. They include
+
+ - __start time__ - YYYY-MM-DDTHH:MM:SSZ
+ - __end time__ - YYYY-MM-DDTHH:MM:SSZ
+ - __observatory code__ - 3-letter observatory code
+ - __data input source__ - {--input-edge, --input-iaga-file, ...}
+ - __data output source__ - {--output-pcdcp-url, --output-iaga-file, ...}
+ - __input data channels__ - {[CHANNEL [CHANNEL ...]]}
+ - __output data channels__ - {[CHANNEL [CHANNEL ...]]}
+ - __data type__ - {variation, quasi-definitive, definitive}
+ - __interval__ - {minute, second, hour, day}
+
+### Examples ###
+
+To retrieve all _raw_ (variation) _H_, _E_, _Z_ and _F_ _minute_ data from
+_Boulder Observatory_ for the entire day of _July 1st 2014_ from an _iaga2002_
+formatted file and output _H_, _E_, _Z_ and _F_ data to a _PCDCP_ formatted
+file:
+
+ geomag.py --type variation --inchannels H E Z F --interval minute \
+ --observatory BOU \
+ --starttime 2014-07-01T00:00:00Z \
+ --endtime 2014-07-01T23:59:00Z \
+ --input-iaga-file BOU20140701vmin.min \
+ --outchannels H E Z F \
+ --output-pcdcp-file BOU2014182.min
+
+To retrieve all _raw_ (variation) _H_, _E_, _Z_ and _F_ _minute_ data from
+_Tucson Observatory_ for the entire month of _March 2013_ from _pcdcp_
+formatted files in a "data-pcdcp" directory and output _H_, _E_, _Z_ and _F_
+data to a group of _iaga2002_ formatted files in a "data-iaga" directory:
+
+ geomag.py --type variation --inchannels H E Z F --interval minute \
+ --observatory TUC \
+ --starttime 2013-03-01T00:00:00Z \
+ --endtime 2013-03-31T23:59:00Z \
+ --input-pcdcp-url file://data-pcdcp/./%(OBS)s%(year)s%(julian)s.%(i)s \
+ --output-iaga-url file://data-iaga/./$(obs)s%(Y)s%(j)s.%(i)s \
+ --outchannels H E Z F
+
+
+---
+### Algorithms ###
+
+There are flags to specify certain algorithms should be run against the data.
+
+#### XYZ ####
+
+`--xyz {geo, mag, obs, obsd} {geo, mag, obs, obsd}`
+
+#### [XYZ Usage](./docs/XYZ_usage.md) ####
+Rotate data from HEZ or HDZ to XYZ and back.
+
+Extensive explanation of all input and output methods:
+[IO Methods](readme_io.md)