Skip to content
Snippets Groups Projects
Commit abc4b72f authored by Eddie McWhirter's avatar Eddie McWhirter
Browse files

Move some instructions from XYZ algorithm to XYZ usage.

parent f1e6d8f6
No related branches found
No related tags found
No related merge requests found
......@@ -5,26 +5,26 @@ Geomag algorithms includes tools to fetch, process, and output geomag data.
### [Install](readme_dependency_install.md) ###
First time install. Walk through dependencies and other considerations.
The project is primarily Python 2.7.
### [Develop](readme_develop_install.md) ###
Development dependencies discussed here. Project is built with Grunt and Node.
Development dependencies discussed here. Project is built with Grunt and Node
and is written primarily in Python 2.7.
### [Usage](readme_usage.md) ###
Details and examples for proper usage. This will help you get started quickly.
Basic usage:
- Use the main script, `geomag.py -h`
- In python scripts, `import geomagio`
- Use the main script, `geomag.py -h`
- In python scripts, `import geomagio`
Supported Algorithms
--------------------
### [XYZ Algorithm](./docs/XYZ_usage.md) ###
---
---
Coding Standards
----------------
......
......@@ -170,29 +170,3 @@ before checking data flags. This is not an issue if data flags are NaN
(not-a-number values), but more typical for Geomag data, these are values like
99999, which can lead to seemingly valid, but erroneous values at times when the
raw data were known to be bad.
> Note: this library internally represents data gaps as NaN, and factories convert
> to this where possible.
## Library Notes
This library references the 3 reference frames as `geo` for
geographic/cartesian, `obs` for observatory and `mag` for magnetic/cylindrical:
- `geo` is XYZ
- `obs` is heZ
- `mag` is HDZ
> Note: within this library all channels are uppercase.
> We use context (ie obs vs. mag vs geo), to differentiate between h,H; e,E; and d,D.
> This mirrors the various data formats, (ie IAGA2002, etc).
The underlying library provides calculations for both the basic conversions,
such as get_get_y_from_mag, which is based off of Y = H sin(D), and higher
level conversions, such as get_geo_from_mag. (Which converts HD to XY).
These are provided by `geomagio.ChannelConverter`.
Upper libraries only provide higher level conversions, ie get_geo_from_mag.
This is the level most users should be accessing.
These are provided by `geomagio.StreamConverter`.
......@@ -3,18 +3,58 @@
geomag.py --xyz {geo, mag, obs, obsd} {geo, mag, obs, obsd}
'geo': ['X', 'Y', 'Z', 'F'],
'mag': ['H', 'D', 'Z', 'F'],
'obs': ['H', 'E', 'Z', 'F'],
'obsd': ['H', 'D', 'Z', 'F']
### Reference Frames ###
There are 3 reference frames in this library.
#### Geographic or cartesian ####
- `geo` is XYZ
- 'geo': ['X', 'Y', 'Z', 'F']
#### Magnetic of cylindrical ####
- `mag` is HDZ
- 'mag': ['H', 'D', 'Z', 'F']
#### Observatory ####
- `obs` is heZ
- 'obs': ['H', 'E', 'Z', 'F']
- 'obsd': ['H', 'D', 'Z', 'F']
### Example ###
To convert HEZF data in pcdcp files to XYZF for Tucson observatory for all of
March 2013 output to iaga2002 files:
geomag.py --xyz obs geo --observatory TUC --starttime 2013-03-01T00:00:00Z --endtime 2013-03-31T23:59:00Z --input-pcdcp-url file://data-pcdcp/./%(OBS)s%(year)s%(julian)s.%(i)s --output-iaga-url file://data-iaga/./$(obs)s%(Y)s%(j)s.%(i)s --type variation --interval minute
geomag.py --xyz obs geo --observatory TUC
--starttime 2013-03-01T00:00:00Z --endtime 2013-03-31T23:59:00Z
--input-pcdcp-url file://data-pcdcp/./%(OBS)s%(year)s%(julian)s.%(i)s
--output-iaga-url file://data-iaga/./$(obs)s%(Y)s%(j)s.%(i)s
--type variation --interval minute
---
### Library Notes ###
> Note: within this library all channels are uppercase.
> We use context (ie obs vs. mag vs geo), to differentiate between h,H; e,E;
> and d,D. This mirrors the various data formats, (ie IAGA2002, etc).
The underlying library provides calculations for both the basic conversions,
such as get_get_y_from_mag, which is based off of Y = H sin(D), and higher
level conversions, such as get_geo_from_mag. (Which converts HD to XY).
These are provided by `geomagio.ChannelConverter`.
Upper libraries only provide higher level conversions, ie get_geo_from_mag.
This is the level most users should be accessing.
These are provided by `geomagio.StreamConverter`.
> Note: this library internally represents data gaps as NaN, and factories
> convert to this where possible.
### [Algorithm Theoretical Basis for "Geomag XYZ"](XYZ.md) ###
Describes the theory behind the XYZ algorithm, as well as some implementation
......
......@@ -9,17 +9,18 @@ specific dependency information.
1. Install __geomagio__
pip install git+https://github.com/usgs/geomag-algorithms.git
pip install git+https://github.com/usgs/geomag-algorithms.git
1. Use __geomagio__
- Use the main script, `geomag.py -h`
- In python scripts, `import geomagio`
- Use the main script, `geomag.py -h`
- In python scripts, `import geomagio`
The [Usage](readme_usage.md) page has more detailed instructions and examples.
[virtual environments]: http://docs.python-guide.org/en/latest/dev/virtualenvs/
---
### Mac ###
......
......@@ -29,6 +29,7 @@ Begin Developing
grunt
---
### Windows ###
......@@ -40,6 +41,7 @@ Begin Developing
[Node]: http://nodejs.org/download/
---
### Add an SSH Key to GitHub ###
......
......@@ -4,70 +4,106 @@ Readme input output details
There are currently 2 supported file formats for input and output in addition
to Edge server input.
### EDGE ###
--input-edge
### Iaga2002 ###
--input-iaga-file FILENAME
--input-iaga-magweb
--input-iaga-stdin
--input-iaga-url
--output-iaga-file FILENAME
--output-iaga-stdout
--output-iaga-url
### PCDCP ###
--input-pcdcp-file FILENAME
--input-pcdcp-stdin
--input-pcdcp-url
--output-pcdcp-file FILENAME
--output-pcdcp-stdout
--output-pcdcp-url
#### EDGE ####
`--input-edge HOST PORT`
Specify an IP address or host name where your Edge lives along with a port.
#### Iaga2002 ####
`--input-iaga-file FILENAME`
Specify the name of the file to read from.
`--input-iaga-magweb`
Data will be pulled from geomag.usgs.gov/data/magnetometer if it exists.
`--input-iaga-stdin`
Use redirection on the command line to send your data in.
`--input-iaga-url`
Use a file pattern to read from multiple IAGA2002 files.
`--output-iaga-file FILENAME`
Specify the name of the file to write to.
`--output-iaga-stdout`
Output will be send directly to the command line.
`--output-iaga-url`
Use a file pattern to write to multiple IAGA2002 files.
#### PCDCP ####
`--input-pcdcp-file FILENAME`
Specify the name of the file to read from.
`--input-pcdcp-stdin`
Use redirection on the command line to send your data in.
`--input-pcdcp-url`
Use a file pattern to read from multiple PCDCP files.
`--output-pcdcp-file FILENAME`
Specify the name of the file to write to.
`--output-pcdcp-stdout`
Output will be send directly to the command line.
`--output-pcdcp-url`
Use a file pattern to write to multiple PCDCP files.
## IO Methods ##
Several methods exist for retrieving and storing data.
### Edge Server ###
--input-edge HOST PORT
#### Edge Server ####
`--input-edge HOST PORT`
Specify an IP address or host name where your Edge lives along with a port.
### Single File ###
--input-iaga-file FILENAME
--input-pcdcp-file FILENAME
--output-iaga-file FILENAME
--output-pcdcp-file FILENAME
#### Single File ####
`--input-iaga-file FILENAME`
`--input-pcdcp-file FILENAME`
`--output-iaga-file FILENAME`
`--output-pcdcp-file FILENAME`
Specify a single file name for the data to be read from or written to.
### Multiple Files ###
--input-iaga-url
--input-pcdcp-url
--output-iaga-url
--output-pcdcp-url
#### Multiple Files ####
`--input-iaga-url`
`--input-pcdcp-url`
`--output-iaga-url`
`--output-pcdcp-url`
URLs can be used to fetch or store groups of data using pattern matching. In
order to use a directory of files on a local machine, just specify "file://"
at the beginning of the pattern.
Patterns that will be matched with information from the data:
- %(i)s : interval abbreviation (sec, min, hor, etc.)
- %(interval)s: interval name (second, minute, hour, etc.)
- %(julian)s : julian day formatted as JJJ
- %(obs)s : lowercase observatory 3-letter code
- %(OBS)s : uppercase observatory 3-letter code
- %(t)s : type abbreviation (v, d, etc.)
- %(type)s : type name (variation, definitive, etc.)
- %(year)s : year formatted as YYYY
- %(ymd)s : time formatted as YYYYMMDD
- __%(i)s__ : interval abbreviation (sec, min, hor, etc.)
- __%(interval)s__: interval name (second, minute, hour, etc.)
- __%(julian)s__ : julian day formatted as JJJ
- __%(obs)s__ : lowercase observatory 3-letter code
- __%(OBS)s__ : uppercase observatory 3-letter code
- __%(t)s__ : type abbreviation (v, q, d, etc.)
- __%(type)s__ : type name (variation, quasi-definitive, definitive, etc.)
- __%(year)s__ : year formatted as YYYY
- __%(ymd)s__ : time formatted as YYYYMMDD
Typical IAGA2002 files are stored as `file://./%(obs)s%(ymd)s%(t)%(i)s.%(i)s`
example: bou20130402vmin.min
Example: bou20130402vmin.min
Typical PCDCP files are stored as `file://./%(OBS)s%(Y)s%(j)s.%(i)s`
example: BOU2013092.min
### Std In and Std Out ###
--input-iaga-stdin
--input-pcdcp-stdin
--output-iaga-stdout
--output-pcdcp-stdout
Example: BOU2013092.min
#### Std In and Std Out ####
`--input-iaga-stdin`
`--input-pcdcp-stdin`
`--output-iaga-stdout`
`--output-pcdcp-stdout`
For standard in, pass the file in with redirection.
For standard in, pass the data in with redirection.
......@@ -29,14 +29,26 @@ _Boulder Observatory_ for the entire day of _July 1st 2014_ from an _iaga2002_
formatted file and output _H_, _E_, _Z_ and _F_ data to a _PCDCP_ formatted
file:
geomag.py --type variation --inchannels H E Z F --interval minute --observatory BOU --starttime 2014-07-01T00:00:00Z --endtime 2014-07-01T23:59:00Z --input-iaga-file BOU20140701vmin.min --outchannels H E Z F --output-pcdcp-file BOU2014182.min
geomag.py --type variation --inchannels H E Z F --interval minute
--observatory BOU
--starttime 2014-07-01T00:00:00Z
--endtime 2014-07-01T23:59:00Z
--input-iaga-file BOU20140701vmin.min
--outchannels H E Z F
--output-pcdcp-file BOU2014182.min
To retrieve all _raw_ (variation) _H_, _E_, _Z_ and _F_ _minute_ data from
_Tucson Observatory_ for the entire month of _March 2013_ from _pcdcp_
formatted files in a "data-pcdcp" directory and output _H_, _E_, _Z_ and _F_
data to a group of _iaga2002_ formatted files in a "data-iaga" directory:
geomag.py --type variation --inchannels H E Z F --interval minute --observatory TUC --starttime 2013-03-01T00:00:00Z --endtime 2013-03-31T23:59:00Z --input-pcdcp-url file://data-pcdcp/./%(OBS)s%(year)s%(julian)s.%(i)s --output-iaga-url file://data-iaga/./$(obs)s%(Y)s%(j)s.%(i)s --outchannels H E Z F
geomag.py --type variation --inchannels H E Z F --interval minute
--observatory TUC
--starttime 2013-03-01T00:00:00Z
--endtime 2013-03-31T23:59:00Z
--input-pcdcp-url file://data-pcdcp/./%(OBS)s%(year)s%(julian)s.%(i)s
--output-iaga-url file://data-iaga/./$(obs)s%(Y)s%(j)s.%(i)s
--outchannels H E Z F
### Algorithms ###
......@@ -44,8 +56,11 @@ data to a group of _iaga2002_ formatted files in a "data-iaga" directory:
There are flags to specify certain algorithms should be run against the data.
#### XYZ ####
--xyz
[XYZ Usage](./docs/XYZ_usage.md)
##### [XYZ Usage](./docs/XYZ_usage.md) #####
Rotate data from HEZ or HDZ to XYZ and back.
Extensive explanation of all input and output methods:
[IO Methods](readme_io.md)
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment