Overhaul EdgeFactory and MiniSeedFactory for Edge data migration
This is a 2nd attempt to overhaul Edge/MiniSeed factories in preparation for a long-awaited Edge data migration. For reference, the original is !382 (closed). This almost certainly more than a "bug fix", so don't merge until any pending bug fixes are merged, and deployed to production.
In summary (from commit messages):
MiniSeedFactory Upgrades:
- configurable scale_factor and float vs. int conversion logic
- configurable sncl_mode for different channel mapping
- multiple read ports allowed for fail-over logic
- configurable write_encoding for setting miniseed data type
- generally more (if not perfectly) compatible with EdgeFactory
EdgeFactory Upgrades:
- configurable scale_factor and float vs. int conversion logic
- configurable sncl_mode for different channel mapping
- multiple read ports allowed for fail-over logic
- generally more (if not perfectly) compatible with MiniSeedFactory
RawInputClient Update:
- handle intervals other than 1s and 1m properly
Controller Update:
- align timestamps more consistently for "update" intervals
- (related) add "interval" option to get_realtime_interval
TimeseriesUtility Updates:
- better handle hour and day intervals in create_empty_trace
- enode_stream now handles "STEIM" encoding
- fixed bug in pad_and_trim_trace
Update filters.py (code for geomag-filter typer app):
- stop using factory.py; just instantiate factories in filters.py...it's only a little extra code, and a lot less confusing
- copy 1-second ObsRio float data (e.g., F, UK*) to RawInput so it is available using EdgeFactory, and also so we have the option in the future to prune 10Hz data from EdgeCWB that has already been archived at IRIS.
- set up calls to _copy_channel to legacy sncls to be removed once Edge data migration is complete
- make all *_filter methods require an instantiated io factory, and set no default options
- other than the temporary write ports for copies of legacy sncl data, allow no default ports in filters.py; either accept the factory defaults, or force user to specify ports with call to geomag-filter app.
Add "tenhertz" to LegacySNCL.py
Use EdgeFactory for all but tenhertz data in /ws/data endpoint:
- assumes updates to filters.py, EdgeFactory, and MiniSeedFactory included in this feature branch
- removes default ports in favor of factory defaults
- tenhertz data will still use MiniSeedFactory, and if channels "U", "V", or "W" are requested, the raw voltage and bin data will be used to calculate nT outputs
To get more than one set of eyes on this MR, I'll mention @swilbur , @awernle , and @bgeels.
Edited by Erin (Josh) Rigler
Merge request reports
Activity
Please register or sign in to reply