- Nov 22, 2022
-
-
Erin (Josh) Rigler authored
Writing unit tests for Controller configurations is tricky, so I didn't do it, and this tiny-but-impactful bug slipped in. I did test this on the stagin server this time, and the processing pipe- line should work after this is deployed.
-
Erin (Josh) Rigler authored
The original version of AverageAlgorithm.py seemed to be written to process the Controller's `--outchannels`. It "worked" for several years because `--outchannels` defaulted to `--inchannels`. But what we actually always wanted was to process `--inchannels`. Mostly this was just a matter of semantics, but when I recently added the ability to average fewer than the full complement of inputs, it was also necessary to change the `can_produce_data()` method to check for "any" instead of "all" channels, which in turn required that the AverageAlgorithm class properly instantiate its `_inchannels` and `_outchannels` class variables, instead of just set them to `None`. To briefly explain the different changes in this commit: - added `Algorithm.__init__(self, inchannels=[channel])` to ensure that can_produce_data() would work when run via programmatic interface. - added `Algorithm.configure(self, arguments)` to AverageAlgorithm.configure() to ensure that can_produce_data() would work when run via Controller.py. - changed all variations of `outchannel` to `inchannel` - cleaned up where class variables were modified inside process() method
-
- Nov 19, 2022
-
-
Erin (Josh) Rigler authored
The method TimeseriesUtility.get_stream_gaps() would over-write gaps[channel] if there were multiple Traces in a stream that were the same channel, but different observatories. This is a typical situation when using the AverageAlgorithm.
-
- Nov 18, 2022
-
-
Erin (Josh) Rigler authored
Recent attempts to change the AverageAlgorithm to allow fewer than the full complement of inputs when calculating a mutli-station average were thwarted by the default can_produce_data() method in the parent Algorithm class, which required all inputs to be present, when what we now want is **any** inputs to be present.
-
- Nov 16, 2022
-
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
Imported Enum to define a CLI parameter with a predefined set of values to choose from. Created a spreadsheets_dir option and factory option.
-
Erin (Josh) Rigler authored
-
- Nov 15, 2022
-
-
Erin (Josh) Rigler authored
-
- Nov 14, 2022
-
-
Erin (Josh) Rigler authored
Previously use the number of traces in the input Stream. This could be fewer than the number of desired observatories if a given input's data was completely missing.
-
Erin (Josh) Rigler authored
The recently added `min_count` option to AverageAlgorithm.py leads to some undesirable behavior when realtime data, with asynchronous inputs, are being processed. By adding the ability to specify an interval over which `min_count` is applied, some of this undesirable behavior can be mitigated. In particular, if the `realtime` option is specified via the controller, and `min_count` is defined, the minimum number of inputs will be allowed only for time steps prior to `(UTCDateTime.now() - realtime)`; the full complement of inputs will be required to calculate averages more recent than that. One drawback is that if an input observatory goes offline for an extended period, the Dst index will be calculated with a persistent lag `realtime` seconds long. A user can always override this admittedly ad-hoc default behavior using the `min_count_start` and `min_count_end` options.
-
- Nov 09, 2022
-
-
Erin (Josh) Rigler authored
- added a `min_count` keyword to the AvergeAlgorithm's __init__() method; - added a --average-min-count option via the add_arguments() classmethod; - set `self.min_count = arguments.average_min_count` in configure() method; - refactored process() method to respect `min_count`, but now it defaults to a requirement that all inputs be valid (this modified a recent merge by @awernle that only required that any inputs be valid; - modified AverageAlgorithm_test.py to properly assess things in light of the change to default behavior just mentioned.
-
- Nov 02, 2022
-
-
Wernle, Alexandra Nicole authored
Renamed variables and put into paragraphs. Asserted that the count correctly shows where there are fewer input data.
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
Changed dst_tot to calculate average regardless of nan values and added a new stream that counts available observatories pertimestep
-
- Oct 24, 2022
-
-
Jeremy M Fee authored
-
Jeremy M Fee authored
-
Jeremy M Fee authored
-
- Sep 30, 2022
-
-
Jeremy M Fee authored
-
Wernle, Alexandra Nicole authored
-
- Sep 22, 2022
-
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
Changed starttime/endtime to be defined by first absolutes reading starttime and last absolutes reading(last commit was supposed to include this but did not)
-
Wernle, Alexandra Nicole authored
Broke up code into smaller functions, changed starttime/endtime to be defined by first absolutes reading starttime and last absolutes reading endtime
-
Wernle, Alexandra Nicole authored
-
- Sep 15, 2022
-
-
Erin (Josh) Rigler authored
The calculation of scale_value in geomagio.residual.calculate() is not quite correct, but that needs to be addressed in a separate commit and merge request. I'm reverting the recent change here so that we don't break the tests.
-
Erin (Josh) Rigler authored
Fixes #75 When geomagio.residual.calculate() is called with adjust_reference set equal to True, the start/end timestamps now both reflect the time of the first absolute measurement (i.e., West Down). This is consistent with the residual absolute spreadsheet calculations.
-
- Aug 20, 2022
-
-
Erin (Josh) Rigler authored
-
- Aug 13, 2022
-
-
Jeremy M Fee authored
-
- Aug 12, 2022
-
-
Jeremy M Fee authored
-
- Aug 11, 2022
-
-
Erin (Josh) Rigler authored
Previously, the endtime-starttime interval being processed by the `run_as_update` method would shrink by 1 (second) with each recursion. This ultimately broke `run_as_update` when processing anything other than 1-second data, but it wasn't obvious because we rarely had to actually recurse. There is a little trickery now to ensure that user- provided starttime and endtime are inclusive of the full first (most recent) update interval, while subsequent, calculated, endtimes are set equal to the prior starttime minus specified output_interval (e.g., 'second', 'minute', etc.).
-
- Jun 01, 2022
-
-
Erin (Josh) Rigler authored
-
- May 27, 2022
-
-
Erin (Josh) Rigler authored
It turns out that none of the factories currently being used in production expects bytes as input, and in fact, Util.read_url() converts everthing to a string anyway, so a BYTES-oriented input would couldn't not possibly be working. So, just remove all use of BYTESIO, and instead use only STRINGIO.
-
- May 18, 2022
-
-
Erin (Josh) Rigler authored
-
Erin (Josh) Rigler authored
This is a bandaid to allow the `--input-url` option to be used to read in single files. It adds zero functionality beyond what already existed when the `--input-file` option was used.
-
- May 03, 2022
-
-
Jeremy M Fee authored
-
Jeremy M Fee authored
-
Jeremy M Fee authored
-
- Apr 27, 2022
-
-
Erin (Josh) Rigler authored
-
- Apr 25, 2022
-
-
Erin (Josh) Rigler authored
The original merge request had an extra indentation space that the black formatter didn't like.
-