- Jan 07, 2025
-
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
Added mode for managing existing spikes. Added logic to create_spike_metadata to allow empty arrays.
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
Changed add_empty_channels to False. Changed check_existing_metadata to return whole metadata object.
-
Wernle, Alexandra Nicole authored
Split run into more functions. Added more functionality such that preexisting spikes are handled here instead of only flag-spikes.
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
Changed channels argument to str. Added add_empty_channels to get_timeseries. Changed split_stream_by_day to split stream, not individual traces.
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
-
- Sep 23, 2024
-
-
Erin (Josh) Rigler authored
A FilterAlgorith.py module function get_nearest_time() is supposed to return the nearest *allowed* time for a given filter "step". This worked fine with steps for second and minute data, whose allowed times are the tops of seconds and minutes. However, for hourly (and daily, and any "average" type step), things failed since the allowed times are the center of the interval (for example, for hourly data, which is the average of all minute samples from 00 to 59, the the allowed time is 29:30). One consequence was that if a user specified an interval with a start and end time that did not encompass a given hour's allowed center time (e.g., start=AA:29:31, end=BB:29:29), the algorithm would still return a sample for time BB:29:30. More generally, requests for average type data would include an extra sample. In this fixed version, if start=AA:29:31, and end=BB:29:29, nothing is returned, as intended (and as always worked for non-average type steps). Furthermoref start=AA:29:30, and end=BB:29:29, a sample for hour AA is generated; if start=AA:29:31, and end=BB:29:30, a sample for hour BB is generated, and if start=AA:29:30, and end=BB:29:30, samples for both AA and BB are generated, all as intended.
-
- Sep 12, 2024
-
-
Erin (Josh) Rigler authored
The method FilterAlgorithm.align_trace() pre-processes the `trace.data` input array for subsequent processing by FilterAlgorithm.firfilter() such that the first input array element corresponds to a time step on which output samples must fall (as defined in the dictionary `step`) minus half the fir window width. In short, it ensures that output samples fall on desired time steps. Prior to this fix, it only worked as intended when input trace's starttime fell on an even time step. A bug became obvious when attempting to filter data from non-Geomag stations that did not have nice time stamps. This fix addresses that issue, and also ensures that `align_trace()` does what was claimed in its own original docstrings, which is to handle trailing misalignments as well. Note: one thing `align_trace()` does NOT do is ensure that all needed input data are available to generate desired outputs. The user is responsible for providing this, but can use the FilterAlgorithm.get_input_interal() method to calculate the actual required input starttime and endtime. The method `align_trace()` will trim or pad with NaNs only enough to align time stamps, and may actually result in `firfilter()` output that are NaNs if the input trace was not adequate.
-
- Sep 11, 2024
-
-
Erin (Josh) Rigler authored
- no longer forces default inchannels and outchannels, and lets the AdjustedMatrix.process() method "do the right thing" if neither is explicitly set by the user - some mostly aesthetic changes that more cleanly separate the vector and scalar (F) adjustements - modified AdjustedAlgorithm.can_produce_data(): a. if any non-F inchannels cannot produce data, return False b. if F inchannel can produce data, return True as long as all non-F inchannels can also produce data c. if this is not desirable (e.g., you want to treat non-F and F channels independently), it is necessary to create two instances of AdjustedAlgorithm, one for non-F inchannels, and one for F.
-
- Aug 16, 2024
-
-
Wilbur, Spencer Franklin authored
-
Wilbur, Spencer Franklin authored
Added severl changes to allow for users to retrieve one minute and one hour data from the avaiable one-second data via the FDSN client.
-
- May 28, 2024
-
-
Erin (Josh) Rigler authored
This simply returns True from can_produce_data(), which is appropriate because 1) a stateful algorithm should be able to procude data as long as it starts from a valid state, adn 2) SqDistAlgorithm itself checks this state. If we ever implement a different stateful algorithm, it should do something similar.
-
- Apr 22, 2024
-
-
Geels, Brendan Ryan authored
-
- Apr 17, 2024
-
-
Geels, Brendan Ryan authored
-
- Dec 08, 2022
-
-
Erin (Josh) Rigler authored
I was lazy the first time and didn't subtract delta from starttime when setting the default min_count_end in the configure method. This led to "flakiness" in Dst when long gaps were encountered in input observatories...exactly the kind of issue min_count_end was intended to mitigate. It should work correctly now.
-
- Nov 22, 2022
-
-
Erin (Josh) Rigler authored
Writing unit tests for Controller configurations is tricky, so I didn't do it, and this tiny-but-impactful bug slipped in. I did test this on the stagin server this time, and the processing pipe- line should work after this is deployed.
-
Erin (Josh) Rigler authored
The original version of AverageAlgorithm.py seemed to be written to process the Controller's `--outchannels`. It "worked" for several years because `--outchannels` defaulted to `--inchannels`. But what we actually always wanted was to process `--inchannels`. Mostly this was just a matter of semantics, but when I recently added the ability to average fewer than the full complement of inputs, it was also necessary to change the `can_produce_data()` method to check for "any" instead of "all" channels, which in turn required that the AverageAlgorithm class properly instantiate its `_inchannels` and `_outchannels` class variables, instead of just set them to `None`. To briefly explain the different changes in this commit: - added `Algorithm.__init__(self, inchannels=[channel])` to ensure that can_produce_data() would work when run via programmatic interface. - added `Algorithm.configure(self, arguments)` to AverageAlgorithm.configure() to ensure that can_produce_data() would work when run via Controller.py. - changed all variations of `outchannel` to `inchannel` - cleaned up where class variables were modified inside process() method
-
- Nov 18, 2022
-
-
Erin (Josh) Rigler authored
Recent attempts to change the AverageAlgorithm to allow fewer than the full complement of inputs when calculating a mutli-station average were thwarted by the default can_produce_data() method in the parent Algorithm class, which required all inputs to be present, when what we now want is **any** inputs to be present.
-
- Nov 16, 2022
-
-
Erin (Josh) Rigler authored
-
- Nov 15, 2022
-
-
Erin (Josh) Rigler authored
-
- Nov 14, 2022
-
-
Erin (Josh) Rigler authored
Previously use the number of traces in the input Stream. This could be fewer than the number of desired observatories if a given input's data was completely missing.
-
Erin (Josh) Rigler authored
The recently added `min_count` option to AverageAlgorithm.py leads to some undesirable behavior when realtime data, with asynchronous inputs, are being processed. By adding the ability to specify an interval over which `min_count` is applied, some of this undesirable behavior can be mitigated. In particular, if the `realtime` option is specified via the controller, and `min_count` is defined, the minimum number of inputs will be allowed only for time steps prior to `(UTCDateTime.now() - realtime)`; the full complement of inputs will be required to calculate averages more recent than that. One drawback is that if an input observatory goes offline for an extended period, the Dst index will be calculated with a persistent lag `realtime` seconds long. A user can always override this admittedly ad-hoc default behavior using the `min_count_start` and `min_count_end` options.
-
- Nov 09, 2022
-
-
Erin (Josh) Rigler authored
- added a `min_count` keyword to the AvergeAlgorithm's __init__() method; - added a --average-min-count option via the add_arguments() classmethod; - set `self.min_count = arguments.average_min_count` in configure() method; - refactored process() method to respect `min_count`, but now it defaults to a requirement that all inputs be valid (this modified a recent merge by @awernle that only required that any inputs be valid; - modified AverageAlgorithm_test.py to properly assess things in light of the change to default behavior just mentioned.
-
- Nov 02, 2022
-
-
Wernle, Alexandra Nicole authored
Renamed variables and put into paragraphs. Asserted that the count correctly shows where there are fewer input data.
-
Wernle, Alexandra Nicole authored
-
Wernle, Alexandra Nicole authored
Changed dst_tot to calculate average regardless of nan values and added a new stream that counts available observatories pertimestep
-
- Mar 18, 2022
-
-
Jeremy M Fee authored
-
- Sep 16, 2021
-
-
Cain, Payton David authored
-
- Mar 26, 2021
-
-
Cain, Payton David authored
-
- Mar 15, 2021
-
-
Cain, Payton David authored
-
- Mar 10, 2021
-
-
Cain, Payton David authored
-
- Mar 03, 2021
-
-
Cain, Payton David authored
-