Skip to content
Snippets Groups Projects
  1. Nov 22, 2022
    • Erin (Josh) Rigler's avatar
      Small fix requiring a length-1 list to be indexed · 5e1ca4cc
      Erin (Josh) Rigler authored
      Writing unit tests for Controller configurations is tricky, so I
      didn't do it, and this tiny-but-impactful bug slipped in. I did
      test this on the stagin server this time, and the processing pipe-
      line should work after this is deployed.
      5e1ca4cc
    • Erin (Josh) Rigler's avatar
      Now process inchannels, not outchannels · e7f9f8ef
      Erin (Josh) Rigler authored
      The original version of AverageAlgorithm.py seemed to be written
      to process the Controller's `--outchannels`. It "worked" for several
      years because `--outchannels` defaulted to `--inchannels`. But what
      we actually always wanted was to process `--inchannels`.
      
      Mostly this was just a matter of semantics, but when I recently added
      the ability to average fewer than the full complement of inputs, it
      was also necessary to change the `can_produce_data()` method to check
      for "any" instead of "all" channels, which in turn required that the
      AverageAlgorithm class properly instantiate its `_inchannels` and
      `_outchannels` class variables, instead of just set them to `None`.
      
      To briefly explain the different changes in this commit:
      
      - added `Algorithm.__init__(self, inchannels=[channel])` to ensure that
        can_produce_data() would work when run via programmatic interface.
      - added `Algorithm.configure(self, arguments)` to AverageAlgorithm.configure()
        to ensure that can_produce_data() would work when run via Controller.py.
      - changed all variations of `outchannel` to `inchannel`
      - cleaned up where class variables were modified inside process() method
      e7f9f8ef
  2. Nov 19, 2022
    • Erin (Josh) Rigler's avatar
      Check for gaps in multi-observatory Stream · c24e2e9c
      Erin (Josh) Rigler authored
      The method TimeseriesUtility.get_stream_gaps() would over-write gaps[channel] if
      there were multiple Traces in a stream that were the same channel, but different
      observatories. This is a typical situation when using the AverageAlgorithm.
      c24e2e9c
  3. Nov 18, 2022
    • Erin (Josh) Rigler's avatar
      Make AverageAlgorithm require **any** inputs · b8523fc4
      Erin (Josh) Rigler authored
      Recent attempts to change the AverageAlgorithm to allow fewer than the full
      complement of inputs when calculating a mutli-station average were thwarted
      by the default can_produce_data() method in the parent Algorithm class, which
      required all inputs to be present, when what we now want is **any** inputs to
      be present.
      b8523fc4
  4. Nov 16, 2022
  5. Nov 15, 2022
  6. Nov 14, 2022
    • Erin (Josh) Rigler's avatar
      Use self.observatories to determine number of inputs · 0dcdead5
      Erin (Josh) Rigler authored
      Previously use the number of traces in the input Stream. This could
      be fewer than the number of desired observatories if a given input's
      data was completely missing.
      0dcdead5
    • Erin (Josh) Rigler's avatar
      Add start/end options AverageAlgorithm's min_count · c946f502
      Erin (Josh) Rigler authored
      The recently added `min_count` option to AverageAlgorithm.py leads to some
      undesirable behavior when realtime data, with asynchronous inputs, are being
      processed. By adding the ability to specify an interval over which `min_count`
      is applied, some of this undesirable behavior can be mitigated.
      
      In particular, if the `realtime` option is specified via the controller, and
      `min_count` is defined, the minimum number of inputs will be allowed only for
      time steps prior to `(UTCDateTime.now() - realtime)`; the full complement of
      inputs will be required to calculate averages more recent than that. One
      drawback is that if an input observatory goes offline for an extended period,
      the Dst index will be calculated with a persistent lag `realtime` seconds long.
      
      A user can always override this admittedly ad-hoc default behavior using the
      `min_count_start` and `min_count_end` options.
      c946f502
  7. Nov 09, 2022
    • Erin (Josh) Rigler's avatar
      Add --average-min-count option to AverageAlgorithm · 4d3d8bd5
      Erin (Josh) Rigler authored
      - added a `min_count` keyword to the AvergeAlgorithm's __init__() method;
      - added a --average-min-count option via the add_arguments() classmethod;
      - set `self.min_count = arguments.average_min_count` in configure() method;
      - refactored process() method to respect `min_count`, but now it defaults
        to a requirement that all inputs be valid (this modified a recent merge
        by @awernle that only required that any inputs be valid;
      - modified AverageAlgorithm_test.py to properly assess things in light
        of the change to default behavior just mentioned.
      4d3d8bd5
  8. Nov 02, 2022
  9. Oct 24, 2022
  10. Sep 30, 2022
  11. Sep 22, 2022
  12. Sep 15, 2022
    • Erin (Josh) Rigler's avatar
      Inadvertent change to scale_value · a0542978
      Erin (Josh) Rigler authored
      The calculation of scale_value in geomagio.residual.calculate() is not
      quite correct, but that needs to be addressed in a separate commit and
      merge request. I'm reverting the recent change here so that we don't
      break the tests.
      a0542978
    • Erin (Josh) Rigler's avatar
      Set proper timestamps in residual calculated absolutes · edc8ba89
      Erin (Josh) Rigler authored
      Fixes #75
      
      When geomagio.residual.calculate() is called with adjust_reference set equal
      to True, the start/end timestamps now both reflect the time of the first absolute
      measurement (i.e., West Down). This is consistent with the residual absolute
      spreadsheet calculations.
      edc8ba89
  13. Aug 20, 2022
  14. Aug 13, 2022
  15. Aug 12, 2022
  16. Aug 11, 2022
    • Erin (Josh) Rigler's avatar
      update interval no longer shrinks with recursion · 6355305d
      Erin (Josh) Rigler authored
      Previously, the endtime-starttime interval being processed by the
      `run_as_update` method would shrink by 1 (second) with each recursion.
      This ultimately broke `run_as_update` when processing anything other
      than 1-second data, but it wasn't obvious because we rarely had to
      actually recurse. There is a little trickery now to ensure that user-
      provided starttime and endtime are inclusive of the full first
      (most recent) update interval, while subsequent, calculated, endtimes
      are set equal to the prior starttime minus specified output_interval
      (e.g., 'second', 'minute', etc.).
      6355305d
  17. Jun 01, 2022
  18. May 27, 2022
    • Erin (Josh) Rigler's avatar
      Remove BYTESIO object creation · e918b4c9
      Erin (Josh) Rigler authored
      It turns out that none of the factories currently being used in
      production expects bytes as input, and in fact, Util.read_url()
      converts everthing to a string anyway, so a BYTES-oriented input
      would couldn't not possibly be working. So, just remove all use
      of BYTESIO, and instead use only STRINGIO.
      e918b4c9
  19. May 18, 2022
  20. May 03, 2022
  21. Apr 27, 2022
  22. Apr 25, 2022
Loading