Update/modify EdgeFactory for new Edge server configurations and SNCL standard

This is an umbrella issue for discussing changes to EdgeFactory that are needed to address planned (some already implemented) changes in the Geomagnetism Program's EdgeCWB server configurations. This includes:

• appropriate default ports as first described in issue #91 (closed), and only partially addressed in MR !225 (merged)
  • defaults should correspond to public EdgeCWB servers, which differ from private servers
  • internal processing pipeline, typer scripts, and CI pipeline configurations need to be modified not to rely on defaults, but explicitly configure ports as needed
• read data using the CWBQuery protocol (obspy's neic client) by default, but allow waveserver (obspy's earthworm client)
  • this flexibility is to help transition off the old Geomag EdgeCWB configurations, so it might be advisable to make the protocol contingent on port designation, and not to add any new class configuration parameters
  • ??? other specific requirements ???
• write data using raw input client (RIC) by default, but allow override using MiniSeedInputClient
  • default port should be one that is "safe" (sort of a sandbox that can be easily purged)
  • RIC and MiniSeedInputClient should not write to the same port
  • ??? other specific requirements ???

I have my own feature branch where many/most of these are already initiated. However, it needs substantial testing before I would be comfortable deploying changes with so much potential to break our production processing pipeline. So, if you have anything to add here, please do, and I will try to incorporate that into said feature branch.

Mentioning a few folks who might be interested in this issue: @bgeels, @awernle, @swilbur

added Discuss To Do labels

assigned to @erigler

changed the description

marked the checklist item write data using raw input client (RIC) by default, but allow override using MiniSeedInputClient as completed

marked the checklist item read data using the CWBQuery protocol (obspy's neic client) by default, but allow waveserver (obspy's earthworm client) as completed

marked the checklist item appropriate default ports as first described in issue #91 (closed), and only partially addressed in MR !225 (merged) as completed

I'm going to close this issue. It is essentially addressed by my recently submitted MR !382 (closed), although some of the proposed details above changed once I started testing the EdgeFactory updates in a simulated production mode. Just to note those here for possible future reference:

• all default "write" ports were removed
  • geomagio scripts will need to be updated to define write ports
  • default protocol is still raw input, but port must be specified explicitly
  • filters.py hard codes a raw input port that will be decommissioned once data migration is complete
• the "CWBQuery" (also known as "Queryserver") protocol is not default, Earthworm/Waveserver is
  • this is to retain backward compatibility with EdgeFactory
  • port 2060 is default for Waveserver, since this is as standard as anything gets to be with Edge
  • this may change once data migrations are complete

closed