Draft: New Instrument class and Observatory class edits

original MR: !279 (closed)

Tagging @bgeels, @nshavers and @swilbur to keep them in the loop.

The goal of this merge request is to develop the metadata framework and backend tools for a frontend inventory manager.

Some confusing background information:

The database, or metadata table assumes the structure of Metadata.py with attributes as columns, this is defined in metadata_table.py. The metadata_history_table preserves the history of a metadata object from the metadata_table.py but copies the same structure. So, all of the Metadata.py attribute values (and therefore the indices) can be redefined except for id which serves as the key to the metadata table. This is important because we don't want to create a new metadata object every time something about an observatory or instrument changes but we want to preserve the history of the changes.

If you look at the Metadata.py you will notice the category attribute, the categories are defined here. Currently, we have a class defined for Reading and Observatory but I would like to further define Observatory and write an Instrument class in this MR. Keep in mind that these classes will define what is in the JSON blob within the Metadata class in the metadata attribute (currently defined as a Dict). Here is an example:

Click to expand

{ "id": 35535,
"metadata_id": null,
"created_by": "copy_observatory.py",
"created_time": "2023-06-27T14:54:50.000000Z",
"updated_by": null,
"updated_time": null,
"starttime": null,
"endtime": null,
"network": "NT",
"station": "BSL",
"channel": null,
"location": null,
"category": "observatory",
"priority": 1,
"data_valid": true,
"metadata": {
"id": "BSL",
"name": "Stennis Space Center",
"agency": "USGS",
"latitude": 30.35,
"elevation": 8,
"longitude": 270.365,
"agency_name": "United States Geological Survey (USGS)",
"declination_base": 215772,
"sensor_orientation": "HDZF"
},
"comment": null,
"review_comment": null,
"status": "new"
},\

The database was designed this way to have freedom defining what is within the JSON blobs and also so that we do not use linked tables. However, we can link elements of a single table (or two, counting the history table) through fields in their JSON blobs.

A major topic of discussion between Josh, Brendan and I so far has been what features should be specifically associated with Instruments and Observatories and how to link them. Here are some thoughts I/we have had:

• Instrument class should include offsets, electronics, sensors, dataloggers, and digitizers.
• the Active_Instruments class will serve as a "bucket" for these items that are currently active and will be defined within an Observatory metadata object but also simultaneously linked to an Instrument metadata object (probably via starttime/endtime).
• Observatory class should include marks and piers, however there can be multiple marks and piers at a site and therefore multiple azimuth values. We may choose to differentiate certain mark and pier arrangements with different location codes, meaning we could have two "BOU" observatory metadata objects with their own metadata_history, but they would have different location codes and contain different mark/pier arrangements.
• We may have a single observatory "bucket" similar to the Active_Instruments class that will contain all observatory metadata for a particular observatory. Meaning, we may have one "BOU" metadata object, that contains and links to all other "BOU" metadata objects with different location codes that I described above.
• I have yet to decide if I think the above point is making things too complicated by having buckets within buckets, and I may just want to have all marks and piers under one observatory metadata object instead of with different configurations under different location codes.