Skip to content

Convert ScienceBase models to Model Catalog models with Pydantic validation

Kurt Kleinrequested to merge
github/fork/fujifilmfan/modelcat-17-convert_sb_models_to_mcat_schema into main

The goal of this PR is to add scripts that pull models from ScienceBase, convert those models to Model Catalog models that conform to previously defined Pydantic models, and write the new models in JSON format.

Summary of changes:

  • Add scripts/get_models_from_sb.py
  • Modify scripts/to_json_schema.py to read downloaded JSON files, call model_metadata_mappers.py to migrate the data, and write the new models back to JSON
  • Add scripts/model_metadata_mappers.py which contains helper functions that do the main work of mapping and writing ScienceBase fields to Model Catalog fields
  • Modify existing tests and add some new ones (tests/test_to_json_schema.py) which just check that the mapping functions return the expected object types; these could easily be expanded by adding test dictionaries with a variety of values (the existing test dictionary began as a real SB item, but I gradually added fields to test certain behaviors)
  • Change other_keywords in the BasicProfile schema to Optional[List[str]] (from Optional[str])
  • Minor formatting changes for internal consistency and/or readability

Merge request reports