updated pydantic, updated pydantic_utcdatetime custom type, added tests
@erigler , @awernle , @bgeels , @nshavers
Update pydantic to the most recent version (2.9.2). The majority of the changes are changing the syntax for optional fields. str = None is no longer valid and the new version uses Optional[str] = None as per this documentation https://docs.pydantic.dev/latest/migration/#required-optional-and-nullable-fields. This is a bit mind bending to me, so please let me know if I interpreted something incorrectly.
Another common change is from UTCDateTime to CustomUTCDateTimeType. @swilbur and I created this new custom type and added it to the pydantic schema in pydantic_utcdatetime.py and there's a corresponding test file pydantic_utcdatetime_test.py to make sure we're allowing all valid inputs.
DataApiQuery and FilterApiQuery have lots of changes because the updated default values. According to this documentation https://docs.pydantic.dev/latest/concepts/validators/#validation-of-default-values, the suggested way to validate default values is by using @model_validator(mode=before). I find this to be a bit clunky and redundant if we're validating the default values every time a class is being instantiated. Instead, I opted to write unit tests here in DataApiQuery_test.py and FilterApiQuery_test.py to make sure defaults are being set as expected.
I also opted to move some of the defaults in line with the field. For example in data_type on DataApiQuery. Before, it was allowing any string with the Union and doing the comparison in the @field_validator. This version of pydantic is a lot more strict and does that validation under the hood so we don't need to explicitly check data_type. You can see this working in the corresponding tests.
There are some more smaller changes that have comments on them either in the code or on the MR.
I'm going to set up time to go over this together. I anticipate it will take some time to review this and make sure everyone is comfortable with it, so no pressure. Please ask all the questions and raise any concerns!
Merge request reports
Activity
- Resolved by Hobbs, Alexandra (Contractor)
Hi Ali and Spencer, can you elaborate on the logic behind circumventing the
@model_validator(mode=before)for validation when the class is instantiated? Perhaps I am misunderstanding, but I heavily rely on the classes being properly validated when I test and run metadata applications. We can discuss this in today's meeting if necessary. - Last reply by Hobbs, Alexandra (Contractor)
assigned to @erigler
requested review from @erigler
