Indicator differences in SB data ( p1_unc_stats) and Oronde's supplied data (p2_unc_agg_summary)

As noted in this MR, we found that the SB data ( p1_unc_stats) and processed Oronde's spreadsheet data (p2_unc_agg_summary) have slight differences in indicators. Namely,

p1_unc_stats contains Other non-white while p2_unc_agg_summary does not.

p2_unc_agg_summary contains amenities & hazard type while p1_unc_stats does not.

See excel spread sheet for more (different cells filled in blue) compare_indicators.xlsx

changed title from Indicator differences in SB data ( {--}p1_unc_stats{--}) and Oronde's supplied data ({--}p2_unc_agg_summary{--}) to Indicator differences in SB data ( p1_unc_stats) and Oronde's supplied data (p2_unc_agg_summary)

changed the description

@eazadpour reminder to QAQC the calculations we are using in the pipeline to generate the data for @aclarke . These data are aggregated at the determinant level, summarizing across the individual indicators. To calculate evidence total the number of studies. Agreement is the max percent of studies that agree in direction (positive, negative, no direction, or unknown). So if there are 5 studies that are positive, 4 studies negative, and 1 unknown, the level of agreement is 50%. It's 50% because 5 studies were positive out of the 10, and agreed on the direction of the relationship. If there were 10 studies, 3 positive, 3 negative, 4 no direction, the level of agreement would be 40% because the biggest bin (no direction) was 4/10.

To do:

• Review pipeline steps that calculate level of agreement and evidence for the determinants and check how they are being calculated
• Compare the output to the table used by the project team to check for consistency in general patterns of what has high evidence and high agreement, low evidence and high agreement
• Add variable to pipeline that creates binned versions of evidence and agreement using the project's definitions.
• Make sure this matches up with that table too
• Share resulting data with Aileen

assigned to @eazadpour

just QAQC'd our steps and we have the correct structure indicator_uncertainty.csv

The evidence_val and level_agreement correspond correctly to the data releases Evidence and Agreement columns:Uncertainty_Summary.csv.

Again, where Evidence: Small = total_studies < 5; Medium = total_studies 5-9; Large,total_studies = > 9 and Agreement: Low = < 50% of models; Medium = >50% & <74% of models; High = >74% of models; NA if the level of agreement could not be calculated as indicator was measured only once.

I plan on sharing with Aileen the p2_unc_agg_summary_csv output csv which is the data aggregated at the determinant level that also has evidence_bin (small, medium, large) and agreement_bin (low, medium, high) columns . I will also commit this directly into the repo so she doesn't need to figure out how to run the R pipeline, as you suggested. Here is that csv:determinant_uncertainty.csv

(Addressed in this MR: !12 (merged))

As another note- I also dropped the uncertainty_aggregated_raw_data.csv which was "data provided by Oronde in relation to "Social vulnerability and water insecurity in the western United States: A systematic review of framings, indicators, and uncertainty" (in review), 12/22/2023" so we strictly only have the data release data in the pipeline

This has been resolved since and we're moving forward using the SB data

closed