Categorical Score Calibrations #589
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…cular dependencies Implements a centralized model rebuilding strategy for Pydantic model rebuilding. Instead of maintaing these model rebuilds in each file, we now can import circular dependencies in an `if TYPE_CHECKING:` block. The model rebuild module will then automatically handle model rebuilds, walking the view_model module and dynamically rebuilding our models based on their sub-classes. This should substantially increase ease of maintainability when adding dependent Pydantic models.
… multipart form data
- Added new SQLAlchemy model `ScoreCalibrationFunctionalClassification` to represent functional classifications associated with score calibrations. - Established relationships between `ScoreCalibration` and `ScoreCalibrationFunctionalClassification`. - Created an association table for many-to-many relationships between functional classifications and variants. - Updated view models to accommodate new functional classification structures, including validation for inclusive bounds. - Enhanced tests to cover new functionality, including creation and validation of functional classifications. - Refactored existing code to ensure compatibility with new models and relationships.
… ScoreCalibration model
…ariants by score range
- Add a property `class_` to score calibration functional classifications. One of `range` or `class_` must be defined - Add validation logic to class based score ranges - Refactor lib code to support both range types - Refactor tests to support both range types TODO: Support for creating variant associations in class based score ranges.
…and adjust related tests
…ClassificationBase
…ionFunctionalClassification
- Added router functionality for validation and standardization of class based calibration files. - Added lib functionality for creation/modification of class based calibrations. - Invoked lib functionality from routers to allow client creation/modification of class based calibrations. - Introduced a new CSV file `calibration_classes.csv` containing variant URNs and their corresponding class names. - Implemented tests for creating and updating score calibrations using class-based classifications. - Enhanced existing test suite with parameterized tests to validate score calibration creation and modification. - Ensured that the response includes correct functional classifications and variant counts.
…ration creation and modification
…dification routes
…_pro - Allow class-based calibration to be defined via hgvs strings - Introduced new test CSV files for calibration classes based on HGVS nucleotide, HGVS protein, and URN. - Enhanced test coverage for score calibration creation and updating, including scenarios for decoding errors and validation errors. - Refactored tests to utilize parameterization for different calibration class files. - Added validation checks for index column selection in calibration dataframes. - Improved error messages for missing or invalid calibration classes.
bencap
force-pushed
the
feature/bencap/538/categorical-calibrations
branch
from
d524416 to
4962f4f
Compare
Collaborator
|
I ran into two unexpected (I think) validation errors. For functional classes with evidence strength, I'm seeing this: When that error isn't present, I run into this: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request implements a major migration of functional range data in MaveDB, moving from a legacy JSONB-based representation to a new normalized relational schema with explicit tables for ACMG classifications, functional classifications, and their associations to variants. The migration includes new Alembic migrations, a data migration script, and supporting code changes for model and enum usage.
The changes also include two related supporting changes: an improvement to the way we handle Pydantic forward references that rebuilds models dynamically upon the import of any Pydantic class and a model loading utility that simplifies route definitions when both JSON and Form data must be supported by routes with optional file uploads.
Key changes:
Database Schema Migration:
acmg_classifications,score_calibration_functional_classifications, and an association table for linking functional classifications to variants. The oldfunctional_rangesJSONB column is renamed and then dropped after migration. [1]], [2]])class_column and renamingclassificationtofunctional_classificationin the relevant table. ([alembic/versions/0520dfa9f2db_rename_functional_ranges_to_functional_.pyR1-R45])Data Migration Script:
alembic/manual_migrations/migrate_jsonb_ranges_to_table_rows.py, a comprehensive script to migrate existing JSONB functional ranges to the new tables, including logic to create ACMG classification records, functional classification rows, and variant associations. The script also supports verification and rollback of the migration. ([alembic/manual_migrations/migrate_jsonb_ranges_to_table_rows.pyR1-R374])Model and Enum Refactoring:
ACMGCriterion,StrengthOfEvidenceProvided) out ofsrc/mavedb/lib/acmg.pyto their own modules, and updated imports to use the new locations. ([src/mavedb/lib/acmg.pyL1-R8])Pydantic Model Circularity:
model_rebuildinsrc/mavedb/__init__.py. ([src/mavedb/init.pyR12-R14])Flexible Loader for multipart/form-data
flexible_module_loader.pywith a generic dependency generatorcreate_flexible_model_loaderand convenience methodjson_or_form_loader. These dependency generators can be used to dynamically create parsers for routes that require support for JSON data and optional file uploads. ([src/mavedb/lib/flexible_calibration_loader.py.pyR185])