End-to-end Azure Databricks retail data engineering project using Medallion Architecture (Bronze, Silver, Gold). Implements Auto Loader, Unity Catalog, Delta Lake, SCD Type 1 & 2 dimensions, and Fact Orders for analytics-ready star schema modeling.

Stock market analytics pipeline with medallion architecture on Databricks

Start Learning From Saylani Institute

Production-style Slowly Changing Dimension (SCD Type 2) pipeline built with Snowflake, dbt, and AWS S3. Demonstrates secure S3 ingestion, layered bronze/silver/gold modeling, dbt snapshots for historical tracking, and analytics-ready views identifying active vs historical records.

End-to-end Metadata-Driven Data Engineering framework built on Azure. Features dynamic SQL/REST API ingestion with range pagination, automated schema mapping, and event-driven orchestration. Implements robust CI/CD via GitHub Actions/YAML and automated failure alerting with Logic Apps. Optimized for scalability and DE best practices.

End-to-end data engineering pipeline using Azure Blob, Data Factory, dbt, Snowflake, and Streamlit for interactive business analytics. (WIP)

End-to-end Databricks LakeFlow (DLT) data engineering project using Medallion Architecture, Auto Loader, CDF, SCD, and Unity Catalog.