Skip to content

🧱 Mason: Decouple Transformer Layers (Fixes DIP)#440

Draft
fderuiter wants to merge 1 commit intomainfrom
mason-decouple-transformer-layers-7450992865170839797
Draft

🧱 Mason: Decouple Transformer Layers (Fixes DIP)#440
fderuiter wants to merge 1 commit intomainfrom
mason-decouple-transformer-layers-7450992865170839797

Conversation

@fderuiter
Copy link
Owner

🧱 Mason: Decouple Transformer Layers (Fixes DIP)

🏚️ Violation: EncoderLayer and DecoderLayer were tightly coupled to concrete implementations of MultiHeadAttention, FeedForward, and LayerNorm, violating the Dependency Inversion Principle and making extensions (e.g., Sparse Attention) impossible without modification.

🏗️ Fix:

  1. Introduced traits module with AttentionMechanism, FeedForwardNetwork, and NormalizationLayer.
  2. Implemented these traits for MultiHeadAttention, FeedForward, and LayerNorm.
  3. Refactored EncoderLayer and DecoderLayer to hold Box<dyn Trait> components.
  4. Added new_with_components for dependency injection.

🔗 Principle: Dependency Inversion Principle (DIP) and Open/Closed Principle (OCP).

🧪 Verification:

  • cargo check passed.
  • cargo test passed (including ai::transformer tests).
  • No changes to external behavior of new constructors.

PR created automatically by Jules for task 7450992865170839797 started by @fderuiter

Extracted `AttentionMechanism`, `FeedForwardNetwork`, and `NormalizationLayer` traits.
Updated `EncoderLayer` and `DecoderLayer` to use `Box<dyn Trait>` components.
Added `new_with_components` constructors for flexible instantiation.
Maintained backward compatibility in `new` constructors.
Verified with existing tests.

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
@google-labs-jules
Copy link
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant