Skip to content

Config Management: Evaluation #557

@AkhileshNegi

Description

@AkhileshNegi

Is your feature request related to a problem?

When running evaluations, users must provide the model name, vector store ID, and prompt as separate inputs each time. This is repetitive and error-prone, especially when running multiple evaluations with the same configuration.

Describe the solution you'd like

Extend the config management feature (added in #430) to support evaluation workflows:

  • Allow saving evaluation parameters (model name, vector store ID, prompt) in a config
  • Enable running evaluations by referencing a config instead of passing individual parameters

This provides consistency with how configs are used elsewhere and reduces friction when running repeated evaluations.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

Status

To Do

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions