Another day, another Awesome List repo. A comprehensive list of Chainforge-related content

Official implementation for "GLaPE: Gold Label-agnostic Prompt Evaluation and Optimization for Large Language Models" (stay tuned & more will be updated)

The prompt engineering, prompt management, and prompt evaluation tool for Python

The prompt engineering, prompt management, and prompt evaluation tool for TypeScript, JavaScript, and NodeJS.

The prompt engineering, prompt management, and prompt evaluation tool for Ruby.

A Streamlit web app that uses a Groq-powered LLM (Llama 3) to act as an impartial judge for evaluating and comparing two model outputs. Supports custom criteria, presets like creativity and brand tone, and returns structured scores, explanations, and a winner. Built end-to-end with Python, Groq API, and Streamlit.

The prompt engineering, prompt management, and prompt evaluation tool for Java.

A few prompts that I am storing in a repo for the purpose of running controlled experiments comparing and benchmarking different LLMs for defined use-cases

A Simple Prompt Optimization Using 3 different algorithms for testing.

A hybrid machine learning system for scoring LLM prompts. Features a BERT-based gatekeeper for structural validation and an LLM-based classifier to ensure semantic intent, delivering consistent empirical metrics for prompt engineering.

An AI-driven system to automatically generate, evaluate, and rank prompts using Monte-Carlo and Elo Ranking system for enterprise-grade Retrieval Augmented Generation (RAG) systems.

The prompt engineering, prompt management, and prompt evaluation tool for C# and .NET

The prompt engineering, prompt management, and prompt evaluation tool for Kotlin.

Runs two simple test prompts against 5 Anthropic models. Visually compares speed, capability, costs.