Skip to content

Conversation

@mmabrouk
Copy link
Member

@mmabrouk mmabrouk commented Feb 9, 2026

Summary

  • Update all 4 frontend chat detection paths to prefer the explicit x-agenta.flags.is_chat OpenAPI extension from the SDK, falling back to the existing messages-property heuristic for older apps
  • Simplify isChatVariantAtomFamily in both appRevision and legacyAppRevision to read pre-computed isChatVariant from schema state instead of re-iterating endpoints

Detection paths updated

Path File Change
Legacy playground genericTransformer/index.ts detectChatVariantFromOpenAISchema() checks operation["x-agenta"].flags.is_chat first
Entity layer appRevision/api/schema.ts extractAllEndpointSchemas() checks flag on all endpoints before heuristic
Entity atoms appRevision/state/runnableSetup.ts Reads schemaState.isChatVariant (already computed with flag priority)
Legacy entity atoms legacyAppRevision/state/runnableSetup.ts Same simplification
New playground requestSchemaMeta.ts hasMessages prefers flag, falls back to properties.messages

Stacked on

Testing


Open with Devin

Update all 4 frontend chat detection paths to prefer the explicit
x-agenta.flags.is_chat flag from the SDK's OpenAPI extension, falling
back to the existing messages-property heuristic for older apps.

Detection paths updated:
- genericTransformer/index.ts: detectChatVariantFromOpenAISchema()
- appRevision/api/schema.ts: extractAllEndpointSchemas()
- appRevision + legacyAppRevision runnableSetup.ts: isChatVariantAtomFamily
  (now reads pre-computed isChatVariant from schema state)
- requestSchemaMeta.ts: hasMessages computation

Stacked on PR #3622 (SDK flags support).
@vercel
Copy link

vercel bot commented Feb 9, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Ready Ready Preview, Comment Feb 9, 2026 11:28pm

Request Review

@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Feb 9, 2026
@dosubot dosubot bot added the Frontend label Feb 9, 2026
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 4 additional findings in Devin Review.

Open in Devin Review

@mmabrouk mmabrouk requested a review from ardaerzin February 9, 2026 23:27
…allback

Fix pre-existing bug surfaced by this PR: ep?.messagesSchema !== null
evaluates to true for null endpoints (undefined !== null is true),
causing every app without the x-agenta flag to be detected as chat.
Change to !!ep?.messagesSchema for correct truthiness check.
@mmabrouk
Copy link
Member Author

mmabrouk commented Feb 9, 2026

Testing Results

Deployment: http://144.76.237.122:8180

What was tested

  • Playground chat detection: Opened a chat app in the playground — chat UI rendered correctly (messages input, conversation history)
  • Add to testset from playground: Ran a chat interaction and saved results to a testset — worked correctly
  • OpenAPI flag verified: Confirmed x-agenta.flags.is_chat: true is present in the OpenAPI spec for the chat app

What's confirmed working

  • Frontend correctly reads x-agenta.flags.is_chat from the OpenAPI spec
  • Chat UI renders when flag is true
  • Backward-compatible heuristic fallback still works (checked via properties.messages detection)

Additional tests needed

  • Non-chat app playground: Open a completion app and confirm it does NOT show chat UI
  • Older SDK app (no x-agenta flag): Verify the heuristic fallback correctly detects chat for apps using older SDK versions that don't emit the flag
  • Comparison view: Test side-by-side comparison of chat variants — should show GenerationComparisonChatOutput
  • New playground path: Test via the new playground (uses requestSchemaMeta.ts / variantFlags.ts) — verify hasMessages is set correctly
  • Entity layer detection: Verify isChatVariantAtomFamily returns correct value for both chat and non-chat revisions

Copy link
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 7 additional findings in Devin Review.

Open in Devin Review


const hasMessages = Boolean(properties?.messages)
// Prefer explicit x-agenta.flags.is_chat, fall back to messages property heuristic
const flagValue = getIsChatFlag(spec as OpenAPISpec, routePath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 getIsChatFlag ignores variant.routePath fallback used by getRequestSchema

When routePath param is undefined, getIsChatFlag builds paths without a route prefix (e.g., /run), but getRequestSchema falls back to (opts.variant as any)?.routePath (web/oss/src/lib/shared/variant/openapiUtils.ts:10). This means the two functions may look up different OpenAPI paths for the same call.

Root Cause

In requestSchemaMetaAtomFamily, both getRequestSchema and getIsChatFlag are called with the same routePath param:

const requestSchema = getRequestSchema(spec, {variant, routePath})
const flagValue = getIsChatFlag(spec, routePath)

However, getRequestSchema internally resolves the route path as:

const routePath = opts.routePath ?? (opts.variant as any)?.routePath

So when routePath is undefined but variant.routePath has a value (e.g., "my-app/v1"), getRequestSchema correctly looks up /my-app/v1/run, while getIsChatFlag looks up /run — a path that likely doesn't exist in the spec. The flag lookup silently fails and returns undefined, causing the code to fall back to the heuristic.

Impact: The explicit x-agenta.flags.is_chat flag is silently ignored whenever routePath is undefined but variant.routePath is set. The heuristic fallback still produces correct results for older apps, but for apps relying on the new flag (e.g., chat variants without a messages property), this could yield an incorrect hasMessages: false.

Suggested change
const flagValue = getIsChatFlag(spec as OpenAPISpec, routePath)
const effectiveRoutePath = routePath ?? (variant as any)?.routePath
const flagValue = getIsChatFlag(spec as OpenAPISpec, effectiveRoutePath)
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Frontend size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant