feat(api): AI services backend — refine prompt tool call #3687

mmabrouk · 2026-02-09T19:05:59Z

Summary

Adds a new REST API at /preview/ai/services/ with an MCP-shaped tool-call contract for AI-powered features
Implements the first tool: tools.agenta.api.refine_prompt — refines a prompt template by calling a deployed prompt in an internal Agenta org
Includes EE permission checks (EDIT_WORKFLOWS), rate limiting (10 burst / 30 per min), and input/output validation

Endpoints

Method	Path	Description
`GET`	`/preview/ai/services/status`	Returns enabled flag + available tools
`POST`	`/preview/ai/services/tools/call`	Executes a tool call

Architecture

Backend calls a deployed prompt in an internal Agenta org via POST {API_URL}/services/completion/run. No direct LLM provider calls — Bedrock credentials live in the internal app config.

Feature is env-var gated (AGENTA_AI_SERVICES_*). When not configured, status returns enabled: false and tool calls return 503.

Files

Backend

api/oss/src/core/ai_services/ — DTOs, HTTP client, service layer
api/oss/src/apis/fastapi/ai_services/ — FastAPI router + models
api/oss/src/utils/env.py — AIServicesConfig
api/entrypoints/routers.py — wiring

Design docs

docs/design/ai-actions/ — spec, plan, context, research, status

What's next

Phase 2: Frontend integration (status query, API client, Refine prompt button in Playground)
Phase 3: Hardening (structured logging, trace_id propagation)

Implement the backend for Chapter 1 of the AI services feature: - REST API with MCP-shaped tool-call contract at /preview/ai/services/ - Single tool: tools.agenta.api.refine_prompt - Thin HTTP client calling deployed prompt in internal Agenta org - EE permission check (EDIT_WORKFLOWS) and rate limiting - Input/output validation for prompt templates - Design docs with spec, plan, context, and research

vercel · 2026-02-09T19:10:46Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
agenta-documentation	Ready	Preview, Comment	Feb 11, 2026 9:02am

devin-ai-integration

Devin Review found 1 potential issue.

View 7 additional findings in Devin Review.

api/oss/src/core/ai_services/service.py

The design reference doc used 'prompt_text' as the input key while the service code, DTOs, spec, and input schema all use 'prompt_template_json'. Align the doc to match.

Remove empty __init__.py file in ai_services module

junaway · 2026-02-11T08:55:57Z

api/oss/src/apis/fastapi/ai_services/router.py

+
+    @intercept_exceptions()
+    async def get_status(self, request: Request) -> AIServicesStatusResponse:
+        allow_tools = True


Is this meant to become an env var or a feat flag ?

junaway · 2026-02-11T08:58:05Z

api/oss/src/apis/fastapi/ai_services/router.py

+            allow_tools = await check_action_access(  # type: ignore
+                user_uid=request.state.user_id,
+                project_id=request.state.project_id,
+                permission=Permission.EDIT_WORKFLOWS,  # type: ignore


Are all tools meant to mutate workflows ?
Should we have VIEW_AI_SERVICES and RUN_AI_SERVICES like we have for RUN_SERVICES, and then leave specific entitlements to specific tools ?

junaway · 2026-02-11T08:59:48Z

api/oss/src/apis/fastapi/ai_services/router.py

+            ):
+                raise FORBIDDEN_EXCEPTION  # type: ignore
+
+        # Router-level rate limit


Out of curiosity,
(1) why not via entitlements ?
(2) why not via the middleware ?

junaway · 2026-02-11T09:01:12Z

api/oss/src/apis/fastapi/ai_services/router.py

+                headers={"Retry-After": str(retry_after)},
+            )
+
+        # Tool routing + strict request validation


Eventually, we might want to push this down to the dispatcher, which would generate a domain-level exception, caught here and turned into an HTTP exception.

Removed the docstring for the AI Services core module.

junaway · 2026-02-11T09:03:37Z

api/oss/src/core/ai_services/client.py

+        Returns: (raw_response, trace_id)
+        """
+
+        url = f"{self.api_url}/services/completion/run"


This turns into {BASE_URL}/api/services/{SERVICE_PATH} instead of {BASE_URL}/services/{SERVICE_PATH}, no ?

junaway · 2026-02-11T09:06:17Z

api/oss/src/core/ai_services/client.py

+                    url=url,
+                )
+                # Surface as tool execution error (caller maps to isError)
+                return {


You might want to create domain-level exceptions via Pydantic models and then raise exception. There are some examples of this throughout the codebase (not enough IMO).

junaway · 2026-02-11T09:06:46Z

api/oss/src/core/ai_services/client.py

+            if isinstance(data, dict):
+                trace_id = data.get("trace_id") or data.get("traceId")
+
+                return data, trace_id


Same thing for the returns data via dtos.

junaway · 2026-02-11T09:09:11Z

api/oss/src/core/ai_services/service.py

+
+class AIServicesService:
+    @classmethod
+    def from_env(cls) -> "AIServicesService":


I'm going to steal this idea of .from_env() clean up the not-yet-inverted dependency to env vars. Thanks !

junaway · 2026-02-11T09:09:58Z

api/oss/src/core/ai_services/service.py

+class AIServicesService:
+    @classmethod
+    def from_env(cls) -> "AIServicesService":
+        config = env.ai_services


I'd avoid this intermediate variables, for readability.

junaway · 2026-02-11T09:10:45Z

api/oss/src/core/ai_services/service.py

+
+        # enabled implies these exist, but keep this defensive.
+        if not api_url or not api_key:
+            return cls(config=config, client=None)


You're still coupling to structure of env vars and the settings in the service, unless you de-structure the config dict. Not a big problem, though, just flagging it.

junaway · 2026-02-11T09:11:53Z

api/oss/src/core/ai_services/service.py

+        if not api_url or not api_key:
+            return cls(config=config, client=None)
+
+        client = AgentaAIServicesClient(


Nice dependency injection.
A purist would move this to adapters, have an interface for it, and would have this as the first implementation.

junaway · 2026-02-11T09:13:04Z

api/oss/src/core/ai_services/service.py

+                ToolDefinition(
+                    name=TOOL_REFINE_PROMPT,
+                    title="Refine Prompt",
+                    description=(


Maybe a _REFINE_PROMPT_HEADER_NAME/DESCRIPTION ?

junaway · 2026-02-11T09:13:32Z

api/oss/src/core/ai_services/service.py

+    async def call_tool(
+        self, *, name: str, arguments: Dict[str, Any]
+    ) -> ToolCallResponse:
+        if name != TOOL_REFINE_PROMPT:


Ah, here it is.
Double defense.

junaway · 2026-02-11T09:14:51Z

api/oss/src/core/ai_services/service.py

+            ],
+        )
+
+    async def call_tool(


As this grow, honestly not too far in the future, this will probably turn into nested dispatchers:
tools.agenta.api.refine_prompt turns into:

dispatch to agenta tools handler
dispatch to api tools handler
dispatch to refine_prompt handler

junaway · 2026-02-11T09:18:25Z

api/oss/src/utils/env.py

+
+    api_key: str | None = os.getenv("AGENTA_AI_SERVICES_API_KEY")
+    api_url: str | None = os.getenv("AGENTA_AI_SERVICES_API_URL")
+    environment: str | None = os.getenv("AGENTA_AI_SERVICES_ENVIRONMENT")


These names would deserve some love:

ENVIRONMENT_SLUG / REFINE_PROMPT_KEY [recommended] (to match the preview entities)

ENVIRONMENT_NAME / APP_SLUG (to match the legacy entities)

vercel bot deployed to Preview February 9, 2026 19:11 View deployment

Merge branch 'main' into feat/refine-ai-feature

0f04e37

devin-ai-integration bot reviewed Feb 9, 2026

View reviewed changes

api/oss/src/core/ai_services/service.py Show resolved Hide resolved

vercel bot deployed to Preview February 9, 2026 19:22 View deployment

fix(docs): align cloud prompt config input key to match service code

e88293d

The design reference doc used 'prompt_text' as the input key while the service code, DTOs, spec, and input schema all use 'prompt_template_json'. Align the doc to match.

vercel bot deployed to Preview February 9, 2026 19:28 View deployment

Merge branch 'main' into feat/refine-ai-feature

be41428

vercel bot deployed to Preview February 9, 2026 22:35 View deployment

mmabrouk mentioned this pull request Feb 10, 2026

feat(frontend): Refine Prompt modal — AI-powered prompt refinement UI #3698

Open

9 tasks

mmabrouk requested a review from jp-agenta February 10, 2026 18:16

mmabrouk marked this pull request as ready for review February 10, 2026 18:16

dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. Backend labels Feb 10, 2026

junaway added 2 commits February 11, 2026 09:54

Update __init__.py

454a876

Delete empty __init__.py file

7ce78e4

Remove empty __init__.py file in ai_services module

vercel bot deployed to Preview February 11, 2026 08:55 View deployment

junaway reviewed Feb 11, 2026

View reviewed changes

Delete docstring from AI Services core module

601bbee

Removed the docstring for the AI Services core module.

vercel bot deployed to Preview February 11, 2026 09:02 View deployment

junaway reviewed Feb 11, 2026

View reviewed changes

feat(api): AI services backend — refine prompt tool call #3687

Are you sure you want to change the base?

feat(api): AI services backend — refine prompt tool call #3687

Conversation

mmabrouk commented Feb 9, 2026 • edited by devin-ai-integration bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Endpoints

Architecture

Files

Backend

Design docs

What's next

Uh oh!

vercel bot commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

devin-ai-integration bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

junaway Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

junaway Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mmabrouk commented Feb 9, 2026 •

edited by devin-ai-integration bot

Loading

vercel bot commented Feb 9, 2026 •

edited

Loading

junaway Feb 11, 2026 •

edited

Loading

junaway Feb 11, 2026 •

edited

Loading