> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cekura.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Improve Evaluators with AI

> Improve evaluators using AI by providing natural language feedback.



## OpenAPI

````yaml post /test_framework/v1/scenarios/scenario-agent/
openapi: 3.1.0
info:
  title: Cekura API
  version: v1
  description: >-
    Complete API documentation for the Cekura platform. This API provides
    endpoints for testing, observing, and evaluating AI voice agents — including
    managing agents, running evaluators, defining metrics, and analyzing call
    quality.
servers:
  - url: https://api.cekura.ai
security: []
paths:
  /test_framework/v1/scenarios/scenario-agent/:
    post:
      tags:
        - test_framework
      description: >-
        Improve evaluators using AI by providing natural language feedback.


        **How it works:**

        1. Send evaluator IDs and your improvement request

        2. AI analyzes and suggests improvements

        3. Review suggestions and apply with `apply_changes=true`


        **`scenarios` field:** pass a list of integer IDs `[101, 102]`, a single
        integer `101`, or the string `"all"` to improve all scenarios for the
        agent. Do not pass IDs as a JSON-stringified list — pass them as a
        native array.


        **Applying large change sets:** when using `apply_changes: true` with
        many scenarios, the `updated_scenarios` payload may be large. If you
        receive an unexpected error, split the apply into smaller batches of 3–5
        scenarios at a time.


        **Response:** Returns `progress_id` for polling status at
        /scenario-agent-progress/


        **Multi-turn improvements:** Use the `context` field to maintain
        conversation history across requests.
      operationId: scenarios-agent-create_2
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ScenarioAgentRequest'
          application/x-www-form-urlencoded:
            schema:
              $ref: '#/components/schemas/ScenarioAgentRequest'
          multipart/form-data:
            schema:
              $ref: '#/components/schemas/ScenarioAgentRequest'
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ScenarioAgentStartResponse'
          description: ''
        '400':
          content:
            application/json:
              schema:
                type: object
                properties:
                  field_name:
                    type: array
                    items:
                      type: string
          description: ''
      security:
        - api_key: []
components:
  schemas:
    ScenarioAgentRequest:
      type: object
      description: >-
        Improve evaluators using AI by providing natural language feedback.


        Provide evaluator IDs and your improvement request, and the AI will
        suggest changes.
      properties:
        user_message:
          type: string
          default: ''
          description: Your improvement request (e.g., 'Make instructions more detailed')
        agent_id:
          type:
            - integer
            - 'null'
          description: >-
            Agent ID - optional, automatically inferred from evaluators if not
            provided
        project_id:
          type:
            - integer
            - 'null'
          description: >-
            Project ID - optional, automatically inferred from evaluators if not
            provided
        session_id:
          type:
            - integer
            - 'null'
          description: Session ID for tracking - automatically created if not provided
        context:
          type:
            - array
            - 'null'
          items:
            $ref: '#/components/schemas/ConversationContextItem'
          description: >

            Conversation history for multi-turn improvements. Allows iterative
            refinement without sessions.


            Example:

            ```json

            [
               {
                  "role": "user",
                  "content": "Add DTMF tool to all evaluators"
               },
               {
                  "role": "assistant",
                  "content": "Added DTMF tool to 3 evaluators"
               },
               {
                  "role": "user",
                  "content": "Make the instructions more detailed"
               },
               {
                  "role": "assistant",
                  "content": "Updated instructions to be more detailed for 3 evaluators"
               }
            ]

            ```
        num_scenarios:
          type: integer
          minimum: 1
          default: 1
          description: Number of scenarios to generate in clarify mode
        knowledge_base:
          type: array
          items:
            type: integer
          description: Knowledge Base file IDs for additional context in clarify mode
        scenarios:
          oneOf:
            - {}
            - type: 'null'
          description: >-
            Evaluator IDs to improve. Pass as a list of integers `[101, 102]`, a
            single integer `101`, or the string `"all"` to improve all
            evaluators for the agent. Do not pass IDs as a JSON-stringified list
            — pass them as a native array.
        apply_changes:
          type: boolean
          default: false
          description: Set to true to apply the AI-suggested changes to your evaluators
        updated_scenarios:
          type: array
          items:
            type: object
            additionalProperties: {}
          description: >-
            AI-suggested changes from previous response - use with
            apply_changes=true to apply them
        agent_selections:
          type: object
          additionalProperties:
            type: integer
            minimum: 1
          description: >-
            Structured agent selections: maps scenario_id (as string key) to
            agent_id. Use this instead of agent_resolution_response.
        agent_resolution_response:
          type: string
          default: ''
          description: 'Deprecated: use agent_selections instead'
        scenarios_needing_agent:
          type: array
          items:
            type: integer
          description: Evaluator IDs that required agent selection
        pending_tool_calls:
          type:
            - array
            - 'null'
          items:
            type: object
            additionalProperties: {}
          description: Tool calls held from a previous agent-clarification response
    ScenarioAgentStartResponse:
      type: object
      properties:
        progress_id:
          type: string
          description: >-
            Use this to poll for results at
            /scenario-agent-progress/?progress_id={id}
        session_id:
          type: integer
          description: Session tracking ID
      required:
        - progress_id
        - session_id
    ConversationContextItem:
      type: object
      properties:
        role:
          enum:
            - user
            - assistant
          type: string
          x-spec-enum-id: 0376a589d982d83c
          description: |-
            Message role - either 'user' or 'assistant'

            * `user` - user
            * `assistant` - assistant
        content:
          type: string
          description: Message content
      required:
        - content
        - role
  securitySchemes:
    api_key:
      type: apiKey
      name: X-CEKURA-API-KEY
      in: header
      description: >-
        API Key Authentication. It should be included in the header of each
        request.

````