Agentic AI overview

AI Agents — mission, tasks, tools, guardrails, publish, public embed, playground, evals, and production controls.

Agents (sidebar: Agents, route /app/agents) are Convoship's LLM-first bots. They reason over a mission and persona, match user intent to Tasks, execute Tools through a real dispatcher, and enforce Guardrails before and after each turn. They are separate from Assistants (/app/assistants), which use scripted intents and flow graphs.

Fresh workspaces start with zero agents. Use Read the playbook on the dashboard (/docs/agent-vs-assistant) when choosing between an Agent and an Assistant.

How a turn works

  1. Task match — keyword match on task examples narrows which tools the LLM may call.
  2. Pre-tool guardrails — AUTH, LIMIT, and BUDGET can block tool execution.
  3. LLM loop — multi-step reasoning with tool_use / tool_result blocks (caps: max_llm_steps_per_conv, max_tool_calls_per_turn).
  4. Tool dispatch — http_post, webhook (HMAC-signed), python_callable, or noop stub until configured.
  5. Post-reply guardrails — REDACT and ESCALATE on the outbound message.
  6. Persist — Playground and public embed write AiAgentConversation + AiAgentTraceEvent (PII-redacted at rest).

Landing dashboard

  • Filter tabs — All, Live, Draft with live counts.
  • Sort — Recently updated, Name A→Z.
  • Agent cards — status, model label, task/tool counts, 7-day conversations, channel chips, auto-resolved %.
  • New Agent — /app/agents/new four-step wizard with draft autosave.

Agent shell (Design + Operate)

TabRoutePurpose
Brain/app/agents/{slug}Mission summary, persona, recent convos, quick stats
Tasks.../tasksIntent cards — example phrase, needs[], tool_names[], SENSITIVE/FALLBACK flag
Tools.../toolsTool catalog + handler editor (kind + JSON config) + test dispatch
Knowledge.../knowledgeRAG sources (file/URL); retrieved in playground and public turns
Persona.../personaTitle, tone chips, voice, boundaries
Memory.../memoryLabeled memory slots for long-running context
Guardrails.../guardrailsAUTH, LIMIT, REDACT, ESCALATE, BUDGET — five defaults seeded at agent create
Deploy.../publishPublish, public token, embed snippet, version history, eval pass rates, rollback
Playground.../playgroundStudio chat with SSE streaming (?stream=true)
Conversations.../convosProduction + playground traffic
Evals.../evalsRegression cases + publish trend chart + per-version drill-down
Audit.../auditAiAgentAuditEvent trail for admin mutations
Analytics.../analyticsKPIs + daily conversation chart

Publish vs draft

  • Draft — editable rows (tasks, tools, guardrails) used in Playground and eval runs.
  • Publish — sets status live, bumps version label (v1 → v2), mints public_token if missing, snapshots AiAgentVersion.
  • Public runtime — embed and POST /v1/public/ai-agents/{public_token}/turn load the published snapshot, not draft edits.
  • Eval gate — when Workspace.eval_pass_threshold > 0, publish runs all evals and blocks if pass rate is below threshold.
  • Rollback — POST .../rollback/{version_id} restores a prior snapshot to the live agent.

LLM provider & BYOK

All agent LLM calls use Claude via ANTHROPIC_API_KEY. Per-workspace BYOK: Workspace Settings → AI Agent LLM — provider must be anthropic; stores an encrypted API key used by playground, evals, and the public channel when configured.

New Agent wizard

See /docs/new-agent-wizard for step-by-step fields. Summary: Mission → Tools catalog → Behavior (persona, caps, guardrails) → Review POST /v1/ai-agents.

  • Tool handlers & dispatch — /docs/agentic-tool-handlers
  • Publish, embed & versions — /docs/agentic-publish-deploy
  • Playground & evals — /docs/agentic-playground
  • Workspace production settings — /docs/agentic-production
  • Public embed SDK — /docs/embed-sdk (AI Agent publicToken path)
  • API routes — /docs/api-reference