Agent Message System

The agent message system is Vertesia's real-time communication layer between agent workflows and client applications. It handles message creation, delivery, streaming, and rendering through a well-defined pipeline.

Architecture Overview

Messages flow through the following pipeline:

User Input --> Signal (UserInput) --> Temporal Workflow
                                          |
                                  Agent/Tool Execution
                                          |
                              postUpdateMessage activity
                                          |
                          POST /runs/:runId/updates
                                          |
                             Redis (list + pub/sub)
                                          |
                  +--------------------------------------+
                  | Phase 1: GET /updates (gzip)         |
                  | Phase 2: SSE /stream (real-time)     |
                  +--------------------------------------+
                                          |
                       ModernAgentConversation UI

Key participants:

• Temporal Workflows orchestrate agent execution and tool calls
• Redis stores message history (up to 1000 messages, 90-day TTL) and delivers real-time updates via pub/sub
• zeno-server exposes REST and SSE endpoints for message retrieval and streaming
• @vertesia/client provides the WorkflowsApi that handles both historical fetch and real-time SSE
• @vertesia/ui provides ModernAgentConversation, the recommended UI component

Message Types

Every agent message has a type field from the AgentMessageType enum (@vertesia/common):

Message Formats

The system uses two message formats for different purposes:

• CompactMessage -- the wire format, optimized for bandwidth (~85% smaller). Used between server and client over SSE/WebSocket.
• AgentMessage -- the UI format, with readable field names. Used by UI components and application code.

The client SDK converts wire messages to AgentMessage automatically, so application code always works with the friendlier format.

CompactMessage (wire format)

Used over the wire between server and client. You should not need to work with this directly.

interface CompactMessage {
    t: AgentMessageType;  // Message type
    m?: string;           // Message content
    w?: string;           // Workstream ID (omitted when "main")
    d?: unknown;          // Type-specific details
    f?: 0 | 1;           // Is final chunk (streaming only)
    ts?: number;          // Timestamp
    i?: string;           // Activity ID (for deduplication)
}

AgentMessage (UI format)

The format used by UI components and application code. This is what you receive in streamMessages callbacks and what ModernAgentConversation works with internally.

interface AgentMessage {
    timestamp: number;
    workflow_run_id: string;
    type: AgentMessageType;
    message: string;
    details?: any;
    workstream_id?: string;
}

Conversion Utilities

The @vertesia/common package provides utilities for working with both formats:

• parseMessage(data) -- Accepts string or object, returns CompactMessage regardless of input format
• toAgentMessage(compact, runId) -- Converts wire format to UI format (done automatically by the client SDK)
• toCompactMessage(msg) -- Converts UI format to wire format
• createCompactMessage(type, options) -- Convenience constructor for server-side code

Real-Time Streaming

The client uses a two-phase streaming strategy for optimal performance:

1. Phase 1 -- Historical fetch: GET /runs/{workflowId}/{runId}/updates returns all past messages as gzip-compressed JSON. This is efficient because HTTP responses support compression while SSE streams do not.

2. Phase 2 -- Real-time SSE: GET /runs/{workflowId}/{runId}/stream?skipHistory=true opens an SSE connection for live updates only. The since parameter ensures no messages are lost between phases.

Reconnection is handled automatically with exponential backoff (1s base, 30s max, 10% jitter, up to 10 attempts).

LLM Token Streaming

During LLM generation, tokens are batched into STREAMING_CHUNK messages at 16ms intervals or 200 characters before being published to Redis. When the final THOUGHT or ANSWER message arrives with a matching activity_id, streaming chunks are replaced by the final message to avoid duplication.

Workstreams

Messages carry a workstream_id (defaults to "main"). This enables parallel agent execution where multiple workstreams run concurrently. The conversation only closes when the main workstream sends COMPLETE or TERMINATED -- other workstreams can complete independently.

Embedding the Conversation UI

The ModernAgentConversation component from @vertesia/ui is the recommended way to display agent conversations in your application. It handles all the complexity of message streaming, reconnection, plan visualization, file uploads, and message rendering.

Basic Usage

Display an existing agent conversation:

import { ModernAgentConversation } from "@vertesia/ui/features";

function MyAgentView({ workflowId, runId }) {
    return (
        
    );
}

Starting a New Conversation

Provide a startWorkflow callback to let users initiate conversations:

import { ModernAgentConversation } from "@vertesia/ui/features";
import { useUserSession } from "@vertesia/ui/session";

function MyAgentView() {
    const { client } = useUserSession();

    const startWorkflow = async (message?: string) => {
        const run = await client.interactions.executeAsync({
            type: "conversation",
            interaction: "MyAgent",
            prompt_data: { task: message },
            interactive: true,
        });
        return {
            run_id: run.runId,
            workflow_id: run.workflowId,
        };
    };

    return (
        
    );
}

Component Properties

File Upload Support

The component supports drag-and-drop and button-based file uploads. Files are uploaded to the workflow's artifact storage and the agent is signaled via FileUploaded.

Document Search Integration

You can integrate a custom document search UI using a render prop:

 (
        
    )}
    selectedDocuments={selectedDocs}
    onRemoveDocument={(docId) => removeDoc(docId)}
/>

Context and Attachments

Fusion Fragment Support

When building data-driven applications, you can provide data for fusion-fragment code blocks in agent responses:

Styling

SDK-Only Integration

If you cannot use the React component (e.g., in a Node.js backend or non-React frontend), use the WorkflowsApi directly from @vertesia/client. See the SDK page for examples of streamMessages and sendSignal.

Key Source Files