LLM Calls

Operaide provides Aktors for LLM calls: text generation, chat, structured output, tool use, embeddings, and speech-to-text. This page covers how to use them in Reaktors and Aktors.

Overview

The AI Aktors work with multiple providers: OpenAI, Azure OpenAI, Anthropic, Google, and others. They are:

• Provider-agnostic: Switch between different AI providers without changing your code
• Type-safe: Full TypeScript support with proper type inference
• Streaming-capable: Support for real-time streaming responses
• Composable: Easily combine with other Aktors in complex workflows

Core AI Aktors

aktorAICall

The primary Aktor for making LLM calls. Supports both standard and streaming responses. Returns string when evaluated by the engine.

import { aktorAICall, aktorAISettingProviderModel } from '@operaide/ai';
import { aktorConst } from '@operaide/aktor';

const response = aktorAICall({
    messages: aktorConst([
        { role: 'system', content: 'You are a helpful assistant.' },
        { role: 'user', content: 'What is the capital of France?' }
    ]),
    providerModel: aktorAISettingProviderModel(),
    // optional:
    // llmOptions: aktorConst({ max_steps: 10 }),
    // tools: myToolSet,
});

aktorCompletePrompt

Template completion for dynamic prompt generation with variable substitution.

import { aktorCompletePrompt } from '@operaide/ai';
import { aktorConst } from '@operaide/aktor';

const completedPrompt = aktorCompletePrompt({
    template: aktorConst('Hello {{name}}, today is {{date}}.'),
    name: args.userName,
    date: aktorConst(new Date().toLocaleDateString())
});

registerChatReaktorDefinition

Convenience wrapper around registerReaktorDefinition for chat-compatible Reaktors. Automatically sets chatCompatible: true, inputSchema (messages array), outputSchema and streamSchema (both z.string()).

import { aktorAICall, aktorAISettingProviderModel, aktorPatchMessages, registerChatReaktorDefinition } from '@operaide/ai';
import { aktorSetting, createAktorComposition } from '@operaide/aktor';
import { z } from 'zod';

const aktorMyChatAssistant = createAktorComposition('aktorMyChatAssistant', ({ messages }) => {
    const systemPrompt = aktorSetting(
        z.string().describe('[textarea]System Prompt'),
        'You are a helpful assistant.',
        'System Prompt'
    );
    return aktorAICall({
        messages: aktorPatchMessages({ messages, system: systemPrompt }),
        providerModel: aktorAISettingProviderModel(),
    });
});

registerChatReaktorDefinition({
    reaktorDefinitionId: 'my-chat-assistant',
    label: 'My Chat Assistant',
    description: 'A simple chat assistant',
    aktor: aktorMyChatAssistant,
});

aktorPatchMessages

Modify existing message arrays by adding system prompts.

import { aktorPatchMessages, aktorUserPromptAsHistory } from '@operaide/ai';
import { aktorConst } from '@operaide/aktor';

const messages = aktorPatchMessages({
    messages: aktorUserPromptAsHistory({
        content: args.userMessage
    }),
    system: aktorConst('You are an expert data analyst.')
});

Message Helpers

aktorMessagesFromSystemAndUser builds a message array from a user prompt and an optional system prompt. When system is omitted, only the user message is included:

import { aktorMessagesFromSystemAndUser } from '@operaide/ai';
import { aktorConst } from '@operaide/aktor';

const messages = aktorMessagesFromSystemAndUser({
    user: args.question,
    system: aktorConst('You are a helpful assistant.'), // optional
});

For cases where you only have one side of the conversation, two lower-level helpers are available:

import { aktorSystemPromptAsHistory, aktorUserPromptAsHistory } from '@operaide/ai';

// System prompt only — useful in agent pipelines where the prompt *is* the instruction
const systemOnly = aktorSystemPromptAsHistory({ system: systemPrompt });

// User message only — combine with aktorPatchMessages to add a system prompt later
const userOnly = aktorUserPromptAsHistory({ content: args.userMessage });

General Usage Patterns

Simple Q&A Workflow

import {
    aktorAICall,
    aktorAISettingProviderModel,
    aktorMessagesFromSystemAndUser
} from '@operaide/ai';
import { aktorConst, createAktorComposition, registerReaktorDefinition } from '@operaide/aktor';
import { z } from 'zod';

const aktorSimpleQA = createAktorComposition('aktorSimpleQA', ({ question }) => {
    const messages = aktorMessagesFromSystemAndUser({
        user: question,
        system: aktorConst('You are a helpful and knowledgeable assistant.')
    });
    const aiResponse = aktorAICall({
        messages,
        providerModel: aktorAISettingProviderModel()
    });
    return aiResponse;
});

registerReaktorDefinition({
    reaktorDefinitionId: 'simple-qa-assistant',
    label: 'AI Assistant',
    description: 'Simple AI-powered question answering',
    inputSchema: z.object({
        question: z.string().describe('The question to ask the AI')
    }),
    outputSchema:  z.string().describe('The AI-generated answer'),
    aktor: aktorSimpleQA,
});

Document Analysis Workflow

import {
    aktorAICall,
    aktorCompletePrompt,
    aktorPatchMessages,
    aktorMessagesFromSystemAndUser,
    aktorAISettingProviderModel
} from '@operaide/ai';
import { aktorConst, createAktorComposition, dedent, registerReaktorDefinition } from '@operaide/aktor';
import { z } from 'zod';

const aktorDocumentAnalyzer = createAktorComposition('aktorDocumentAnalyzer',
    ({ document, analysisType }) => {
        const analysisPrompt = aktorCompletePrompt({
            template: aktorConst(dedent`
            Please analyze the following document and provide a {{analysisType}} analysis:
            Document:
            {{document}}
            Analysis Type: {{analysisType}}
            Please provide a detailed analysis including:
            1. Key findings
            2. Main themes
            3. Actionable insights
        `),
            document,
            analysisType
        });
        const messages = aktorPatchMessages({
            messages: aktorMessagesFromSystemAndUser({
                user: analysisPrompt
            }),
            system: aktorConst('You are an expert document analyst with expertise in extracting meaningful insights from text.')
        });
        return aktorAICall({
            messages,
            providerModel: aktorAISettingProviderModel()
        });
    }
);

registerReaktorDefinition({
    reaktorDefinitionId: 'document-analyzer',
    label: 'Document Analyzer',
    description: 'AI-powered document analysis',
    inputSchema: z.object({
        document: z.string(),
        analysisType: z.enum(['summary', 'detailed']).describe('Type of analysis to perform on the document')
    }),
    outputSchema:  z.string().describe('The AI-generated answer'),
    aktor: aktorDocumentAnalyzer,
});

Structured Output with aktorAIGenerateObject

When you need the LLM to return structured data instead of free-form text, use aktorAIGenerateObject. You provide a Zod schema and the model returns a typed object — no JSON parsing needed.

import { aktorAIGenerateObject, aktorAISettingProviderModel, aktorMessagesFromSystemAndUser } from '@operaide/ai';
import { aktorConst, createAktorComposition, registerReaktorDefinition } from '@operaide/aktor';
import { z } from 'zod';

const SentimentResultZ = z.object({
    sentiment: z.enum(['positive', 'negative', 'neutral']).describe('Overall sentiment'),
    score: z.number().min(0).max(1).describe('Confidence score'),
    keywords: z.array(z.string()).describe('Key themes identified'),
});

const aktorSentimentAnalyzer = createAktorComposition('aktorSentimentAnalyzer',
    ({ feedback }) => {
        return aktorAIGenerateObject({
            messages: aktorMessagesFromSystemAndUser({
                user: feedback,
                system: aktorConst('Analyze the sentiment of this customer feedback.'),
            }),
            providerModel: aktorAISettingProviderModel(),
            schema: aktorConst(SentimentResultZ),
        });
    }
);

registerReaktorDefinition({
    reaktorDefinitionId: 'sentiment-analyzer',
    label: 'Sentiment Analyzer',
    description: 'Analyzes sentiment of customer feedback',
    inputSchema: z.object({ feedback: z.string() }),
    outputSchema: SentimentResultZ,
    aktor: aktorSentimentAnalyzer,
});
// Returns: { sentiment: 'positive', score: 0.87, keywords: ['fast', 'reliable'] }

Use .describe() on each schema field to guide the LLM on what each field should contain.

Tool Use / Function Calling

You can give the LLM tools to call during a conversation. The model decides when and how to call them based on the user's request.

Defining Tools with aktorToTool

aktorToTool wraps an aktor as an AI tool. It separates two kinds of inputs:

• parameters — a Zod schema of values the LLM provides (e.g. a search query)
• dependencies — values you provide externally as Aktors (e.g. a connection name)

import { aktorToTool, aktorToolSet, aktorAICall, aktorAISettingProviderModel, aktorMessagesFromSystemAndUser } from '@operaide/ai';
import { aktorConst, createAktorFunction } from '@operaide/aktor';
import { z } from 'zod';

// 1. Define the underlying aktor function
const searchDocuments = createAktorFunction(
    'searchDocuments',
    async ({ query, database }: { query: string; database: string }) => {
        if (!query) return { results: [] }; // Guard for graph traversal (see note below)
        // ... actual search logic
        return { results: [{ title: 'Result 1', content: '...' }] };
    }
);

// 2. Wrap as an AI tool
const searchTool = aktorToTool({
    aktor: searchDocuments,
    description: 'Search the company document database',
    parameters: z.object({
        query: z.string().describe('The search query'),
    }),
    dependencies: {
        database: aktorConst('production'),
    },
});

// 3. Combine tools into a ToolSet and pass to aktorAICall
const tools = aktorToolSet({ search: searchTool });

const response = aktorAICall({
    messages: aktorMessagesFromSystemAndUser({ user: userQuery }),
    providerModel: aktorAISettingProviderModel(),
    tools,
    llmOptions: aktorConst({ max_steps: 10 }),
});

Set max_steps in llmOptions to control how many tool call rounds the LLM can perform before returning a final response.

note

Graph traversal: The aktor system performs an initial "empty call" during graph construction. At this point, LLM parameters have Zod default values ("" for strings, 0 for numbers, undefined for optionals). Guard against these empty values in your aktor function — for example, return early if query is empty.

Combining ToolSets

Use aktorCombineToolSets to merge multiple ToolSets:

import { aktorCombineToolSets } from '@operaide/ai';

const allTools = aktorCombineToolSets({
    search: searchToolSet,
    communication: emailToolSet,
});

If tool names collide, later ToolSets override earlier ones.

Wrapping Pre-Built Tools with aktorTool

If you already have a Vercel AI SDK Tool object (created with the tool() helper from the ai package), use aktorTool to wrap it as an Aktor so it can be passed to aktorToolSet:

import { aktorTool, aktorToolSet } from '@operaide/ai';

const weatherTool = aktorTool(existingVercelAiTool);
const tools = aktorToolSet({ weather: weatherTool });

Streaming

aktorAICall supports streaming out of the box. To enable streaming in your reaktor, add a streamSchema to the reaktor definition:

const aktorStreamingAssistant = createAktorComposition('aktorStreamingAssistant',
    ({ question }) => {
        return aktorAICall({
            messages: aktorMessagesFromSystemAndUser({ user: question }),
            providerModel: aktorAISettingProviderModel(),
        });
    }
);

registerReaktorDefinition({
    reaktorDefinitionId: 'streaming-assistant',
    label: 'Streaming Assistant',
    inputSchema: z.object({ question: z.string() }),
    outputSchema: z.string(),
    streamSchema: z.string(), // Enables streaming output
    aktor: aktorStreamingAssistant,
});

When streamSchema is defined, the framework automatically uses aktorAICall's streaming mode, which yields text chunks in real time. No code changes are needed inside the Aktor — the same aktorAICall handles both modes.

For chat-compatible reaktors, registerChatReaktorDefinition sets streamSchema: z.string() automatically.

Embeddings

Generate vector embeddings for text using aktorAIEmbedding:

import { aktorAIEmbedding, aktorExtractEmbedding, aktorAISettingEmbeddingModel } from '@operaide/ai';

const embeddingResult = aktorAIEmbedding({
    value: textToEmbed,
    providerModel: aktorAISettingEmbeddingModel(),
});

// Extract the embedding vector as a JSON string
const embeddingVector = aktorExtractEmbedding({ result: embeddingResult });

aktorAISettingEmbeddingModel() uses a separate default from aktorAISettingProviderModel() — it reads from the platform's DEFAULT_EMBEDDING_PROVIDER configuration.

Speech-to-Text

Transcribe audio files using aktorSpeechToText:

import { aktorSpeechToText } from '@operaide/ai';

const transcript = aktorSpeechToText({
    audio: audioDataUrl, // base64 data URL, e.g. 'data:audio/mp3;name=recording.mp3;base64,...'
});
// Returns the transcript as a string

The audio input must be a base64 data URL. Supported formats include MP3, WAV, and M4A. The model and provider are configured platform-wide via DEFAULT_TRANSCRIPTION_MODEL and DEFAULT_TRANSCRIPTION_PROVIDER.

File Handling in Messages

Two helpers let you work with files attached to chat messages:

import { aktorFilesFromLastMessageAsFiles, aktorFilterFiles } from '@operaide/ai';

// Extract files from the last user message as File objects
const files = aktorFilesFromLastMessageAsFiles({ messages });

// Remove file parts from messages, replacing them with a placeholder text
const textOnlyMessages = aktorFilterFiles({ messages });

• aktorFilesFromLastMessageAsFiles finds the most recent user message and converts any FilePart entries to File objects for further processing (e.g. PDF conversion).
• aktorFilterFiles strips file parts from all messages, adding "FILES IS INCLUDED IN CONTEXT" as a text placeholder. This is useful when forwarding messages to models that do not support file inputs natively.

LLM Options Reference

Control model behaviour by passing llmOptions to aktorAICall or aktorAIGenerateObject:

llmOptions: aktorConst({
    temperature: 0.7,
    max_tokens: 500,
    max_steps: 10,
})

Option	Type	Description
temperature	number	Randomness (0–2). Lower values are more deterministic.
top_p	number	Nucleus sampling threshold (0–1).
top_k	number	Top-k sampling (provider-specific).
max_tokens	number	Maximum tokens in the response.
max_steps	number	Maximum tool call iterations for agentic workflows.
max_retries	number	Retry failed API requests.
seed	number	Random seed for reproducible outputs.
presence_penalty	number	Penalize new topics (–2 to 2).
frequency_penalty	number	Penalize token repetition (–2 to 2).
response_format	'json'	Request JSON-formatted output (not all providers support this).

Best Practices

1. Prompt Engineering

Write structured, explicit prompts. Use dedent to keep template literals readable and aktorCompletePrompt for variable interpolation:

// Good: Structured, clear prompts
const structuredPrompt = dedent`
    You are a professional email writer.
    Task: Write a follow-up email based on the meeting notes.
    Meeting Notes:
    {{meetingNotes}}
    Requirements:
    - Professional tone
    - Include action items
    - Keep under 200 words
    - Include a clear subject line
    Format your response as JSON:
    {
        "subject": "Email subject line",
        "body": "Email body content"
    }`;

// Avoid: Vague, unstructured prompts
const vaguePrompt = "Write an email about the meeting: {{notes}}";

For more techniques (zero-shot, few-shot, chain-of-thought), see Prompt Engineering.

2. Use Structured Output Instead of Manual Parsing

When you need JSON from the LLM, prefer aktorAIGenerateObject over parsing aktorAICall output manually. The schema guarantees a valid, typed result:

// Good: Let the framework handle parsing and validation
return aktorAIGenerateObject({
    messages,
    providerModel: aktorAISettingProviderModel(),
    schema: aktorConst(MyOutputSchemaZ),
});

// Avoid: Manual JSON.parse on free-form LLM text
const raw = aktorAICall({ messages, providerModel });
const parsed = processJSON({ response: raw }); // fragile

3. Make Prompts Configurable with Settings

Use aktorSetting with a [textarea] descriptor so operators can tune prompts at deployment time without changing code:

const systemPrompt = aktorSetting(
    z.string().describe('[textarea]System Prompt'),
    'You are a helpful assistant.',
    'System Prompt'
);

4. Guard Tool Functions Against Graph Traversal

When using aktorToTool, the aktor system performs an initial empty call during graph construction. LLM parameters will have Zod defaults ("" for strings, 0 for numbers). Return early to avoid unintended side effects:

const myTool = aktorToTool({
    aktor: createAktorFunction('myTool', async ({ query }) => {
        if (!query) return { results: [] }; // Guard for graph traversal
        return await executeSearch(query);
    }),
    description: 'Search documents',
    parameters: z.object({ query: z.string().describe('Search query') }),
    dependencies: {},
});

Summary

Aktor	Purpose	Returns
aktorAICall	Text generation and chat	string
aktorAIGenerateObject	Structured output with Zod schema	Typed object
aktorCompletePrompt	Template variable substitution	string
aktorAIEmbedding	Vector embeddings	EmbedResult
aktorExtractEmbedding	Extract embedding vector from result	string (JSON)
aktorSpeechToText	Audio transcription	string
aktorToTool	Wrap an aktor as an LLM tool	Tool
aktorTool	Wrap a pre-built Vercel AI SDK tool	Tool
aktorToolSet	Combine tools into a ToolSet	ToolSet
aktorCombineToolSets	Merge multiple ToolSets	ToolSet
aktorPatchMessages	Prepend system prompt to messages	CoreMessage[]
aktorMessagesFromSystemAndUser	Create message array from system + user	CoreMessage[]
aktorSystemPromptAsHistory	Create message array from system prompt only	CoreMessage[]
aktorUserPromptAsHistory	Create message array from user message only	CoreMessage[]
aktorFilesFromLastMessageAsFiles	Extract files from messages	File[]
aktorFilterFiles	Strip files from messages	CoreMessage[]

For provider and connection configuration, see AI Provider and Connection Management.