ply-agent.h
Agent Harness

ply-agent.h defines a C++ API for interacting with AI agents. Applications create Transcript objects and pass them to Agent objects; the agent's job is to extend the transcript in a logical way. It does this by communicating with a remote inference server and running local tools.

The steps for interacting with agents are as follows:

1. Create a new Transcript object containing the user's prompt.
2. Create a new Agent object, passing in the Transcript, the desired inference provider and a set of tools for the agent to use. The agent runs in a background thread.
3. Receive TranscriptEvent objects back from the Agent.
4. As each event comes in, call applyTranscriptEvent and perform any application-specific handling.
5. Once Agent::isWorking() returns false, no further events will be received and the Agent can be safely destroyed.

To send a followup prompt, create a new Transcript object, assign the previous Transcript as its parent, then create another Agent object.

Using ply-agent.h requires linking your project with libcurl for HTTPS support. Instructions for installing libcurl can be found in the documentation for building and running the agent sample.

Transcript

A transcript consists of a sequence of turns, with each turn consisting of a sequence of messages. These concepts are represented by the Transcript, Transcript::Turn and Transcript::Message classes.

Each message in the transcript is associated with a Transcript::Role, which can take any of the following values:

Every Transcript object holds a reference to a parent Transcript object, allowing you to link them together into a graph. Each node in this graph corresponds to a followup prompt sent by the user.

Agent

The Agent class represents an agent running in a background thread. As the agent runs, it generates TranscriptEvents, which are buffered internally until the application calls pollForEvents, waitForEvents or waitForCompletion. Only one thread is allowed to call pollForEvents, waitForEvents or waitForCompletion at a time.

You can destroy an Agent at any time as long as there are no racing member function calls from other threads. If the agent is still running at destruction time, it's immediately canceled.

Agent::Agent(const Agent::Settings& settings)

Constructor. The agent starts running in a background thread. Agent::Settings has the following data members:

const Transcript* startTranscript	The transcript used to start the agent.
EndPoint endPoint	Identifies the inference provider, protocol and model.
ToolSet toolSet	Specifies the system prompt, working directory and available tools.
bool enableHttpLog	Enables HTTP-level logging (for debugging purposes). Default is false.

ToolSet has the following data members:

String systemPrompt	The system prompt passed to the agent.
Set<Owned<Handler>> handlers	The tool handlers available to the agent.
String workingDirectory	The agent's working directory.

Array<TranscriptEvent> Agent::pollForEvents()

Returns all currently buffered events without waiting, or an empty array if no events are buffered.

Array<TranscriptEvent> Agent::waitForEvents(s32 maxTimeInMillis)

Waits until at least one event is available, then returns all buffered events. A negative argument waits indefinitely.

Array<TranscriptEvent> Agent::waitForCompletion(s32 maxTimeInMillis)

Waits until the agent stops working or the time limit is reached, then returns all buffered events. A negative argument waits indefinitely.

bool Agent::isWorking()

If true, the agent can still return more events. false means the agent has finished running, the buffer is empty and no more events will arrive. This function can be called by any thread at any time.

void Agent::cancel()

Stops running the agent. No new events will be generated after this function returns, but any events already buffered remain available for consumption. This function can be called by any thread at any time. If another thread is waiting inside waitForEvents or waitForCompletion, that thread will immediately return.

If cancel is called while a tool call is running in the background, the tool call might not stop immediately. Tool calls can continue running briefly after cancel returns, but they'll be stopped as soon as possible and won't generate any further TranscriptEvents.

TranscriptEvent

Transcript changes are received as a stream of TranscriptEvent objects. The agent never modifies the original Transcript object directly; instead, the application must call applyTranscriptEvent for each event it receives.

void applyTranscriptEvent(Transcript* transcript, const TranscriptEvent& event)

Modifies transcript by applying the given event.

Applications are free to perform additional application-specific handling in response to each event. To facilitate this, TranscriptEvent exposes the following data members:

TranscriptEvent::Operation can have any of the following values:

Tools

The tools available to an agent are defined by filling in ToolSet::handlers.

Several built-in tools are available. To add them to a ToolSet, call any of the following functions. Each function returns a pointer a new ToolSet::Handler owned by the ToolSet.

ToolSet::Handler has the following data members:

When a tool handler is added, its permittedDirectories is initially empty. The application can add directories before creating the agent. All built-in tools currently enforce these permissions except the shell tool, which should be used with caution; ideally in a sandboxed environment. (Note: An auto-approve mode for the shell tool is planned.)

Defining Custom Tools

In addition to the built-in tools, applications are free to create their own tools to integrate more closely with agents. For example, a tool to count the number of bytes in a string can be implemented as follows.

void byteCountToolHandler(ToolContext* toolCtx, Transcript::Message* toolCall,
                          const json::Node& arguments) {
    // Validate the argument.
    const json::Node& textArg = arguments.get("text");
    if (!textArg.isText()) {
        toolCtx->appendResponse(toolCall, "Error: 'text' argument is required.");
        return;
    }

    // Add response text to the transcript.
    toolCtx->appendResponse(toolCall, String::format("{} bytes", textArg.text().numBytes()));
}

void addByteCountTool(ToolSet* toolSet) {
    // Describe the tool and its arguments.
    Owned<ToolSet::Handler> tool = Heap::create<ToolSet::Handler>();
    tool->name = "byte_count";
    tool->description = "Return the length of a string in bytes.";
    ToolSet::Parameter& textParam = tool->parameters.append();
    textParam.name = "text";
    textParam.description = "Text to measure";
    textParam.type = "string";
    textParam.required = true;
    tool->handler = byteCountToolHandler;
    toolSet->handlers.insertItem(std::move(tool));
}

ToolContext

Each time a tool is invoked, it receives a ToolContext object. ToolContext provides the following member functions:

StringView ToolContext::getWorkingDirectory() const

Returns the agent's working directory.

ArrayView<const String> ToolContext::getPermittedDirectories() const

Returns the directories that the tool is permitted to access.

void ToolContext::appendResponse(Transcript::Message* toolCall, StringView text)

Adds text to the tool response in a thread-safe manner. Can be called more than once to stream a response. Each call to appendResponse creates a new TranscriptEvent and buffers it in the Agent so that the application receives it as soon as possible. The complete tool response won't be sent to the remote inference server until the next turn.

bool ToolContext::isCanceled() const

Returns whether the agent has been canceled.

bool ToolContext::setCancelCallback(Functor<void()>&& callback)

If the agent hasn't already been canceled, registers a cancellation callback and returns true. Otherwise, if the agent was already canceled, clears any existing cancellation callback and returns false. The callback will be invoked from the client thread when Agent::cancel() is called.

void ToolContext::clearCancelCallback()

Clears the registered cancellation callback.

Long-running tools should call isCanceled() periodically and return promptly when it becomes true. A tool blocked in an interruptible operation can use setCancelCallback() to register a callback that unblocks it. The callback must be cleared before the tool handler returns.