Skip to main content

Getting_Started_with_Agents.md

Turn the Brainframe into, multi‑channel chatbot/voice/video agent — without adding a third‑party agent platform. This guide gives you a thin, drop‑in “chat orchestrator” that sits in front of your existing Central LLM Gateway, calls your shared feature modules, and logs to internalmonologue / reasonlog / cognitivealignment exactly as your docs prescribe. 【7†source】

Who this is for​

  • You already have RABS (aka Reggie) with the Brainframe, Model Matrix, Prompt Assembly Engine, and System Independence Model in place. 【7†source】
  • You want one Reggie brain with many “mouths” (web chat, staff/admin chat, SMS/MMS, voice, avatar/video). 【7†source】

Architecture at a glance​

Channel (web | staff | admin | sms | voice | video)
   │
   ▼            (normalize)              (tools wrap shared modules)
 /reggie/message ───────► LLM Gateway ───────────────► Tool Executor ──► features/*
   │                         (System→Situation→Welfare)                  (deterministic)
   │                                      │
   ├── emit: internalmonologue (USER_INPUT, LLM_CALL, SMS_SENT, etc.)    【7†source】
   └── persist: reasonlog (+ cognitivealignment when reviewed)           【7†source】

Key principle: The interface is transport; the agent is your backend. Voice/SMS/WebRTC only change I/O, not the brain. 【7†source】

Prerequisites​

  • Central LLM Gateway with three‑tier prompts (System → Situation → Welfare) and PII redaction. 【7†source】
  • Shared, deterministic feature modules used by both Operations and Reggie (System Independence Model). 【7†source】
  • Eventing & storage: internalmonologue, reasonlog, cognitivealignment, context buckets, pgvector RAG tables. 【7†source】
  • Optional comms providers already in your stack: Twilio (SMS/Voice) and LiveKit (WebRTC). 【7†source】

Step 0 — Core concepts to align on​

  • One unified message contract for all channels.
  • One HTTP entry point (POST /reggie/message).
  • One tool executor that only calls your existing feature modules.
  • Always log to internal telemetry; persist compact reasoning to reasonlog; route reviews to cognitivealignment. 【7†source】

Step 1 — Define the Unified Message Contract​

Use this shape for every inbound event (web, staff, admin, SMS, voice, video).

{
  "channel": "web|staff|admin|sms|voice|video",
  "session_id": "uuid",           // one per ongoing conversation
  "user_id": "uuid|null",
  "context": "public|staff|admin",
  "input": {
    "type": "text|stt|event",
    "text": "hello, I need to change Friday pickup",
    "media": null
  },
  "meta": {
    "twilio": {},
    "livekit": {},
    "ip": "203.x.x.x",
    "timezone": "Australia/Sydney"
  }
}

Store session_id → thread in DB/kv so any channel continues the same conversation. 【7†source】

Step 2 — Create the Orchestrator Endpoint​

Implement POST /reggie/message in your backend. It normalizes input, builds the Situation pack (RAG + context buckets + mind‑map references), invokes the Gateway, handles tool calls, records logs, and delivers a reply. 【7†source】

// POST /reggie/message
app.post('/reggie/message', authOptional, async (req, res) => {
  const msg = normalizeInbound(req.body);                 // Step 1
  const conv = await getOrCreateConversation(msg.session_id);

  await logEvent('comms', 'USER_INPUT', { ...msg });      // internalmonologue 【7†source】

  const situation = await buildSituation(conv, msg);      // RAG + buckets + mindmap 【7†source】

  const llmStream = await gateway.invoke({
    task: 'reasoning.v2',
    situation,
    style_tokens: ['PLAIN']
  });                                                     // Central LLM Gateway 【7†source】

  const reply = await handleRealtimeToolCalls(llmStream, { conv, user: msg.user_id });

  await writeReasonlogAndMaybeReview(reply.reasoning);    // reasonlog + cognitivealignment 【7†source】

  await deliverOutbound(msg.channel, msg, reply);         // SMS/Web/Voice/Video transport
  res.json({ ok: true });
});

Step 3 — Expose Tools that Wrap Your Shared Modules​

The LLM can “act” only through these functions. Internally, they call the same pure modules Operations uses (no direct, free‑form DB access). 【7†source】

const tools = {
  schedule_shift: async ({ staffId, start, end }) => features.roster.scheduleShift({ staffId, start, end }),
  find_transport: async ({ bookingId }) => features.transport.allocate({ bookingId }),
  send_sms:       async ({ to, body }) => comms.sms.send({ to, body })
};

Govern tool visibility by context/role; route risky outcomes to the review queue as defined by your monitoring levels. 【7†source】

Step 4 — Memory & Logging (What to Store, Where)​

  • Thread state: minimal recent turns for coherence.
  • Evidence: RAG snippets + the mind‑map paths used. 【7†source】
  • Telemetry: emit LLM_CALL, SMS_SENT, VOICE_SEGMENT, etc., then summarise via your scheduled job. 【7†source】
  • Reasoning: compact steps to reasonlog (no raw chain‑of‑thought); link verdicts via cognitivealignment. 【7†source】

Step 5 — Channel Adapters (thin wrappers)​

Twilio SMS/MMS​

// Twilio webhook → normalize → /reggie/message
app.post('/twilio/sms', verifyTwilio, async (req, res) => {
  const inbound = {
    channel: 'sms',
    session_id: stableHash(req.body.From),
    user_id: null,
    context: 'public',
    input: { type: 'text', text: req.body.Body },
    meta:  { twilio: req.body }
  };
  await fetchLocal('/reggie/message', { method:'POST', body: inbound });
  res.type('text/xml').send('<Response/>');
});

Outbound: deliverOutbound('sms', msg, reply) uses the Twilio REST client to send reply.text to msg.meta.twilio.From.

Web chat (Public / Staff / Admin)​

Send via WebSocket or POST /reggie/message. Include user_id and context to unlock role‑scoped tools.

Voice (Twilio Voice or LiveKit)​

  • Twilio: (Media Streams) → streaming STT → /reggie/message with type:'stt' → TTS back to the call.
  • LiveKit: same loop over WebRTC with lower latency and multi‑party; attach TTS as an audio track.
    Both paths only change the transport; the orchestrator and Gateway are identical. 【7†source】

Step 6 — Prompt Glue (Situation Pack)​

When assembling situation for the Gateway, include:

  • Prime Directives + optional welfare tokens (e.g., empathy/plain‑language). 【7†source】
  • Context buckets for the active entities (participant, staff, vehicle, route). 【7†source】
  • RAG snippets from communications_log, incident/shift notes with freshness weighting (time‑decay). 【7†source】
  • Mind‑map paths so the model cites exact table/field names. 【7†source】

This keeps answers grounded and auditable.

Safety & Governance (built‑in to your stack)​

  • Gateway injects Prime Directives, redacts PII, and meters tokens/cost. 【7†source】
  • Monitoring Levels gate high‑risk actions (Review‑All → Spot‑Check → Autonomous). 【7†source】
  • Every call logs to internalmonologue; reviews write to cognitivealignment. Dashboards can track alignment & cost. 【7†source】

Local Dev Checklist (90‑minute loop)​

  1. POST /reggie/message (echo tool only) → see reply in web chat.
  2. Wire Twilio SMS webhook → receive and reply.
  3. Add two tools (send_sms, schedule_shift) that wrap existing modules.
  4. Persist reasonlog (+ fake reviewer to test cognitivealignment).
  5. Swap in real RAG and context buckets in buildSituation.
  6. Optional: connect LiveKit room for voice; verify turn‑taking and barge‑in.

Next Milestones​

  • Channel‑agnostic transcripts: consistently thread and store across SMS/Web/Voice.
  • Staff/Admin role gates: map tools to roles; audit tool calls.
  • Observability: Grafana panels for llm_tokens, cost_usd, alignment_mean_24h, STT/TTS latency. 【7†source】
  • “Reggie Is Alive” blog: post weekly changelogs & lessons learned from internalmonologue summaries. 【7†source】

Appendix A — Minimal Orchestrator (Express)​

// generated by Reggie :: purpose: unify chat/voice/sms into one entrypoint
import express from 'express';
import { gateway } from './brainframe/gateway.js';
import { buildSituation } from './brainframe/situation.js';
import { tools, handleRealtimeToolCalls } from './orchestrator/tools.js';
import { logEvent, writeReasonlogAndMaybeReview } from './orchestrator/audit.js';
import { deliverOutbound, normalizeInbound } from './orchestrator/io.js';

export const app = express();
app.use(express.json());

app.post('/reggie/message', async (req, res) => {
  const msg  = normalizeInbound(req.body);
  await logEvent('comms', 'USER_INPUT', msg);                // internalmonologue

  const situation = await buildSituation(msg);               // RAG + buckets + mindmap
  const stream = await gateway.invoke({ task:'reasoning.v2', situation, style_tokens:['PLAIN'] });

  const reply = await handleRealtimeToolCalls(stream, { tools, user: msg.user_id });
  await writeReasonlogAndMaybeReview(reply.reasoning);       // reasonlog + cognitivealignment

  await deliverOutbound(msg.channel, msg, reply);
  res.json({ ok:true });
});

This file is intentionally tiny: all the smarts live in your Gateway, Brainframe, and shared features. 【7†source】

Why this fits RABS perfectly​

  • Respects the Central LLM Gateway as the single choke‑point. 【7†source】
  • Uses the System Independence Model (same modules power Ops and Reggie). 【7†source】
  • Writes into the cognition loop you already documented (monologue → reasonlog → alignment). 【7†source】
  • Twilio/LiveKit are transports only — no extra agent provider needed. 【7†source】