Skip to content

Chapter 5 — BlockFlow · AI nodes · Agent node

5.2.6.3 Supervision & Learning — Traces, JSON & Rollout

What gets recorded and how it becomes training data, how the configuration is serialised into the node, and the governed path from teacher-led operation to student autonomy.

Trace collection and learning data

Trace collection records the information needed to explain model selection and build future evaluation or training data — model and profile versions, relevant prompt and tool-schema snapshots, the candidate decision, the teacher verdict, the selected decision, the normalised tool result and the final outcome.

  • Rejected candidates may be stored in the trace, but never in canonical conversation history.
  • Successful turns are useful as positive examples; failure incidents require curation and must not automatically become positive training samples.
  • Secrets are redacted, and production data eligibility is governed by an approved learning-data policy.
  • Synthetic example generation runs after the live request — typically for teacher replacements, validation failures or explicit user corrections.
  • Fine-tuning hyperparameters and GPU or deployment details belong to the model training and deployment layer, not to this Agent tab.

JSON integration

The complete configuration is added under the existing node data object at data.learning . When enabled, the nested teacher/student configuration becomes authoritative for supervised execution. Legacy model fields may remain for migration compatibility, but the supervised core must not merge them with the nested configuration at request time.

{
"type": "chatbot",
"data": {
"model": "legacy-model-value",
"temp": 0,
"max_output_tokens": 2000,
"learning": {
"config_version": "1.0",
"enabled": true,
"mode": "STUDENT_GATED",
"teacher": { "profile_ref": "..." },
"student": { "profile_ref": "..." },
"supervision": { "policy_ref": "..." },
"handover": {},
"tool_safety": {},
"tracing": { "policy_ref": "..." },
"privacy": { "data_policy_ref": "..." },
"synthesis": {},
"deployment": { "policy_ref": "..." }
}
}
}

Serialisation uses real JSON booleans and integers, converts UI percentages to decimal rates from 0.0 to 1.0, serialises absent optional values as null , includes immutable profile versions, and never serialises raw credentials or runtime metrics.

Validation and dry-run

Workflow publication fails when an enabled configuration is incomplete or would weaken a required safety boundary — a missing teacher or student profile, incompatible tool-calling capabilities, a protected action that could execute without required review, invalid timeout or sample values, missing policies, or raw secrets embedded in the configuration.

  • Validate — a read-only pass over schema, profiles, capabilities, credential references, policies, MCP metadata and evaluation releases.
  • Dry-run — shows the student proposal, validation result, effective review rule, teacher verdict, selected decision, trace fields and expected latency. It never executes a real side-effecting MCP tool.

Runtime status

Runtime status is read-only and comes from the core observability API rather than the design JSON. It distinguishes the configured mode from the effective mode, and surfaces model and profile versions, provider health, evaluation status, review and replacement rates, supervision latency, protected-action coverage, trace and learning queue health, and any degraded-mode reason.

Migration and rollout

Existing Agent nodes continue to work unchanged when the feature is disabled. When supervision is first enabled on an existing node, the migration flow detects the current model settings and explicitly asks whether the existing model is intended to become the teacher or the student — it does not infer this silently.

  • Start new production use cases in Phase 1 with trace capture enabled under an approved privacy policy.
  • Optionally run the student in a small shadow sample to gather comparison data without affecting users.
  • Move to Phase 2 only after the student passes representative evaluation, tool-effect metadata is complete, fail-closed paths are tested, and rejected candidates cannot leak into history, TTS, routes or execution.
  • Progress to Phases 3 and 4 only through governed deployment and evaluation policy, with automatic regression or rollback available when quality drifts.

The rule that matters

The student output is a candidate — not a committed conversation event and not an executable action — until deterministic validation and any required teacher review select it. That boundary enables safe teacher takeover, reliable tool execution, reproducible traces and future student improvement without coupling the Agent node to a specific business workflow.

Source basis — OpenBrain BlockFlow Agent Node: Supervision & Learning GUI and JSON Configuration Specification, v1.0 (2026-08-12).