Most AI learning tools default to a chat interface: learner asks, model answers, and the quality of the experience is whatever today's response happens to be. I wanted to test the opposite architecture, one where the application owns the learning process end to end and the language model gets exactly one narrow job inside it.
That question became Processfolio, a prototype that records how a learner's thinking develops across a structured problem. It's a personal project sitting at the intersection of my software engineering background and my graduate work in education.
The first playable scenario asks a learner to recommend a school policy for generative AI. They form an initial position, identify what evidence they'd need, review a set of deliberately imperfect source material, respond to a changed condition, revise their recommendation, and complete a near-transfer task in a related scenario.
The engineering problem was adding contextual AI support (support that actually responds to what the learner wrote) without letting the model touch the sequence, the grading, or the reasoning itself.
Why not a chat interface
A chat interface hands the model three jobs at once: deciding what happens next, deciding how much help to give, and generating the actual text. Any one of those going wrong degrades the whole activity: the model answers the question outright, skips a step, or over-explains. And there's no clean way to intervene short of a better prompt and a hope.
I wanted those three jobs separated, so I could make hard guarantees about two of them and treat the third as replaceable.
Product constraints
I started with a small set of non-negotiable constraints:
- A learner must save an attempt before requesting support.
- Earlier responses are never overwritten.
- Help gets more structured across repeated requests, but the learner still writes the response.
- The model cannot advance the activity, choose a support level, grade anything, or supply a policy recommendation.
These constraints decided the architecture before I wrote a line of AI-facing code. Processfolio uses a deterministic state machine for progression and treats the language model as a bounded, swappable dependency rather than the thing driving the session.
The scope stays intentionally narrow: one playable project, session state in the browser, no auth, no database, no classroom management, no payments. Keeping the surface area small let me validate the core interaction (bounded AI support inside a deterministic activity) before building a platform around it.
System architecture
Five parts, each with one job:
- Typed, educator-authored project content: scenario, stage prompts, source material, challenge cards, three levels of scaffold templates.
- An append-only event log: the single source of truth for a session.
- A deterministic reducer: derives current stage, gating, and next support level from that log.
- A guide interface: two implementations, mock and API, behind one response shape.
- A server route: validates context and calls the model.
learner action
│
▼
typed event ──────────► event log (append-only)
│
▼
deterministic reducer
(stage · gating · next support level)
│
┌─────────────────┴─────────────────┐
▼ ▼
mock guide API guide
(returns prepared scaffold) (POST /api/guide → model)
│ │
└─────────────────┬───────────────────┘
▼
same response shape
(question · level · source)
│
▼
new hint event
→ back into the log
Project content (the scenario, prompts, materials, challenges, scaffold templates) lives in local typed data, and the model has access to nothing beyond what the server hands it: no ability to create a stage, replace material, or invent a different support sequence.
The reducer is the only thing that decides current stage, whether the learner can continue, and which support level comes next: all pure functions over the event history, no side effects, no model calls.
Mock and API, the two guide implementations, return the exact same response shape. The mock guide just returns the prepared scaffold directly, which keeps the whole experience playable with zero API cost. The state machine doesn't know or care which guide produced the text, which is the point: the model is a dependency, not a foundation.
Modelling the learning process as events
Every meaningful action (submission, hint, challenge, revision, reflection, transfer response) is appended as a typed event. Here's a simplified version:
type ProcessfolioEvent =
| { type: "submission"; stage: Stage; content: string }
| { type: "stage_completed"; stage: Stage }
| { type: "challenge_introduced"; challengeId: string }
| {
type: "hint";
level: 1 | 2 | 3;
templateId: string;
reason: HelpReason;
scaffold: string;
source: "ai" | "fallback" | "mock";
}
| { type: "revision"; content: string }
| { type: "reflection"; content: string };
This is a lightweight event-sourcing pattern: the log is the source of truth, and everything else (current stage, available help, the final Processfolio) is derived by replaying it through pure functions.
That's what makes the product goal work: a revision gets appended after the original recommendation, never written over it. The final Processfolio can reconstruct exactly what a learner believed first, what evidence they picked, what support they asked for, and what they changed their mind about.
Right now the log lives in sessionStorage: a refresh
restores the journey, a restart clears it. A production version needs
durable server-side storage, session ownership, and migrations, but the
event-sourced shape doesn't change; only where it's persisted does.
Deterministic progression and scaffold escalation
The learning journey is a fixed sequence: initial position → evidence plan → investigation → complication → revision → near transfer → reflection and summary. Writing stages require a complete saved submission before the learner can move on; there's no partial credit for progression.
Help escalates independently of stage progression. The learner picks why they're stuck, and the first request in a stage gets Level 1, the second gets Level 2, the third gets Level 3. There's no skipping to a stronger scaffold and no requesting a fourth. A Level 1 scaffold reads like:
Which claim in your recommendation depends on a source you haven't checked yet?
A Level 3 scaffold pushes harder:
If you had to explain the policy to the group that benefits least, what would they ask you to justify?
The app picks the level by counting hint events already recorded for the current stage. The model receives that level and the matching template as fixed inputs, with no mechanism to change either. And a hint never marks a stage complete; asking for help and finishing the stage are two separate events, deliberately.
Building bounded context
A useful scaffold needs enough history to be relevant. Sending the entire event log would balloon cost, add noise to the prompt, and hand the model information it doesn't need. So there's a projection layer that turns event history into a bounded request, roughly:
interface GuideRequest {
stage: Stage;
latestAttempt: string;
priorAttempts?: string[]; // capped, not the full history
priorScaffolds?: string[]; // last three, max
materialIds?: string[]; // resolved server-side, never trusted as-is
challengeId?: string;
helpReason: HelpReason;
}
Two details matter here. First, the guide always reads the latest saved submission. Unsaved text sitting in the form is excluded, so a learner can't edit a draft after saving, request a scaffold, and get one built on text that never actually entered the Processfolio. Second, the browser only sends identifiers for materials and challenges, never their content. The server resolves those IDs against the trusted, educator-authored catalog, so a modified client request can't redefine a source, swap the active challenge, or smuggle in a different scaffold template.
The guide request pipeline
A request to POST /api/guide runs through several checks
before it ever reaches the model:
POST /api/guide
│
▼
1. schema validation shape · stage · field lengths · material IDs
│
▼
2. resolve trusted content server-side lookup, ignores client-sent content
│
▼
3. calculate expected level count hint events for this stage
│
▼
4. verify scaffold sequence prior scaffolds must form a valid chain
│
▼
5. select template educator-authored, matched to level
│
▼
6. construct prompt stage + bounded context + scaffold objective
│
▼
7. call model ──────► timeout / rate limit / malformed output ───┐
│ ok │
▼ │
8. validate structured output ──► fails semantic check ───────────┤
│ passes │
▼ ▼
return AI-generated scaffold return prepared fallback scaffold
The prompt itself is deliberately constrained: purpose of the Processfolio, current stage, bounded prior thinking, relevant source material, the learner's saved attempt, and the scaffold's objective. Nothing more. It caps the model to one short guiding question, and the model can't hand over a policy, grade the learner, pull in outside evidence, or claim mastery.
On the way back in, the browser re-validates, checking that the returned support level and template ID match what it selected locally. Every generated sentence ends up wrapped in several independent boundaries before a learner sees it.
Treating model output as untrusted input
Valid JSON isn't the same as an acceptable scaffold. A response can pass structured-output parsing and still violate what the activity is supposed to do, so there's a semantic validation pass after parsing that checks length, question structure, and recommendation-like language. Fail that check, and the server returns the original educator-authored scaffold instead. Same fallback fires on a timeout, a rate limit, or malformed output. The browser adapter gives the model an eight-second window and one attempt, with no retries.
Every hint event records its own provenance:
type ScaffoldSource = "ai" | "fallback" | "mock";
That's surfaced to the learner as AI-generated guiding question, Prepared guiding question, or Prototype guiding question, and preserved in the Processfolio.
I watched this matter in practice during browser testing: the first
two model-generated questions passed validation cleanly, the third
failed with a sanitized semantic_validation_error
(invalid-scaffold), and the API silently returned the
prepared Level 3 scaffold instead. The learner never saw a gap. That's
the fallback path doing exactly what it's for.
Building provisional capability observations
The final summary includes a small capability map: evidence use, stakeholder awareness, assumptions, responses to a changed condition. It's entirely deterministic right now, searching specific submission fields for configured evidence and creating an observation only when it can attach an exact excerpt from the learner's own writing. A learner listing stakeholders like this:
Students, teachers, multilingual learners, students with disabilities, families, administrators, and students with limited access to devices or tutoring.
...produces a provisional stakeholder-awareness observation with that exact line attached beneath it. These are evidence pointers, not scores: no semantic assessment, no mastery claim. A future version could interpret a wider range of responses with an actual model in the loop, but that needs real evaluation and explicit rules for uncertainty first. Putting a model here before that groundwork would produce the appearance of assessment without the evidence to back it.
Testing the slice
Unit tests cover the deterministic core: stage progression, required submissions, append-only history, help gating, support escalation, hint exhaustion, evidence validation, session restoration, capability-observation excerpts.
The AI path adds its own layer: request and response schemas, bounded-context projection, trusted-content resolution, support-level verification, prompt construction, structured-output parsing, timeouts and network failures, model versus fallback provenance, invalid model responses, duplicate-request prevention, and protection against unsaved form content leaking into a scaffold.
ESLint, Vitest, TypeScript's compiler, and a production Next.js build all run before anything ships. None of that tells me whether the interaction actually feels coherent, though. For that I still run it by hand, start to finish: initial position through three scaffold requests, revision, near transfer, reflection, final summary.
Current limitations
This is one vertical slice, not a platform. Sessions live in a single browser tab: no accounts, no classroom view, no durable records, no permissions. Real rate limiting is still missing from the API route, and it needs to be there before more than a handful of testers touch it. Coverage in the capability map is only as good as its deterministic rules, which is to say: narrow. And the AI scaffolds themselves have only been exercised against a small set of authored journeys, not real learner behavior.
The next phase is evaluation, not features: a bank of varied learner responses (incomplete reasoning, conflicting claims, copied language, minimal answers, unexpectedly sharp interpretations), each paired with an expected scaffold objective and a list of outputs that would be unacceptable. That's what would let me actually measure model acceptance rate, fallback frequency, constraint violations, and whether the generated questions are any good.
What this clarified
The LLM integration turned out to be the small part. Almost all the engineering work went into the boundary around it: a trustworthy event history, a deterministic reducer, bounded context projection, server-side content resolution, output validation, a real fallback path, and provenance tracking on every generated sentence.
That boundary is what makes the model replaceable, and what makes the prototype still work when the model isn't available at all.