Written by the ENSI Foresight Division on a library of 107 primary documents — deployed-system evaluations, agent architectures, benchmarks, learning-analytics codes of practice and regulatory instruments — downloaded and indexed in the “AI for Teaching at CTU” library.
Almost every university discussion of artificial intelligence is a discussion about a chatbot. Should students use one, may staff use one, which one should we license. This is a category error of a familiar kind — the same error as asking, in 1995, whether the university should have a website. The website was not the change. The change was that every administrative and scholarly process the institution ran would eventually be rebuilt around networked access, and the organisations that treated the web as a publication channel spent a decade being slowly reorganised by the ones that treated it as a substrate.
Teaching is not a single activity that a tutor-bot either helps with or does not. It is a production system with a dozen distinguishable functions, each with its own inputs, its own quality criterion and its own cost structure. Someone designs the course. Someone explains the material. Someone answers the question that gets asked forty times a semester. Someone marks the formative work. Someone notices that a student has stopped attending. Someone certifies, under their professional signature, that a named human can do a thing. Someone decides whether any of it worked. Those are different jobs, and generative AI has a radically different relationship to each of them. Some it can absorb almost entirely; some it can amplify by an order of magnitude; some it must be kept out of, for reasons that are pedagogical, legal, and — in the case of certification — constitutive of what a university is for.
The unit of design, therefore, is not a chatbot. It is an operating model: which functions are run by agents, which by humans, which by humans supervising agents, what data flows between them, and who is accountable when the whole thing produces a wrong answer to a student at midnight. This report specifies that model.
The evidence that it is buildable is no longer speculative, and it is unusually honest. Georgia Tech has run Jill Watson across its online master’s programme for years and published the operational numbers: more than 4,000 students across more than a dozen classes, more than 500 teacher hours saved, question coverage rising from about 21% at 80% precision in 2017 to over 96% coverage at over 86% precision by autumn 2019, and a companion system that generates a new agent for a fresh syllabus in about 25 hours. The LLM-era rebuild reports 76.7% answer accuracy against 31.3% for a generic assistant baseline on the same evaluation — which is the single most important architectural finding in this library, because it says the value is in the grounding, not the model. Harvard’s CS50 has served approximately 211,000 students and 10 million queries at $1.50 per student per year, with 94% of students finding the tools helpful. These are not pilots. They are production systems with published failure rates.
And the failure rates are the reason this report is structured around an operating model rather than a shopping list. CS50’s own evaluation reports that roughly 2.1 million of those 10 million responses — 22% — contained code blocks despite the system being instructed not to give solutions, rising to 48% at conversation level. The team names the mechanism: instruction dilution, in which a long conversation progressively erodes the authority of the system prompt until the pedagogical constraint simply stops binding. Their remedy was not a cleverer prompt. It was a human-feedback loop with teaching fellows reviewing and correcting the agent’s behaviour. Which means the working architecture was never “an agent”. It was an agent plus a staffed quality process, and every institution that budgets for the first and not the second is buying the leak without the benefit.
There is a second argument for the agentic frame, and for a technical university it is the more important one. The profession these students are entering is being rebuilt on exactly this substrate. The GitHub–Microsoft–MIT randomised trial found developers completing tasks roughly 56% faster with an AI pair programmer. The NBER field study of 5,179 support agents found +34% for novices and near-zero for experts, with the mechanism being diffusion of expert tacit knowledge. The BCG field experiment found consultants inside the AI frontier producing work rated 40% higher and, on a task just outside it, 19 percentage points less likely to be correct than colleagues with no AI at all. Whatever an engineering degree certifies in 2030, it will be exercised by a person who works by specifying, directing, verifying and integrating the output of machine systems. A university that teaches about that while teaching by nineteenth-century means is teaching the content and withholding the method.
So the reframe: an agentic university is not one that has bought AI. It is one whose teaching runs on the same substrate as the practice it certifies — deliberately, visibly, and with the human accountabilities specified rather than assumed. The students learn the discipline and, at the same time, learn what it is to work in a system where a competent machine does a large share of the first draft and a human is answerable for the result. That second lesson is the graduate attribute the labour market will actually pay for, and it cannot be delivered by a module.
One caution before the architecture, because the enthusiasm in this field is dangerous. The complementarity literature — Hemmer and colleagues’ review formalising when a human–AI team beats either alone — reports an empirical record that is frequently disappointing: many studies find the team underperforming the better of its two members. Complementarity is not automatic; it is engineered, and it is engineered mostly by deciding correctly where the boundary sits. Most of the design effort in what follows is boundary work.
The engine in brief
Eight agent archetypes do the work, and they are not interchangeable: the Gateway Tutor (constrained, course-grounded, the highest-value one); the Course Concierge (logistics, cheapest to build, immediate staff relief); the Feedback Agent (formative only, never summative); the Lab and Simulation Agent (digital twins where physical capacity is the constraint); the Studio Critic (critique, never generation); the Instructional Design Agent (drafts syllabi, slides and assessment for human authorship); the Early-Warning Agent (triggers offers of help, never decisions); and the Evidence Agent (runs the trials and audits the others).
Four data strata carry the system: the course corpus, the interaction stream, the progression record and the outcome ledger. The interaction stream is the asset nobody is capturing and the one that compounds.
Four human roles must exist by name, or the architecture degrades: the course owner, the agent steward, the evidence lead and the certifying academic. The last one is not delegable — not culturally, but legally.
The graduate this produces is an engineer defined by four capabilities the machine does not have: verification, frontier judgement, unassisted core reasoning, and accountability for a result they did not personally generate.
Six failure modes are documented rather than hypothetical: instruction dilution, model drift across cohorts, vendor dependency, evidence theatre, fairness drift, and the hollowing of the apprenticeship — the last being the one nobody has solved.
How this report is organised
The architecture is presented in four layers, from the visible to the structural. First the agent archetypes, each with what it does, the published evidence that it works, its hard boundary — the thing it must never be allowed to do — and its human counterpart. Then the data layer, because agents without grounding are the failure case the Georgia Tech numbers quantify most starkly. Then the operating model: the named human roles and the accountability structure, which is where every deployment in the literature actually succeeds or fails. Then the graduate profile and the failure modes, which are the same subject viewed from opposite ends.
Two rules govern the whole design and are worth stating before the detail, because every archetype below is an application of them. Rule one: agents may generate, explain and flag; humans certify. The EU AI Act’s Annex III makes this a legal boundary as well as a professional one, classifying as high-risk any system used to evaluate learning outcomes, determine admission, assign educational level, or monitor examination behaviour. Rule two: every agent is grounded in institutional material and instrumented from the first day. An agent that answers from the model’s general knowledge is a liability; an agent whose behaviour is not logged and evaluated cannot be improved, defended, or shut down on evidence.
The eight archetypes
1. The Gateway Tutor — the constrained explainer
What it does. Answers a student’s question about the material of a specific course, at any hour, grounded in that course’s own lecture notes, problem sets, worked solutions and past examinations — and does so by withholding. One step at a time; a question back before an answer forward; never the final result to an assessed problem.
The evidence. This is the archetype with the strongest support in the entire library, and the support is unusually clean. Kestin and colleagues’ randomised crossover trial in Harvard’s PS2 physics course found students learning more than twice as much in less time with a pedagogically-constrained AI tutor than in an expert-led active-learning class — the comparison being against the best-evidenced form of human undergraduate physics teaching, not against a bad lecture. The World Bank’s six-week Nigerian RCT returned 0.31 standard deviations. The pooled meta-analytic effect across 35 experimental studies and 4,193 participants is g = 0.670. And the pre-LLM baseline — Ma, Adesope, Nesbit and Liu’s meta-analysis across 107 effect sizes and 14,321 learners — establishes what intelligent tutoring systems achieved before: real gains over large-group instruction, but not over individual human tutoring. The current generation is producing larger effects in weeks, in domains nobody hand-authored.
The hard boundary. It must not solve assessed work, and the constraint must be treated as an engineering requirement with a measured failure rate rather than a line in a prompt. CS50’s published 22% code-leakage rate is the number to design against, and instruction dilution is the named mechanism: in long conversations the system prompt loses authority. Practical countermeasures visible in the deployments — conversation-length limits and re-anchoring, retrieval that returns the scaffolding material rather than the solution, refusal classifiers running as a separate check rather than as prompt text, and sampled human review.
Its human counterpart. A named teaching assistant per course reviewing sampled conversations weekly. This is the CS50 lesson stated as an organisational requirement: the human-feedback loop was the fix, not the prompt.
Where it goes first. The course with the worst failure rate, because every compression finding in this library — NBER’s +34% for novices, Tutor CoPilot’s +9 percentage points for students of the weakest tutors — says the effect concentrates at the bottom of the distribution.
2. The Course Concierge — logistics and syllabus
What it does. Answers the question that is not about the material: when is the deadline, what is the resit policy, which room, what counts toward the grade, is the lab report due before or after the exam. This is a startlingly large share of what teaching staff actually spend time on and it carries no pedagogical value whatsoever.
The evidence. The SyllabusQA work in this library is the reference — 63 syllabi, 5,078 question–answer pairs, retrieval-augmented baselines and, importantly, a factuality metric, because a confidently wrong answer about an examination date is worse than no answer. Georgia Tech’s operational figure belongs here too: Jill Watson’s 500-plus teacher hours saved across a dozen classes is overwhelmingly this category of question, not deep conceptual tutoring.
The hard boundary. It must never invent a policy. The correct failure behaviour is escalation with an explicit “I do not have this in the course documents”, which requires the retrieval corpus to be authoritative and current — meaning someone owns keeping it current.
Its human counterpart. The study office, which stops answering the same forty questions and starts answering the hard ones.
Why it goes first in practice. It is the cheapest archetype to build, it has the fastest visible payoff for staff, and — the political point — it builds institutional trust in the technology on a use case where the downside of an error is a corrected deadline rather than a corrupted understanding of thermodynamics. Every deployment sequence in the literature that succeeded started somewhere low-stakes.
3. The Feedback Agent — formative only
What it does. Gives a student a substantive response to a draft, a solution attempt, a design or a piece of code — against the actual rubric — before it is submitted for a grade. Not a mark. A critique.
The evidence. The computing-education literature carries the strongest signal: the ITiCSE working group’s survey covers automated feedback extensively, and the pattern documented in the ASEE study — students submitting their own solution alongside the AI’s and accounting for the difference — converts feedback into the object of study rather than a substitute for effort. The EducationQ benchmark, a multi-agent teacher/learner/evaluator evaluation over 1,498 questions across 13 disciplines, supplies the most useful and least comfortable finding for procurement: teaching ability does not scale linearly with model size. The biggest model is not automatically the best teacher, which means model selection has to be evaluated on teaching behaviour rather than assumed from benchmark scores.
The hard boundary. It must not grade. This is the sharpest line in the whole architecture and it is drawn in law, not only in pedagogy: Annex III of the AI Act classifies systems used to evaluate learning outcomes as high-risk, with the full apparatus of risk management, documentation, logging and human oversight attaching. A formative agent that quietly becomes the basis of a mark has silently reclassified itself and the institution’s obligations along with it.
Its human counterpart. The academic who sets the rubric and owns the summative judgement. In practice, the useful arrangement is the Tutor CoPilot pattern — the agent coaches the human marker rather than the student, which is the configuration that produced measured learning gains at roughly $20 per tutor per year.
4. The Lab and Simulation Agent — where physical capacity is the constraint
What it does. Runs guided experimental work in simulation: a digital twin of the plant, the circuit, the structure or the process, with an agent that sets the task, watches the student’s actions, asks why, and injects the fault that makes the lesson land.
The evidence. The gAI-PT4I4 framework in this library is the strongest published template — generative AI combined with low-fidelity digital twins, VR and retrieval-augmented generation for personalised experiential learning in an Industry 4.0 training context. The AITEE agentic tutor for electrical engineering is the discipline-specific pattern: graph-based retrieval over course content plus tool use, applied to circuit analysis, which is exactly the shape of problem a technical university has a thousand instances of.
The hard boundary. Simulation must not silently replace the physical laboratory. An engineer who has only ever debugged a simulation has not encountered the thing that makes engineering difficult — that the model is not the world, that the connector is loose, that the tolerance stack-up ate the margin. The correct framing is capacity extension: unlimited rehearsal before scarce bench time, and unlimited variation afterwards, so the physical session is spent on what only physical presence teaches.
Its human counterpart. The lab supervisor, whose time is redirected from setup and standard procedure toward the moments where physical judgement is formed.
5. The Studio Critic — critique, never generation
What it does. Sits with a design student — architecture, mechanical design, systems design — and interrogates the design: what is this load path, why this material, what happens at the interface, what did you rule out and why. The oldest and best pedagogy in engineering, which is the crit, made continuously available.
The evidence. The ASEE study on integrating image-generative AI into conceptual design in a CAD class is the closest classroom protocol available, and the CDIO paper on project-based assessment in the generative-AI era supplies the rubric pattern — grading design decisions and their justification rather than the artefact’s authorship.
The hard boundary, and it is a real one. In design disciplines the temptation runs the wrong way: image generation is spectacular, immediate and hollow. A student who generates forty facade options has not designed anything; they have shopped. The agent’s job is to make the student defend, not to make the student options. This is the archetype where the boundary is hardest to hold because the violation is the most attractive, and it is worth noting that the Faculty of Architecture has the lowest study-failure rate at a technical university precisely because the crit is relentless — this archetype should extend that method, not dilute it.
Its human counterpart. The studio tutor, whose scarce judgement moves to the reviews that matter once the routine interrogation is continuously available.
6. The Instructional Design Agent — drafts for human authorship
What it does. Produces the first version of the things academics spend enormous unrecognised time on: a syllabus aligned to stated outcomes, lecture scaffolds, slide sets, problem sets with worked solutions, rubrics, and — most valuably — variants, so that the same assessment can exist in twenty forms.
The evidence. The multi-agent instructional-design paper in this library describes a pipeline generating syllabi, lecture scripts, slides and assessments end to end, and quantifies how much of the teaching-faculty workload agents can absorb. Georgia Tech’s Agent Smith is the deployed proof of the same principle at the agent-construction layer: it produces a working Jill Watson for a fresh syllabus in about 25 hours, where the equivalent bespoke build had previously been a research project.
The hard boundary. The output is a draft with an author, and the author is a person. This matters more than it sounds. An academic who accepts a generated syllabus has outsourced the one act — deciding what a course is for — that constitutes academic authority, and has done so invisibly, because the artefact looks the same either way. The ESG’s quality-assurance requirements attach to a named academic’s judgement about programme design; a syllabus nobody actually chose is not accredited work, however plausible it reads.
Its human counterpart. The course owner, whose time moves from producing artefacts to deciding and defending them. This is the archetype with the largest raw efficiency gain and the largest risk of quiet hollowing, and those two facts are the same fact.
Where the real value sits. Not in the first syllabus but in variant generation for assessment, which is what makes secured assessment affordable at scale. Twenty equivalent versions of an examination, generated and human-checked, changes the economics of the certification lane that the two-lane assessment model depends on.
7. The Early-Warning Agent — triggers help, never decisions
What it does. Watches the progression signals a university already collects — attendance, submission timing, formative performance, engagement with materials — and flags a student who is drifting toward failure, early enough for an offer of help to matter.
The evidence, and the caution attached to it. The archetype works: this is the one place where a technical university may already be running an agent without having named it. The pattern is well-established in the learning-analytics field, whose canonical reference — the SoLAR Handbook of Learning Analytics — covers predictive modelling, institutional adoption and the ethics of exactly this use.
But the caution is severe and specific. Gardner and colleagues’ study of temporal and between-group variability in college dropout prediction shows model performance degrading across cohorts and across student subgroups. A model trained on last year’s students underperforms on this year’s, and does so unevenly — typically worse for the subgroups an equity-minded institution most wants to serve. A prediction system deployed once and left running is not a stable instrument; it is a decaying one, and it decays fastest where the consequences are worst.
The hard boundary, which is where the law sits. The output is an offer of help to a human, never an input to a decision about the student. The moment a risk score influences admission, progression, level assignment or resource allocation, the system moves into the AI Act’s Annex III high-risk categories with the full compliance apparatus attached. Keeping it strictly on the support side of that line is both the legal and the ethical design, and it is easy to cross by accident — a dashboard that a study officer uses to decide who gets a place on a support programme has crossed it.
Its human counterpart. A study officer or tutor who makes the contact. The agent’s output is a name and a reason; the intervention is a person.
The non-negotiable operational requirement. Annual re-validation with subgroup fairness analysis published, in the manner the Jisc Code of Practice for Learning Analytics requires — responsibility, transparency, consent, minimising adverse impact, stewardship. An unaudited prediction model running on students is a liability that grows quietly.
8. The Evidence Agent — the one that audits the others
What it does. Instruments every other archetype: assembles the comparison conditions, tracks the pre-registered outcomes, monitors the leakage and drift rates, runs the subgroup analyses, and produces the evaluation that decides whether each agent is expanded, fixed or switched off.
The evidence. This archetype exists because the standards exist and the field ignores them. The What Works Clearinghouse Procedures and Standards Handbook defines what counts as evidence in education — design requirements, attrition thresholds, baseline equivalence, effect-size computation — and the EEF evaluator guide provides the operational playbook of protocols, pre-registration, analysis plans and process evaluation. Measured against either, most published claims about AI in higher education are satisfaction surveys with no comparison condition. Meanwhile the deployment papers that are honest — CS50’s leakage figures, Georgia Tech’s coverage-and-precision time series — are honest precisely because someone was measuring continuously rather than writing a case study afterwards.
The hard boundary. It reports to academic governance, not to the programme it evaluates. An evaluation function owned by the people whose project is being evaluated produces evidence theatre, which is failure mode four below.
Its human counterpart. A small unit with methodological authority — the people who would referee this work if it arrived as a paper.
Why it belongs in the architecture rather than in the appendix. Because the field’s central asymmetry is that the population needed to settle its open questions sits in every gateway lecture theatre, and almost nobody is collecting from it. The institution that instruments its own deployment ends up owning evidence nobody else has — which is publishable, fundable, and, in a field where most participants are guessing, a durable reputational asset.
The data layer — four strata
Agents without grounding are the failure case, and the Georgia Tech numbers quantify it more sharply than any argument: 76.7% answer accuracy for the course-grounded system against 31.3% for a generic assistant baseline on the same evaluation. The difference is not the model. It is the corpus. What follows is the minimum data architecture the eight archetypes require.
Stratum one — the course corpus. Lecture notes, slides, problem sets, worked solutions, past examinations, permitted textbook material, laboratory manuals, the syllabus and the rules. Per course, curated, versioned, with rights cleared. This is unglamorous work and it is the actual bottleneck: Agent Smith’s 25 hours per syllabus is mostly this. It is also where institutional politics concentrate, because it requires academics to hand over materials many regard as personal. The framing that works is that the corpus is the course’s, not the lecturer’s, and that a course whose materials cannot be assembled is a course with a continuity problem regardless of AI.
Stratum two — the interaction stream. Every question a student asks an agent, every point at which the agent failed or escalated, every conversation where the student gave up. This is the asset nobody is capturing and it is the one that compounds. A university has never before been able to see, at scale and in the student’s own words, exactly where its teaching is unclear — not inferred from examination performance months later, but recorded at the moment of confusion. Ten million CS50 queries is a map of what is hard about introductory computer science that no amount of pedagogical intuition could produce. The strategic point for any institution using a vendor’s general assistant is blunt and is stated correctly in CTU’s own 2023 methodological instruction: no AI tool used at the university is operated by the university, and all user–tool communication is visible to the operator. Which means that today, this asset is accruing to someone else.
Stratum three — the progression record. The institutional student data that already exists — enrolment, attendance, submissions, marks, resits, withdrawal. It feeds the early-warning agent and it is the outcome measure for every trial. It is also the stratum with the heaviest data-protection load, since it is identifiable, sensitive and consequential. The Jisc code of practice and the EDPS orientations on generative AI define the handling regime; the practical rule is that the interaction stream and the progression record should be joinable only under a defined governance process, not by default.
Stratum four — the outcome ledger. What the institution concluded, and on what evidence: which agents were evaluated, against what comparison, with what effect, and what was decided. This is the stratum universities never build, and its absence is why institutional memory in teaching innovation is roughly two years long. Every deployment, its trial, its result and its disposition, in one place, published.
The governance that binds the four together. One register, one owner. For each system: its Annex III classification; whether the institution is provider or deployer; its data flows; its human-oversight arrangement; its quality-assurance route; and the academic accountable for it. Built before the first pilot, this is modest work. Retrofitted after the third, it is a project.
The build sequence and what it costs
An architecture is not a plan. The order in which these components are built determines whether the programme accumulates trust or spends it, and the published deployments are unusually consistent about what that order should be. Three principles govern it, and all three cut against institutional instinct.
Build in ascending order of consequence, not descending order of efficiency. The systems that look like the biggest wins — automated grading, admissions triage, proctoring — are precisely the ones the AI Act’s Annex III classifies as high-risk, and precisely the ones where an early failure is unrecoverable in trust terms. A university whose first visible AI deployment marks someone unfairly has lost the argument for a decade. Start where an error costs a corrected deadline.
Build depth in one course before breadth across many. Georgia Tech’s coverage-and-precision series is the argument: from about 21% of questions answered at 80% precision in 2017 to over 96% coverage at over 86% precision by autumn 2019. That improvement came from iteration on a working system in real classes, not from a wider rollout. A university that launches twelve mediocre course agents simultaneously has twelve mediocre course agents; one that gets a single one to Georgia Tech’s numbers has a template, an evaluation harness, and staff who believe it.
Build the evaluation harness before the thing it evaluates. This inverts normal practice and is the single most valuable procedural commitment available, because retrofitting a comparison condition is impossible — once every student has the tutor, the counterfactual is gone forever.
The resulting sequence has four stages, and a realistic institution should expect it to take about two years to complete for a first faculty.
Stage one — the Concierge and the harness, months one to four. Build the course-logistics agent over syllabus and rules material, because it is the cheapest archetype, the SyllabusQA work gives a reference implementation with a factuality metric, and the downside of error is bounded. In parallel, stand up the evaluation function and the AI Act register. The deliverable is not really the agent; it is a working corpus pipeline, a logging architecture, an evaluation set, and an institution that has now done this once.
Stage two — one Gateway Tutor, months four to nine. One course, chosen on failure rate rather than on the enthusiasm of its lecturer, with the constraint set specified by the course owner, a named agent steward running weekly sampled review, and a randomised waitlist design so half the cohort receives it in semester one and half in semester two. The deliverable is an effect size on a progression outcome — which almost nobody in Europe currently has.
Stage three — Feedback and Lab agents, months nine to eighteen. Formative feedback against the rubric, and simulation-based practice where physical laboratory capacity is the binding constraint. Both are meaningful step-ups in pedagogical consequence and both should wait until the review loop has been running long enough to have caught its first systematic failure.
Stage four — Instructional Design and Early Warning, months twelve to twenty-four. The design agent because it needs academics who already trust the system to accept drafts from it without accepting them uncritically; the early-warning agent because it needs the fairness-audit machinery in place first, and because it sits closest to the high-risk boundary.
The economics, using the published numbers
The cost structure of this engine is genuinely unusual, and misreading it is the commonest planning error — in both directions.
Inference is close to free at university scale. Harvard’s CS50 reports approximately $1.50 per student per year across 211,000 students and 10 million queries; an earlier cohort figure was about $1.90 per student per month at 15,000–20,000 prompts per day, with a per-prompt cost of about five cents. Stanford’s Tutor CoPilot ran at roughly $20 per tutor per year. For a faculty of three thousand students, a fully-used tutoring layer costs less than a single conference budget. Any business case that founds itself on model licensing costs has misidentified the expense.
Corpus preparation is the real per-course cost, and it is one-off. Georgia Tech’s Agent Smith produces a working agent for a fresh syllabus in about 25 hours — an honest figure to plan against, covering material assembly, rights clearance, structuring and initial evaluation. Twenty-five hours per course, then maintenance at each syllabus revision.
Supervision is the real recurring cost, and it is the one that gets cut. CS50’s response to a 22% code-leakage rate was teaching fellows reviewing and correcting agent behaviour; Georgia Tech’s climb from 21% to 96% coverage was iteration by people. Plan a fraction of a teaching-assistant post per course in production, permanently. This is roughly the cost of the marking hours the feedback agent frees, which makes the programme close to cost-neutral in staffing terms — but only if the freed hours are consciously redeployed into supervision and contact rather than quietly absorbed as savings.
The saving is real but is not headcount. Georgia Tech’s 500-plus teacher hours across a dozen classes is the best-documented figure, and it is overwhelmingly logistics and repeated-question load. That is a genuine, large recovery of academic time. It is not a staffing reduction, and any institution that books it as one will find within two years that it has removed the supervision capacity the system depends on and is running an unmonitored agent on its own students.
The return that actually justifies the programme is attrition. Every other line above is a modest efficiency. A university that loses roughly a third of a first-year cohort is losing the recruitment cost, the teaching cost and the entire future contribution of those students — against an intervention whose evidence base says the effect concentrates precisely in that population. No productivity saving in the administrative use cases comes within an order of magnitude of it, which is why the sequence puts a gateway course second and grading nowhere.
The operating model — four roles that must exist by name
Every deployment in this library that worked has the same structural feature, and every account of one that stalled has its absence: named humans with defined authority over specific agents. Ithaka S+R’s interview work across nineteen universities describes adoption happening in isolated pockets driven by individual enthusiasm and dying on institutional friction; the multi-institution barriers study shows the obstacles operating independently at individual, departmental and institutional level, so fixing one changes little. What follows is the minimum role set. It is deliberately small, because a governance structure nobody can staff is a governance structure that does not exist.
The course owner. The academic accountable for what a given course’s agents say. They approve the corpus, they set the pedagogical constraints — what the tutor must refuse, at what point it should escalate — and they own the consequences. This role is not new; it is the course guarantor with an additional and quite specific responsibility. The reason it must be explicit is that the alternative is the default, in which the agent’s behaviour is determined by whoever wrote the system prompt, usually a technologist, usually months ago, usually without the course in front of them.
The agent steward. The person who runs the human-feedback loop: sampling conversations weekly, correcting behaviour, escalating systematic failures. In CS50 this is a teaching fellow function, and the published account is unambiguous that it is what fixed the leakage problem — not prompt engineering, but staffed review. At a realistic ratio this is a fraction of a teaching-assistant post per course, permanently. It is the line item most likely to be cut in a budget round and the one whose removal silently converts a working system into a broken one, because nothing visible changes for several months.
The evidence lead. Owns the comparison conditions, the pre-registrations, the drift audits and the publication. Reports to academic governance rather than to the AI programme, for the reason given above. Three to four people can run this for an entire university, which makes it the cheapest structural component and the one that determines whether the institution ends up with knowledge or with anecdotes.
The certifying academic. The person who signs, under professional responsibility, that a named human holds a capability. This role is not delegable to an agent, and the constraint is legal as well as cultural: Annex III of the AI Act classifies systems evaluating learning outcomes, determining admission and assigning educational level as high-risk, and the ESG’s quality-assurance framework rests on human academic judgement being identifiable and accountable. The deeper point is that certification is the university’s actual product. Explanation, feedback and practice can all be substantially automated without the institution losing what it is; certification cannot, because a degree is a statement one institution makes to strangers about a person, and its value is entirely a function of who is answerable for it.
The accountability rule that binds them. For every agent in production, a single name against each of: corpus, constraints, review loop, evaluation, and the QA route. An agent without all five is not governed; it is merely running.
The graduate this machine produces
If teaching is rebuilt on this substrate, the question that follows immediately is what the resulting engineer is actually good at — because it will not be the same list as before, and pretending otherwise is how a curriculum becomes ceremonial. The evidence in this library supports four capabilities, and each is defined by being something the machine specifically does not supply.
Verification. The ability to establish, by independent means, whether a machine-produced result is correct in one’s discipline. This is the load-bearing competence of the entire agentic era and it is currently taught almost nowhere as an explicit skill. The Microsoft Research and Carnegie Mellon study of 319 knowledge workers found that generative AI shifts effort toward verification and integration — and simultaneously found that higher confidence in the AI predicts less critical-thinking effort, which means the shift is toward a task people are increasingly disinclined to perform. For an engineer this is not a productivity question. A structural calculation, a control loop, a dosage algorithm, a safety interlock: being unable to tell that the plausible answer is wrong is how people are harmed. Digital Promise’s Understand / Evaluate / Use framework is the most teachable operationalisation available.
Frontier judgement. The ability to tell which side of the capability boundary a given task sits on — where these systems are strong, where they fail, and how you know before you find out. The Dell’Acqua, Mollick and Lakhani field experiment supplies the definitive image: inside the frontier, 758 consultants produced work rated 40% higher in quality; on a task just outside it, they were 19 percentage points less likely to reach the correct answer than colleagues working with no AI at all. The tool did not merely fail to help; it degraded performance, because the boundary is invisible from the inside and the failures are fluent. This is teachable, it is assessable, and the assessment is straightforward: give the student tasks on both sides of the line and mark them on whether they knew which was which.
Unassisted core reasoning. The ability to do the discipline’s foundational thinking without assistance, demonstrated at defined points. Not because unassisted work is how they will practise — it is not — but because the verification competence above is parasitic on it. You cannot check what you could never have derived. This is the pedagogical justification for the secured lane of the two-lane assessment model, and it is a stronger justification than integrity: the reason to examine an engineer unaided is not to catch cheats but to establish that the foundation exists on which their judgement of machine output rests.
Accountability for results one did not personally generate. The professional stance of signing off on work produced by a system. Every engineer already does a version of this — with finite-element packages, with library code, with a supplier’s datasheet — which is precisely why engineering is better placed than most disciplines to teach it. What is new is the fluency and generality of the thing being signed off, and the fact that its errors do not look like errors.
What follows for curriculum. These four are not a module. A module teaches AI literacy as content; these are dispositions that only form by being required repeatedly, in the student’s own discipline, across a degree. The frameworks that supply the vocabulary already exist and should be adopted rather than reinvented — UNESCO’s AI Competency Framework for Students with its twelve competencies at Understand / Apply / Create levels, the joint OECD–European Commission AI literacy framework, DigComp 2.2’s AI-specific examples, and the AI Literacy Heptagon’s translation of the abstract discourse into higher-education learning objectives. What none of them supply, and what a technical university must add, is the discipline-specific verification standard: what counts as checking, in circuits, in structures, in code, in control.
Six ways this fails
Every failure mode below is documented in the library rather than imagined, and each has a countermeasure that must be designed in rather than added later. They are ordered from the most tractable to the least.
1. Instruction dilution — the guardrail stops binding. The best-documented failure in educational AI, and the one with a published number. Harvard’s CS50, running the most pedagogically careful deployment on record, reports that 22% of ten million responses contained code blocks despite instructions not to give solutions — 48% at conversation level, roughly 635,000 of 1.3 million conversations. The mechanism is that in a long exchange the system prompt’s authority decays against the accumulating context, and the model reverts to being maximally helpful, which in a teaching context means maximally unhelpful. Countermeasure: treat the constraint as an engineered system with a measured failure rate, not as prose — conversation re-anchoring, refusal checks running outside the prompt, retrieval that returns scaffolding rather than solutions, and the staffed sampling loop that CS50 identifies as the actual fix. And publish the rate, because a leakage figure nobody measures is a leakage figure that grows.
2. Model drift across cohorts and subgroups. Predictive components decay, and they decay unevenly. Gardner and colleagues show dropout-prediction performance degrading both temporally and between student groups, typically worst for the groups an institution most wants to help. A model validated once and left running is a slowly failing instrument whose failures are invisible in the aggregate. Countermeasure: annual re-validation with published subgroup analysis, under the Jisc learning-analytics code of practice, and a standing rule that the output triggers an offer of support rather than a decision.
3. Vendor dependency. The general assistant is convenient, the enterprise data-protection terms are genuine, and the trajectory is a university whose teaching runs on a system it does not operate, whose pedagogical behaviour is set by someone else’s product roadmap, and whose richest teaching-improvement dataset accrues to a third party. The observation is made most crisply in CTU’s own methodological instruction: no AI tool used at the university is operated by the university. Countermeasure: the layered stack — a commercial general assistant as the floor, a university-built, course-grounded, instrumented layer above it, with the interaction stream retained institutionally. The retrieval layer is not hard engineering; a technical university can build it to a higher standard than it can buy it.
4. Evidence theatre. The most likely failure by a wide margin, because it is comfortable and produces good slides. The programme deploys, collects satisfaction data, reports high enthusiasm, and concludes success — while never establishing a comparison condition or measuring learning. The published deployment literature is riddled with this: CS50’s 94%-found-it-helpful is a real and useful number that says nothing whatever about whether anyone learned more, and its authors do not claim otherwise. Measured against the What Works Clearinghouse standards or the EEF’s evaluator guide, most claims in this field would not qualify as evidence. Countermeasure: the evidence lead reports to academic governance, not to the programme; no deployment goes live without a pre-registered outcome and a comparison condition; and the outcome ledger is published including the failures.
5. Fairness drift in assessment. An institution that abandons detection but keeps its instincts will find informal suspicion migrating into marking. The detector evidence is the warning: Weber-Wulff and colleagues found fourteen tools neither accurate nor reliable, and Liang and colleagues found GPT detectors misclassifying roughly 61% of non-native English speakers’ essays as AI-generated while performing near-perfectly on native writing. The same bias — fluency read as authenticity — operates in human assessors, unmeasured. For any institution with a large international cohort this is a live equity exposure. Countermeasure: measure the misconduct-rate disparity between international and domestic students and publish it; move certification into secured components where identity, not style, is the evidence; and never reinstate detection informally after withdrawing it formally.
6. The hollowed apprenticeship — and nobody has solved this one. The compression finding that makes the tutoring case so strong contains its own long-term problem. AI raises the floor: +34% for novices against near-zero for experts in the NBER field study; the largest gains for below-average performers at BCG; +9 percentage points for students of the weakest tutors. Simultaneously, Stanford’s Digital Economy Lab finds employment declining specifically among young workers in the occupations most exposed to AI — the entry-level technical roles where novices have historically become experts. So the technology compresses the novice–expert performance gap while eroding the jobs in which that gap was closed. A graduate can now perform like a competent junior on day one and may find no junior role in which to become a senior. The university cannot fix the labour market. But it can notice that the apprenticeship function is migrating upstream into the degree, and that this changes what the final years of an engineering programme are for: less coverage, more supervised responsibility for real work with real consequences — which is what the capstone, the industrial project and the laboratory were always the seed of, and which CESAER’s challenge-based direction and the CDIO tradition already describe. Countermeasure: none that is sufficient. The honest position is that this is the open problem, that it is structural rather than institutional, and that a university which has thought about it for three years will be better placed than one that has not.
Close — what the engine is for
It is worth restating what all of this is actually in service of, because an architecture document can easily read as though the architecture were the point.
The eight archetypes, the four data strata and the four named roles exist to do one thing: move scarce human attention to where only human attention works. A technical university’s real scarcity has never been information — the material has been in books for a century — and it has never been enthusiasm. It is the number of hours in which an experienced engineer sits with a student who is stuck and asks the question that reorganises their understanding. Every hour that person currently spends answering when the resit is, explaining the same misconception for the fortieth time, marking a formative draft, or assembling a slide deck is an hour not spent doing the thing nobody else can do. The engine is a mechanism for buying those hours back. It is not a mechanism for having fewer of those people.
That distinction is the whole difference between an agentic university and an automated one, and it will be decided in budget meetings rather than in strategy documents. The published evidence gives the honest version of the trade: Georgia Tech’s 500 saved teacher hours are real; CS50’s $1.50 per student per year is real; and both institutions staffed the review loop that made the systems work. An institution that takes the savings and skips the loop has not built this engine. It has installed the failure mode and called it a transformation.
The last argument is the one that should decide the timetable. The evidence base in this field is thin, the standards for evidence are published and largely ignored, and the population needed to settle its central questions is sitting in every gateway lecture theatre in Europe. Whoever instruments first, defines the field — not through better technology, which is a commodity, but by being the institution that can say, with a control group, what happened. For a technical university that already publishes on machine learning, that is not a stretch into unfamiliar territory. It is the application of its own standards to its own teaching, which is a smaller step than it sounds and a much larger change than it looks.
