Research Note
Institutional AI Tutor Pilot Framework
Evaluate a narrow learning or work task before evaluating a platform. Novel output is not a success measure.
In this article
Institutional AI Tutor Pilot Framework
Pilot principle
Evaluate a narrow learning or work task before evaluating a platform. Novel output is not a success measure.
Pilot charter
| Area | Required record |
|---|---|
| Job | One low-risk learning or onboarding outcome |
| Participants | Included and excluded groups, consent, support, withdrawal |
| Sources | Approved manifest, rights, versions, conflicts, data classes |
| Account | Product, edition, administrator, region, model, applicable terms |
| Access | Identity, roles, sharing, export, public access, deletion |
| Baseline | Current learning or task performance without the tool |
| Fidelity | Sampled claim and citation audits |
| Outcome | Comparable independent assessment or task result |
| Accessibility | Participant testing, accommodations, barriers, alternatives |
| Integrity | Permitted assistance, attribution, assessment rules |
| Incidents | Reporting, containment, correction, notification, recovery |
| Exit | Stop, revise, expand, export, delete, and ownership decision |
Evidence
Vendor documentation can establish stated features and terms. It cannot establish the organization's actual configuration, legal compliance, accessibility, learning impact, source quality, or policy fit.
The organization must verify account type, administrative controls, sharing behavior, retention, connected services, feedback behavior, and data handling.
Use a subject expert to audit citations and claims. Use an educator or process owner to define and assess the outcome. Use privacy, security, accessibility, legal, copyright, labor, procurement, and records expertise as required by context.
Decision
Stop when the task is unsuitable, sources are unauthorized, participants cannot withdraw, access cannot be controlled, incidents cannot be handled, or outcomes cannot be measured.
Expand only after the organization has evidence for the named task and a clear owner for continued review.
Sources
Follow the evidence.
- blog.google: notebooklm new features december 2024blog.google
- blog.google: notebooklm audio overviewsblog.google
- daltonanderson.ghost.io: googles ai tutor the future of personalized learningdaltonanderson.ghost.io
- edu.google.com: ai notebooklmedu.google.com
- journals.sagepub.com: fulljournals.sagepub.com
- NIST Generative AI Profilenvlpubs.nist.gov
- open.spotify.com: 1YXy6yyC4u2mnqeARVB5qvopen.spotify.com
- studentprivacy.ed.gov: privacy and education technologystudentprivacy.ed.gov
- support.google.com: notebooklmsupport.google.com
- support.google.com: 16164461support.google.com
- support.google.com: 16213268support.google.com
- support.google.com: 16212820support.google.com
- support.google.com: 16179559support.google.com
- support.google.com: 16215270support.google.com
- support.google.com: 16322204support.google.com
- support.google.com: 17003757support.google.com
- support.google.com: 17004255support.google.com
- daltonanderson.net: googles ai tutor the future of personalized learningdaltonanderson.net
- NIST AI Risk Management Frameworknist.gov
- www2.ed.gov: ai reportwww2.ed.gov
- youtu.be: 6BwWKkZ7aeAyoutu.be