Learning Signal Brief #3: Thin AI Evidence, Authentic Work and Making for Others
This week's signal: Schools are adopting student-facing AI faster than the evidence can say what works. NPR and a Stanford review describe that gap, and a large study of Chinese secondary students links self-directed AI use to lower exam scores. The design items respond with practice: a math coach's four-step AI planning routine, more prototyping inside project units, and a test for whether work has a real audience. The makerspace and career-pathway items share the same thread of real users, employers and consequences.
10 articles, 2 short takes and a quote of the week. About 9 minutes to read.
In this issue
AI and Agentic AI in Teaching and Learning
Districts try student-facing AI with little evidence of what works
Stanford finds only 20 rigorous studies of AI in K-12
A 27,000-student study links student AI use to lower exam scores
Instructional Design and Curriculum
A math coach gives teachers four steps for planning lessons with AI
Spencer says project units need more prototypes, not just more time
A Getting Smart essay offers a test for authentic project work
Makerspaces, STEM and STEAM
Students rebuild a ride-on toy car for a child with mobility challenges
Two Illinois high schools open makerspaces with 3D printers and laser cutters
Innovation and Entrepreneurship
Texas leaders: AI raises the value of judgment and self-management
Danbury High moves every student into a career academy
AI and Agentic AI in Teaching and Learning
1. Districts try student-facing AI with little evidence of what works
“Schools are experimenting with AI with little evidence or policy to guide them,” by Lee V. Gaines, NPR, September 28, 2026
NPR visited Dayton, Ohio, where students at an alternative school talked with Jordan, a chatbot that presents as angsty. Instead of asking how students feel, Jordan asks them for advice and grades their answers for engagement and authenticity. Dayton paid SchoolAI $72,600 last year and renewed for $78,650.
The district's outside consultant, Mike Kentz, says the pilot produced enough positive evidence to keep testing but is not ready for the whole district. “Every test, every experiment that I try to measure in some sort of concrete way, I can only get as far as perception,” he says. Wichita's AI specialist says that district has no answer yet on what metric to use, so student-facing tools stay limited.
NPR cites a Stanford review of more than 800 papers that found research on helping students is extremely limited. This is a news report built on district staff and consultants, and the Dayton evidence is perception, not outcomes.
Why it matters: Define what counts as success before launching any student-facing AI pilot, and treat student and staff perception as a starting point, not proof.
2. Stanford finds only 20 rigorous studies of AI in K-12
“The Evidence Base on AI in K-12: A 2026 Review,” Stanford SCALE research summary, publication date not shown on the page
Stanford SCALE reviewed more than 800 papers on AI and K-12 in its research repository as of October 2025 and found only 20 high-quality causal studies of whether AI tools changed outcomes for students or educators. Most research treats students as the users, and the causal studies cluster in math.
Students often perform better while using AI tools, but results are mixed when they are assessed without AI support. Tools with pedagogical guardrails, such as tutors that give hints or guide reasoning, show more promising outcomes than general chatbots that give answers directly. Early studies of teacher-facing tools suggest AI may reduce lesson-preparation time while maintaining instructional quality.
The summary lists large gaps: no high-quality causal studies of student AI use in U.S. K-12 classrooms, and mostly short-term outcomes. This item relies on the authors' own summary, not the full report.
Why it matters: For leaders: Ask vendors whether their tool gives hints or answers, and ask for results from assessments students took without the AI.
3. A 27,000-student study links student AI use to lower exam scores
“The Penalty Students Pay for Using AI,” by Benjamin Riley, Education Next, September 23, 2026 (adapted from his Cognitive Resonance Substack)
Benjamin Riley, founder of Cognitive Resonance, writes about a June 2026 working paper by David Strömberg, Victor Lei and Yanhui Wu, using data from about 27,000 students in grades seven to 12 in China.
Between 2023 and 2025, about 80 percent of the students started using generative AI, and after five months about half of those students were fully outsourcing homework. AI users spent less time on homework and scored higher on it, but lower on closed-book exams. For students who adopted AI two or more years before testing, the estimated declines were 24 percent on the high school entrance exam and 18 percent on the college entrance exam.
The researchers write that “our findings show that generative AI, which is likely to become a prevalent technology for education, has a substantial negative impact on student learning.” Riley notes the estimated penalty fell from around 25 percent in early 2023 to around 16 percent by June 2025.
This is a commentary by an AI critic on a single working paper from China. Riley rebuts the claim that AI helps students who study for the same time, saying the 75-minute spike involves 20 students among almost 27,000.
Why it matters: For leaders: Compare closed-book results with homework scores when you review student AI use, since the two moved in opposite directions.
Instructional Design and Curriculum
4. A math coach gives teachers four steps for planning lessons with AI
“Crafting Quality Math Lessons with AI Assists,” by Jen Knudsen, MiddleWeb, September 27, 2026
Math curriculum consultant Jen Knudsen describes four steps for planning a lesson with ChatGPT, Gemini or Claude, drawn from dozens of lessons she wrote with AI over the past year. Step one is a planning table covering pedagogy, topic, resources, key features and special supports, including two students who need support. The table is meant to override the AI's default style.
Knudsen reports that ChatGPT and Gemini drift toward gradual release, while Claude often drifts toward guided discovery. Three drafts from three AI models covered only two cases of adding unlike fractions and missed the most common one, denominators that share a factor, like 1/6 + 3/8, until she named it. She also warns, “do not trust the answer key it creates.”
This is one practitioner's account of her own method, with no student results or comparison group. Knudsen now trains teachers in AI lesson planning, and she says the steps may not save time but do save labor.
Why it matters: For teachers: Put your teaching method and two target students into the prompt, then check each draft for missing problem types and misconceptions.
5. Spencer says project units need more prototypes, not just more time
“How Quantity Can Improve the Quality of Project-Based Learning,” by John Spencer, Spencer Education, September 29, 2026
Spencer describes a training where a university dean said industry partners audited the school's fully project-based program. They found students worked too slowly, were weak at rapid prototyping and worked in isolation, so the school added more prototyping and shorter projects.
He cites studies on quantity and quality. In a brainstorming study by Paul Paulus and colleagues, the group told to produce as many ideas as possible generated not only more ideas but better ones. In a Stanford study led by Steven Dow, novice designers who made several prototypes at once produced ads that performed better on expert ratings and click-through data. Spencer notes that some research finds diminishing returns.
His proposal is to add “more” to each phase of a project: more questions, ideas, prototypes and feedback, including short rapid prototyping sprints. This is a practitioner blog post that summarizes the studies briefly and offers no classroom results of its own.
Why it matters: Add a one-to-two-day rapid prototyping sprint, with several quick versions, before students commit to a final product in your next project unit.
6. A Getting Smart essay offers a test for authentic project work
“Why the Experience of Experience Matters: Designing learning through experiences that actually have an impact,” by Chris Unger, Getting Smart, October 1, 2026
Chris Unger, who teaches a doctoral course on experiential learning, argues that many school tasks borrow the look of real work without a real audience, purpose or consequence. He uses the phrase “faux experience” for examples such as an essay only the teacher will read or a business plan with no possible client.
He says authenticity grows when work addresses a real need, carries consequences beyond a grade, draws feedback from people affected by it and requires revision. He cites a meta-analysis of 66 studies finding project-based learning produces gains over traditional instruction, with effects that depend heavily on how well it is designed.
His school examples come from the America Succeeds Research Practice Collaborative, a 16-month look that included interviews with more than 200 students, educators and industry partners. Unger says its case studies are illustrative, not proof of a single right model. This is an opinion essay by a guest author.
Why it matters: Test your next project with Unger's questions: who besides the teacher will see, use or respond to the work, and where will students get feedback?
Makerspaces, STEM and STEAM
7. Students rebuild a ride-on toy car for a child with mobility challenges
“What Happens When Student Innovation is Rooted in Empathy,” by Sara Mehrabi, Digital Promise, September 30, 2026
Teacher Sara Mehrabi describes a Ciena Solutions Challenge project at Dove Virtual Academy. Students redesigned a battery-powered ride-on toy car for a young girl with mobility challenges, because the original required pressing a foot pedal. Before anyone picked up a tool, Mehrabi asked her students, “Who are we building this for?”
Students researched adaptive mobility devices and accessibility, then spent several weeks brainstorming, testing prototypes, rewiring electrical systems and modifying the controls. Mehrabi says their questions shifted from circuits to the child, such as “Will she be able to drive by herself?” They delivered the car to the child and her family at Oklahoma Children's Hospital.
This is a teacher's first-person account published by Digital Promise, which promotes the challenge and invites educators to register projects. It reports no learning measures.
Why it matters: For teachers: Open an engineering project by asking who the design is for, and have students research that person's needs before they choose a solution.
8. Two Illinois high schools open makerspaces with 3D printers and laser cutters
“Unit 5 makerspaces put creation in students’ hands,” by Mateusz Janik, The Pantagraph, September 24, 2026
McLean County Unit 5 opened makerspaces at Normal West and Normal Community high schools, funded by more than $50,000 from the Unit 5 Educational Foundation. The spaces have laser cutters, heat presses, large-format printers and 3D printers. Students are designing an object to position a fan, banners for a school club and rocket prototypes.
Normal West technology teacher Brock Keller says students learn to interpret designs, use software and export files to machines to make items such as study guides, flashcards, posters and T-shirts. His department's motto is “Hands on, minds on.” A QR code on a sheet of “smart material” sets the laser cutter's parameters automatically.
The makerspace is not fully running yet. Networking, training and standard operating procedures remain, and Keller plans for students to write the user guides and operating procedures before they train other students, with full implementation hoped for by the second semester. Superintendent Kristen Weikle says students can learn from designs that fail: “They learn just as much when things don’t go right.”
This is a local report from the program's first weeks, with no learning results.
Why it matters: For leaders: Have students write the user guides and operating procedures and train classmates, as Keller plans, so the space becomes a schoolwide project.
Innovation and Entrepreneurship
9. Texas leaders: AI raises the value of judgment and self-management
“Portrait Of A Graduate In The AI Age,” by Ray Bendici, Tech & Learning, September 25, 2026, reporting on a regional leadership summit panel
Four Texas district leaders, from Pflugerville, Arlington, Pasadena and Belton ISDs, discussed what a Portrait of a Graduate should include in the age of AI. Arlington's Kyle Berger named human judgement and critical thinking as skills growing in importance. Pflugerville's Quintin Shepherd added leadership, including how to lead humans and agents.
On what schools should stop, Belton's Frank Murray asked, “Why are we not grading along the process rather than just the final product?” On what to start, Shepherd said schools should teach students “how to make a job, not get a job.” Murray described encouraging a student who built a successful site of his own instead of saying no.
This is a transcript of four leaders' opinions with no outcome data. Murray's claim that the top skill by 2030 will be using AI for work is stated without a source.
Why it matters: For leaders: Pick one graded assignment this term and add credit for the process, such as drafts and feedback, not only the final product.
10. Danbury High moves every student into a career academy
“How a Connecticut district gave its high school a career-connected revamp,” K-12 Dive, Lessons in Leadership interview with Superintendent Kara Casimiro, date not shown on the page
Danbury Public Schools replaced a traditional model at Danbury High School, the largest high school in Connecticut, with 21 career pathways. Casimiro says there is no opt-out: every student takes part in academy work. A ninth-grade exploration course introduces the pathways, which include business and entrepreneurship.
Each academy has a college-preparatory sequence with a career theme, embedded industry credentials and an advisory board with industry and higher education, and the district is building work-based learning. Casimiro says students without family career networks see fewer careers, and that “the career track is really for everyone.”
This is an interview with the superintendent about her own district's program, and it gives no outcome data. She says graduation rates were starting to slip before the redesign and cites engagement and graduation benefits seen in other districts' models.
Why it matters: For leaders: Before building pathways, visit districts already running them, and seat employers and colleges on an advisory board from the start.
Short takes
Microsoft Learning Zone is a Windows app that turns a teacher's prompt and source materials into an editable interactive lesson. Tech & Learning's explainer says it generates an outline first so teachers can adjust the structure before the full lesson is built. Microsoft says it is included with Microsoft 365 Education licenses at no added cost, but teachers who create lessons need a Copilot+ PC. Luke Edwards, Tech & Learning. This is a product explainer, not an evaluation of learning. Read the original →
Ross Hartley, who teaches computer science and robotics at New Albany Middle School in Ohio, builds a week around a theme. In his Winter Olympics unit, students laser cut a wood flag, coded games in Scratch or MakeCode Arcade, and designed 3D medals in Tinkercad, choosing their own platforms. Hartley, CSTA. This is a teacher's self-reported practice with no student results. Read the original →
Quote of the week
Robin Lake directs the Center on Reinventing Public Education, which tracks AI adoption in K-12 schools. She was describing how districts and educators across the country are using AI, from lesson planning to chatbots that give students feedback to a humanoid robot at a San Diego charter school.
“It feels very Wild West in terms of the randomness.”
— Robin Lake, in NPR's September 28 report on schools experimenting with AI
The Learning Signal Brief summarizes the week's most useful articles on innovation in K-12 education, chosen from more than 100 sources. Summaries are original, quotes are short and attributed, and every item links to the original. Compiled with AI assistance and reviewed by Ryan Keser.
Comments