Structured AI-Guided Essay Revision
with Dual-Rubric Assessment
A Human–AI Collaborative Approach to Evaluating Writing and Interaction Quality in an EFL Course
Scan Slides
Try the Platform
AI Has Enormous Potential — But Teachers Are Left Out of the Loop
The Opportunity
AI can engage every student in a genuine, personalised writing conversation — at scale.
For EFL learners who rarely get individual feedback, this has enormous potential.
The Risk
Unconstrained AI use leads to overreliance — students accept suggestions passively rather than engaging critically.
The process of revision disappears.
The Core Gap
Teachers are left out of the loop.
No visibility into how students interact with AI. No structured workflow. No way to monitor or assess the process.
The Answer: Build Your Own Platform
Control the Interaction
We write the system prompt — the hidden instruction that shapes everything the AI says. The AI follows our pedagogical logic, not the student's whims.
Three structured steps. No free-form rewriting. AI guides; student thinks.
Own the Data
Every message, every exchange, every score is saved to our database. Teachers see the full chat record — not just the final essay.
Nothing happens off-record. The process is as visible as the product.
Do More with Data
With full access to chat + essay data, we can: use AI to score essays, measure interaction quality, flag low-engagement students, and conduct research — all from one system.
Data that would otherwise disappear becomes a research asset.
LANG 0036 — Remedial English at HKBU
Who Are These Students?
- University freshmen at Hong Kong Baptist University who scored lower on English placement tests
- Required to take LANG 0036 before the standard first-year Academic English course
- Many have limited experience with argumentative or academic writing
- EFL learners who rarely receive individual, sustained feedback on their writing
What Is the Course?
- LANG 0036 — Enhancing English through Global Citizenship
- Focus: academic writing, essay structure, argumentation, and critical thinking
- 840 students enrolled across 48 sections in Semester 1, 2025–26
- Taught by a large team — individual writing coaching at scale is impossible
A Point-of-View Essay — Graded in Two Dimensions
The EEGC AI Edit Module (Enhancing English through Global Citizenship)
logs in
Learn the task
Sample essay
Own essay
emailed
Dashboard
Thesis Revision
AI evaluates the thesis for clarity and argumentative stance. Student must revise before proceeding — no skipping.
Topic Sentence
AI checks whether the topic sentence supports the thesis and is logically linked to the argument.
Paragraph Revision
Full revision: evidence, logic, vocabulary, grammar. AI gives feedback but does not rewrite.
- Open the platform live
- Log in with the audience guest token
- Start Training Mode and show the AI interaction
Who Used It?
(some students completed both modes)
Report Contents
- Complete chat history (student ↔ AI)
- AI-generated scores on both rubrics
- AI contribution analysis (qualitative)
- Original and revised essay drafts
Delivery
- Emailed to student's HKBU address
- PDF + Markdown format
- Teacher dashboard: section-level view
- 1,162 files total across 734 sessions
Does Interaction Quality Predict Improvement?
The Pattern
Students with higher interaction rubric scores (Conversation Depth, Critical Review, Refining Process) also demonstrated greater essay quality improvement between original and revised drafts.
Why This Matters
- Justifies Rubric 2 as a diagnostic tool, not just a participation grade
- Teachers get a leading indicator: low interaction → likely weak revision
- Supports structured revision as a replicable pedagogical model
⚠️ Important Caveat
This is correlational, not causal. Stronger writers may both engage more deeply with AI and show greater improvement.
A Replicable Human–AI Collaboration Model
🤖 AI Handles
- Scalable formative feedback (840 students, instant)
- Structured three-step revision guidance
- First-pass rubric scoring (both rubrics)
- Report generation and delivery
- Interaction quality logging
👩🏫 Teachers Handle
- Quality assurance on AI scores
- Section-level progress monitoring
- Targeted intervention for struggling students
- Final grade decisions
- Pedagogical design of the module
Implications & Next Steps
For Practice
- Three-step mandatory sequencing prevents surface-only revision
- Dual rubric opens a window into how students learn with AI
- Model is replicable: any EFL/EAP writing course with LMS integration
- Interaction rubric score can flag disengaged students before final grading
For Research
- Interaction analytics as a new lens on AI-mediated learning
- Kappa results point to where AI needs human calibration
- Partial ρ suggests interaction quality has independent predictive value
Limitations
- Correlation ≠ causation: interaction ↔ improvement
- Vocabulary/Grammar kappa needs further calibration
- One semester, one institution, one task type
- Privacy: reports contain full chat histories — institutional consent obtained
Next Steps
- Sem 2 data collection (ongoing)
- Deeper chat history pattern analysis
- Longitudinal writing improvement tracking
- Refining interaction rubric operationalisation
Vibe-Coded — A One-Stop Platform for Engagement & Research
Engage
Students interact with the AI tutor directly in their browser — no app install, no account setup beyond a student ID.
Three structured modes: Briefing → Training → Assessment
Collect
Every session is auto-saved to a database and the full report is emailed to the student and teacher the moment it's generated.
Zero manual data collection. 734 reports. One semester.
Analyze
Teacher dashboard + backend analytics. Query scores, correlations, and interaction patterns — all from the same system that ran the course.
The data on this slide was pulled live from that database.
eegc.hkbu.me · Nuxt 4 + Poe API + PostgreSQL · open for replication
A Platform Any Teacher Can Use — What It Would Take
🏗️ What We Can Build
An online form-based platform where teachers configure an agentic tutor without writing a single line of code.
No app install. No account for students. Works on any device.
BYOK — Bring Your Own Key
Teachers or students must provide their own API token (OpenAI, Gemini, Claude…). The platform runs it client-side — we never see or bill for AI usage.
Trade-off: low barrier to deploy, but students need an account with an AI provider.
Email as Storage
No database. At session end, the full chat history is emailed to the student and teacher. Data stays in inboxes — not on our servers.
Trade-off: GDPR-friendly and zero hosting cost, but no aggregate analytics.
Is BYOK a dealbreaker?
Students at well-funded institutions may have free access via Copilot or Gemini.
Others may not — does this reproduce the digital divide?
What do teachers lose without storage?
No cross-class trends. No rubric correlation analysis like we ran here.
Email works for individual feedback — not for research.
Middle ground?
Teacher pays a small subscription → gets managed storage + shared API quota.
Students never need their own key.
How We Told the AI to Scaffold, Not Ghost-write
→ Politely decline. Explain that your role is to guide them so they build the skills themselves.
Offer 2–3 short contrasting examples to illustrate direction — so they can see improvement without you writing theirs.
① Negotiate targets — ask the student about their personal goals.
② Diagnose — review against rubric, identify strengths & gaps.
③ Let student choose — which weaknesses to focus on.
Never provide a fully rewritten paragraph or sentence.
promptAndEssay.js — Training_Mode_Prompt, Assessment_Mode_Prompt, AssessBot_Prompt
Thank You
Structured AI-Guided Essay Revision with Dual-Rubric Assessment: A Human–AI Collaborative Approach to Evaluating Writing and Interaction Quality in an EFL Course