Moderation Without Borders

Peer moderation for every educator, everywhere

A white paper by Moderate Technologies Limited

Executive summary

Every grade is an evidence-based, but personal and subjective judgement, and every such judgement benefits from a second opinion. Moderation, the practice of double-checking another educator’s work to make sure it has been graded correctly, is how education protects fairness, objectiveness and empiric impartiality. Yet it is still usually done face to face, at fixed times, with colleagues from the same region.

Moderate replaces that model with one simple online space where educators from kindergarten to post-doctoral level receive friendly, anonymised, peer feedback from subject specialists around the world.

Why moderation matters

Doubts about the consistency of grading are more than a century old. Brookhart et al. (2016), reviewing a century of research, report that early studies generally judged teachers’ grades unreliable, attributing the variation to task difficulty, grading criteria, teacher strictness or leniency, and teacher error. Even specialists diverge: when teachers trained in assessing writing graded the same essay, scores ranged from 50 to 93 out of 100 (Brimi, 2011). The encouraging news, reported in the same review, is that clear rubrics and proper training can produce impressive inter-rater reliability. Moderation is where that consistency is built.

Fairness has a second dimension. A meta-analysis of 20 experimental studies involving 1,935 graders found a significant bias effect (g = 0.36) when graders knew irrelevant facts about a student (Malouff & Thorsteinsson, 2016). Anonymised work removes those cues.

Moderation also does more than catch errors. Bloxham, Hughes and Adie (2016) identify four purposes: equity, justification, accountability and community building. Sadler (2013) goes further, arguing that the ultimate aim is “calibrated” academics, whose shared sense of standards grows with every moderation conversation.

The hidden costs of traditional moderation

Face-to-face moderation needs a room, a date and a stack of scripts. Each carries a price.

Time. Full-time teachers in England already report working 52.4 hours a week on average (Department for Education, 2024). A moderation meeting adds travel, coordination and waiting to the discussion itself, and because moderation is repeated for every task (Sadler, 2013), so is the friction.

Money. Venues, travel, refreshments and cover for released staff are real line items, whereas online professional learning is typically less expensive than in-person alternatives (Slagoski, 2019). Moderate removes the room, the journey and the cover.

Opportunity. Hours spent finding a shared slot are hours not spent planning, giving feedback to students, researching or resting. Educators who teach a subject alone, or work in a small or remote institution, may have no local peer to moderate with at all, so the opportunity is lost entirely.

The echo-chamber problem

Because in-person moderation ties educators to those nearby, moderators often share the same training, syllabus and habits, and shared blind spots can pass for consensus. Standards live largely in tacit understandings that are hard to transfer between people (O’Donovan, Sadler & Reimann, 2024), which suggests the remedy is a wider conversation, not a closer one. Calibration in UK higher education already draws on academics from across institutions (Advance HE, n.d.). Online networks, meanwhile, let teachers learn with distant peers (Macià & García, 2017) and are recognised as an important source of professional development (Macià & García, 2016; Trust, Krutka & Carpenter, 2016). Moderate brings that reach to moderation itself: every educator can receive valuable feedback from educators of the same discipline from all over the world.

How Moderate works

Moderate has no convoluted process. Whether the work is a primary-school story, a high-school essay, a master’s dissertation or a post-doctoral manuscript, the steps are the same.

1. Upload. An educator uploads anonymised student material.

2. Comment in the document. Peers from the same discipline leave comments exactly where a question or concern arises.

3. Discuss in threads. Each comment can grow into a thread where uploader and reviewer explain their reasoning and learn from each other.

4. Talk it through. Moderate’s private instant messenger lets two educators take a discussion further, directly and privately.

5. Grade. The process culminates in the reviewer grading the anonymised material, giving the uploader a clear second judgement.

There are no meetings to schedule, no time zones to reconcile and no forms to complete; educators contribute whenever it suits them. Feedback is friendly and peer to peer, and it flows both ways: those who review also sharpen their own judgement. Feedback is among the most powerful influences on learning, though its impact depends on how it is delivered (Hattie & Timperley, 2007), and a constructive, collegial setting gives it the best chance of landing well.

Moderation compared

Moderate removes the main constraints of face-to-face moderation. Where traditional moderation needs a shared date, time and place, Moderate needs no scheduling. Where it incurs travel, venue and cover costs, Moderate has no room, journey or cover to pay for. Where reviewers are local colleagues, Moderate opens the door to educators of the same discipline worldwide, so reach is global rather than regional. And where the process means meetings, agendas and travel, Moderate asks only that educators upload, comment, discuss and grade.

Conclusion

Moderation works best when it is anonymous, frequent and open to many voices. Traditional methods make it slow, costly and local. Moderate makes it simple, light on time and cost, and global, so that an educator anywhere in the world, at any level from kindergarten to post-doctoral research, can have their grading checked by peers who care about getting it right.

References

Advance HE. (n.d.). Who is involved in calibration? https://advance-he.ac.uk/how-calibration-carried-out

Bloxham, S., Hughes, C., & Adie, L. (2016). What’s the point of moderation? A discussion of the purposes achieved through contemporary moderation practices. Assessment & Evaluation in Higher Education, 41(4), 638–653. https://doi.org/10.1080/02602938.2015.1039932

Brimi, H. M. (2011). Reliability of grading high school work in English. Practical Assessment, Research & Evaluation, 16(17).

Brookhart, S. M., Guskey, T. R., Bowers, A. J., McMillan, J. H., Smith, J. K., Smith, L. F., Stevens, M. T., & Welsh, M. E. (2016). A century of grading research: Meaning and value in the most common educational measure. Review of Educational Research, 86(4), 803–848. https://doi.org/10.3102/0034654316672069

Department for Education. (2024). Working lives of teachers and leaders: Wave 2 summary report. GOV.UK. https://www.gov.uk/government/publications/working-lives-of-teachers-and-leaders-wave-2/working-lives-of-teachers-and-leaders-wave-2-summary-report

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487

Macià, M., & García, I. (2016). Informal online communities and networks as a source of teacher professional development: A review. Teaching and Teacher Education, 55, 291–307. https://doi.org/10.1016/j.tate.2016.01.021

Macià, M., & García, I. (2017). Properties of teacher networks in Twitter: Are they related to community-based peer production? International Review of Research in Open and Distributed Learning, 18(1).

Malouff, J. M., & Thorsteinsson, E. B. (2016). Bias in grading: A meta-analysis of experimental research findings. Australian Journal of Education, 60(3), 245–256. https://doi.org/10.1177/0004944116664618

O’Donovan, B., Sadler, I., & Reimann, N. (2024). Social moderation and calibration versus codification: A way forward for academic standards in higher education? Studies in Higher Education, 49(12), 2693–2706. https://doi.org/10.1080/03075079.2024.2321504

Sadler, D. R. (2013). Assuring academic achievement standards: From moderation to calibration. Assessment in Education: Principles, Policy & Practice, 20(1), 5–19. https://doi.org/10.1080/0969594X.2012.714742

Slagoski, J. D. (2019). Professional learning practices and beliefs of an online community of English language teachers. Contemporary Issues in Technology and Teacher Education, 19(1).

Trust, T., Krutka, D. G., & Carpenter, J. P. (2016). “Together we are better”: Professional learning networks for teachers. Computers & Education, 102, 15–34.