Grants and prizes
The review cycle, in numbers#
| Figure | What it is |
|---|---|
| ICC 0.26 | Measured agreement between independent reviewers across 23,414 ratings at a national science fund (PLOS One) |
| 2× | Application volume growth at a major national funder since 2017, while award rates fell from 36% to 19% (UKRI, via LSE Impact Blog) |
| 18+ months | End-to-end decision cycles at major schemes |
| Midnight | When your confidentiality policy meets your most tired reviewer, with a free chatbot one tab away |
The consistency problem here is documented, not hypothetical.
Read this first: the AI policy question#
Two different rules, and they have different answers.
NIH NOT-OD-23-149 bars reviewers from uploading confidential proposals to consumer AI tools. So does EvalLens: that rule exists because individual reviewers paste applications into public chatbots, unlogged and retained by whoever runs the tool. An organizer-governed perimeter is a different object, and the distinction is one to walk your counsel through rather than assume.
The ERC guidance of March 2026 goes further: reviewers may not delegate the assessment of scientific merit to AI at all. That is a boundary no perimeter fixes, and it is not argued with. In programs under rules like these, EvalLens runs the administrative half of the round only — intake, completeness, eligibility, comparability checks and the record — while merit stays entirely with your reviewers.
Prize programs, foundation calls and competitions that set their own rules can use the scored first read as well.
If you are not sure which side your call sits on, that is the first question to settle — before any document moves.
The appeal file#
Three years later, someone asks why an application scored 4.2. Today the answer lives in scanned scoresheets and a departed reviewer's inbox.
With a record, staff opens that application in one click: the AI read and the panel score preserved side by side, each finding tied to the quote and page it came from, and the human sign-off attached.
A worked example on Implementation readiness: score 7.8, anchor band 7–8 — finding three funded pilot sites; two report continuation funding, quote "…three pilot deployments; two districts renewed…" · page 14, evidence strength strong, panel action confirmed by panel, open question logged for the applicant interview.
Field-level change history and rubric versioning are on the roadmap, and where that line sits today is stated before purchase rather than after.
The seven steps#
1 · Your rubric, locked before the call opens. Criteria, anchor descriptions, weights and eligibility rules configured in one working session, then applied identically to every application that will ever arrive. Procedural fairness by construction — and a documented methodology you can publish.
2 · Applications flow from your existing intake. Submittable, SurveyMonkey Apply, Fluxx, SmartSimple, OpenWater or your own forms. Applicants change nothing.
3 · The administrative screen runs itself. Completeness and eligibility checked against your rules, gaps flagged info / warning / critical. Staff handles exceptions, not the pile.
4 · Independent reviewer roles, named honestly. Not people: independent AI reviewer roles, each reading the full proposal through its own lens, composed per program including domain-matched technical reads. Each scores blind to the others, evidence before score, and every model's read is logged — so "why is this reviewer qualified" has an answer too. Your human panel's conflict-of-interest and recusal workflow stays exactly where it is.
5 · Panels read briefs, not piles. Every proposal arrives pre-read with comparable scores, laid-out evidence and ranked open questions. Judges read briefs rather than the full stack of PDFs, so expertise goes to judgment on the borderline.
6 · The committee decides. Finalists and awards are built from your panel's scores; the AI reads stay advisory. The selection memo is generated from the live review record rather than reconstructed for the board.
7 · The record survives the round. Independent reads where reviewers never see each other's scores, disagreement surfaced to the panel rather than averaged away, and bias made inspectable through score distributions by geography and organization size on request.
Applicant-facing commitments#
- Never trained on. Applicant documents are processed only for your evaluation. Contractual.
- Disclosed to applicants. Template disclosure language is provided for your call documents, so applicants know exactly how AI assists the review.
- Governed, not improvised. A governed alternative for the first read beats not knowing what your reviewer pool does at midnight.
Next steps#
- What EvalLens does not do — the boundaries to quote in a policy conversation.
- Security and privacy — the perimeter argument, in detail.
- Criteria and weights — anchors, and why they move agreement.