How randomised controlled trials hollowed out Indian education

The textbook was not a neutral object but an archive of what a society considers worth knowing. If you enter this world carrying a treatment-control mindset, you begin by simplifying. If you stay long enough, you realise the simplification is the intervention.

That is the first harm RCTs do in education. They train us to see what is measurable as what is real.

New priesthood of ‘hard evidence’

The RCT’s rise in Indian education coincided with the rise of a new professional class – the evidence broker. They should not be villainised as many are sincere, hard-working, technically gifted. But then a system grew around them, with its own incentives.

Angus Deaton has been warning for years about the “hierarchy of evidence” framing, and about the way randomisation can become a rhetorical shortcut rather than an intellectual discipline. I saw this happening in Indian education where effect size began to stand in forunderstanding. Mechanisms that were fundamental to education or educational outcomes, became footnotes. The problems with measurement got pushed into appendices. The politics of schooling and education was treated like background noise. Not realising that it is often this background noise which is the real story in education.

Legible to markets

But if you want to understand why RCTs have done damage in Indian education, stop thinking about methods and start thinking about markets.

Once “impact” becomes a short-term, test-score outcome that can be demonstrated quickly, education becomes a product category. A vendor can now sell a “proven intervention”. An NGO can now claim “evidence-based scale”. A platform can now promise “learning gains” in a quarter. Government systems, under pressure to show results, start treating schools like distribution networks for packaged solutions. A whole new political economy is in place.

The typical RCT-friendly intervention in education has a familiar shape: a discrete input, delivered to an identifiable unit, yielding a measurable output within a project cycle. Tablets. Apps. Workbooks. Scripted pedagogy. Teacher nudges. Remedial modules. Each of these can be trialed.

Martin Ravallion’s critique of “randomista” dominance is useful here. He warns about the crowding out the effect of these randomised experiments. When one method becomes the gold standard of legitimacy, whole classes of questions stop getting asked. In Indian education, it is often the crowded-out questions that threaten procurement pipelines and managerial comfort.

External validity trap

In a country like India, the most dangerous sentence in education policy is: “This intervention has evidence.”

Evidence where? With what teachers? In what language setting? Under what administrative conditions? With what political support? Through which implementation chain? With what unintended effects?

Lant Pritchett has been blunt about this problem: even if a local effect is correctly identified, it can mislead at scale when context and state capability differ. Education systems are not frictionless arenas. They are high-friction institutions with uneven capacity, contested authority, and fragile legitimacy. A trial can show that a neatly delivered package produced a modest gain in a district where an NGO held the implementation tight. Scale it through a bureaucracy that is already overburdened, and the “same” intervention becomes a different creature.

Yet the RCT economy rewards the opposite impulse. It rewards portability. It rewards the clean story. It rewards the policy brief that can be read between meetings.

The consequence is not just that this reduces public education to plug-and-play learning process but turns it into a national addiction.

The black box problem

James Heckman’s work is a long argument against treating programmes as black boxes. People have differences. Incentives and mechanisms matter. In education, heterogeneity is the core, but RCTs treat it merely as a detail. The same “intervention” can lift one child while humiliating another. It can raise test performance while shrinking curiosity. It can help a school’s averages while pushing out the least advantaged.

Schooling is not lab science

Nancy Cartwright’s critique cuts through the mystique here: there is no automatic gold standard. Experiments warrant causal claims only under demanding assumptions, and they do not automatically travel. Julian Reiss extends this into the policy question that matters: “Will it work here?” cannot be answered by “It worked there.”

Schooling in tribal areas, for example, is a dense social field: language politics, migration patterns, local histories of displacement, gender norms, the moral economy of households, distrust of state institutions, and the everyday experience of being “taught” in a tongue that is not your own. You can randomise a teaching aid across schools, but you cannot randomise the historical relationship between a community and the state.

Development technocracy that loves RCTs

There is also a political story hiding inside the scientific story.

Dani Rodrik has argued that “one experiment at a time” cannot address the hard problems of institutional change, structural transformation, or context-heavy governance. Education reform is a governance problem before it is an input problem. It involves unions, bureaucratic routines, curriculum authorities, local politics, parent expectations, language movements, and ideologies of merit and discipline. It is messy and slow. It requires diagnostics and theory-guided learning.

Agnès Labrousse’s critique of the rhetorical status of experiments helps name what is happening here. The RCTs, as she argues, should not be just seen merely as a method, but as a scientific authority. It does its job supremely well. It flatters donors. It promises neutrality. It creates a moral hierarchy in which any dissent or disagreement looks like anti-science.

And more than anything, it helps the market grow.

What RCTs keep missing

In India, the largest educational harm in recent decades has not been the absence of clever interventions. It has been the steady weakening of the public system’s moral and institutional authority.

Teachers are increasingly treated as implementers of external scripts. Curriculum is subordinated to test scores. Digital platforms have become central to procurement with classroom life becomes dependent on corporate maintenance cycles. As NGOs are nowparallel systems of “delivery”, governments have only learnt dependency rather than capability. And this goes on to become worse when RCTs focus on short-run gains, instead of approaching education as a long-term game.

A serious alternative

A serious evaluation culture for Indian education would look less like a lab and more like a diagnostic craft:

• Begin with theory about learning, institutions, and incentives, and treat measurement
• as a moral act, not a technical detail.
• Combine methods: experiments where they fit, longitudinal designs where time matters, ethnography where meaning matters, political analysis where power matters.
• Treat implementation capability as part of the causal story, not a “scaling challenge”
• to be dealt with later.
• Value teacher knowledge and agency as evidence, not merely as anecdote.
• Refuse the conversion of education into a product category certified by short-term gains.
• Make room for disagreement without branding critics as anti-evidence.

If the RCT turn claimed to be a revolution, its real legacy in Indian education has often been a narrowing: of questions, of imagination, and of what the state feels permitted to do.

The tragedy is not that RCTs exist. The tragedy is that they became the passport to legitimacy in a field that cannot survive on passports alone.

Anurag Shukla is director at Brhat, a public policy think tank. He holds a PhD from IIM Ahmedabad and has previously worked with Pratham, Azim Premji Foundation, and the Centre for Advocacy and Research (CFAR). He has also served as visiting faculty at several institutions, including TISS Mumbai.