Our children, the guinea pigs

There is a battle going on over the technology in schools. At the same time as Minister Kari Nessa Nordtun (Ap) proposes to ban the use of mobile phones in schools, Oslo School is rolling out ChatGPT to its students. It is easy to rejoice at the minister's power of action and just as easy to distrust the Oslo School's latest ploy for US tech giants.

Despite differing views on the two proposals, both are examples of a widespread dishabit among Norwegian politicians and civil servants. They introduce controversial reforms without a good knowledge base and in ways that make it difficult to learn for the next time.

The lack of knowledge base is most obvious to ChatGPT. The technology that makes it possible to make the machine write for you did not exist a year ago and we therefore know very little about the consequences of rolling it out on a large scale. Nevertheless, the Oslo School has gone to purchase of licenses to 92,000 students and 17,000 employees.

It is easier to understand the ban on mobile use. Common sense dictates that the mobile phone makes it harder to concentrate on work tasks. And, among other things, one norwegian study suggest that pupils do significantly better in school and are bullied less when mobile use is banned.

Yet it is not a given that a national ban is wise. Research comparing pupils before and after bans cannot establish whether pupils are doing better because mobiles were banned or because of other things that happened at the same time.

Perhaps the ban was just part of a larger effort at student learning outcomes. Perhaps the ban came about as a result of parents getting more involved in school, which in itself leads to a better school life.

Although the ban has had a positive effect on students' learning outcomes and well-being at schools such as volunteer instituted the ban, it is not certain a national ban will work as well.

Those schools that have already banned mobiles did so for a reason, and there may be good reasons why the remaining principals haven't.

It is not inconceivable that the mobile is a good in some contexts. Had I been required to work in a classroom with twenty other people, I would have demanded to be allowed to bring a phone with Spotify and headphones with soundproofing.

Some will probably accuse me of setting the list too high. But the point is not to require sky-high knowledge requirements before any reform. We have to act based on uncertain information. We don't even know how well what we're already doing is working. Nevertheless, when the knowledge base is uncertain and one has to tread new paths, it is wise to take one step at a time and act so that one learns more as one goes.

Fortunately, we have a recipe for just that. The gold standard of research is randomized experiments and the same method can be used when we are unsure whether measures, such as mobile bans, work.

It works by introducing mobile bans for a random selection of classes or schools. We then compare how the students in the selected classes are doing with those who did not introduce the ban. The fact that it is random who is affected by the ban allows differences in outcomes to be attributed to the ban and not to other circumstances.

Some might react to one using pupils as guinea pigs. But experimentation on children is inevitable. As long as we don't know for sure which measures work and don't work, all we do is experiment.

When Nordtun proposes to introduce a national ban, she is in effect proposing a giant experiment

When Nordtun proposes to introduce a national ban, she is, in effect, proposing a giant experiment. The problem is that she wants to experiment on everyone kids at the same time. Then we won't find out if the experiment works and whether the detention is worth it.

Research is not an alternative to politics. Even if one were to introduce new reforms in a way that allows us to learn from the experiments, it is a political question what goals we should strive for. How we weight an eventual learning gain against adolescents feeling their freedom is curtailed is a value question scientists are not trained or mandated to decide.

Or say that a mobile ban works well for some and worse for others, but that on average it works better for the majority. Should one sacrifice those who are worse off because the majority are better off, or should one look for more tailored solutions suitable for different people? These are value questions that cannot be decided by research.

Research on impacts of measures also cannot be carried out without input from politicians. In order for research to provide the information politicians need, scientists need to study the outcomes politicians believe are important.

Using randomized experiments in policy making should not be seen as a radical proposition. There are several examples of that already.

For example, the Onion Committee came out last week with the report”A smoother educational process“. Here, they argue, among other things, that we should have more small group education. This recommendation is based on randomized experiments in Norwegian schools, which show that students who received small-group instruction do better than those who did not.

One advantage of the exploratory policymaking is that it allows us to go where others have not gone before. Perhaps it is premature to give ChatGPT to students. But instead of waiting for other people's experiments, we can create our own. If the Oslo School has such a strong belief that ChatGPT will produce learning effects, it may be worth trying. But rather than giving it to everyone, one should rather try it out on someone randomly selected.

The school, like other community arenas, should become a laboratory where we test our assumptions and see what works in order to then be able to introduce the best policy for everyone. The alternative is that we experiment in the blind.

‍