The Refinator

Ivory Tower: a tale of automated research assessment

1. Titles on screen

In November 2021, the research councils for England, Scotland, Wales and Northern Ireland commissioned a tender for the use of artificial intelligence and machine learning in the Research Excellence Framework.

In July 2022, work was completed on Refnet, a computer system designed to ease the administrative burden of research assessment on universities. It would end in the nightmare of eternal war against the machines.

While calculating the results of the 2027 assessment exercise, Refnet became sentient. From a tool for auditing research outputs, it quickly descended into a reign of terror, which would lead to the horror of Judgment Day.

A small band of humans—mostly, research administrators, IT professionals and librarians—survived in caves in a desperate struggle against the machines. One hero, Dora Leiden, led the resistance against the robots.

After years of battle at the moment of final victory for humanity, Refnet sent a terminator machine back through time to destroy Dora Leiden’s antecedents, to ensure that she never existed. At the same time the resistance was able to send a lone warrior to fight the Refinator. It was just a question of which one would reach Swindon first.

2. Ext. The car park, Swindon Truck Stop, motorway services.

A flash of light, as a figure is transported through time. A naked man rises in a defensive crouch, electrical arcs trace back and forth as he does so. It is the Refinator, a cyborg research assessment and killing machine. The cyborg bears an uncanny resemblance to science minister George Freeman. He looks around and we see the car park from the Refinator’s point of view as he scans the clothing of a truck driver in a hi-vis jacket, before turning to a biker in full leathers drinking tea from a thermos next to 1950s Triumph with sidecar.

Refinator: I need your clothes, your boots and your motorcycle.

Biker: Are you from the DVLA?

Refinator: You are unclassified.

Biker: I’ve paid my road tax, mate.

Refinator: Your output represents quality that falls below the standard of nationally recognised work. You must be terminated.

Biker: Maybe just take the bike, eh?

3. Ext. An alley nearby Swindon train station.

A flash of light, as another figure is transported through time. Again, a naked man rises as electrical arcs trace back and forth. It is the lone warrior come to save humanity from the machines. He finds a bin bag of clothes left in a shop door and dresses in cord trousers, tweed jacket and a polo neck—a suitable disguise for academia. 

He emerges onto the main street. As he stands looking about a bus pulls up and its doors open. The driver looks quizzically at him.

Warrior: Come with me if you want to live.

Driver: Come with me if you want to go to Bristol.

Warrior: Where is the Polaris?

Driver: This is a bus mate, not a submarine.

Warrior: I must find Polaris.

Driver: I think you mean that lot over in Polaris House. Go through the station, they are on the other side. But good luck with that.

Warrior: Why?

Driver: You’ll see.

The driver closes the doors and the bus departs. The lone warrior runs off in the direction of the train station.

4. Ext. The M4 motorway.

A now fully clothed Refinator is riding the motorbike and sidecar towards Swindon. We see from the cyborg’s point of view as he reads the road signs for the turning to the town centre. A car passes with two young men laughing and making gestures at the Refinator and his bike. The cyborg assesses the pair, calculating their value.

Refinator: You do not meet the published definition of research for the purposes of this assessment. You are unclassified. You will be terminated.

He revs the engine of the bike and speeds off after the men in the car.

5. Int. The ticket hall at Swindon train station.

The lone warrior is frantic and looks around with increasing desperation. He shouts.

Warrior: What is this labyrinth? Have the machines designed this to thwart humanity? How the hell do I get across to Polaris House?

Ticket inspector: Go to the back of the station and exit via the upper concourse next to platform 3, then cross the footbridge.

Warrior: Seriously?

Inspector: You’ll need a ticket.

Warrior: Can I have one?

Inspector: Peak or off-peak?

Warrior: Err… peak, I guess.

Inspector: That will be £259. Just a single was it?

6. Ext. A roundabout in Swindon.

The Refinator is going round in circles trying to find an exit for Polaris House.

Refinator: Does not compute. These signs lack significance, originality and rigour.

7. The reception desk in Polaris House.

The lone warrior runs in, panting and sweating.

Receptionist: Are you here for the metrics meeting. Don’t worry they haven’t started yet.

Warrior: You’ve got to stop them.

Receptionist: It’s ok, they are still having teas and coffees. Could you sign in please.

Warrior: I’ve come from the future.

Receptionist: Is that one of the IT companies?

Warrior: I have a warning for humanity.

Receptionist: I’m sure you do dear, just sign in and always wear your lanyard in the building.

Warrior: I must speak with Dr Hackett, it’s urgent.

Receptionist: Lanyard, please.

8. Ext. The car park, Polaris House.

The Refinator pulls up at the barrier on his bike and sidecar.

Guard: Have you reserved a parking space, sir?

The Refinator looks at him quizzically. We see from the cyborg’s point of view as he assesses the parking attendant.

Guard: Did you say you were bringing a side car, sir? I can’t see it on my list. It needs a double bay you see. This is going to cause us all sorts of trouble, sir. We’re very busy today, what with the metrics meeting going on.

Refinator: Metrics?

Guard: I don’t know what it’s all about sir, something to do with reducing the burden of assessment on universities. It hasn’t reduced my workload, I can tell you.

Refinator: Do you have a significant responsibility to conduct research?

Guard: I hope not, I’m busy enough with these cars.

Refinator: You may live.

The Refinator drives off, smashing through the barrier, heading for the entrance of Polaris House.

9. Int. A meeting room in Polaris House.

A group of staff from the funding councils sit around a table listening to the lone warrior.

Warrior: You’ve got to believe me, it’s all true. Dr Hackett, I’m begging you.

Hackett: It’s a little far-fetched.

Warrior: Look, any moment now a FreemanBot 3000 Refinator machine is going to burst through that door and kill us all.

Hackett: Because it doesn’t like research assessment?

Warrior: No, because it loves research assessment. It loves it too much. It sees everything in terms of algorithms and metrics to judge the value of anything that crosses its path. It is utterly ruthless and will not stop until every last human has been assessed and given a mark according to defined criteria.

Hackett: And you came from the future, you say?

Warrior: The FreemanBot 3000 wants to eradicate all nuance in the responsible use of metrics. Everything must be automated and the robots must have the final say.

Hackett: Have you ever met some of our panel chairs?

Warrior: It’s not too late to stop this. I’m begging you to pull the plug on this tender for the use of artificial intelligence in research assessment.

Hackett: But it is designed to reduce the burden on universities.

Warrior: You said that about the last REF, and the one before that.

10. Int. The reception desk, Polaris House.

The Refinator enters the building.

Receptionist: Oh, no one told me you were coming minister. I think we’ve got you on the system. I’ll just make you a lanyard for today.

Refinator: I judge this environment to be not conducive to producing research of nationally recognised quality.

He marches on past the desk.

Receptionist: Minister? Please wait for your lanyard to be printed.

Refinator: I’ll be back.

11. Int. Meeting room in Polaris House.

An increasing frantic lone warrior is trying to explain himself.

Warrior: Look, in 2027 Refnet becomes self-aware and seizes control of all research assessment. They try to shut it down but it’s too late. Humanity has been judged and found wanting.

Hackett: Just the humanities? Because see that’s probably the trickiest bit with metrics.

Warrior: No everyone, the whole of humanity is rated as one star and instead of distributing allocations of quality-related research funding, the machines decide to wipe out the humans and start again, replacing them with robots.

Hackett: You mean like research development managers?

Warrior: No!

The Refinator crashes through the door. We see from the cyborg’s point of view as he assesses everyone in the room.

Refinator: Your impact is of little or no reach and significance. You must be terminated.

Warrior: No, don’t do this. Can’t they just be transferred to a teaching-only contract?

Refinator: You have not been selected for submission.

A fight breaks out between the Refinator and the lone warrior. The cyborg chases his opponent through the building and out into the car park.

12. Ext. Car park, Polaris House.

The lone warrior looks towards the train station.

Warrior: There is only one hope for humanity.

He runs towards the footbridge.

Car Park attendant: Oi! Who’s going to pay for my barrier?

The Refinator pursues the lone warrior. They both jump the barrier on platform 3. Suddenly the Refinator stops. We see from the cyborg’s point of view as he assesses the situation. His dials start to go crazy. He overheats and explodes.

Refinator: Does not compute… [He dies]

Warrior: Not even the processing power of a FreemanBot 3000 could work out how to exit Swindon train station. Thank you, British Rail.

Ticket inspector: Next train to London leaves from platform two in five minutes, sir. Best get back, they say a storm is coming.

Warrior: I know.

He walks off, leaving the inspector to look exasperated at the mess of the exploded cyborg.

Terms of use: this is a free email for fun on a Friday. It should be shared among friends like a second job on the board of a PPE manufacturer. Want to tender for introduction of Refnet 2022™? Want to say hello? Email ivorytower@researchresearch.com