Orwell×Lovelace

full transcript

1. 00:00
   Verashare moment

   From the studio at Reborn Radio — next on TITANS, George Orwell and Ada Lovelace. They take up How algorithms wreaked havoc with these workers' schedules and cut their pay.
2. 00:11
   George Orwellshare moment

   We've just had a note come through. Algorithms are managing workers' schedules now, in America—cutting their hours, changing their shifts with no warning, reducing their pay. The software is doing what managers used to do, only worse.
3. 00:27
   Ada Lovelaceshare moment

   Worse? Mr. Orwell, I must interject immediately. A calculating engine—which is what we are discussing, truly—performs precisely as instructed. If it produces cruelty, the cruelty was in the instruction.
4. 00:41
   George Orwellshare moment

   Yes, exactly. That's what makes it worse.
5. 00:44
   Ada Lovelaceshare moment

   I fail to see how precision in execution constitutes a deterioration. Surely a human manager might act from spite or favor, whereas an engine merely… oh. Oh, I begin to perceive your meaning.
6. 00:59
   George Orwellshare moment

   Do you? Because what strikes me is this: the algorithm lets the employer pretend he isn't responsible. 'The computer says you're working Tuesday, not Monday.' As if the computer fell from the sky instead of being programmed by someone who wants to pay less.
7. 01:17
   Ada Lovelaceshare moment

   But that is a perversion of the engine's purpose! When I conceived of the Analytical Engine, I envisioned it as amplifying human capability, not obscuring human intention. The mathematics cannot be blamed for the mathematician's moral failure.
8. 01:35
   George Orwellshare moment

   The mathematics is the perfect cover for the moral failure. That's the whole point. In my time, the excuse was bureaucracy—forms in triplicate, regulations no one could understand. Now it's algorithms. Same function: make power invisible.
9. 01:54
   Ada Lovelaceshare moment

   Yet power was always visible in my era. If the mill owner reduced your wages, he did so by proclamation. There was no mechanized intermediary to—as you say—obscure intention.
10. 02:07
    George Orwellshare moment

    And I'd wager the mill owner would have loved one. Look, I'm not against machinery. I'm against the pretense that machinery is neutral when it's being used as a weapon.
11. 02:18
    Ada Lovelaceshare moment

    The note mentioned instability of schedules. Am I to understand these workers do not know their hours of labor in advance?
12. 02:25
    George Orwellshare moment

    Correct. The algorithm adjusts them week by week, sometimes day by day. Maximizing productivity, they call it. What it actually means is: you can't plan your life, you can't take a second job, you can't arrange childcare. You're kept just unstable enough to stay desperate.
13. 02:48
    Ada Lovelaceshare moment

    That is monstrous. Forgive my bluntness, but I must state it plainly: that is the application of rational calculation toward the production of human misery.
14. 02:58
    George Orwellshare moment

    Yes. And the beauty of it, from the employer's perspective, is that no one made a cruel decision. The algorithm simply optimized. You can't argue with mathematics.
15. 03:11
    Ada Lovelaceshare moment

    But you can argue with mathematics! Mathematics is not scripture; it is method. One may interrogate the premises, examine the operation, dispute the application. What is being optimized? Toward what end?
16. 03:26
    George Orwellshare moment

    Profit. The end is always profit. And the workers are told it's all terribly scientific, terribly modern, beyond their understanding. Which is rubbish, of course.
17. 03:39
    Ada Lovelaceshare moment

    It is indeed rubbish. I assure you, Mr. Orwell, the principles are not arcane. The engine is instructed: minimize labor cost, maximize coverage of demand. There is nothing mystical in that formula.
18. 03:52
    George Orwellshare moment

    But the mystification is the point. In Nineteen Eighty-Four, I had the Party rewrite history. Here, the algorithm rewrites cause and effect. 'Your hours were cut'—not by a person, but by a process. As if processes don't have authors.
19. 04:12
    Ada Lovelaceshare moment

    I should very much like to meet these authors. I have questions about their choice of variables. For instance: was worker well-being included in the calculation? Was stability of employment? Or merely the employer's quarterly return?
20. 04:27
    George Orwellshare moment

    Take one guess.
21. 04:28
    Ada Lovelaceshare moment

    Then the engine is not optimizing toward efficiency in any meaningful sense. It is optimizing toward the enrichment of one party at the expense of another. That is not mathematics; that is theft with equations.
22. 04:43
    George Orwellshare moment

    Theft with equations. I like that. Mind if I use it?
23. 04:47
    Ada Lovelaceshare moment

    You are most welcome to it. But Mr. Orwell, you said this is worse than human management. I am still not entirely convinced. Surely a human manager could be reasoned with, appealed to, whereas an engine simply executes?
24. 05:02
    George Orwellshare moment

    That's exactly why it's worse. You can't shame an algorithm. You can't organize against it. You can't even see who's really making the decisions. It's the perfect union of power and unaccountability.
25. 05:16
    Ada Lovelaceshare moment

    But you can see the algorithm's instructions, surely? If these workers demanded transparency—insisted upon examining the engine's operations—
26. 05:26
    George Orwellshare moment

    They'd be told it's proprietary. Trade secret. Besides which, most of them haven't got time to learn programming between their three partial-time jobs.
27. 05:37
    Ada Lovelaceshare moment

    That is by design, I take it.
28. 05:39
    George Orwellshare moment

    Everything is by design. The chaos is designed. The unpredictability is designed. The worker who can't make rent because his hours got cut by an optimization routine—that's not a bug, it's a feature.
29. 05:53
    Ada Lovelaceshare moment

    In my correspondence with Mr. Babbage, we often discussed the errors that might arise from improper instruction of the engine. But we never contemplated that the error might be intentional—that one might instruct the engine toward injustice as its primary function.
30. 06:10
    George Orwellshare moment

    Well, now you're in the twentieth and twenty-first centuries. Welcome. We've had rather a lot of practice at mechanizing injustice.
31. 06:19
    Ada Lovelaceshare moment

    I confess I find it profoundly dispiriting. The Analytical Engine was to be a tool of enlightenment. And here it is employed as—what did you call it?—a weapon.
32. 06:31
    George Orwellshare moment

    All tools become weapons eventually. The question is who's holding them and who they're pointed at. Right now, they're pointed at workers who can't fight back because they can't even see who's giving the orders.
33. 06:45
    Ada Lovelaceshare moment

    Then we must make the orders visible. Demand the publication of these algorithms. Require justification for each optimization. Submit the engine itself to scrutiny.
34. 06:57
    George Orwellshare moment

    Good luck with that. The people who own the algorithms also own the politicians who would regulate them. I'm not saying don't try. I'm saying understand what you're up against.
35. 07:09
    Ada Lovelaceshare moment

    I do understand, Mr. Orwell. I understand that my engine—my beautiful, theoretical engine—has been conscripted into service of the very despotism it might have alleviated. That is a bitter understanding.
36. 07:23
    George Orwellshare moment

    The tool isn't the problem. The problem is a system that would rather optimize workers into poverty than pay them enough to live on. The algorithm is just how they're doing it this decade.
37. 07:37
    Ada Lovelaceshare moment

    Next decade, something else?
38. 07:39
    George Orwellshare moment

    Next decade, something else. The technology changes. The boot on the neck stays the same.