S08E04: finally, someone wants to match things

This will be a reasonably technical post about the fulfilment of my life’s ambition: people wanting to match things.

For as long as I’ve had a professional career I have been banging the same extremely boring drum: matching things is hard, humans are bad at it, and anything larger than a 10 x 10 grid should be handled by a machine. I’ve written software and pitched the same idea literally three or four times. I built a website showcasing how difficult it is (the pitch if you want to know the why, the challenge if you want to see if you can beat the computer)

And now, at long last, someone has picked it up. Enter John:

John runs a mentoring programme for LGBTQ+ people in the Civil Service. It’s fantastically popular: around 400 mentors are matched to more than 500 mentees. It’s also a huge job for anyone, especially someone as busy as John. The number of potential matches – putting aside that some of them are totally impossible – is more than 200,000. Imagine a grid that big! It’s like the biggest, most impossible sudoku in the world.

It’s a lot like that, in fact, because matching people is incredibly difficult. It’s incredibly easy to prove you’ve got it wrong, a feeling you’ll remember from leaning over a fellow commuter’s morning newspaper and realising they’ve put a 4 twice in the same column, poor sod, with their tongue sticking out as their pen speeds across the page. Oh, no. They’ve realised. Shame.

But getting it right is bloody hard. You’ve got to actually step through it. There are no shortcuts. You’ve just got to mechanically compare rows, and columns, and check the constraints, and juggle the possibilities of a 7 or a 2 in a particular cell. Matching people is like that. Except with, as I said, 200,000 cells.

People have thought about matching things for ages, except they call them assignment problems. The algorithm that I’ve spent the last, ooh, 5 years? working with is called the Kuhn-Munkres algorithm, or the Hungarian algorithm. It rocks. It’s definitely my favourite assignment optimisation algorithm, I would say. Stable marriage is my least favourite for its heavy heteronormative overtones. Sometimes women propose to men!

(That’s a niche assignment algorithm joke, don’t sweat it)

Writing this code will hopefully reduce the amount of work he has to do from 30-ish hours to about 5 minutes, if we include the time setting up the code and typing the commands. It’s reduced his effort by a factor of 360. I think that’s awesome. It’s what I’m been trying to get people to see for the past five years: that this is difficult for a human, because it involves taking literally thousands of steps through an algorithm. Not because it’s particularly hard: most humans can do a sudoku, and I recently completed a logic puzzle in Russian purely because I knew the words for ‘not’ and ‘is’. But in thousands of steps, each comprising an iteration over a 200,000-cell grid, everyone would make a mistake. It’s only human.

Computers are not human. They do not make mistakes. They are the ideal candidates for doing a billion steps perfectly. Technologies like these make tasks like this completely, boringly, trivial. I think that’s something worth exploring, because the more of this trivial, boring nonsense we automate, the more of our incredible human potential we unlock.

Check out my code (or checkout my code and offer fixes!) here: https://www.github.com/jonodrew/mentor-match

My team is closing down, and my draft closedown document currently has the format of a last will and testament. We’re trying to collate wisdom we’ve gathered, advice to those we’ve left behind, and bequeathments: things we’re handing on to other people. My team found this a little morbid at first, but I’m talking them round: death is not to be feared and an ending can be both celebrated and mourned.

We’ve completed the work that we were formed to do and, like a human cell, I want us to efficiently wrap ourselves up and stop running. Zombie processes are not welcome, at least not in teams that I run, and we can learn important lessons from the natural world and how to view processes and cells that outlive their usefulness –

– and I note that sometimes that usefulness is a question of the cell’s usefulness to do the job: the job of your brain thinking must always be done, but the cells that carry out that function get tired and lose their effectiveness. It’s okay to say that we, as a cell, have reached the end of our usefulness, or that the job we’re doing no longer needs to be done.

These endings are natural and commonplace. The opposite is zombies. And nobody wants zombies.

I’m learning to touch-type – that’s why these weeknotes are so late – and it’s like trying to relearn how to swim while doing a triathlon. Turns out I’ve been typing wrong for about 20 years, and it’s starting to bite me in the butt. This is personal technical debt: a fix that made perfect sense at the time is now causing pain, and unpicking it is going to cause more pain because I’ve accidentally built a career on it.

The temptation to leave it alone, or patch it by stretching more or something, is strong. But I suspect this will keep getting worse if I don’t address it. And this is just me! Imagine if instead of me, this was technology supporting multiple lines of business. I’ve just got muscle memory: consider the institutional muscle memory you’d need to change.

It’s really easy to just demand change from people, but moments like this are great reminders to me to be humble in approaching these problems.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s