A theoretical solution for solving a very niche problem

A problem I’ve bleated about before for ages. If you run a graduate scheme, and you spend a lot of staff time trying to move people around, I have a solution for you.

Many graduate schemes enforce a process whereby candidates are moved around the organisation. A shallow understanding of the entire business gives a good starting point, and the Civil Service is no different. Across its multiple schemes, candidates are moved across Whitehall.

The problem this creates – and I’ll admit it’s a niche one – is that you have to put people into roles that will improve them. That will increase their skills in certain areas. In this post I’m going to explore how you might do this, and then consequently how you might automate it. It’s going to be intensely boring to almost everyone, but if I’m really lucky somebody will commission me to actually implement it. For that to happen I have to put it out there, so apologies to all regular readers.

This will, hopefully, provide a solution to the problem of moving candidates around your organisation so that they gain a wide breadth of skills in a way that doesn’t cost you multiple real people’s time.

The approach

Suppose your organisation has a number of areas in which we want your graduate to grow – let’s say Finance, Operations, Digital, and Policy. Each of the roles we put forward will score, out of 5 (this number is picked on the basis that I like it; it could be anything), how much of each area you might cover if you were doing it. For example:

Head of Finance

Finance: 5/5
Operations: 2/5
Digital: 1/5
Policy: 0/5

Each candidate is unique and interesting and will know, approximately, what they know about.

Lyssa Penner

Finance: 2/5
Operations: 4/5
Digital: 2/5
Policy: 4/5

We could then apply some mathematics and come up with a score for how compatible Lyssa is for the Head of Finance role. We might want to weight the areas where she has a lower score, for example:

Compatibility: Lyssa Penner x Head of Finance

Finance score: (5-2) * 5 = 15
Operations score: (5-4) * 2 = 2
Digital score: (5-2) * 1 = 3
Policy score: (5-4) * 0 = 0

Total score: 15 + 2 + 3 + 0 = 20

By subtracting her current score in each areas from the top score, we weight more heavily in favour of areas Lyssa doesn’t know about.

Now, if there’s only one role and only one candidate, then this is a fairly quick bit of work for whoever’s running this scheme: you match Lyssa to the Head of Finance role and then you go and have a cup of tea.

Even if there are two roles, and two candidates, well:

Head of FinanceHead of Digital
Lyssa2021
Sotirios2219

Either Lyssa will do the Head of Finance role or Sotirios will. We’ve gone from 1 possibility to 2. Easy.

At 3, though – and this is assuming that calculating these weighted scores takes no time at all – we’ve got to pick from 6 different options: because the first candidate picks from 3 roles, the second candidate picks from 2, and the last candidate gets whichever 1 is left. 3 x 2 x 1 = 6

In mathematics, this is called a factorial, and is represented as the highest number of choices followed by an exclamation point: !. So in this case, with three candidates and three roles, there are 3! different solutions.

Because we’re multiplying all the time, these numbers get big quickly. With ten candidates, and ten roles, we have 10! solutions. 10! is 3,628,800: three million, six hundred and twenty-eight thousand, eight hundred different possible solutions. The grid is still fairly small – you could fit it on a piece of A4 paper – but the number of possible solutions is already wildly unmanageable.

But suppose your graduate scheme was incredibly impressive, and your organisation was massive. Let’s suppose you had 100 candidates, and 100 excellent, well-vetted roles to put them into. You can see where this is going, but let’s do the numbers anyway. 100! is 100 x 99 x 98…

The number of solutions with 100 candidates and 100 roles – a grid only 100 x 100 – is 9.3 x 10157. 93 with 156 zeroes after it. It looks like this:

93000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

and there are no good ways to imagine how big it is. I can’t express to you how big this number is. You know the made up number ‘a googol’, which means “a really big number”? It’s bigger than that. A lot bigger. A googol is only about 70!.

(By the way, I got exactly 156 zeroes above by getting a computer to do it, because I don’t trust my human fingers to press a button exactly 156 times. Boring, repetitive tasks are the kinds of tasks for which we should use computers)

My point is that even if you’ve got your perfect 100 x 100 grid with every score perfectly worked out you now have to find a way of checking each solution to find out which one is the best one.

And you have to do it every 6 to 9 months.

And you have to do it close enough to the actual moving date that the role is still the same as the one that was submitted, and the person’s understanding of our key areas hasn’t changed. So let’s say…6 weeks before everyone gets shuffled around.

Cool. Six weeks. More possible solutions to this problem than there are particles in the universe, and 30 working days to find the best one.

Cool cool cool.

Potential solutions

Okay, don’t panic. There are ways around this. Approaches to solving this problem are ‘algorithms’: that is, processes or rules that you follow. There are a couple of algorithms we could use:

  1. Pick at random
  2. Pick the highest scoring role for candidate 1, then eliminate it. Pick the highest scoring role from the remaining roles for candidate 2. Repeat until you run out of candidates.
  3. Try to optimise for the best overall score, rather than best individual scores.

An algorithm is a repeatable loop of instructions that you can carry out to complete a task. We use something called Big O notation to explain how long an algorithm will take to run in the worst case. For example, in the first case, the algorithm runs in O(2n): for every candidate, you’ll randomly pick a role and then eliminate that role from the pool. Because you’ll carry out 2 interactions for each piece of data, the algorithm will take 200 steps, or 2n loops, to complete. We’re assuming here that picking at random takes the same amount of time each loop.

Option 2 is slightly more complex. We’d have to find the highest scoring result in our column and, assuming our column isn’t sorted – because we’ve just written it down for now – it’ll take n steps to find the highest score, because we’ll need to go through every single option. Then we’ll select it, and score it out. So for the first candidate, there are n + 1 + 1 steps. However, for the second step, there’s (n-1) + 1 + 1, because we only need to examine 99 options as one has been binned. That means the last choice will just be 2 steps: picking the only one left and scoring it out. It will be (n-n) + 1 + 1. Calculating the sum of 0 to n is easy thanks to a handy formula, so we know that the overall complexity of this method is (n2 + n)/2 + 2n

In short, it’s more complicated than the first method, but will definitely result in some great outcomes for those candidates who are selected first. Great for Ana, but probably not ideal for Zaynab.

The third approach uses an algorithm called the Hungarian, or Munkres, algorithm. It is quite complicated, and has a worst-case runtime of O(n3). That means, in the worst case scenario, it will take 1003 or 1 million steps to find the optimal solution. By comparison, our first option only takes 200 steps and our second would only take 5,250. 1 million seconds is is about 34 working days, which is a smidgeon too long for our deadline of six weeks.

Still, it might be worth it. Optimising for the best overall score might mean that candidates almost never get their best option – but also that they’ll never get the worst possible option either. This is probably preferable to option 2, where the candidate at the end gets whatever’s left.

Enter the computer

Up above I employed a computer to type zero 156 times. Boring tasks like that, where you have to repeat an instruction or set of instructions, are pretty much ideal for computers. Can we therefore teach our computer our algorithms, and achieve our aim?

Yes.

In fact we can teach the computer the algorithm and have it complete this exercise in a day.

That means candidates and role sponsors can update their templates up to the day before the deadline. It removes the need for people in your organisation to try to remember every possible candidate and role combination. It makes the whole process transparent.

If you’d like to talk some more about how I can remove the need for three full-time members of staff to shuffle papers around every 6-9 months, you can leave a comment or speak to me on Twitter.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s