NaBloPoMo #12

I was hoping to build a library. I’m going to write about how far I got, where the barrier is, and what I plan to do next. This follows from the introduction to Munkres I wrote earlier this month.

The algorithm requires manipulation of the elements in a grid as well as it’s rows and columns. I’m going to start this series by explaining how I tried to make the rows and column manipulable[mfn]completely guessed at that, genuinely amazed it’s a real word[/mfn] by extending the pandas class Series.

This is probably going to be moderately technical. I’m going to be walking through the code so there’s no need to go and look for it, but if you’d like to you can find it on Github.

import pandas as pd

class PandasMunkresAccessor:
    def __init__(self, pandas_obj):
        self._obj = pandas_obj
        self.covered = False

    def cover(self):
        Marks all elements in this series as 'covered' and also itself
        self.covered = True

    def uncover(self):
        Marks all elements in this series as uncovered and also itself
        self.covered = False

This is the first class I’m using. It uses a decorator, which is a sort of magic thing we have in Python. I’m going to avoid explaining decorators for the moment[mfn]because I don’t fully understand them myself yet[/mfn] and just say that this is a way of extending an existing class in Pandas – the Series class. It adds two methods and an attribute that I can access through Series.munkres – you can see where the “munkres” part comes from in the second line of code.

The way I’ve modified it is by adding an attribute, covered. There are also quasi “setter” and “getter” methods – one that covers all the objects in the Series and itself, and one that does the reverse.

I’ve added this because part of the algorithm involves covering rows and columns in a grid, and I thought this would be the simplest way to implement that in code. I am not yet sure whether I’m right, so I’m writing tests to check my understanding of how a Series is actually initialised.

(This is by far the best thing about writing code: if you don’t know how a thing works, you can construct a test to check. Most things in life aren’t like that.)

You’ll notice that when the extended Series.cover method is called, it calls another method – .apply. This is a method in the base Series class, and it allows you to apply a function to every element in the series. In this case, I’m applying a method from the MunkresElement class, which is just as well because all the items in the Series are MunkresElements. We know this because when we initialised the object in which we find the Series, we fill it with MunkresElements.

November is National Blog Posting Month, or NaBloPoMo. I’ll be endeavouring to write one blog post per day in the month of November 2019 – some short and sweet, others long and boring.

NaBloPoMo #11

We are more than a third of the way through the month and I’m starting to hit the first barrier. This is a very tiring exercise.

Here’s one I’ve been ruminating on since last weekend.

Excel workbooks are where you should look for your next internal product.

The most effective strategy you can have for building internal, and possibly external tools, is running a workshop for your colleagues to improve their Excel skills. Ask them to bring their problems. Welcome them.

You will find that one team is running a thousand-person development programme with spreadsheets that reference each other in ways that will make your head spin. You will discover convergent evolution in the form of ten different, but functional, versioning systems. You will find out that there is a hole in a firewall because someone once wrote some VBA that needed database access, and you will need a quiet moment by yourself as you question your life choices.

But if you reframe these as opportunities – as low-hanging fruit where business processes can be smarted up, made more streamlined, and more repeatable – then you will have a user-centered roadmap of the internal tools you should be building.

I believe wholeheartedly that spreadsheets, where there is autonomy and power to change fields and do powerful calculations on the fly, are the bleeding edge evidence of what your organisation needs. A tiny example – if your organisation is using 14 spreadsheets to keep track of its room bookings, then you have identified a user need that could be met by the market but instead is being met internally with a custom product.

If you poke that long enough, by the way, the question becomes “Why do people feel they need to meet this user need through a massive, custom-built spreadsheet, the upkeep of which has become 25% of someone’s job, when there’s a bunch of products out there that can meet this need?” and that’s a different but equally important question.

Now, since spreadsheets are highly decentralised in terms of power – the person using them can add columns and remove them – they are quite different from your next step, which is an MVP database-backed service. So the second thing I’d counsel here is that your MVP absolutely must have a way of giving back a slice of the data in a CSV, so that it can be put back in Excel. If you insist instead that the new product will do everything, you’ll get low uptake (because it won’t if it’s an MVP) or you’re angling for a three-year million pound contract and that’s not an MVP, that’s your retirement fund. Stop it.

Your new product will need to almost be a micro service, a little slice where data goes in – possibly even in CSV format – and comes out again, and somewhere else some processing happens. Maybe emails get sent. Maybe some complicated calculation happens.

Or maybe a form that used to be emailed out to everyone in Excel that could instead be offered as a web interface. Basically, carve out a tiny slice of the user’s journey and replace it with something they’ll prefer to use.

In mapping parlance, you’re taking an element of a value chain and moving it to a product space. The more you do this, the more value you’ll be able to move up the value chain – value here being your colleagues’ time.

Of course with the extra time they’ll make more spreadsheets, but then that’s sort of the point of the bleeding edge. It’s always moving.

“Well, in our country,” said Alice, still panting a little, “you’d generally get to somewhere else—if you ran very fast for a long time, as we’ve been doing.”

“A slow sort of country!” said the Queen. “Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”

Lewis Carroll, Through the Looking-Glass

November is National Blog Posting Month, or NaBloPoMo. I’ll be endeavouring to write one blog post per day in the month of November 2019 – some short and sweet, others long and boring.

NaBloPoMo #10

I’ve had a purple beard for less than a week, and in that time at least once a day somebody has complimented me on it.

And that’s nice. Don’t get me wrong, it feels absolutely bloody lovely to have people compliment you on how cool you look. All the same, I’m struck by the sudden change in character of otherwise dour Londoners. Why has my purple beard got tongues wagging?

So this lead to a sudden startling conclusion: that I am experiencing a tiny peek into the life of anyone who deviates from the white heterosexual male appearance. Further than that, I think what I’m experiencing is that deviation from that perceived “norm” is considered public performance. And public performance means that critique is permissible or even welcomed.

The phrase I’m thinking of is:

Well, if you didn’t want people to notice, you wouldn’t do it, would you?

a dickhead, probably

And to be fair I do, kind of, want people to notice. Suddenly attractive people are chasing me down the street and asking if I have plans for dinner. Babies laugh uproariously. Beauties on Vespas stop traffic to stare.

But: that’s a choice I’ve made. The assumption that everything that deviates from the “norm” is for public consumption – be it the hair of people of colour; the clothing choices of women; or the appearance of people who don’t feel comfortable at the extremes of the gender spectrum – is, for me, at the root of street harassment.

November is National Blog Posting Month, or NaBloPoMo. I’ll be endeavouring to write one blog post per day in the month of November 2019 – some short and sweet, others long and boring.

NaBloPoMo #9

My mentee asked me recently what an API is, and my fellow traveller-in-NaBloPoMo Alex asked me to try to cover a crunchy technical thing.

Let us see if we can solve both of these.

This blog is put out as the limit of my current knowledge. If I’m wrong, or you think I’ve skimped on detail, I’d be really grateful if you could let me know so I can make it better. I won’t be covering versioning APIs, although it is important and I recommend reading up on it.

Okie dokes.

An API is a description of how a thing can be interacted with programmatically. That is, by a computer-y shaped thing, rather than a person-y shaped thing. If a person-y shaped thing is interacting with your thing you will need a User Interface, or UI.

This is true at every level of abstraction.

This is also made slightly annoying by the way Python disregards concepts like encapsulation and instead lays itself open for anyone to abuse its internals, like a dog that rolls onto its back and exposes its soft belly because it trusts you implicitly.

There are, however, some nice conventions that discourage those who are not aggressively invested in fiddling with the insides. But I’m getting ahead of myself. Let’s talk about the API for a class, Dog.

class Dog:
    def __init__(self, age, colour):
        self.colour = colour
        self._age = age
        self.__goodest_boi = True

    def age():
        self._age += 1

    def bad_boy():
        print("Impossible, is always goodest boi")

Alright. Here’s our Dog class. There are three attributes: colour, _age, and __goodest_boi. There are also methods to change these attributes. Well, to try and change these attributes.

The API of this object is the methods and attributes other computer-y type things can use to try to manipulate the object. In this case, they can reach in and alter colour by just setting it, eg

>>>rufus = Dog(age=1, colour='a sort of white with black smudges')
>>>rufus.colour = 'a muddy brown with eyes like amber'
# rufus has been playing in the river again!
'a muddy brown with eyes like amber'

so colour is part of the API of this object.

We can also set Rufus’ age directly, because a single underscore prefixed to an attribute is more of a guideline than a rule:

>>>rufus = Dog(age=1, colour='a sort of white with black smudges')
>>>rufus._age = 14
# it's weird how time flies; you blink and the puppy is now 
# a hoary old fellow with a frosted chin 
# who still raises his head when you call

The single underscore suggests to other developers that they ought not to try messing around with this attribute, so I would say it’s not part of the API.

We’ve also got a method to make our Dog older, called age, which is called as normal:

>>>rufus = Dog(age=1, colour='a sort of white with black smudges')
# happy birthday Rufus! 🎉

So our age method is definitely in the API, because we can use it to manipulate the object.

We’ve also got a method bad_boy, for when our Dog has been naughty. Since it’s a public method it forms part of the API. Let’s see what happens when we call it:

>>>rufus = Dog(age=1, colour='a sort of white with black smudges')
'Impossible, is always goodest boi'

Oh no! We can’t punish our Dog! But wait. We could directly manipulate the attributes above. Let’s try to change __goodest_boi to False:

>>>rufus = Dog(age=1, colour='a sort of white with black smudges')
>>>rufus.__goodest_boi = False
AttributeError: 'Dog' object has no attribute '__goodest_boi'
# gasp! but we definitely put that attribute in when we made Dog!
# it is essential to Dog, and should not be accessed by anyone
# without it, what is Dog but Wolf?

It turns out we can’t easily mess around with a double underscored attribute. Instead, we get an error and our Python interpreter pretends it’s never heard of such a thing.

It can be hacked around, but I’m not going to show you how. You’ll only try to make my Dog into a Bad Dog and that’s not a good use of technology.

The point of this slightly meandering stroll through a fictional Dog class is to show that the API of a Python class comprises its publicly accessible methods and attributes.

Alright, says my mentee, but that’s not what people mean when they say ‘API’ is it?

Bugger. No, it’s not. It’s a terminological inexactitude. When people say ‘API’ they mean ‘web-based API’. But it’s the same concept: it’s the collection of public methods, this time exposed over the internet, that computer-y things can use to interact with your thing (except this time ‘your thing’ is probably your software as a whole, rather than individual classes).

For example, suppose you had to register your Dog with an organisation for some reason. You might want to do that programmatically or, rather, your vet would probably want any Dogs that passed through the surgery to be automatically registered. So there might be a web-based API for registering a Dog that the vet’s surgery’s appointment booking software would link to.

In Flask, which is a Python web framework, it might look like this:

from flask import Flask
from app.models import Dog 
# our Dog is inside a file called
from app import db 
# oh yeah, we've got a database

app = Flask(__name__)

@app.route('/api/register', methods=['POST'])
def register():
    if not request.json or not 'age' in request.json:
        # it's okay if there's no colour, but we can't 
        # register ageless dogs. 
        # So if there's no age we abort the request
    dog = Dog(
            age = request.json.get('age'),
            colour = request.json.get('colour', '')
            # defaults to empty string if no colour
    return 201

if __name__ == '__main__':

So as you can see, the only API the software is exposing is a method to register a new Dog. There’s no access to the Dog API.

Alright. That’s a big, boring block of text about APIs. What did I get wrong? What else could I talk about? You should let me know.

November is National Blog Posting Month, or NaBloPoMo. I’ll be endeavouring to write one blog post per day in the month of November 2019 – some short and sweet, others long and boring.

NaBloPoMo #8

Hansard is, theoretically, a comedy. It is a comedy because it makes you laugh.

One of my best friends has remarked to me in the past that the best comedy says things about the world. She’s right, too, and this is the best comedy. This is comedy that softens you up with laugh after laugh before sticking in the knife and giving it a damn good twist.

There are many spoilers ahead. There is also mention of suicide.

Continue reading

NaBloPoMo #7

I went and had my hair professionally dyed.

It was a genuinely lovely experience, and I recommend all of my fellow men try experiences where people do lovely things to your appearance. It feels great – I had a genuine endorphin rush as I left. Alfie, the stylist, was thoughtful and genuine. It was a lovely experience.

I am trying to reflect on why I did it.

Now there are no single causes and effects, I think, when making decisions like these. Stuff just builds up and then we express it. But in reflecting on what the stuff is I can find out whether hair colouring is the outcome, or a symptom of something deeper. I think it’s important to do this, because knowing yourself is valuable and I think perhaps I don’t know myself as well as I’d like.

To steal from dear Bilbo Baggins: I don’t know half of me half as well as I should like; and I like less than half of me half as well as I deserve.

Altering my outward appearance is a manifestation of control. In fact, I think it’s fair to say that this is quite well known. When some part of your life feels out of control, it’s comforting to take actions that prove you have agency. Are some parts of my life out of my control?

Yes: I’m moving roles at work, and I can’t be certain that I’ll be good at the new job. My relationship with my ex is evolving, and I’m not sure of the direction it’ll go from here. I am supposed to be moving house, and have been in that state of limbo for coming up to 9 months. There is…stuff…going on at work. And my autism spectrum disorder assessment is finally looking like it’ll come to a conclusion.

These are things that are outside of my control, and knowing this makes me feel somewhat powerless. I thought I was okay with that. I think I am okay with it, but all the same, I’m displaying behaviour that suggests I’m trying to exert control.

Counterpoint: none of these things are particularly new. What else is going on?

A couple of my friends have dyed hair and it looks epic. I’m not normally one for peer pressure but, listen, hair that’s different styles or colours is awesome. Self-expression is awesome. I’m working in an organisation that welcomes self-expression, and I don’t think I’ve ever really taken advantage of it, short of the occasional tv-themed t-shirt. While I work somewhere that respects that, I’d like to take advantage of it.

I’ve also, at the age of almost thirty, started to give a shit about how I look. This is because I’ve discovered that you can outsource the problem of “what things looks good with other things” to a computer or another person, get those things delivered to you, and get them tailored so they fit you.

Having clothes that actually fit is a revelatory experience.

Having clothes that you feel good in because you look good is another revelatory experience, and that leads me to start questioning other things. I like my cerulean sweater. I like it paired with a thin salmon tie. I am discovering colours and, as with all newly converted, I am perhaps overzealous in my application. Why not have colour everywhere? Why not, indeed, try a beard that’s purple?

Another influence is the comic Alice Fraser, who in a recent show has a fantastic bit where she talks about retirement as making a bet. It’s making a bet that one day, when you are older, you will be free to do things that you always wished you’d done and that those things will not taste like dust. You are betting that at the age of 70 your body will still be able to roll in the grass or carry you up mountains or learn new languages.

And when you put it like that it sort of feels like a ridiculous bet. I might not have any hair left at all when I get there. Why not do things now? Why not enjoy things while you can?

Finally: I’m bi. I don’t know what that looks like, and I’m not even sure if it’s a factor, but I’ve noticed that I’m sort of fed up with people assuming I’m straight. And it’s not great that deviation from white male appearance marks you out as queer, but since it does then this is a tiny little rebellion on my part to stake out my space.

All of this came together, and here I am. Surprising myself in the mirror and liking myself more than half as well as I deserve.

November is National Blog Posting Month, or NaBloPoMo. I’ll be endeavouring to write one blog post per day in the month of November 2019 – some short and sweet, others long and boring.

NaBloPoMo #6

Pandas-Munkres: Part 1

Munkres is an algorithm for solving the assignment problem. I’ve talked about the problem before, so this blog is going to focus on why I’m trying to solve it. It’s a slice of personal history and will hopefully demonstrate who I am, and the kind of person I am, and why I’ve fixated on this for so long.

Continue reading

NaBloPoMo #5

I have become the person who, at unconferences with government folk, bangs on about Wardley maps. Who every single time pitches ‘Mapping 101’. It’s now become so rote that I jokingly, not jokingly, offered to do a TED talk on it.

There are two good things that come out of this for me. The first is the joy of presenting a subject to new people and them telling me they get it. The second is the joy of presenting and evolving my own examples, iterating the talk so that it’s always better.

However: I also worry that I’m pigeonholing myself. I’m the guy who does the thing, and being the guy who does the thing means I’ve got to do the thing. It adds a constraint to what I talk about – or, rather, it feels like it adds a constraint. I know that plenty of people wouldn’t mind if I didn’t do it[mfn]indeed, some might applaud the self-restraint[/mfn], and they’d support me if I pitched something else – so a fairly large component of this pressure is coming from within. I’m putting pressure on myself to be the guy who does the thing.

I may be teetering on the edge of self-discovery here, so let’s wind the rope back in for fear of finding something familiar.

What I am also finding, though, is that every year there’s a little bit more understanding in the community. I’m finding doors opening that I’d have had to push at previously, and there are now invitations in my inbox rather than politely confused rejections. People ask my opinion about things.

(ahah, my friend, are we perhaps finding whence the pressure springs?)

My personal saturation point for talking about the basics of this thing has long since passed, and I have to pay attention to that. Twice in a recent talk I found myself covering more complex topics before I’d properly explained the basics, and it’s a good reminder that while I’ve been talking about this for two years the audience may not have heard about it until twenty minutes ago. There is value in doing the basics again and again, and what feels like the same boring repetition to me should feel to the audience like something fresh and novel.

Reframing it as a performance – an interactive performance, but at its core a performance nonetheless – is helping me to remember that point. Hamlet must be thoughtful every night; Godot must wait; the Play That Goes Wrong must go wrong in precisely the same way.

Maybe what feels like overcommunication to you looks like confidence to your audience.

November is National Blog Posting Month, or NaBloPoMo. I’ll be endeavouring to write one blog post per day in the month of November 2019 – some short and sweet, others long and boring.

NaBloPoMo #4

I’m listening to a podcast an ex introduced me to, because my relationship with my ex is weird[mfn]everyone’s relationships with their exes is weird, I’m not special[/mfn]. It’s an insight into the practices of a therapist called Esther Perel, and something she said really stuck out for me.

She used a common metaphor for psychic trauma, equating it with physical trauma, but something about the way she expressed it suddenly hit home for me.

Psychic injuries are like physical injuries, because they require care by professionals. They also, which I hadn’t considered before, require aftercare. They require physio.

Physio is a tough and lengthy process, and it requires a dedicated professional to help you repair what was broken. It’s quite difficult work. People train very hard to become physios, and they exchange their labour for money with consent.

I’m going to put aside a discussion about consent under a capitalist system, and instead focus on the fact that consent is a necessary element of this exchange.

It is not fair to expect a physio to help with your healing if they do not consent to it, because it is not fair to expect anyone to work without consent. For the purposes of this metaphor, ‘work’ includes emotional labour.

Even if you’re really hurting, you don’t get to co-opt someone else to help you heal. You’ve got to ask. They’ve got to say yes.

With a physical injury it’s fairly easy to tell if someone’s trying to get you to do the work to help them get better. It’s fairly easy to decide whether you agree to do it, putting aside existing dynamics in the relationship.

I think it’s harder to tell when someone’s using you to help with the repair of a psychic injury. To be frank I think it’s hard to know that you’re leaning on someone to help repair it until one day you wake up and realise that you don’t need that person’s support any more.

You’ve used that person to help heal that injury[mfn]or, at least, you think it’s healed – there’s a separate question of whether taking this approach results in a clean mend or a brittle one[/mfn] and you’ve done it without their consent, because they didn’t realise that was all you were doing. Maybe you knew and maybe you didn’t, but now you’re facing the truth that you used someone to meet your needs and now you have no idea what to do.

I think it’s perfectly valid behaviour, following a break-up, to go through emotional physio. To try going on dates with different people; to find your confidence and your sexuality again. To have multiple sexual partners. But I think it’s also important to recognise that this is a kind of catharsis, a kind of physio, and that the consent of the people involved is important.

Consent is sexy, and there’s plenty of folks out there who’ll gladly help you flex that flirty muscle. But be honest and open, so that they know what they’re getting into and can give their informed consent.

November is National Blog Posting Month, or NaBloPoMo. I’ll be endeavouring to write one blog post per day in the month of November 2019 – some short and sweet, others long and boring.

NaBloPoMo #3

Can you be outcome focussed when examining an emergent space?

This was a point that was raised yesterday at Open Data Camp. There’s a good outline of the disagreement in this blog:

In essence, for me this comes back to what I mentioned yesterday – that whatever the “X” is in your “X Strategy” is the product you’re delivering. But, further, if the space you’re exploring is emergent – is in the Genesis space on a map – then it’s almost impossible to say what would be built on top of it. The point I made was that going to the Moon got us a load of extra cool stuff (like Velcro, although I’m not actually sure that’s true) but the US didn’t set up NASA in order to get Velcro.

Equally, setting up GDS has consequently produced the Digital Marketplace, but if you started with the desired outcome of “Save money on government procurement” then I don’t think you’d have got to “Create a new business unit in the Cabinet Office dedicated to digital delivery” as a solution.

The emergent space is, for me, a space where you need a fairly strong-willed leader who’s just going to say “Do this thing, because I am the boss and I would like it done”. This is a very good attitude to have in an emergent space, where it’s mostly gut instinct and lots of bets will lose. Having a strong conviction that X is the right thing to do because of personal belief or political convictions or whatever is helpful in making things happen.

It’s also the worst possible attitude to take in a well-understood space where we have data and vast stores of experience to draw on, because that’s when a more considered approach is absolutely vital. Mavericks are not needed when the product or service moves into the ‘Product/Rental’ space (although of course if there’s not someone in your organisation who’s scanning the horizon and trying out emergent stuff there’s a real danger of stagnation).

If you’re building a product, I think it’s easy to identify the datasets you wish were open. You can map those and then apply tactics to try to open them up:

  • You can second a member of staff to the relevant organisation and task them with internal advocacy
  • You can advocate from outside, leveraging the learnings of organisations like the Open Data Institute
  • You can advocate to the top and ask them to apply downward pressure
  • If the data is available but not open – for example trapped in PDFs – you can build it yourself (whether you open it up or keep it for yourself at that point is a business decision)

However, if we’re looking more broadly at a strategy to make more data open, particularly government data, then it kind of depends where you are as to what levers you can pull. I’d maintain that in some spaces opening data is in a Genesis space because it’s so novel. In those spaces stories are not enough, and it may be valuable to have tools available that reduce the barriers to people getting it done.

In the end though I still believe that where Open Data is in Genesis the best thing to do is to try to influence the most senior people and encourage them to demand it. Or, alternatively, become the most senior people for a couple of years.

Be the change you want to see.

November is National Blog Posting Month, or NaBloPoMo. I’ll be endeavouring to write one blog post per day in the month of November 2019 – some short and sweet, others long and boring.