Using python to analyse a Trello board

Things this is:

  • a post about using python 3 to parse JSON files
  • a love letter to code
  • a reminder for myself
  • shameless self-promotion
  • a theoretical approach to calculating story points per sprint
    • only use this power to help your team reflect: if you use it to compare teams across your organisation, something bad will happen to you
      • not really bad, obviously, but it’ll be moderately irritating, like you won’t be able to find a pound for your shopping trolley or you’ll be halfway to the shops when you remember you’ve forgotten your wallet

Things this is not:

  • how to make burndown charts from your Trello boards
    • if you want me to make a thing that does that, it will cost you a sizeable ongoing donation to #BlackLivesMatter

I have been dying to write code for weeks. The problem with wanting to write code is finding a problem that you need to solve. Luckily for me, Sam had a problem. But getting content out of an actual website is monstrous. If only I could grab it in some kind of machine-readable format…

Hot diggety dog!

Looking over the original data source there’s a few things that stand out to me straight away. Cards are a thing, and cards have labels. Cards exist in columns, and those columns have names. Sam’s labelled most of these columns with ‘Week commencing’ and then a date in a ‘date/month/year’ format.

One of Sam’s questions was to analyse what she’d been reading, and I figured her labels might be a really good way of analysing that. First things first: I grabbed the source data as JSON. There’s a lot in there!

(If you’d like to follow along, you can: https://github.com/jonodrew/labels-by-column/blob/master/reader.py)

The Card

I started by making a Card object. Python can be used in an object-oriented way, like Java or Scala, or in a more functional way like F#. It doesn’t do either particularly well, but that’s fine, because neither do I. My Card class takes a dictionary of indeterminate size and pulls out two key things.

class Card(object):
    def __init__(self, **data):
        self.labels: list = [label.get('id') for label in data.get('labels')]
        self.column_id = data.get('idList')

This code says that when you create a new Card object, you have to pass it a dictionary (a kind of data structure in python). Out of that dictionary, the object will grab a list of labels and then iterate through them, only grabbing their ids. It will also grab the id of the column that the Card was in.

For example:

>>>data = {'idList': '1', 'labels': [{'id': '1', 'name': 'COVID-19', 'color': 'grey'}]
>>>c = Card(**data)
>>>c.labels
['1']
>>>c.column_id
'1'

As you can see, we’ve lost some data about the labels assigned to this card – namely the name and color [sic, US spelling]. Although we care about the name, we don’t care about it right now. Later on we can write a method to turn it back into a human-readable name.

This was a bad decision, because it turns out that’s quite fiddly to do. I would have been better off making a Label object and putting everything in there, thereby making it easier to change things around later.

The Board

I did make a Board object, and I admit that it’s not very beginner-friendly as it uses a slightly more advanced python feature called comprehension.

class Board(object):
    def __init__(self, trello_json_file_path: str):
        with open(trello_json_file_path, 'r') as trello_file:
            self.trello_content = json.load(trello_file)
        self.labels = {label.get('id'): label for label in self.trello_content.get('labels')}
        self.columns = {column.get('id'): column for column in self.trello_content.get('lists') if '/' in column.get('name')}  # all the columns we care about
        self.cards = filter(lambda x: x.column_id in self.columns.keys(), all_cards)  # all the cards in all the columns we care about!

Okay, so on the blog it looks really messy and horrible. There are two different kinds of comprehension here, and they’re basically a one-line for loop. Rather than writing:

collected_bees = []
for bee in bonnet:
    if bee.friendly:
        collected_bees.append(bee)

I can just write:

[bee for bee in bonnet if bee.friendly]

and I get a list of friendly bees.

However, I’ve made that far more complicated:

{column.get('id'): column for column in self.trello_content.get('lists') if '/' in column.get('name')}

This creates a dictionary, rather than a list. A dictionary is made of key-value pairs (like a real dictionary!). This one is equivalent to:

new_dictionary = {}
for column in self.trello_content.get('lists'):
    if '/' in column.get('name'):
    # this checks if a column name has a slash, ie is a date
        new_dictionary[column.get('id')] = column

The last line adds a new key-value pair, where the key is the id of the column and the value is…what? It’s actually another dictionary, containing all the extra information about the column. In the dictionary comprehension, you can see that’s the first thing we do. With comprehensions we sort of do everything back-to-front.

So now we have dictionaries nested inside dictionaries. Матрёшка dictionaries, if you will, except some of the inside dictionaries are bigger than the containing dictionaries.

Alright. We’re doing pretty well, so now there are a couple of other cool python things I’m going to bring in: lambda and filter.

filter first: famously fickle and fiddly, I find filter furprisingly useful.

filter takes a function and an iterable. We’re going to use a list for our iterable, and we’ll make the list by doing a list comprehension:

all_cards = [
    Card(**card) for card in self.trello_content.get('cards')
]

This creates a Card object from each lump of card data we pull out of the board.

Then we’re going to filter that list by checking if the card is in one of the columns we care about:

filter(lambda x: x.column_id in self.columns.keys(), all_cards)

We do this by checking if the card’s column_id is in the list of keys from the columns dictionary. Notice the lambda – another line-saving feature of python, it allows me to define a little function there and then. If I didn’t use the lambda, I’d need to write something like:

def column_id_is_one_I_care_about(self, card):
    return card.column_id in self.columns.keys()
filter(column_id_is_one_I_care_about, all_cards)

I’ve realised as I’ve written this that it takes too many steps – I could have used a slightly messier list comprehension like this:

[
    Card(**card) for card in self.trello_content.get('cards')
if card.get('idList') in self.columns.keys()
]

But that’s kind of the thing. There are always different ways of doing things, and for me down here at the level I’m at the most important thing is readability.

Also – and I can’t stress enough how important this is – I like it better.

This is already 1,000 words long and quite complicated, so I’m going to pause here. We’re only at line 18, but hopefully things will get simpler from here.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s