I spent some time with my mentee this week – though, truth be told, she’s more senior than me now. Mostly we just hang out and do cool stuff. I’ve also picked up another mentee, and we’re going on a functional programming journey together. He’ll be talking to his colleagues with deadly earnestness about monads before long, just you wait.

This week I’ve been working on a pure python, pure CDK project with my mentee. The AWS CDK is a way of modelling and the building architecture, and often deployable code, in your primary language. That means in an hour we had an API up and running. As in deployed. From nothing.

That’s really weird. Great, but weird. It also means radically rethinking how we do a lot of things. For me, for example, it means letting go of hard-won knowledge of the intricacies of Flask, and Celery, and a myriad other technologies. It also means fundamentally rethinking how I structure my code, when the very idea of “globals” is so laughably distant. There are no globals. There’s 15 minutes of compute time, somewhere, potentially miles away from the previous 15 minutes. Incredible.

Though to be fair, I’ve actually found that just means reapplying my experience with Celery and distributed systems. Just…bigger. And harder. And now with a timer. Agh.

On Thursday I got to see my mentee talk through what we’d done and answer questions, and she was absolutely amazing. I also spoke to some folks who want to use the matching system for real in another organisation. They’re enthusiastic, both about the tool and what we could do next. I always feel like the most negative person in the room in these meetings, because I keep saying no. The Venn diagram of things it could do and things it will do (given that I’m writing it in my spare time) is a tiny dot in the centre of a vast circle. I don’t like it, but given I’m the one who’ll actually have to implement it, I feel like I have to. Tips and tactics for dealing with this would be welcomed.

At the same time, an old friend and former colleague showed a rough draft of an AWS-native approach for the mentoring software. He really, really knows his stuff, so I’m excited to see how we can use this to further drive down the costs of the software, and to make it even more efficient. From what I’ve seen, it’s looking like another 85-90% time reduction when running our Really Big Task. What a doll. What a queen. You can follow along that particular challenge on my GitHub repo.

On the topic of the above, a small note to myself about how AWS Lambda, AWS Lambda Layers, and Python can work together. This is a bit technical.

Let’s look at some code:

# file: index.py

from copy import deepcopy

from tasks import async_process_data


def async_process_data_event_handler(event: dict, context):
    unmatched_bonus = event.get("unmatched bonus", 6)
    mentees = event["mentees"]
    mentors = event["mentors"]
    matched_mentors, matched_mentees, bonus = async_process_data(mentors, mentees, unmatched_bonus)
    return {
        "mentors": [mentor.to_dict_for_output() for mentor in matched_mentors],
        "mentees": [mentee.to_dict_for_output() for mentee in matched_mentees],
        "unmatched bonus": bonus
    }

This will be a lambda function. As you can see, it’s pretty bare-bones, and that’s on purpose. Testing Lambda functions, it turns out, is a total pig when you have dependencies. I have defined a few dependencies at the top.

copy is a native Python library, so there are no issues importing that. However, tasks is actually a file inside a neighbouring module on my local machine. This is a problem. When I tell the AWS command-line interface to wrap this function up to be run in their cloud, it does not take the dependencies. And then it complains that the dependencies are not there.

Seems like there’s an obvious enough solution, but I am 100% certain that’s because I don’t know enough about this.

Instead, the approach one has to take is to separately install the dependencies as a LayerVersion. This is where I’m a little hazy: I’ve put the code I want to import in a folder called “python”, because I think that’s what the docs say. I am open to the possibility that I’m wrong, and I could have named it anything.

Now, luckily for me there’s a Python-specific flavour of Function and LayerVersion, in a downloadable package called aws-cdk-lib.aws-lambda-python-alpha. I really like it, especially as it seems I can bung in a requirements folder at the root of the folder containing my shared code and get all of those things installed for free as well. The end result is that I can access

All of this synthesizes and works online. However, my IDE is not at all happy. While for AWS the tasks module is now a neighbour at the same level in the file hierarchy, for my local development environment it’s inside a folder. That means that locally my IDE wants me to call this path python.tasks. If I do that and deploy the code, AWS complains that it’s never heard of this python module and what do I think I’m playing at.

Current solution? In my IDE, I can mark a directory as “Sources Root”. I don’t know what it means, but it seems to do the trick – fooling my IDE into thinking that tasks is at the same level as the file I’m importing it into. I would love to know if there’s a better solution.

I didn’t write my team’s weeknotes in my usual rapid fashion this week, and we didn’t have anything finished in time for show and tell. I think there’s a link there, and it’s bothering me.