Building serverless data pipelines with AWS Lambda, SQS, and DynamoDB has become a common pattern for processing large volumes of messages. The architecture is straightforward: SQS delivers batches of messages to Lambda, which processes each one and tracks its status in DynamoDB. When something goes wrong with a message, Lambda marks it as failed in the database, allowing it to be retried. However, Lambda timeouts can break this pattern in a critical way, orphaning SQS messages and leaving your system in an inconsistent state.

Audio and video processing in serverless environments like AWS Lambda has traditionally been challenging due to the lack of native support for media tools. However, we can now package powerful media processing libraries like FFmpeg directly into our deployments by containerising Lambda functions with Docker. In this post, we'll explore how to effectively use FFmpeg within AWS Lambda by examining a proof of concept transcription service we recently built for a client.

FastAPI's asynchronous capabilities make it one of the best performing Python web frameworks available, something that has contributed to its meteoric rise in popularity over the past few years. However, because of its use of asynchronous code, testing a FastAPI application can be more complex than a standard synchronous API. In this post, we'll explore how to use pytest and pytest-asyncio to write effective async tests for your application.

With a plethora of integrations and add-ons, Slack is more than just a simple chat app, and often serves as a notification centre for all sorts of observability and monitoring applications such as Sentry, Datadog and AWS CloudWatch. In this post we show how to create a Slack notifier to push CloudWatch alarms to a Slack channel using AWS Chatbot and the AWS Slack integration. All configured using Python CDK of course!

Management of database schema changes can be a significant challenge, especially in serverless environments. Automated, continuous deployment of application and infrastructure code is commonplace; managing database migrations in a similar way can greatly increase development speed and minimise issues with what can otherwise be a highly error-prone procedure.

For rapid development of APIs with an anticipated spiky usage pattern, almost nothing beats the combination of FastAPI and AWS Lambda for scalability and cost. In this post, we go over a simple FastAPI application, using DynamoDB as the persistence layer, and deploying serverlessly on AWS Lambda. All composed using Python CDK, of course!

AWS Lambda Power Tuning is a great tool to analyse how the speed and cost of your Lambda functions vary with configured memory. While running performance tuning on a Lambda function we built for a client's microservice, we came across an interesting edge-case involving Lambda aliases, Python package bundling and file permissions!

When deploying a Django app into production, serving static files is something that requires some thought - we can't just leave it to the development server anymore! Using barely more than 100 lines of Python CDK code, we can create a low-cost, low-latency distribution network for our static files with Amazon S3 and CloudFront.

If you work in multiple Bitbucket accounts on the same machine, a small amount of upfront effort configuring your SSH keys will significantly improve your developer experience.