Tagged with Web Services

Web services including pushing and pulling services.

Building a basic multiprocess worker in python

I’ve recently been working quite a bit on a worker application in python. This worker picks up tasks over a gRPC service executes those tasks, and publishes the results via gRPC. Because of python’s Global Interpreter Lock (GIL), threads don’t help for CPU intensive workloads. Instead, I’m using multiprocessing to run additional processes per worker and get better CPU utilization at the cost of using a ton of memory. When a python process opens a child process it can also share state with the child process. The most interesting of these are synchronization and communication primitives like Event, Queue, Pipe, and Lock.

Rustling exercise notes

This Easter I had some time to dedicate to working through the rustling exercises. I still consider myself a novice rust developer. I frequently don’t know rust idioms, and spend a lot of time struggling to get an idea working at the fidelity of a prototype. Once I have something working, I’m able to revise and refine it incrementally, but getting the initial idea out is tough.

Building time limited workers with SIGALRM

Recently, I’ve been working on a system that needs to run tasks scheduled by either web requests, or other tasks. These tasks are trusted but sometimes problematic code. While developers generally write good code, we all make mistakes and the occasional slow job that can run for multiple hours gets written. These slow tasks can create problems if the volume of them goes up as they can consume all available resources.

Feature flagging for CakePHP

I wanted to share a new CakePHP plugin that provides feature flagging. In a Saas environment, I’ve been a long time advocate for using feature flags to release changes instead of deploys. Feature flags let you better manage release risk by allowing you deliver code with new features disabled.

Introducing Stickler-CI

I’d like to introduce a project I’ve been working on over the past few months. Stickler-CI helps automate the tedious process of ensuring coding standards are followed during pull requests. Like many teams, FreshBooks uses pull requests as a way to solicit feedback from other developers, ensure consistent coding practices and catch bugs before they can cause real problems.

Moving to linode

I recently moved hosting for this blog and my wife’s site to a shiny new linode. I did this because I wanted to have control over the versions of PHP, and other server software I ran. I had some issues with email blacklists at my previous host. I chose linode as I’ve heard nothing but good things from friends and my operations folks at work recommended them.