The UNDERRATED Open Source Powering My HomeLab


The UNDERRATED Open Source
Powering My HomeLab

This issue is brought to you by:

Secure Your AI Future at Snyk Launch 2025

Join Snyk Launch to discover how to establish a foundation to build securely and confidently in the age of AI.

Kestra, an open-source automation platform that's been a game-changer for my homelab and, frankly, could be for a lot more.

It's one of those tools that flies under the radar for too many people, but packs so much goodness inside.

I've seen my fair share of automation (and deployments) horror stories – from a six-terminal manual file-swap in production (yes, really) to the feels-like-winning-but-terrible Jenkins (sorry, Jenkins fans, if you even exist).

The landscape has improved with tools like GitHub Actions and GitLab CI, which are great for their specific CI/CD niches, which are, at the end of the day, quite limited, and for a good reason.

But what if you need more?

What if you want to automate everything else, from simple scheduled tasks to complex, event-driven workflows across dozens of different services, without juggling a dozen different tools?

That's the massive headache Kestra aims to solve.

Many setups I see involve a patchwork of CI tools for build-and-deploy, maybe Airflow for data pipelines if they're ambitious, a separate IFTTT-like service for simple integrations, and a pile of custom scripts gluing things together.

This often turns into a maintenance nightmare.

A tangled web of dependencies, and a constant battle with tool-specific quirks and languages.

Remember Groovy scripts, my Jenkins users friends? No, thank you.

This approach just doesn't scale efficiently and often leaves you wishing for a single, coherent system. (And yeah, a wish to never see a single line of groovy ever again).

This is where Kestra steps in

And why I'm genuinely excited about it.

It’s not just another CI/CD, or automation tool: it defines itself as a language-agnostic workflow automation platform that can orchestrate anything.

Think of it as the central nervous system for all your automated tasks.

The fact that it's language agnostic is due to its simple container-based architecture, BUT, in all honesty, you don't even have to care: kestra comes with hundreds of plugins (over 600 at the time of writing these lines), I tried hard searching for something they don't already have a plugin for, and it wasn't easy.

Getting started is ridiculously easy – a single docker run command, and you're operational in about 20 seconds:

// mounting the docker socket to be able to run containers

docker run –pull=always –rm -it -p 8080:8080 –user=root \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp kestra/kestra:latest server local

No complex setup, no steep learning curve just to see "Hello World."

But the real power lies in its flexibility and some features that aren't always obvious if you don't dive deep:

True Universal Orchestration

Kestra runs every task in a Docker container.

This is huge.

It means you can write a task in Python, Go, Bash, or whatever language you need.

If it runs in Docker (can't you think of something that doesn't...?), Kestra can orchestrate it.

This makes it incredibly versatile, from managing cloud deployments with AWS or BigQuery plugins to configuring servers or even interacting with your Google Sheets (yep 😅).

Remote Triggers with Meaningful Feedback

One of the most brilliant features is the ability to trigger any flow with a simple curl command:

curl -X POST \
http://localhost:8080/api/v1/executions/team/hello_world \
-F greeting=”hey there”

What's more, the API call returns a JSON response with all the execution details, so you can parse it, send results to other services and basically control the flow and tweak it to your liking.

This is an absolute goldmine for automation.

You can kick off complex workflows from anywhere – a script, another application, even your phone – and get immediate, parsable feedback.

This one is a personal favourite of mine, because it's not only genius and different that any other automation system, it also runs in the terminal, remotely, from a script, or a local shell!

Secrets Management for Everyone (Yes, Free Tier Too!):

A common pain point with open-source tools offering premium tiers is how they handle secrets.

Kestra keeps UI-based secret management for its premium users.

However, for those of us running the open-source version, there's a straightforward way to handle secrets securely: environment variables.

If you're using Docker Compose (and you can easily convert the docker run command):

Kestra will pick it up, keep the value safe, and make it available in your flows.

Just remember to base64 encode the secret value:

echo "somesecretkey" | base64

This is a crucial tip for using Kestra effectively without paying for premium features.

Blueprints & No-Code

For those who want to get started quickly or prefer a visual approach, Kestra offers a blueprint catalog with pre-baked pipelines for common tasks.

Plus, there's a no-code editor, allowing you to build flows by filling in forms rather than writing YAML from scratch, which is a nice touch for accessibility.

AI-Powered Flow Generation (The "MCP"): I initially tried generating pipelines with AI, and noticed that LLMs always mess up Kestra YAMLs.

Turns out, Kestra is ahead of the game here, they actually do have an MCP server, that you can use to generate pipelines from prompts.

This is a fantastic development, making it even easier to create complex flows by just describing what you want.

Not isn't just for deploying applications

It can be your IFTTT server, integrate with your smart home, talk to OpenAI (I even made it generate some truly terrible AI "art"!), or manage Kubernetes resources in response to triggers – effectively letting you build custom autoscalers.


While no tool is perfect (I did encounter some hiccups with the Telegram plugin's response parsing, though the message still sent!), Kestra's open-source nature, its incredible versatility, and these powerful (sometimes less obvious) features make it a great choice for anyone serious about automation, whether in a homelab or a larger org.

It truly feels like the "DevOps Toolbox" project I had in mind when starting my channel and this newsletter.


Thank you for reading.
Feel free to reply directly with any question or feedback.
Have a great weekend!

ESPRESSO FRIDAYS

Every once in a while I send hand picked things I've learned. Kind of like your filter to the tech internet. No spam, I promise!

Read more from ESPRESSO FRIDAYS

I've Been Using AWS Wrong for YEARS... For years, my approach to AWS felt like a battle. As a DevOps engineer and later and architect, building infra always involved a tedious process of carefully building templates and structure, reviewing, deploying, testing and iterating over and over. I’d either spend hours clicking through the console or writing endless infrastructure code, always feeling like I was one misconfiguration away from a headache. It turns out, I was making it much harder than...

You've been lied to about self hosting... This issue is brought to you by: Auth0, my auth provider for the last 6 years. Join their free virtual dev_day on June 18th to learn how to secure AI agents and applications. Save your free spot That title might sound a bit aggressive, but this isn't about hating on hosting platforms. It's about loving the freedom, control, and cost-savings that come from owning your deployment process, without giving up the slick, easy experience we all love. And...

How DHH Solved Deploying to Production (with Open Source) Ever felt depressed by the sheer complexity of getting your application live and serving users? You’re not alone. But what if deploying to production, even (or especially) across multiple servers, could be straightforward and more importantly, free? That’s the reality DHH, the creator of Ruby on Rails and CTO of Basecamp & HEY, wanted to create, and he delivered with an open source tool called Kamal. DHH’s approach to technology always...