Digest #152: Prometheus Exposure, Terraform Tricks, re:Invent 2024 Takeaways, and Kubernetes Tools

Learn to manage cloud costs, secure Prometheus setups, optimize Terraform, configure AWS Load Balancers, and use Kubernetes tools for better workflows.

Welcome to this week’s edition of the DevOps Bulletin!

This week, we unpack the exposure of over 300,000 Prometheus instances, examine trends shaping cloud cost management, and uncover details about PUMAKIT, a new Linux rootkit using advanced stealth techniques. We explore the role of shared storage in modern architectures, analyze how CPU utilization affects performance, and consider the benefits of intentionally breaking systems to strengthen resilience. You’ll also find insights on how incident.io prepares for downtime and a practical framework of four North Star metrics for platform engineering teams.

Our tutorials provide step-by-step guides on running databases in GitHub Actions, streamlining incident management with Terraform, and configuring the AWS Load Balancer Controller on EKS. You will also learn how to build infrastructure with Tilt, Linkerd, and Nginx, take a closer look at Slack audit logs, and use practical text-to-SQL for analytics. Finally, you will discover new ways to optimize Terraform workflows, including tips for managing Landing Zone Accelerators on AWS and handling “data source will be read during apply” Terraform messages.

This week’s open-source highlights feature tools designed to simplify your workflow. Teleform enables local debugging of AWS Lambda functions while interacting with live infrastructure. Kubernetes Spec Explorer offers a detailed view of Kubernetes resources, tracks change across versions, and links to helpful documentation. gah simplifies GitHub release installations without requiring sudo, and Holos introduces a new approach to configuration management in Kubernetes by implementing the rendered manifests pattern.

All this and more in this week’s DevOps Bulletin—don’t miss out!

Newsworthy Stories

• Over 300K Prometheus instances exposed

• State of cloud costs

• New Linux Rootkit PUMAKIT uses advanced stealth techniques to evade detection

• The case for shared storage

• Breaking down CPU speed: How utilization impacts performance

• Break stuff on purpose

• How we page ourselves if incident.io goes down

• 4 North Star metrics for Platform Engineering teams

AWS re:Invent 2024 Takeaways

I’ve made a quick slideshow of my key takeaways—check it out! What was your favorite announcement or moment?

Running Kubernetes in the public sector

Tutorials of the week

• Run database in GitHub Actions

• Streamlining Incident Management with Terraform

• Building your infrastructure with Tilt, Linkerd, and Nginx

• Monolithic architecture of Terraform & IaC

• Slack audit logs and anomalies

• Practical text-to-SQL for data analytics

• How to configure the AWS Load Balancer Controller on EKS

• Using Terraform with Landing Zone Accelerator on AWS

• Terraform 'data source will be read during apply' messages

Projects of the week

Highlighting cool DevOps projects to keep an eye on:

• Teleform is a Terraform-native tool that lets you run and debug AWS Lambda functions locally while seamlessly interacting with live AWS infrastructure.

• Kubernetes Spec Explorer is a tool for exploring Kubernetes resources, tracking version changes, accessing examples, and linking to official docs—all in one place.

• gah is a GitHub Releases app installer that does not require sudo!

• Holos is a configuration management tool for Kubernetes implementing the rendered manifests pattern.