Digest #139: Self-Hosting Databases, Postgres Memory Optimization, Kubernetes Success and FinOps Tools
Stay ahead with insights on cloud cost management, the self-hosting debate, Kubernetes' success story, adopting AWS features with Terraform, Raspberry Pi's public debut.
Welcome to this week’s edition of the DevOps Bulletin!
Get the latest updates on Vista Equity Partners' acquisition of Pluralsight, the journey of Kubernetes' success, and a daring hijack of a sovereign state's top-level domain. Discover the latest in Linux commits and much more.
In our tutorials, explore the new Linux Commits Search tool, compare Curl vs Wget, and learn to manage Postgres memory effectively. Automate PostgreSQL partitioning with pg_cron and pg_partman, and dive into AWS Cloud Native Pipelines for robust testing tools. Shield PostgreSQL from the FreeBSD OOM Killer, back up Git repositories to S3, and prevent CloudFormation orphaned resources. Optimize AWS resources with CLI FinOps tools, deploy Lambda in Docker, and handle long-running tasks with AppSync.
Check out our highlighted projects of the week, including a toolkit for hosting CloudFront, a networking analysis tool, and more.
Explore tips on transparency and control in cloud spending, how to self-host databases, and the importance of proper resource tagging.
Newsworthy Stories
Stay informed with the latest news impacting the DevOps and SRE world:
Do you self-host your database?
There was a discussion on HackerNews last week about whether you self-host your database. The choice between self-hosting and managed databases depends on project size, budget, and technical needs. Self-hosting offers control and savings, while managed services provide convenience and support for larger, critical applications.
Tutorials of the week
Linux Commits Search: A new tool for searching through Linux kernel commits efficiently. This can be a game-changer for developers and system admins alike.
Curl vs Wget: Curl supports more protocols and platforms, while Wget excels in recursive downloads and simplicity.
Optimizing Postgres Memory Usage: Learn how to manage Postgres memory effectively, including shared_buffers, work_mem, and maintenance_work_mem.
Automate PostgreSQL Partitioning: Learn how to automate interval partitioning in Aurora PostgreSQL using pg_cron and pg_partman, integrated with AWS Lambda and SNS for proactive monitoring and notifications of job failures.
AWS Cloud Native Pipelines: Supports major code hosting platforms, integrates static code analysis, and provides robust testing tools.
Preventing FreeBSD OOM Killer from Terminating PostgreSQL: Learn how to use the protect command in FreeBSD to shield PostgreSQL from the Out of Memory (OOM) Killer.
Back Up Your Git Repositories to S3: Ensure your code is safe by learning how to back up your Git repositories to Amazon S3.
Preventing CloudFormation Orphaned Resources: Learn how to use AWS CloudFormation's "force delete" feature to keep your account clean and reduce costs.
CLI FinOps Tools: Automated AWS resource optimization tools, designed to save time and reduce costs.
Deploy Lambda in Docker: Learn how to package and deploy AWS Lambda functions using Docker containers, enabling any language or framework, easier dependency management, and larger deployment packages.
Executing Long-Running Tasks with AppSync: Learn how to handle long-running tasks in AWS AppSync to ensure smooth and efficient operations.
Using the TOFU Terraform AzAPI Provider: Discover how to use the TOFU Terraform AzAPI provider to efficiently find all subnets in your environment.
Projects of the week
Highlighting cool DevOps projects to keep an eye on:
Beta9 is an open-source platform for running scalable serverless GPU workloads across cloud providers.
Batfish is a network configuration analysis tool that can find bugs and guarantee the correctness of (planned or current) network configurations.
Dblab is an interactive terminal-based UI application for PostgreSQL, MySQL, SQLite3, and Oracle.
Mycorrhiza Wiki is a lightweight file-system wiki engine that uses Git for keeping history.
LumoSQL is a reliable and secure database management system that is fully open-source and improves on the performance on classic SQLite.
CloudFront Hosting Toolkit, an open-source command line tool to help developers deploy fast and secure frontends in the cloud.
Tip of the week
Transparency and control in cloud spending are key principles of FinOps. Here are some tips to help manage and optimize your cloud infrastructure:
Resources Inventory: Maintain a unified view of active resources across all cloud providers (AWS, GCP, Azure, and Kubernetes). This helps in tracking and managing resources efficiently.
Maintain Tags Hygiene: Ensure every resource is properly tagged. Proper tagging drives optimal utilization and helps eliminate idle assets, ensuring you're not paying for resources you don't use.
Empower Engineering to Take Actions: Monitor cloud usage across different teams. Whether it's DevOps, Marketing, or any other department, knowing who is using what and how it affects your bottom line is crucial for effective cost management.
Tailwarden excels by providing the above features and more.