This is a series. You can find part 1 here and part 3 here. We were left on part-1 with a fully functional log pipeline. As I said, we’re going to check how to achieve a single shared index in OpenSearch. Shared Index If you recall, on the previous part we set a field on each document we were indexing in OpenSearch by leveraging the FluentD record_transformer plugin. main-fluentd-conf.yaml kind: ConfigMap apiVersion: v1 metadata: name: fluentd-es-config namespace: logging labels: addonmanager....

In this post I want to summarize some important Kafka producer’s configurations that usually are not taken in consideration when dealing with Kafka. max.block.ms: This timeout controls how long the producer may block when entering the send() method. This configuration provides an upper bound timeout for time spent waiting for metadata from the broker. Hence this timeout can be triggered when the producer’s send buffer is full or when topic metadata is not available....

I have been using Tekton since the last year and it amazed me for how much is easy to bootstrap CI pipelines with it. I will not go deep about how Tekton works (take a look at the documentation here ), but it’s important to mention that it executes your CI code in isolated Docker containers: for example, if your pipeline is composed by the steps “git-clone” and “docker-build”, each of these steps will execute its code inside a container, so for the first step we can use the alpine/git Docker image and for the docker-build we can use gcr....

This is a series. You can find part 2 here and part 3 here. Managing multiple Kubernetes clusters is not so easy; even more managing the logs that are produced from these clusters. The architecture that I want to show you is still a WIP but on the right track. Let’s start from this scenario: 15 Kubernetes clusters (that we will call Tenants) where Spring Boot based microservices are running. We need to provide to the developers a central logging dashboard where they can navigate and correlate logs; in this case we will use OpenSearch (formerly known as Open Distro for ElasticSearch)....

Podman is an OCI Open Containers Initiative specification compatible container-engine part of RedHat Linux which can be used instead of the Docker runtime. Podman in contrast to Docker is daemonless: it interacts directly with the containers, image registry and storage through the runC container runtime process just by using the CLI (podman); you can even alias podman as alias docker=podman without any problem! How to install Podman on MacOs Note: this guide is tested on a Macbook Pro 2019 (Intel)....

Having to deal with jvm based projects deployed on Kubernetes expose you to a different set of problems: performance, compatibility, logback issues, jvm flags etc. Here I want to show you how I deploy Spring boot (Java 11+) based applications on Kubernetes, starting from the application itself to the deployment and monitoring with Grafana. The application Let’s say we’re dealing with an application that needs to expose http endpoints; first of all we want download from https://start....

During my career happened to face this issue: how to build once a react-based application and run it through different environments (staging, qa, dev, production) with a single docker image? Usually to change a variable (for example a URL) you need to rebuild the application and redeploy on the target environment; as you may know, this isn’t very flexible and it may require other actions such as CDN cache clear, coordinate between teams etc....

Sometimes you will face a scenario where you have a Kubernetes cluster monitored with Prometheus and different services that live outside the K8S cluster; you would like to have these monitored with Prometheus as well, so how you do that with a ServiceMonitor? We can create a service without selectors and manually defined Endpoints. Having a single monitoring interface can be beneficial for a lot of reason: one of them is to use the monitoring tools we already use in the cluster, without setting up different monitoring stacks across VMs, K8S clusters etc (and the burden of managing them)...

Hello everyone, I am Justin, a DevOps Enthusiast. Here I will post some notes about Kubernetes and DevOps related resources. All opinions/everything in this blog are my own and do not reflect any company in any mean.