Project

Scalable Jenkins on Kubernetes

We migrated Jenkins from a single EC2 host to dynamic Kubernetes build agents on AWS EKS using Karpenter, Jenkins Cloud agents, and Jenkinsfiles.

  • AWS
  • EKS
  • Jenkins
  • Kubernetes
  • Karpenter
  • CI/CD
  • Kaniko
Scalable Jenkins on Kubernetes architecture diagram preview
Business context

BlueOrbit (pseudonym) had a single EC2-hosted Jenkins master/agent model with increasing build load, mixed dependencies, and growing operational complexity. The existing setup worked early on, but it was becoming hard to scale and maintain for multiple teams and projects.

Core outcomes
  • Shifted build capacity from fixed VM behavior to elastic Kubernetes agents.
  • Added cost control with spot-first node families and fallback nodes.
  • Introduced codified Jenkins pipelines for repeatable build workflows.
  • Improved visibility in CI pipelines and reduced manual maintenance overhead.
Problems to solve
  • Single EC2 instance limited build parallelism and isolation.
  • Different projects required different runtime stacks and toolchains.
  • Freestyle jobs in Jenkins UI were hard to track and maintain.
  • Limited pipeline visibility and weak change management practices.
Architecture direction
  • Keep existing AWS EKS footprint and isolate Jenkins agents in a dedicated namespace.
  • Use Karpenter for dynamic node provisioning and right-sized build nodes.
  • Use Jenkins Kubernetes plugin for ephemeral build pods.
  • Use Kaniko inside build pods to avoid privileged docker on hosts.

Reference architecture

Source-control and pipeline events enter Jenkins and fan out to Kubernetes build agents. Agents execute containerized toolchains and push artifacts to ECR.

Open full size
Jenkins Kubernetes pipeline architecture
Implementation notes
  • Created dedicated EKS namespace for Jenkins build agents.
  • Configured scoped IAM roles for ECR and deployment actions.
  • Provisioned Karpenter node pools with spot-first policies and fallback on demand.
  • Replaced freestyle jobs with Jenkinsfiles for versioned, auditable pipelines.
  • Used Kaniko for image builds to keep CI safe and repeatable.

Technologies used

AWS EKS Karpenter Jenkins Kubernetes Kaniko ECR IAM

Detailed case study

Read the full educational implementation guide with architecture decisions, migration steps, and reusable checklists.

Read full case study

Lessons learned

Moving CI to Kubernetes agents is most effective when CI architecture is combined with explicit job standardization. The largest gains came from codifying pipelines and controlling node capacity through intent-based autoscaling instead of one fixed build host.