Introduction to Kubernetes Plugin Development

Kubernetes, the behemoth of container orchestration, is not just powerful but also highly extensible. One of the most exciting ways to extend its capabilities is by developing custom plugins using Go. In this article, we’ll dive into the world of Kubernetes plugin development, guiding you through the process with a mix of theory, practice, and a dash of humor.

Why Go?

Before we begin, let’s address the elephant in the room: why Go? Go, or Golang, is the language of choice for many Kubernetes developers because it’s the same language in which Kubernetes itself is written. This makes the ecosystem incredibly cohesive, and the official client libraries are some of the most well-maintained out there.

Setting Up Your Development Environment

Prerequisites

To start developing Kubernetes plugins, you’ll need a few things:

  • A working Kubernetes setup. If you don’t have a multi-node cluster, tools like minikube or microk8s can help you set up a local environment.
  • Basic knowledge of Go. If you’re new to Go, it’s time to get familiar; it’s a language that’s both simple and powerful.
  • A working Go modules directory. This is essential for managing dependencies in your Go projects.

Installing Client-Go

The client-go library is your gateway to interacting with the Kubernetes API. Here’s how you can install it:

mkdir k8sjobs
cd k8sjobs && go mod init github.com/k8sjobs
go get k8s.io/client-go@latest

This will set up your Go module and install the client-go library. You can verify the installation by checking the go.mod file:

module github.com/k8sjobs

go 1.15

require k8s.io/client-go v0.20.2 // indirect

Developing Your First Kubernetes Plugin

Step-by-Step Guide

1. Create Your Go Project

Start by creating a new Go project. Here’s an example of how you might structure your project:

.
├── LICENSE
├── README.adoc
├── cmd
│   └── main.go
├── go.mod
└── go.sum

2. Write the Plugin Code

Let’s create a simple plugin that displays the Kubernetes server version. Here’s what your main.go file might look like:

package main

import (
    "github.com/bmuschko/kubectl-server-version/cmd"
    "k8s.io/cli-runtime/pkg/genericclioptions"
    "os"
)

var version = "undefined"

func main() {
    serverVersionCmd := cmd.NewServerVersionCommand(genericclioptions.IOStreams{
        In:  os.Stdin,
        Out: os.Stdout,
        ErrOut: os.Stderr,
    })
    if err := serverVersionCmd.Execute(); err != nil {
        os.Exit(1)
    }
}

This code sets up a new command using the Cobra library and executes it, handling any potential errors.

3. Build and Install the Plugin

To make your plugin discoverable by kubectl, you need to build it and place it in your system’s PATH. Here’s how you can do it:

go build -o kubectl-server_version main.go
sudo mv kubectl-server_version /usr/local/bin/

On macOS, the binary is moved to /usr/local/bin/. Ensure this directory is in your system’s PATH.

Making the Plugin Discoverable

Your plugin name should have the prefix kubectl-, and any dashes in the command name should be replaced with underscores. For example, kubectl-server-version becomes kubectl-server_version when building the binary.

Creating Kubernetes Jobs with Client-Go

Sometimes, you might need to create Kubernetes jobs programmatically. Here’s how you can do it using the client-go library.

Example Code

Here’s an example of a command-line tool that creates a Kubernetes job:

package main

import (
    "context"
    "flag"
    "log"
    "os"
    "path/filepath"
    "strings"

    batchv1 "k8s.io/api/batch/v1"
    v1 "k8s.io/api/core/v1"
    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    kubernetes "k8s.io/client-go/kubernetes"
    clientcmd "k8s.io/client-go/tools/clientcmd"
)

func connectToK8s() *kubernetes.Clientset {
    home, exists := os.LookupEnv("HOME")
    if !exists {
        home = "/root"
    }
    configPath := filepath.Join(home, ".kube", "config")
    config, err := clientcmd.BuildConfigFromFlags("", configPath)
    if err != nil {
        log.Fatalf("Failed to build config: %v", err)
    }
    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        log.Fatalf("Failed to create client: %v", err)
    }
    return clientset
}

func main() {
    jobName := flag.String("job-name", "", "Name of the job")
    containerName := flag.String("container-name", "", "Name of the container")
    entrypoint := flag.String("entrypoint", "", "Entrypoint of the container")
    flag.Parse()

    if *jobName == "" || *containerName == "" || *entrypoint == "" {
        log.Fatal("All flags must be provided")
    }

    clientset := connectToK8s()
    job := &batchv1.Job{
        ObjectMeta: metav1.ObjectMeta{
            Name: *jobName,
        },
        Spec: batchv1.JobSpec{
            Template: v1.PodTemplateSpec{
                Spec: v1.PodSpec{
                    Containers: []v1.Container{
                        {
                            Name:    *containerName,
                            Image:   "busybox",
                            Command: []string{*entrypoint},
                        },
                    },
                },
            },
        },
    }

    _, err := clientset.BatchV1().Jobs("default").Create(context.TODO(), job, metav1.CreateOptions{})
    if err != nil {
        log.Fatalf("Failed to create job: %v", err)
    }
    log.Println("Job created successfully")
}

This code connects to a Kubernetes cluster and creates a new job based on the provided flags.

Debugging Kubernetes Services from GoLand

If you’re using GoLand for your development, you can also debug your Kubernetes services directly from the IDE.

Steps to Debug

  1. Prepare Your YAML Files: Ensure you have the necessary YAML files for your deployment. For example, db.yaml and web.yaml files that define your services and deployments.

  2. Build and Deploy: Build your Docker container and deploy it to Kubernetes using the green arrow in the editor gutter of GoLand.

  3. Adjust for Debugging: Make necessary changes to your web.yaml file to enable debugging. This typically involves adjusting the Dockerfile and building it with the debug configuration.

  4. Launch the Debugger: Use Run | Debug … | Kubernetes Service to launch the Go Remote debug configuration. This will allow you to debug your service as if it were running locally.

Conclusion

Developing Kubernetes plugins with Go is a powerful way to extend the capabilities of your Kubernetes cluster. From creating custom kubectl plugins to programmatically managing Kubernetes resources, the client-go library provides a robust set of tools.

Here’s a flowchart summarizing the key steps in developing a Kubernetes plugin:

graph TD A("Set Up Development Environment") -->|Install client-go| B("Create Go Project") B -->|Write Plugin Code| C("Build and Install Plugin") C -->|Make Plugin Discoverable| D("Test Plugin") D -->|Optional: Create Kubernetes Jobs| E("Use client-go to Create Jobs") E -->|Optional: Debug from GoLand| B("Debug Kubernetes Services")

With these steps and a bit of practice, you’ll be well on your way to becoming a Kubernetes plugin development master. Happy coding