About

To cache images, Kubernetes Image Puller creates a Daemonset on the desired cluster, which in turn creates a pod on each node in the cluster consisting of a list of containers with command sleep 720h. This ensures that all nodes in the cluster have those images cached. The sleep binary being used is golang-based (please see Scratch Images). We also periodically check the health of the daemonset and re-create it if necessary.

The application can be deployed via Helm or by processing and applying OpenShift Templates. Also, there is a community supported operator available on the OperatorHub.

Configuration

Configuration is done via env vars pulled from ./deploy/helm/templates/configmap.yaml, or ./deploy/openshift/configmap.yaml, depending on the deployment method. The config values to be set are:

Env Var	Usage	Default
CACHING_INTERVAL_HOURS	Interval, in hours, between checking health of daemonsets	"1"
CACHING_MEMORY_REQUEST	The memory request for each cached image when the puller is running	10Mi
CACHING_MEMORY_LIMIT	The memory limit for each cached image when the puller is running	20Mi
CACHING_CPU_REQUEST	The CPU request for each cached image when the puller is running	.05 or 50 millicores
CACHING_CPU_LIMIT	The CPU limit for each cached image when the puller is running	.2 or 200 millicores
DAEMONSET_NAME	Name of daemonset to be created	kubernetes-image-puller
NAMESPACE	Namespace where daemonset is to be created	kubernetes-image-puller
IMAGES	List of images to be cached, in this format <name>=<image>;...	Contains a default list of images, but should be configured when deploying
NODE_SELECTOR	Node selector applied to pods created by the daemonset, provided in this format '{"key":"value"}'	'{}'
IMAGE_PULL_SECRETS	List of image pull secrets, in this format pullsecret1;... to add to pods created by the DaemonSet. Those secrets need to be in the image puller's namespace and a cluster administrator must create them.	""
AFFINITY	Affinity applied to pods created by the daemonset, in this format '{"nodeAffinity":{ ... }}'	'{}'
KIP_IMAGE	The image puller image to copy the sleep binary from	quay.io/eclipse/kubernetes-image-puller:next
TOLERATIONS	Tolerations applied to pods created by the daemonset, provided in this format '[{"operator":"Exists"}]'	'[]'

Configuration - Helm

The following values can be set:

Value	Usage	Default
deploymentName	The value of DAEMONSET_NAME to be set in the ConfigMap, as well as the name of the deployment	kubernetes-image-puller
image.repository	The repository to pull the image from	quay.io/eclipse/kubernetes-image-puller
image.tag	The image tag to pull	next
serviceAccount.name	The name of the ServiceAccount to create	k8s-image-puller
updateStrategy.type	The updateStrategy type to use when restarting the Deployment	Recreate
priorityClassName	The updateStrategy type to use when restarting the Deployment	""
configMap.name	The name of the ConfigMap to create	k8s-image-puller
configMap.images	The value of IMAGES to be set in the ConfigMap	// TODO create a reasonable set of default containers
configMap.cachingIntervalHours	The value of CACHING_INTERVAL_HOURS to be set in the ConfigMap	"1"
configMap.cachingMemoryRequest	The value of CACHING_MEMORY_REQUEST to be set in the ConfigMap	"10Mi"
configMap.cachingMemoryLimit	The value of CACHING_MEMORY_LIMIT to be set in the ConfigMap	"20Mi"
configMap.cachingCpuRequest	The value of CACHING_CPU_REQUEST to be set in the ConfigMap	.05
configMap.cachingCpuLimit	The value of CACHING_CPU_LIMIT to be set in the ConfigMap	.2
configMap.nodeSelector	The value of NODE_SELECTOR to be set in the ConfigMap	"{}"
configMap.imagePullSecrets	The value of IMAGE_PULL_SECRETS	""
configMap.affinity	The value of AFFINITY to be set in the ConfigMap	"{}"
configMap.tolerations	The value of TOLERATIONS to be set in the ConfigMap	"[]"

Configuration - OpenShift

The following values can be set:

Parameter	Usage	Default
SERVICEACCOUNT_NAME	Name of service account used by main pod	k8s-image-puller
IMAGE	Name of image used for main pod	quay.io/eclipse/kubernetes-image-puller
IMAGE_TAG	Tag of image used for main pod	next
DAEMONSET_NAME	The value of DAEMONSET_NAME to be set in the ConfigMap	"kubernetes-image-puller"
DEPLOYMENT_NAME	The name of the image puller deployment	"kubernetes-image-puller"
CACHING_INTERVAL_HOURS	The value of CACHING_INTERVAL_HOURS to be set in the ConfigMap	"1"
CACHING_MEMORY_REQUEST	The value of CACHING_MEMORY_REQUEST to be set in the ConfigMap	"10Mi"
CACHING_MEMORY_LIMIT	The value of CACHING_MEMORY_LIMIT to be set in the ConfigMap	"20Mi"
CACHING_CPU_REQUEST	The value of CACHING_CPU_REQUEST to be set in the ConfigMap	.05
CACHING_CPU_LIMIT	The value of CACHING_CPU_LIMIT to be set in the ConfigMap	.2
NAMESPACE	The value of NAMESPACE to be set in the ConfigMap	k8s-image-puller
NODE_SELECTOR	The value of NODE_SELECTOR to be set in the ConfigMap	"{}"
IMAGE_PULL_SECRETS	The value of IMAGE_PULL_SECRETS	""
AFFINITY	The value of AFFINITY to be set in the ConfigMap	"{}"
TOLERATIONS	The value of TOLERATIONS to be set in the ConfigMap	"[]"

Installation - Helm

kubectl create namespace k8s-image-puller

helm install kubernetes-image-puller -n k8s-image-puller deploy/helm

To set values, change deploy/helm/values.yaml or use --set property.name=value

Installation - OpenShift

Openshift special consideration - Project Quotas

OpenShift has a notion of project quotas to limit the aggregate resource consumption per project/namespace. The namespace that the image puller is deployed in must have enough memory and CPU to run each container for each node in the cluster:

(memory/CPU limit) * (number of images) * (number of nodes in cluster)

For example, running the image puller that caches 5 images on 20 nodes, with a container memory limit of 5Mi, your namespace would need a quota of 500Mi.

Installing the image puller

oc new-project k8s-image-puller

oc process -f deploy/openshift/serviceaccount.yaml | oc apply -f -

oc process -f deploy/openshift/configmap.yaml | oc apply -f -

oc process -f deploy/openshift/app.yaml | oc apply -f -

To change parameters, add -p PARAM=value to the oc process command, before piping to oc apply.

Building

Makefile

# Build Go binary:
make build
# Make docker image:
make docker
# The above:
make
# Clean:
make clean

The provided Makefile has two parameters:

• DOCKERIMAGE_NAME: name for docker image
• DOCKERIMAGE_TAG: tag for docker image

Manual

Build:

CGO_ENABLED=1
BINARY_NAME=kubernetes-image-puller

GOOS=linux go build -v -o ./bin/${BINARY_NAME} ./cmd/main.go
GOOS=linux go build -a -ldflags '-w -s' -a -installsuffix cgo -o ./bin/sleep ./sleep/sleep.go

Make docker image:

DOCKERIMAGE_NAME=kubernetes-image-puller
DOCKERIMAGE_TAG=next

docker build -t ${DOCKERIMAGE_NAME}:${DOCKERIMAGE_TAG} -f ./build/dockerfiles/Dockerfile .

Testing

Once built and published to a registry, you can test FIPS compliance using https://github.com/openshift/check-payload#scan-a-container-or-operator-image

To run the unit tests:

make test

End to end tests require kind. Note that kind should not be installed with go get from this repository's directory.

cd $HOME && GO111MODULE="on" go get sigs.k8s.io/[email protected] && cd ~-

./hack/run-e2e.sh

Will start a kind cluster and run the end-to-end tests in ./e2e. To remove the cluster after running the tests, pass the --rm argument to the script, or run kind delete cluster --name k8s-image-puller-e2e.

Scratch Images

The image puller also supports pre-pulling the scratch images. Previously the image puller was not able to pull scratch images, as they do not contain a sleep command.

However, the daemonset created by the Kubernetes Image Puller now:

1. creates an initContainer that copies a golang-based sleep binary to a common kip volume.
2. creates containers volumeMounts set to the kip volume, and with command set to /kip/sleep 720h

As a result, every container (including scratch image containers) uses the provided golang-based sleep binary.

Trademark

"Che" is a trademark of the Eclipse Foundation.