Deis on Google Container Engine with Private GCR.io registry

Wed, Aug 9, 2017



Deis allows you to quickly deploy applications using just a few short commands:

Create, Configure and Pull.

Since your app is deployed as standard Kubernetes pods, you can take advantage of all the robust capabilities k8s offers, coupled with the simplicity of deploying to a Paas.

In this tutorial we’ll walk through the initial setup of Deis on Google Container Engine and then deploy a custom app from a private docker registry using Google Container Registry.

Throughout this tutorial I export variables that are used frequently in the commands so you can copy and paste the commands as much as possible. Be sure to review these exports though and tweak as needed.


Get your Project and Cluster Ready

Project Option 1: Use an existing project

export PROJECT=yourprojectname
gcloud config set project ${PROJECT}

Project Option 2: Create a new Project

Create a random project name and set your billing ID before spinning up your project

TIP: Get a list of billing accounts with gcloud alpha billing accounts list

export PROJECT=$(whoami)-$(date +%y%m%d)-deis
export BILLING=00AA00-BB00BB-00CC00

Create the project

gcloud projects create ${PROJECT}
gcloud alpha billing accounts projects link ${PROJECT} \
--account-id=${BILLING}

gcloud config set project ${PROJECT}

Enable some APIs

gcloud service-management enable compute-component.googleapis.com \
--project=$PROJECT
gcloud service-management enable container.googleapis.com \
--project=$PROJECT

Create a Cluster

export CLUSTER=${PROJECT}-cluster
export ZONE=us-west1-c

TIP: Get a list of zones using gcloud compute zones list --project=${PROJECT}

gcloud container clusters create ${CLUSTER} \
--project=${PROJECT} \
--zone=${ZONE} \
--machine-type=custom-2-4096 \
--image-type=GCI \
--num-nodes=2 \
--quiet

Get the credentials

gcloud container clusters get-credentials $CLUSTER \
--project=${PROJECT} \
--zone=${ZONE}

Double check the nodes are live

kubectl get nodes

Install Helm & Deis Workflow

With a project built and cluster spun up, it’s time to install Deis Workflow in our cluster

Helm Install

Helm is useful for managing a wide variety of predefined and customizable packages in Kubernetes. It’s a great addition to your Kubernetes toolbox. Here we’ll use it to install Deis itself.

For formal docs head over and read: Installing Helm

For me on mac this was just brew install kubernetes-helm

or just run this script


curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > get_helm.sh
chmod 700 get_helm.sh
./get_helm.sh

Next we need to install a helm controller called Tiller in our cluster. This is as simple as

helm init

NOTE: Pause a moment before continuing. It will take a short minute for Tiller to get started

DEIS Workflow

You’ll need the command line client locally as well as installing Deis in your cluster. To install the command line client locally follow these instructions

Tell Helm where to find Deis Workflow and then perform the installation.

helm repo add deis https://charts.deis.com/workflow
helm install deis/workflow --namespace deis

Watch the pods start up. The following command will update as each pod changes state. You’re waiting to see 11 for all the pods. This will report out until everything is up.

kubectl --namespace=deis get pods -w

Configure DNS

Deis requires a base DNS name to build all the apps off of. There are a couple models available. If you’re just testing this out, you can use a dynamic service as demonstrated with nip.io in Option 1. Alternatively if you’re looking to use this for more robust implementations, you’ll want to implement actual DNS records as indicated in Option 2. Both options require the load balancer IP so lets get that first.

Get the LoadBalancer IP

kubectl --namespace=deis describe svc deis-router | grep LoadBalancer

for me this returned Type: LoadBalancer LoadBalancer Ingress: 35.197.9.174

export LB_IP=35.197.9.174
export DEIS_ROOT=apps

OPTION 1: Mock DNS with NIP.io

Nip.io is a service that takes allows you to simply use an IP address in a domain name to make fully qualified domain name lookups. No additional configuration is needed, simply inject your LoadBalancer IP in front of nip.io and you’re good to go

export DEIS_DOMAIN=${DEIS_ROOT}.${LB_IP}.nip.io

OPTION 2: Real DNS

For more formal uses you may choose to configure your DNS with google. In this example we’ll configure a wildcard record to point any subdomains to our load balancer

NOTE: This example assumes you’ve already setup a DNS zone in this Google project for your use. ZONE_NAME and BASE_DOMAIN name below should come from that configuration

export ZONE_NAME=<YOUR ZONE NAME>
export BASE_DOMAIN=<YOUR DOMAIN>

export DEIS_DOMAIN=${DEIS_ROOT}.${BASE_DOMAIN}

Configure Google Cloud DNS to point to your LoadBalancer

gcloud dns record-sets list --zone=${ZONE_NAME} --project=${PROJECT}

gcloud dns record-sets transaction start -z=${ZONE_NAME} --project=${PROJECT}
gcloud dns record-sets transaction add -z=${ZONE_NAME} --name="*.${DEIS_DOMAIN}." --type=A --ttl=300 ${LB_IP}
gcloud dns record-sets transaction execute -z=${ZONE_NAME} --project=${PROJECT}

gcloud dns record-sets list --zone=${ZONE_NAME} --project=${PROJECT}

Finalize Deis Setup

Create an admin account and login

curl http://deis.${DEIS_DOMAIN}/v2/ && echo
deis register http://deis.${DEIS_DOMAIN}

TIP: This step registers the admin account and logs in. To login later without registering simply call deis login http://deis.${DEIS_DOMAIN} and provide your credentials

Deploy a test App

This step is just to demonstrate everything is functioning using a sample application provided

Create the app

export DEIS_TEST_APP=mytestapp

deis create mytestapp --no-remote

Deploy a sample image to your new app

deis pull deis/example-go -a ${DEIS_TEST_APP} 
curl http://${DEIS_TEST_APP}.${DEIS_DOMAIN}

You should see: Powered by Deis

Delete the Test app

deis apps:destroy -a ${DEIS_TEST_APP} --confirm=${DEIS_TEST_APP}

Deploying Custom apps from Private Registry

You may not want to publish your apps to pubic repos like docker hub. For this demo we’re going to utilize the private Google Container Registry included with your Google Cloud Project.

We’ll provide access to GCR.io both locally and from Deis, then use it to publish and deploy a custom image.

Generate Access Credentials

Here we’re going to create a service account to access Google Container Registry, both from our local workstation as well as from the Deis application itself .

The following step will create and download the json locally.

gcloud iam service-accounts create gcrreg --display-name "GCR.io Key" --project=${PROJECT}
gcloud iam service-accounts keys create --iam-account gcrreg@${PROJECT}.iam.gserviceaccount.com key.json

We need to assign the right permissions for the service account to access gcr. This happens to simply be google storage rights.


gcloud projects add-iam-policy-binding ${PROJECT} --member serviceAccount:gcrreg@${PROJECT}.iam.gserviceaccount.com --role roles/storage.admin

Now lets use those credentials with locally with docker so we can push our image up.

docker login -u _json_key -p "$(cat key.json)" https://gcr.io

NOTE: we downloaded key.json and used it here in this step we’ll use it again in just a moment when we tell Deis how to access gcr.io so don’t delete it yet.

Build and Push a custom app to GCR

If you have a Dockerized app available go ahead and use it here. If not, I’ve got a simple Go app you can tweak for this demo.

Get the code

git clone https://github.com/cgrant/docker-go-sample.git

Build the image

docker build . -t gcr.io/${PROJECT}/example-image
docker run -d -p 80:80 gcr.io/${PROJECT}/example-image
open http://localhost
docker stop $(docker ps -a -q -f "ancestor=gcr.io/${PROJECT}/example-image")

Push the image

docker push gcr.io/${PROJECT}/example-image

Create and Configure the App in Deis

Here first we’re going to create a new app in Deis, and tell it which port our app is on. This is a quirk of using the private registry.

Define an app for image and specify what port. Use any name you see fit.

export DEIS_APP=real_app_name

Create the app in Deis and configure it

deis create ${DEIS_APP} --no-remote
deis config:set PORT=80 -a ${DEIS_APP}

Now tell your app to use the service account from earlier when accessing gcr.io

deis registry:set username=_json_key password="$(cat key.json | jq -c .)" -a ${DEIS_APP}

Deploy the image from GCR

deis pull gcr.io/${PROJECT}/example-image -a ${DEIS_APP} 
curl http://${DEIS_APP}.${DEIS_DOMAIN}

That’s it you’ve got a custom app deployed from a private Google Container Registry image

Wrap Up

You’re up and running with Deis! From here on out you can easily update this app or create new ones with just a few commands: Create, Configure Pull.

I’ve barely scratched the surface here so checkout the Deis Workflow Documentation for even more configuration options.

Clean Up

You can delete the cluster with

gcloud container clusters delete ${CLUSTER}

or simply delete the entire project with

gcloud projects delete ${PROJECT} --quiet
#google, #gcp, #kubernetes, #gke, #deis, #gcr,