Objectives
Developing microservices in Go can be a fun experience using all of the 12-factor application packages available in the community; however, if you can’t debug your services using a sophisticated debugger like delve, then you’re going to experience quite a bit of pain. I recently ran into this problem while developing Chimera, my game microservices project running in Docker. I couldn’t find any resources online outlining how to effectively set up a larger microservices project in Docker and Visual Studio Code.
This article outlines how I solved those problems and set up my Go microservices in Docker and Visual Studio Code. At the end of this article, you’ll know how to:
- Set up multiple build configurations (debug and release) in docker files
- Set up a multi-stage build system for a services image
- Configure services and builds using Docker Compose from Visual Studio Code
- Run and attach the delve debugger using dynamic ports and port mappings
- Get a list of running services to attach your debugger to within Visual Studio Code
Chimera
Setting up multi-stage builds in Docker with build configurations
The first step in debugging Go microservices in Visual Studio code is setting up the Docker image with the delve debugger. Rather than having two separate dockerfiles, I chose to have build configurations that are driven by Docker Compose and an env file.
The code below is a quick example of my first draft for building services for Chimera. The image accepts environment variables / arguments for the build config and delve debug server port. This port will be remapped to dynamic ports using Docker Compose. A run script is created for the entry point of the service image (allows environment variables at execution time to be used with a command crafted at build time).
# Spirited © All Rights Reserved
# See copyright file at project root for full details
FROM golang:1.18.3 AS builder
WORKDIR /usr/src/chimera/
# Set argument defaults
ENV CHIMERA_BUILD_CONFIG "release"
ARG CHIMERA_BUILD_CONFIG=$CHIMERA_BUILD_CONFIG
ENV CHIMERA_DEBUG_PORT 40000
ARG CHIMERA_DEBUG_PORT=$CHIMERA_DEBUG_PORT
# Stage 1: Copy and build services and dependencies. Download external dependencies first
# so this step can be skipped for most builds. Then, build services and share the same
# container to be inherited by all other builds.
RUN go install -v golang.org/x/tools/cmd/stringer@latest
RUN go install -ldflags "-extldflags '-static'" -v github.com/go-delve/delve/cmd/dlv@latest
COPY go.mod ./go.mod
COPY go.sum ./go.sum
RUN go mod download && go mod verify
COPY lib/ ./lib/
COPY service/ ./service/
RUN go generate ./...
RUN set -eux; mkdir /usr/bin/chimera; echo "#!/bin/bash" > /usr/bin/chimera/run.sh
RUN set -eux; echo "echo Launching Chimera..." >> /usr/bin/chimera/run.sh
RUN set -eux; echo "echo Build directory contents:" >> /usr/bin/chimera/run.sh
RUN set -eux; echo "ls -li /usr/bin/chimera/; echo " >> /usr/bin/chimera/run.sh
# Switch on the build configuration
RUN set -eux; case "$CHIMERA_BUILD_CONFIG" in \
"debug") \
go install -ldflags "-extldflags '-static'" -gcflags "all=-N -l" -v ./service/...; \
echo "echo Source directory contents:" >> /usr/bin/chimera/run.sh; \
echo "ls -li /usr/src/chimera/; echo " >> /usr/bin/chimera/run.sh; \
echo "echo Starting \$CHIMERA_SERVICE_NAME service in debug mode..." >> /usr/bin/chimera/run.sh; \
echo "./dlv --listen=:$CHIMERA_DEBUG_PORT --headless=true --continue" \
"--accept-multiclient --api-version=2" \
"exec /usr/bin/chimera/\$CHIMERA_SERVICE_NAME" >> /usr/bin/chimera/run.sh; \
;; \
"release") \
go install -ldflags "-extldflags '-static'" -v ./service/...; \
echo "echo Starting \$CHIMERA_SERVICE_NAME service in release mode..." >> /usr/bin/chimera/run.sh; \
echo "/usr/bin/chimera/\$CHIMERA_SERVICE_NAME" >> /usr/bin/chimera/run.sh; \
rm -rf /usr/src/chimera/; \
mkdir /usr/src/chimera/; \
echo "" > /usr/src/chimera/nil; \
;; \
*) echo >&2 "error: unsupported build configuration '$CHIMERA_BUILD_CONFIG'"; exit 1 ;; \
esac
# Stage 2: Copy the compiled service binaries from the previous stage and prepare an
# image to be tagged as a new release of Chimera.
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /usr/bin/chimera/
ENV CHIMERA_SERVICE_NAME "service"
ARG CHIMERA_SERVICE_NAME=$CHIMERA_SERVICE_NAME
# Necessary for running Go executables that are compiled with glibc.
RUN mkdir /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2
COPY --from=builder /go/bin/ /usr/bin/chimera/
COPY --from=builder /usr/src/chimera/ /usr/src/chimera/
COPY --from=builder /usr/bin/chimera/run.sh /usr/bin/chimera/run.sh
RUN set -eux; chmod +x ./run.sh
ENTRYPOINT ["sh", "./run.sh"]
Four things you should focus on in the image above: the Go version is the same as the Go version installed on the development machine, services get compiled with optimizations off, delve gets added to the run script, and source files get copied to the final image.
Configuring builds and services with Docker Compose
Chimera uses Docker Compose to build and start services with dependencies (and eventually to deploy services to a Docker Swarm across Raspberry Pis). Environment variables used to build and start services are defined in .env, which sits at the root of the project along with the service image’s dockerfile and compose.yml.
The .env file below defines the build config and ports used in Docker Compose:
# Spirited © All Rights Reserved
# https://spirited.io
chimera_build_config=debug
# Port configurations
chimera_auth_port_front=9960
chimera_auth_port_start=9961
chimera_auth_port_end=9981
chimera_debug_port=40000
chimera_debug_port_start=40000
chimera_debug_port_end=41000
In Chimera, two networks are defined: a front network exposed by the load balancers and a back network utilized by the microservices. Debug ports are only exposed on the back network. Docker Compose will automatically utilize the definitions in .env for build and start commands. See the compose file below for an example with a single Chimera service (the auth service before adding elastic stack dependencies):
# Spirited © All Rights Reserved
# https://spirited.io
version: '3.8'
networks:
backend:
driver: bridge
frontend:
driver: bridge
services:
auth:
image: chimera/service
build:
context: .
args:
- chimera_build_config
- chimera_debug_port
environment:
- chimera_service_name=auth
networks:
- backend
ports:
- $chimera_auth_port_start-$chimera_auth_port_end:$chimera_auth_port_front
- $chimera_debug_port_start-$chimera_debug_port_end:$chimera_debug_port
With the compose file set up, services can be built and started by right clicking the file in the Visual Studio file explorer. If you don’t already have the Docker or Tasks Shell Input extension (required in the next section), then you can create a new extensions.json file in the .vscode folder with the following to have them recommended:
{
"recommendations": [
"augustocdias.tasks-shell-input",
"irongeek.vscode-env",
"ms-azuretools.vscode-docker",
"golang.go"
]
}
To make it easier to build and start services, I set up my build action to run the following task:
{
"version": "2.0.0",
"cwd": "${workspaceFolder}",
"tasks": [
{
"label": "Compose Services",
"type": "shell",
"command": "docker",
"args": ["compose", "up", "--build", "--detach"],
"group": {
"kind": "build",
"isDefault": true
},
},
],
}
Attaching the debugger to running services using dynamic ports
Now that Docker is set up to run the services from Visual Studio Code, the final step is to create a list of available services to attach to using the debugger. To create a list of services, I use the Tasks Shell Input extension.
The input’s command in the launch.json file below uses Go text templates to parse out debug ports from running docker containers. Since text templates don’t support regexp, it instead uses supported functions like range, slice, split, index, and if to isolate debug ports for the selection. The Tasks Shell Input extension expects the following format:
<value>|<label>|<description>|<detail>
The value for this input is the debug port to connect to. In the configuration portion of the launch.json file, the port number is chosen using the input selector. The remote path points to the source files path within the container rather than the source files on the local machine.
{
"version": "0.2.0",
"inputs": [
{
"id": "findService",
"type": "command",
"command": "shellCommand.execute",
"args": {
"command": "docker ps --filter \"ancestor=chimera/service\" --format \"{{ range (split .Ports `,`)}}{{ range (slice (split . `:`) 1) }}{{ if eq (index (split . `->`) 1) `40000/tcp` }}{{ index (split . `->`) 0 }}{{end}}{{end}}{{end}} | {{.Names}} | {{.ID}} | {{.Ports}}\"",
"fieldSeparator": "|",
"description": "Select the service to attach to",
"useSingleResult": false,
}
},
],
"configurations": [
{
"name": "Attach to Service",
"type": "go",
"request": "attach",
"mode": "remote",
"remotePath": "/usr/src/chimera/",
"host": "127.0.0.1",
"port": "${input:findService}",
"trace": "log"
}
]
}
When running the service with replicas, the following menu gets created when launching the “Attach to Service” task:
Each time the launch task is executed, this menu appears and allows you to select one service to attach to. Multiple services can be attached to at a given time. Enjoy your service debugging!
