Build cross-platform executables in Go

If you have been working on the Go language, then you might probably know about the go build <filename> command which builds an executable file for the given filename's main function. This executable can then be run via the command line or just like any usual exe files.

But this build command generates an executable file which is only supported for the current machine's platform and architecture on which it runs. So if you run the command from a 64-bit Mac, it will generate a macOS-based executable, which can be only run on x86_64 MacOS machines.

Let's try this out

Create a go file helloworld.go with a very basic program -

package main
import "fmt"

func main() {
   fmt.Println("Hello World!!")
}

Go to the terminal and run

go build helloworld.go

This will generate the executable binary named helloworld in the same directory. To check the type of this file, run file helloworld , it will print below

➜ file helloword
helloword: Mach-O 64-bit executable x86_64

Since I build it from my Mac, it says it's a 64-bit Mac-O executable.

Where does the problem arise

We usually want to run these executables from our integration or production environments, where instances are Linux / ubuntu based, most of the time. If we have a go executable file, we don't need to have the Go installed on the system to run it, just like any executable.

If I copy this executable file from my local machine to the integration/production VMs and try to run the executable, it will throw an executable format error, as the executable file type was MacOS based but I tried to run it on Linux machines.

Solution

One approach to solve this is to copy the go program in the VM, install go and then run go build command from within that VM. But this is not usually recommended as it adds the additional overhead of installing Go and other supported packages inside the VMs whenever we spin up a new VM instance.

Another approach is to find out a way if we can build a Linux executable from our Mac itself so that we just copy that executable to our VM and run it.

Fortunately, we can. Go support cross-compilation! 🤟

Just run the below command and we have a go executable for the Linux platform

env GOOS=linux GOARCH=amd64 go build helloworld.go

GOOS - Operating system for which we want to build this file
GOARCH - The architecture of the OS for which we want to build this file

Once this command is succeeded, check the file type again using file command. It will show something like below

➜ file helloword
helloword: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked

ELF**(Executable and Linkable Format),** is a common standard file format for executable files.
LSB executable stands for Linux Standard Base executable.

With this, we just built a Linux executable file from our MacOS.

We can then copy this executable to your remote VM if required using the below scp command

➜ scp helloworld <IPAdressOfYourVM>:

Now, we can just ssh into the VM and run the executable

./helloworld

Issues with packages which does not support cross-compilation

There are some packages which do not support cross-compilation, and if you are trying to build your script(which uses those packages) for a different platform, the build will fail and it will throw an error. One such package is confluent-kafka-go(still an open issue 🙈) which does not support cross-compilation.

So in such cases, we can use docker to build it inside the Linux container itself instead of cross-compiling from MacOS.

Create a new Dockerfile with the below content in the same directory which has the go script and go.mod file

FROM golang:1.15

# create a build dir
RUN mkdir /build
WORKDIR /build

ENV CGO_ENABLED=0

# copy module dependencies
COPY go.mod .
COPY go.sum .

# copy the project itself
COPY . .

If you also want to install some packages in this image (example: git), then add that to the above Dockerfile at line 2. Example:

FROM golang:1.15

# Install git...
RUN apt-get install git

# create a build dir
RUN mkdir /build
..
..
..

Generate a docker image from the above Dockerfile using the below command

docker build -t go-cli-script .

Run the above-created image inside a container and open a bash shell in this container with which we can interact, using the below command

docker run -it go-cli-script /bin/bash

This command will open up the bash CLI from within the Linux container. Execute the below command in this CLI to generate the Linux executable.

go build helloworld.go

After performing the above step, a new linux executable would be added in the same directory inside the container.

Now, we want to copy this executable file from the container to the host machine,

To do this, run the below docker command in your local machine terminal-

docker cp <container id>:/build/helloworld <local machine directory where you want to copy>

The container id in the above command should be of the container we created using docker run command above. You can also get the container id by running docker ps -a command.

Check the copied file’s type in the local machine using file helloworld command, it will show LSB(Linux Standard Base) executable as expected.

If you found this article helpful, please comment and share this article so that it reaches others. 😃 You can subscribe to my newsletter to get an email notification on my latest posts. 💻

Let's connect 💫. You can follow me on

Neel Vikmani