Go: development tools as dependencies

Tags: #go #dependencies

README.md

There are multiple ways to deal with non-application dependencies (i.e. “tools” that your project needs).

go tool
- caveats and issues
- multiple module files
tools.go
go run

go tool

As of Go 1.24 (Feb 2025)

To add a new tool:

go get -tool golang.org/x/lint/golint
go get -tool github.com/mgechev/revive@latest

To run the tool:

go tool golint -h
go tool golang.org/x/lint/golint -h # in case of naming overlap

To see a list of all tools:

go tool

To update all tools:

go get -u tool

If you check the go.mod you’ll see a new tool syntax:

module testing-tools

go 1.23.4

tool (
    github.com/mgechev/revive
    golang.org/x/lint/golint
)

caveats and issues

Now, there is a problem (sort of), which is that you’ll see a bunch of indirect dependencies showing up in the go.mod.

This is because these are the dependencies that your “tools” need.

I’m less concerned about that as a side-effect of using the new go tools feature, but I appreciate it’s not ideal.

My concern being: it’s more mental overhead.

You don’t know if these indirect dependencies are transient dependencies used by your application dependencies, or if they’re dependencies for the “tools” you’ve installed.

The reason I’m not usually that fussed by this is because I only really care about the “direct” dependencies, and those are always clear because they don’t have // indirect following them.

So the following instructions are only relevant if you really care about this.

multiple module files

There is another option on the table that we can use, and it doesn’t appear to be too much additional maintenance or mental overhead, which is great. But it does have a downside (see the IMPORTANT note at the end of this section).

Essentially, the approach is to have a separate modfile for tools.

It means we’d have multiple files now, like this…

go.mod
go.sum
tools.mod
tools.sum

[!NOTE] If you give the tools.mod a unique module name, let’s say go.mod uses github.com/example/foo, and so you make tools.mod use github.com/example/foo/tools then be aware that the use of go mod is going to make your tools.mod think it needs the module from go.mod and it’ll add it as a dependency (this makes things weird in special cases), so it might be worth making the module name the same between go.mod and tools.mod.

To install a new tool:

# instead of...
go get -tool github.com/mgechev/revive

# we do...
go get -modfile=tools.mod -tool github.com/mgechev/revive

[!TIP] To remove a tool you can do the above but set the version to @none.

And if we want to use that tool we have to make sure to specify the modfile:

$ go tool revive --version
go: no such tool "revive"

$ go tool -modfile=tools.mod revive --version
version 1.7.0

Having to specify the -modfile flag isn’t a big issue for me as I typically have go tool abstracted inside various Makefile targets, so I’m only ever calling a Makefile target (or in the case of stringer have it codified in the go generate directive in the code itself).

As far as updating tools, you can either do it a dependency at a time or all of them at once:

# instead of...
go get -u -tool github.com/mgechev/revive@latest
go get -u tool

# we do...
go get -u -modfile=tools.mod -tool github.com/mgechev/revive@latest
go get -u -modfile=tools.mod tool

Same for listing the installed tools:

# instead of...
go tool

# we do...
go tool -modfile=tools.mod

[!TIP] Can also try go list -modfile=tools.mod tool

To verify the integrity of the tool dependencies:

go mod verify -modfile=tools.mod

Here’s an associated Makefile:

.PHONY: deps-app-update
deps-app-update: ## Update all application dependencies
	go get -u -t ./...
	go mod tidy
	if [ -d "vendor" ]; then go mod vendor; fi
	
.PHONY: deps-outdated
deps-outdated:  ## Lists direct dependencies that have a newer version available
	@go list -u -m -json all | go tool -modfile=tools.mod go-mod-outdated -update -direct
	
TOOLS = \
	cuelang.org/go/cmd/cue \
	github.com/client9/misspell/cmd/misspell \
	github.com/go-delve/delve/cmd/dlv \
	github.com/mgechev/revive \
	github.com/psampaz/go-mod-outdated \
	github.com/stealthrocket/wasi-go/cmd/wasirun \
	github.com/stern/stern \
	github.com/tetratelabs/wazero/cmd/wazero \
	golang.org/x/lint/golint \
	golang.org/x/tools/cmd/stringer \
	golang.org/x/tools/go/analysis/passes/nilness/cmd/nilness \
	golang.org/x/vuln/cmd/govulncheck \
	honnef.co/go/tools/cmd/staticcheck \
	mvdan.cc/gofumpt \

.PHONY: tools
tools:
	@$(foreach tool,$(TOOLS), \
		if ! go tool -modfile=tools.mod | grep "$(tool)" >/dev/null; then \
			go get -modfile=tools.mod -tool "$(tool)"@latest; \
		fi; \
	)

.PHONY: tools-update
tools-update:
	go get -u -modfile=tools.mod tool
	go mod tidy -modfile=tools.mod

[!IMPORTANT] This approach keeps the main go.mod and go.sum clean of any tool dependencies, but not the other way around. So the tools.mod and tools.sum will ultimately contain all the dependencies from the main go.mod (that is a side-effect of running go mod tidy -modfile=tools.mod as go mod always consults the main go.mod, hence all of its dependencies end up in your tools.mod and tools.sum).

This is unavoidable. There is no way to get around it (trust me, I’ve tried 😅).

Now, this isn’t the end of the world as the tools directive is still at the top of the tools.mod and is very clear as to what “tools” are installed, but yeah, you’ll also see a bunch of require directives (related to your main Go project) as well, unfortunately.

One thing you could do, is only run the go get -u -modfile=tools.mod tool command, which would keep your tools.mod clean, and would only update tools.sum with the relevant updated dependencies. The problem with that is the old dependencies aren’t cleaned out. e.g. if you updated tool “foo” from version 1.0 to 2.0 then both versions appear in your tools.sum (this is why we have go mod tidy to ensure only 2.0 is present in the tools.sum). So one approach would simple be to manually clean up the go.sum everytime after running go get -u -modfile=tools.mod tool – it’s not that difficult as you just look for the new tool version added and remove the old one, but it’s a manual process and that sucks).

UPDATE Aug 2025:
I was having issues with importing a public package (e.g. pkg/package_name) from within my own module. It was related to compiling a Protobuf schema into a .go stub file that I wanted to import and use within my own module’s code. And I found the following approach helped me to avoid an issue where go mod tidy wanted me to go get my own package (which didn’t make any sense to me). The way I avoid that was to copy my tools.mod and tools.sum into a temp directory and then cd there and run go mod tidy and it appeared to work fine. What’s weird is I’m pretty sure I tried this approach before and it didn’t work as far as keeping out my main application’s dependencies from the tools dependencies. So this might be the way to go.

.PHONY: tools-update
tools-update: ## Update dev tools
	@$(foreach tool,$(TOOLS), \
		echo "updating $(tool)"; \
		go get -u -modfile=tools.mod -tool "$(tool)"@latest; \
	)
	go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
	@echo "Tidying tool dependencies..."
	@mkdir -p .tmp_tools
	@cp tools.mod .tmp_tools/go.mod
	@if [ -f tools.sum ]; then cp tools.sum .tmp_tools/go.sum; fi
	@cd .tmp_tools && go mod tidy
	@mv .tmp_tools/go.mod tools.mod
	@if [ -f .tmp_tools/go.sum ]; then mv .tmp_tools/go.sum tools.sum; fi
	@rm -rf .tmp_tools

tools.go

[!NOTE] For more details on code generation in a general sense, refer to:
https://gist.github.com/Integralist/8f39eb897316e1cbeaf9eff8326cfa59

The following file internal/tools/tools.go uses a build tag to avoid the dependencies being compiled into your application binary…

//go:build tools

// Package tools manages go-based tools that are used to develop in this repo.
package tools

import (
	_ "github.com/nbio/cart"
	_ "github.com/nbio/slugger"
	_ "github.com/psampaz/go-mod-outdated"
	_ "github.com/stealthrocket/wasi-go/cmd/wasirun"
	_ "github.com/tetratelabs/wazero/cmd/wazero"
	_ "golang.org/x/lint/golint"
	_ "golang.org/x/tools/cmd/stringer"
	_ "golang.org/x/vuln/cmd/govulncheck"
)

//go:generate go install github.com/nbio/cart
//go:generate go install github.com/nbio/slugger
//go:generate go install github.com/psampaz/go-mod-outdated
//go:generate go install github.com/stealthrocket/wasi-go/cmd/wasirun
//go:generate go install github.com/tetratelabs/wazero/cmd/wazero
//go:generate go install golang.org/x/lint/golint
//go:generate go install golang.org/x/vuln/cmd/govulncheck
//go:generate go install golang.org/x/tools/cmd/stringer

Notice the go:generate comments? Yup, we invoke them like so (notice the -tags flag):

tools: internal/tools/tools.go
	go generate -v -x -tags tools ./internal/tools/...

go run

An alternative to this approach is to use go run directly, which downloads tools to a cache but doesn’t install them and yet still gives you explicit versioning consistency across developer’s machines…

//go:generate go run golang.org/x/tools/cmd/stringer@v0.25.0 -type=Scope -linecomment

I then invoke go generation with:

.PHONY: go-gen
go-gen: ## Invoke go generate
	@# The `-x` flag prints the shell commands that `go generate` runs.
	go generate -v -x ./mustang/status/...

[!TIP] If you’re developing whilst offline, then one advantage the tools.go pattern has is that it works whilst offline because the tool is explicitly installed. But to work around that with go run you can set export GOPROXY=direct and as long as you have the module in your local cache you’ll be able to use it.