« Back to Index

Windows: Terminal Development

View original Gist on GitHub

Tags: #terminals #windows #os #dev #nvim

0. Terminals for Windows.md

The following notes are based around a CLI tool I had built in Go that would read a configuration file containing a ‘build script’ and would open a subshell to run the script.

I had reports from Windows users that it wasn’t working. I was using cmd.exe as the Terminal (as it’s the default for Windows, so we know it’s always available), but it doesn’t support the POSIX command substitution syntax (e.g. $()).

The simplest solution ended up being to avoid command substitution (which in my case was possible as the build script was doing $(npm bin)/webpack and that could be changed to npm exec webpack), but at the time of this investigation I was not aware of npm exec (or npx for that matter) and so I was looking at different terminals for Windows to understand which of them supported ‘command substitution’.

I inevitably stumbled into a nightmare.

Here are few terminals available (non-exhaustive list)…

  1. Command Prompt (Cmd.exe)
  2. Cygwin
  3. Git for Windows (e.g. provides a BASH emulation used to run Git from the command line)
  4. PowerShell
  5. Windows Subsystem for Linux (WSL)
  6. Windows Terminal

NOTE: Windows Terminal looks to be a nice terminal interface, but the shell itself appears to just be PowerShell. So it might not be worth testing against if you already test your code against PowerShell.

PowerShell

I tried to switch to using powershell.exe (which although not installed by default, is the next simplest shell to get installed for Windows) and I discovered it doesn’t support the ‘logical AND’ operator &&. This means you have to use a more arcane -and syntax, but this would make my build script incompatible with other shells that are POSIX compatible.

In an example project we could have configured our CLI code to pattern match && and then, when on a Windows OS, replaced it with -and but I then discovered PowerShell fails to execute a binary using the syntax $(npm bin)/webpack (I would hit this issue again later when testing the new Windows Terminal).

To avoid && we an do the following in Bash bin=$(npm bin); $bin/webpack but in PowerShell you can’t assign the result of a command substitution to a variable.

Cmd.exe

Interestingly, cmd.exe does support && but as we know, it fails to support $(). There is an alternative syntax but I couldn’t get it to work (even with a simple example), let alone something more complex like:

for /F "tokens=*" %n IN ('npm bin') DO @(%n\webpack)

But this is a very confusing syntax. Also, I’m not sure how to ‘chain’ the command.

Having to logic hop around this sort of code would be a project maintenance/sustainability concern (if we were to go to these lengths to support building on Windows).

It also means to ensure our scripts stays compatible with the majority of users on actual POSIX/Bash shells we’d have to have the CLI parse the build script and identify any $() usage and replace it with the above more complex syntax, which would be a very brittle solution (especially if there was nested usage of $()).

Cygwin

I didn’t test with Cygwin. I’ve used it in the past for testing but it is a specialised tool that provides an isolated environment to work within and so that could cause problems if there are expectations for the CLI to be able to access files outside of that environment.

Also, from my understanding Cygwin is less commonly used nowadays and so it’s unlikely we would want to require users to install it just to use the CLI. Ideally a tool that doesn’t require an isolated environment would be better.

Git for Windows

I installed Git for Windows but found it also created its own isolated environment, and I also couldn’t figure out how to share files from my host Windows machine to it.

Windows Subsystem for Linux (WSL)

Installing WSL requires Windows 10+, which might exclude some Window developers (looks like Windows 10 was released in 2015 so maybe users on an earlier version is not a concern). WSL also sets up an isolated environment but what is interesting is that it also (partially at least) integrates with cmd.exe and PowerShell (e.g. you can pass commands to be executed to the isolated environment from the host environment).

This integration is not a panacea, as although a simple execution like wsl ls -a (which prints all files from my host Windows machine) works fine, when I try to execute something like wsl ls $(npm bin) it fails to access the directory. This means a more complex mounting of files into the WSL is necessary and again highlights the ‘isolated environment’ concern I had with Cygwin and Git for Windows.

Windows Terminal

When installing the new Windows Terminal, it appears to be an enhanced PowerShell (it actually shows as ‘PowerShell’ in the terminal), as it supports && as well as other posix/Bash flavoured syntax. Although executing ls $(npm bin) worked (unlike WSL), the moment I tried $(npm bin)/webpack to execute the binary within the directory, the command execution failed because of some unsupported syntax (i.e. the forward slash was not recognised). Meaning if we have to modify our scripts, then it isn’t going to be compatible with other shells. This error also occurred with the standard PowerShell.

Makefile

Also worth noting that my project’s Makefile had a line like the following:

GO_FILES = $(shell find cmd pkg -type f -name '*.go')

This didn’t work on Windows as there is no find command available. So I had to fix this like so:

ifeq ($(OS), Windows_NT)
	SHELL = cmd.exe
    .SHELLFLAGS = /c
	GO_FILES = $(shell where /r pkg *.go)
	GO_FILES += $(shell where /r cmd *.go)
else
	GO_FILES = $(shell find cmd pkg -type f -name '*.go')
endif

Additionally, our CLI program uses cmd.exe but the CI environment where we run our integration tests was running PowerShell!

This means the above Makefile change would break our CI as some of the POSIX shell code was working fine in PowerShell, and then we suddenly changed things to be cmd.exe (e.g. I had a variable that worked with PowerShell but failed in cmd.exe: ./{cmd,pkg}/... and so I had to change it to ./... to avoid issues).

1. Command Prompt.md

C:\Users\markm> cmd.exe /C "echo 123 && echo 456"
123
456	

C:\Users\markm> mkdir go
C:\Users\markm> type nul > main.go
C:\Users\markm> dir
C:\Users\markm> Xcopy <src> <dest> /E/H/C/I

2. Set PATH.md

set PATH "%PATH%;C:\NewDirectory"

3. Neovim.md