« Back to Index

Go: TOML Examples with custom Marshaller

View original Gist on GitHub

Tags: #go #serialization

1. README.md

The following files demonstrate how the CLI will handle the input toml.

Here are the files:

• example.toml: it represents similar content to what fastly.toml holds and we expect the CLI to manage.
• main.go: a reduced test case go program that attempts to decode/encode the provided toml to see how it handles things by default.
• output: shows the output of the go program.

Summary

The TOML library will continue to produce more verbose output, and will overwrite the users toml with the verbose variation (only for fastly.toml, as that’s the only file that CLI manages).

Custom Marshaller

I’ve written a custom marshaller but there are some issues to keep in mind.

First, here’s the implementation of the custom marshal:

func (m Manifest) MarshalTOML() ([]byte, error) {
	var b bytes.Buffer

	b.WriteString("managed = " + m.Managed + "\n\n")
	b.WriteString("[testing.backends]\n")

	for k, v := range m.Testing.Backends {
		line := fmt.Sprintf("%s = { url = \"%s\" }\n", k, v.URL)
		b.WriteString(line)
	}

	return b.Bytes(), nil
}

Here are the issues:

1. The order of the backends are non-deterministic so they won’t necessarily match what the user entered originally.
2. This is a very simple implementation and so we don’t analyse the original syntax. This means if the user writes their toml over multiple lines, then the CLI will overwrite that with the more concise version defined in MarshalTOML.
3. The user is going to mostly copy/paste toml between fastly.toml and their environment files. Meaning: there will be a difference between the fastly.toml (which the CLI will overwrite the toml with more verbose toml syntax) and the environment files that the CLI doesn’t manage.

example.toml

managed = "by CLI"

[testing.backends]
foo = { url = "https://foo.com/"}
"bar.baz" = { url = "https://barbaz.com/"}
qux = { url = "https://qux.com/"}

main.go

package main

import (
	"fmt"
	"io"
	"log"
	"os"

	"github.com/pelletier/go-toml"
)

func main() {
	fpath := "example.toml"

    	// NOTE: trying both Decode/Encode and Unmarshal/Marshal to see if there were any differences (there aren't).

	md := decode(fpath)
	toml.NewEncoder(os.Stdout).Encode(md)

	mu := unmarshal(fpath)
	b, _ := toml.Marshal(mu)
	fmt.Println(string(b))
}

func decode(fpath string) Manifest {
	fmt.Println("\nDECODE")

	var m Manifest

	f, err := os.OpenFile(fpath, os.O_RDWR|os.O_CREATE, 0755)
	if err != nil {
		fmt.Println("decode: %w", err)
	}

	toml.NewDecoder(f).Decode(&m)

	fmt.Printf("\n%+v\n\n", m)

	return m
}

func unmarshal(fpath string) Manifest {
	fmt.Println("\nUNMARSHAL")

	var m Manifest

	f, err := os.OpenFile(fpath, os.O_RDWR|os.O_CREATE, 0755)
	if err != nil {
		fmt.Println("unmarshal: %w", err)
	}

	b, err := io.ReadAll(f)
	if err != nil {
		log.Fatal(err)
	}

	toml.Unmarshal(b, &m)

	fmt.Printf("\n%+v\n\n", m)

	return m
}

type Manifest struct {
	Managed string  `toml:"managed"`
	Testing Testing `toml:"testing"`
}

type Testing struct {
	Backends map[string]Backend `toml:"backends"`
}

type Backend struct {
	URL string `toml:"url"`
}

output

DECODE

{Managed:by CLI Testing:{Backends:map[bar.baz:{URL:https://barbaz.com/} foo:{URL:https://foo.com/} qux:{URL:https://qux.com/}]}}

managed = "by CLI"

[testing]

  [testing.backends]

    [testing.backends."bar.baz"]
      url = "https://barbaz.com/"

    [testing.backends.foo]
      url = "https://foo.com/"

    [testing.backends.qux]
      url = "https://qux.com/"

UNMARSHAL

{Managed:by CLI Testing:{Backends:map[bar.baz:{URL:https://barbaz.com/} foo:{URL:https://foo.com/} qux:{URL:https://qux.com/}]}}

managed = "by CLI"

[testing]

  [testing.backends]

    [testing.backends."bar.baz"]
      url = "https://barbaz.com/"

    [testing.backends.foo]
      url = "https://foo.com/"

    [testing.backends.qux]
      url = "https://qux.com/"