Tags: #go
Package io provides basic interfaces to I/O primitives. Its primary job is to wrap existing implementations of such primitives, such as those in package os, into shared public interfaces that abstract the functionality, plus some other related primitives.
Because these interfaces and primitives wrap lower-level operations with various implementations, unless otherwise informed clients should not assume they are safe for parallel execution.
Example functionality: read section of a file into memory
package main
import (
"fmt"
"io"
"log"
"strings"
)
func main() {
r := strings.NewReader("some io.Reader stream to be read\n")
buf := make([]byte, 4)
if _, err := io.ReadFull(r, buf); err != nil {
log.Fatal(err)
}
fmt.Printf("%s\n", buf) // "some"
}
Package ioutil implements some I/O utility functions.
Example functionality: read entire file into memory
import "io/ioutil"
func main() {
dat, err := ioutil.ReadFile("/tmp/dat")
if err != nil {
panic(err)
}
}
Package bufio implements buffered I/O. It wraps an io.Reader or io.Writer object, creating another object (Reader or Writer) that also implements the interface but provides buffering and some help for textual I/O.
Example functionality: scan a file line-by-line, but grab the text from the 3rd line
func parseNodes(input io.Reader) (string, error) {
var txt string
count := 0
line := 3
scanner := bufio.NewScanner(input)
for scanner.Scan() {
count++
if count == line {
txt = scanner.Text()
}
if scanner.Text() == "END" {
break
}
}
if err := scanner.Err(); err != nil {
logger.Println("Scanner Error: ", err.Error())
return "", err
}
return txt, nil
}
The bufio package implements a buffered reader that may be useful both for its efficiency with many small reads and because of the additional reading methods it provides.
Package os provides a platform-independent interface to operating system functionality. The design is Unix-like, although the error handling is Go-like; failing calls return values of type error rather than error numbers. Often, more information is available within the error. For example, if a call that takes a file name fails, such as Open or Stat, the error will include the failing file name when printed and will be of type *PathError, which may be unpacked for more information.
Example functionality: opening a file with very granular control over how, and what parts, are read
file, err := os.Open("file.go") // read access
if err != nil {
log.Fatal(err)
}
// the file's data can then be read into a slice of bytes...
data := make([]byte, 100)
count, err := file.Read(data)
if err != nil {
log.Fatal(err)
}
fmt.Printf("read %d bytes: %q\n", count, data[:count])
Type os.File represents a file on the local system. It implements both io.Reader and io.Writer and, therefore, can be used in any streaming IO contexts.
The os package exposes three variables, os.Stdout, os.Stdin, and os.Stderr, that are of type *os.File to represent file handles for the OS’s standard output, input, and error respectively.
Function io.Copy() makes it easy to stream data from a source reader to a target writer. It abstracts out the for-loop pattern (we’ve seen so far) and properly handle io.EOF and byte counts.
io.WriteString provides the convenience of writing a string value into a specified writer.
Types io.PipeWriter and io.PipeReader model IO operations as in memory pipes. Data is written to the pipe’s writer-end and is read on the pipe’s reader-end using separate go routines.
Go supports buffered IO via package bufio which makes it easy to work with textual content. For example, we could use bufio.Reader.ReadString to read the content of a file line-by-line but using a specific delimeter like '\n'.
Package ioutil, a sub-package of io, offers several convenience functions for IO. For example, the function io/ioutil.ReadFile can load the content of a file into a []byte.
Package httputil provides HTTP utility functions and interfaces, some of which relate to IO and buffers. For example, httputil.BufferPool is an interface for getting and returning temporary byte slices for use by io.CopyBuffer (CopyBuffer is identical to Copy except that it stages through the provided buffer, if one is required, rather than allocating a temporary one. If buf is nil, one is allocated; otherwise if it has zero length, CopyBuffer panics). This can also be tied back into resources such as net/http/httputil.ReverseProxy which provides a BufferPool field such that it can help with copying HTTP response bodies in a more efficient way.
Examples grouped together on Play… https://play.golang.org/p/4gtvoCJENwr
reader := strings.NewReader("Clear is better than clever")
p := make([]byte, 4)
for {
n, err := reader.Read(p)
if err != nil {
if err == io.EOF {
fmt.Println(string(p[:n])) //should handle any remainding bytes.
break
}
fmt.Println(err)
os.Exit(1)
}
fmt.Println(string(p[:n]))
}
proverbs := []string{
"Channels orchestrate mutexes serialize",
"Cgo is not Go",
"Errors are values",
"Don't panic",
}
var writer bytes.Buffer
for _, p := range proverbs {
n, err := writer.Write([]byte(p))
if err != nil {
fmt.Println(err)
os.Exit(1)
}
if n != len(p) {
fmt.Println("failed to write data")
os.Exit(1)
}
writer.Write([]byte(", ")) // just so we can read the output a bit better
}
fmt.Println(writer.String())
Copy and ReadFilepverbs := new(bytes.Buffer)
pverbs.WriteString("Channels orchestrate mutexes serialize\n")
pverbs.WriteString("Cgo is not Go\n")
pverbs.WriteString("Errors are values\n")
pverbs.WriteString("Don't panic\n")
file, err := os.Create("./proverbs.txt")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer file.Close()
// copy from reader data into writer file
if _, err := io.Copy(file, pverbs); err != nil {
fmt.Println(err)
os.Exit(1)
}
dat, _ := ioutil.ReadFile("./proverbs.txt")
fmt.Print("\n", string(dat))
os.File, and write to os.Stdout using Copyfile1, err := os.Open("./proverbs.txt")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer file1.Close()
// Copy to Stdout is going to cause output to be immediately displayed (e.g. no need for fmt.Print style functions)
if _, err := io.Copy(os.Stdout, file1); err != nil {
fmt.Println(err)
os.Exit(1)
}