How to Efficiently Read a File Line by Line in GO

1. Package and Necessary Imports
2. Read Line by Line in Go
3. Complete Code to Read a File Line by Line in Go
4. Configure Scanner Behavior in Go

File I/O, specifically reading and writing to files is a fairly common functionality of programming languages. Oftentimes, you need to read a file line by line. GO provides the bufio package which basically implements buffered I/O. bufio provides helpful types and functions particularly for reading and writing to text files.

Package and Necessary Imports

package main

import (
	"bufio"
	"fmt"
	"log"
	"os"
)

We will need several imports from the GO standard library:-

• os - for platform-independent interface to OS functionality
• fmt - implementing formatted I/O functionality
• log - standard logging package
• bufio - supports buffered I/O

Read Line by Line in Go

Before reading the file, it first needs to be opened using the os.Open() function which returns a pointer type to the file. The test.txt file shown in the code snippet needs to exist beforehand (put the path to where your file is).

The bufio.NewScanner(file) function returns a scanner type that has functions that support reading through the file.

To read through line by line on the file, we need to use two methods on the new Scanner - Scan, which advances the scanner to the new token (in this case the new line), and Text(or Byte) which reads the most recent token that was generated when Scan was called.

If any errors were encountered when reading the file, they can be handled by a call to Err() method on the new Scanner which will return the first non End of File error encountered by the Scanner.

func main(){
    // open the file
    file, err := os.Open("test.txt")

    //handle errors while opening
    if err != nil {
        log.Fatalf("Error when opening file: %s", err)
    }


    fileScanner := bufio.NewScanner(file)

    // read line by line
    for fileScanner.Scan() {
        fmt.Println(fileScanner.Text())
    }
    // handle first encountered error while reading
    if err := fileScanner.Err(); err != nil {
        log.Fatalf("Error while reading file: %s", err)
    }

    file.Close()
}

Complete Code to Read a File Line by Line in Go

The complete code in main.go to read a file line by line is provided below.

You need to type:

    $ go run main.go

to run the program in the terminal.

package main

import (
	"bufio"
	"fmt"
	"log"
	"os"
)

func main() {
	// open the file
	file, err := os.Open("test.txt")

	//handle errors while opening
	if err != nil {
		log.Fatalf("Error when opening file: %s", err)
	}

	fileScanner := bufio.NewScanner(file)

	// read line by line
	for fileScanner.Scan() {
		fmt.Println(fileScanner.Text())
	}
	// handle first encountered error while reading
	if err := fileScanner.Err(); err != nil {
		log.Fatalf("Error while reading file: %s", err)
	}

	file.Close()
}

Configure Scanner Behavior in Go

The Scanner type has a Split function which accepts a SplitFunc function to determine how the scanner is going to split the given byte slice. The default SplitFunc is the ScanLines which returns each line of the text, stripped of the trailing end-of-line marker.

For example, we could split using words as shown in the code snippet below,

scanner.Split(bufio.ScanWords)  //configure how the scanner behaves