Json in Golang

Monday, March 23, 2020 • 8 minutes to read

TutorialGolang2020golangjson

When you start a project which interacts with the outer world, it requires data exchange. To make a project successful this interaction must be simple and efficient.

What is Data Exchange?

Data exchange is the process of taking data structured under a source schema and transforming it into data structured under a target schema, so that the target data is an accurate representation of the source data. - Wikipedia

In simple terms, convert the data in the format which the receiver understands.

There are many data exchange formats like XML, JSON, HTML, CSV etc. Among all, JSON has become ubiquitous for web applications.

What is JSON?

JSON (JavaScript Object Notation) is a lightweight format that is used for data interchanging. It is based on a subset of Javascript language. An object is an unordered set of name/value pairs.

An Example of JSON

1{
2  "title": "How to use JSON in golang?",
3  "type": "Tutorial",
4  "publisher": "codesource.io",
5  "tags": ["golang", "json"],
6  "published": true
7}

Now, we know what is JSON. It is time to use it with golang. In this tutorial, we will explore different golang types like string, int, struct, array, slice, maps.

Parse JSON in Golang ⚙️

JSON is JavaScript Object Notation and golang can't use it in its original format. For this, golang has provided an encoding/json package in the standard library.

In golang, we use struct to represent json.

For example: In golang

1type Employer struct {
2    Name string
3    Employee []int
4}

In JSON

1{
2  "name": "string",
3  "employee": []
4}

We will use encode/json package to convert json into struct and struct into json. For this we will use 2 functions:

  1. Marshal (Encode) : Convert golang struct into json format.
  2. Unmarshal (Decode): Convert json into golang struct

It is more like mapping the parameters than conversion. The default Go types for decoding and encoding JSON are

  • bool for JSON boolean
  • string for JSON string
  • int/float64 for JSON number
  • nil for JSON null

Encode (Marshal) 👨‍💻

To encode JSON data we use the Marshal function.

1func Marshal(v interface{}) ([]byte, error)

Marshal function accepts an empty interface and returns an array of byte and error message.

 1package main
 2
 3import (
 4    "encoding/json"
 5    "fmt"
 6    "log"
 7)
 8
 9func main() {
10    type Person struct {
11        Name     string
12        Age      int64
13        Location string
14    }
15
16    person := Person{
17        "Jon", 27, "London",
18    }
19
20    // encode into JSON
21    b, err := json.Marshal(person)
22
23    if err != nil {
24        log.Fatalf("Unable to encode")
25    }
26
27    // Marshal returns []byte
28
29    fmt.Println(string(b))
30}

Try It

Output

1{ "Name": "Jon", "Age": 27, "Location": "London" }

We are encoding Person struct in JSON format. First, create a new object of Person as person. Then, encode the person in JSON using json.Marshal. If everything goes well, then the err will be nil and b is the representation of person in []byte format.

Only data structures that can be represented as valid JSON will be encoded:

  • JSON objects only support strings as keys; to encode a Go map type it must be of the form map[string]T (where T is any Go type supported by the json package).
  • Only the exported fields (those that begin with an uppercase letter) of the struct can be encoded in JSON.
  • Cyclic data structures are not supported; they will cause Marshal to go into an infinite loop.
  • Pointers will be encoded as the values they point to (or ‘null’ if the pointer is nil).

In golang Uppercase represent that a field is exported or public.

Take a look at the Person struct,

1type Person struct {
2        Name     string
3        Age      int64
4        Location string
5    }

Change the Location field to lowercase location.

1type Person struct {
2        Name     string
3        Age      int64
4        location string
5    }

Try it

Output

1{ "Name": "Jon", "Age": 27 }

To map the struct field to the json tag.

For example, you have struct field as Name but you want to map it as firstName in json. To do this, you can tag the struct field.

Syntax

1FieldName type `json:"tagname"`

⚠️ Don't give any space between json:"tagname", else it will throw an error. struct field tag json: "firstName" not compatible with reflect.StructTag.Get: bad syntax for struct tag value

Tag the Name field as firstName.

1type Person struct {
2        Name     string `json:"firstName"`
3        Age      int64
4        Location string
5    }

Try it

Output

1{ "firstName": "Jon", "Age": 27, "Location": "London" }

Omitempty

We have a special json tag as omitempty. If a field is set as omitempty then it will not encode that field to json if it is empty.

For example, set Location field as omitempty.

 1type Person struct {
 2        Name     string `json:"firstName"`
 3        Age      int64
 4        Location string `json:"location, omitempty"`
 5    }
 6
 7person := Person{
 8		Name: "Jon",
 9		Age: 27,
10	}

Try it

Output

{"firstName":"Jon","Age":27}

Decode (Unmarshal) 👨‍💻

To decode JSON data we use the Unmarshal function.

1func Unmarshal(data []byte, v interface{}) error

Unmarshal accepts an array of byte and an interface and returns the error message. This interface is the struct to which the JSON decode.

 1package main
 2
 3import (
 4    "encoding/json"
 5    "fmt"
 6    "log"
 7)
 8
 9func main() {
10    type Person struct {
11        Name     string
12        Age      int64
13        Location string
14    }
15
16    j := []byte(`{"name":"Jon","age":27,"location":"London"}`)
17
18    var p Person
19
20    err := json.Unmarshal(j, &p)
21
22    if err != nil {
23        log.Fatalf("Unable to decode the json")
24    }
25
26    fmt.Println(p)
27}

Try it

Output

{Jon 27 London}

To store the decoded data in the struct, Unmarshal will look the json case-insensitive keys in the exported struct fields. As in the above case, name is in lowercase and is mapped to the exported Name field.

Similar, to encoding you can add json tag to the struct field. For example:

Tag the Name field with json:"firstName". Now, it will map firstName to the Name.

1type Person struct {
2        Name     string `json:"firstName"`
3        Age      int64
4        Location string
5    }

Try it


Decoding the arbitrary JSON 👨‍💻

In the above examples, we knew the JSON structure and we mapped it to the struct.

What if you don't know the JSON structure? 🤔

In the json all the keys must be string. It means we can use the map type for arbitrary data.

The encoding/json package uses

  • map[string]interface{} to store the arbitrary JSON objects
  • []interface{} to store the arbitrary JSON arrays.

In map[string]interface{} the keys are string and values are interface{}. It is an empty interface. The interface{} (empty interface) type describes an interface with zero methods. In short, it can accept all the types.

The default Go types are:

  • bool for JSON booleans,
  • float64 for JSON numbers,
  • string for JSON strings, and
  • nil for JSON null.

Consider the JSON object as

1{
2  "name": "Jon",
3  "location": {
4    "country": "England",
5    "city": "London"
6  },
7  "hobbies": ["photography", "writing"]
8}

The Unmarshal function will parse it into a map whose keys are string and values are empty interface.

 1map[string]interface{} {
 2    "Name": "Jon",
 3    "Location": map[string]interface{} {
 4        "Country": "England",
 5        "City": "London",
 6    },
 7    "Hobbies": []interface{} {
 8        "photography",
 9        "writing",
10    },
11}

All the values are of type interface{}. To access the underlying type of interface, we have to use type assertion.

For example:

1var name interface{}
2name = "Jon"
3
4result := name.(string)
5fmt.PrintF("Type is %T, value is %s", result, result")

Try it

 1package main
 2
 3import (
 4    "encoding/json"
 5    "fmt"
 6    "log"
 7)
 8
 9func main() {
10
11    jsonObj := `{
12                "name": "Jon",
13            "age" : 27,
14                "location": {
15                        "country": "England",
16                  "city": "London"
17                },
18               "hobbies": [ "photography", "writing"]
19        }`
20
21    var person map[string]interface{}
22
23    err := json.Unmarshal([]byte(jsonObj), &person)
24
25    if err != nil {
26        log.Fatalf("Unable to encode")
27    }
28
29    for k, v := range person {
30        switch v := v.(type) {
31        case string:
32            fmt.Println(k, v)
33        case float64:
34            fmt.Println(k, v)
35        case map[string]interface{}:
36            for i, ival := range v {
37                fmt.Println(i, ival)
38            }
39        case []interface{}:
40            for i, ival := range v {
41                fmt.Println(i, ival)
42            }
43        default:
44            fmt.Println(k, v)
45        }
46    }
47}

Try it

Output

country England
city London
0 photography
1 writing
name Jon
age 27

Map is not indexed, so the order will always be different.


Streaming Encoders and Decoders

The encoding/json provides Decoder and Encoder types to support the common operation of reading and writing streams of JSON data.

1func NewDecoder(r io.Reader) *Decoder
2func NewEncoder(w io.Writer) *Encoder

In the below example,

  • It will read the stream of JSON data from an io.Reader
  • removes the Age field from each object
  • writes the objects to an io.Writer
 1package main
 2
 3import (
 4	"encoding/json"
 5	"log"
 6	"os"
 7	"strings"
 8)
 9
10func main() {
11
12	jsonStream := `{"Name":"Jon", "Age":27, "Location": "London"}
13		{"Name":"Bruce", "Age":35, "Location": "Gotham"}`
14
15	reader := strings.NewReader(jsonStream)
16	writer := os.Stdout
17
18	dec := json.NewDecoder(reader)
19	enc := json.NewEncoder(writer)
20
21	for {
22		var v map[string]interface{}
23		if err := dec.Decode(&v); err != nil {
24			log.Println(err)
25			return
26		}
27		for k := range v {
28			if k == "Age" {
29				delete(v, k)
30			}
31		}
32		if err := enc.Encode(&v); err != nil {
33			log.Println(err)
34		}
35	}
36}

Try it

Output

{"Location":"London","Name":"Jon"}
{"Location":"Gotham","Name":"Bruce"}
2009/11/10 23:00:00 EOF

Code Walkthrough

1reader := strings.NewReader(jsonStream)

Create a type of io.Reader using the strings package.

1dec := json.NewDecoder(reader)
2enc := json.NewEncoder(writer)

Create a new decoder which reads the data from the reader. Create a new encoder which writes the data to the writer.

Rest of the code is self-explanatory. Decode the arbitrary json in map[string]interface{}.


Conclusion

In this tutorial, we explored the json package. There are many other amazing functions provided by the encoding/json. Like MarshalIndent, it will indent the encoded json. Please check out the official website to learn more.


TutorialGolang2020golangjson

PreviousHow to Send Email in Golang

NextUnderstand Quicksort the easy way