Use yq to parse, filter, and generate YAML

2022-Mar-16 • by David Norton

Many engineers have found it necessary to manipulate JSON while scripting, or just while poking around on the command line. An essential tool for that is jq - you may already be aware of it.

But what about YAML? It can be converted to JSON, so is there a similar tool for YAML? Yes! Two, actually. There is a Python-based tool named yq, and now a more portable (built with Go) tool also named yq, and they are both mostly compatible with the jq interface! Since I prefer portable binaries to Python packages, I will be focusing on the latter tool.

(Note: until quite recently, this version of yq did not use the same CLI command interface as jq. That, however, has changed as of version 4.18.1. This is also a good reminder that not everybody follows SemVer for breaking changes -- so please pin your dependencies whenever you can.)

Installing

On Mac:

brew install yq

Others: read the docs

Some examples

Consider the following multi-document YAML file:

#manifest.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: game-config
  labels:
    foo: bar
data:
  allowed: "true"
  enemies: aliens
  lives: "3"
---
apiVersion: v1
kind: Pod
metadata:
  name: api-test-pod
  labels:
    foo: baz
spec:
  containers:
    - name: test-container
      image: k8s.gcr.io/busybox
      command: [ "/bin/sh", "-c", "env" ]
      envFrom:
        - configMapRef:
            name: game-config
  restartPolicy: Never

Here are some examples of various commands you can run:

$ cat manifest.yaml | yq .metadata.name
game-config
---
api-test-pod

$ cat manifest.yaml | yq .metadata.labels
foo: bar
---
foo: baz

$ cat manifest.yaml | yq 'select(.metadata.labels.foo == "bar") | .metadata.name'
game-config

$ cat manifest.yaml | yq 'select(.metadata.labels.foo == "bar") | .metadata.labels' -o json
{
  "foo": "bar"
}

$ cat manifest.yaml | yq 'select(.kind == "Pod") | .metadata.namespace = "bar" | .metadata'
name: api-test-pod
labels:
  foo: baz
namespace: bar

I'm not going to do a better job than the docs -- I hope you found this a helpful introduction, and I invite you to give it a try and learn more!