GitBook: [v2.x] 10 pages modified

removed repeated word

.gitbook/assets/convert.md

@@ -0,0 +1,60 @@
+## Yaml to Json
+To convert output to json, use the --tojson (or -j) flag. This is supported by all commands.
+
+Each matching yaml node will be converted to json and printed out on a separate line.
+
+Given a sample.yaml file of:
+```yaml
+b:
+  c: 2
+```
+then
+```bash
+yq r -j sample.yaml
+```
+
+will output
+```json
+{"b":{"c":2}}
+```
+
+Given a sample.yaml file of:
+```yaml
+bob:
+  c: 2
+bab:
+  c: 5
+```
+then
+```bash
+yq r -j sample.yaml b*
+```
+
+will output
+```json
+{"c":2}
+{"c":5}
+```
+
+## Json to Yaml
+To read in json, just pass in a json file instead of yaml, it will just work :)
+
+e.g given a json file
+
+```json
+{"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}
+```
+then
+```bash
+yq r sample.json
+```
+will output
+```yaml
+a: Easy! as one two three
+b:
+  c: 2
+  d:
+  - 3
+  - 4
+```
+

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,48 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+version of yq: 
+operating system:
+
+**Input Yaml**
+Concise yaml document(s) (as simple as possible to show the bug)
+data1.yml:
+```yaml
+this: should really work
+```
+
+data2.yml:
+```yaml
+but: it strangely didn't
+```
+
+**Command**
+The command you ran:
+```
+yq merge data1.yml data2.yml
+```
+
+**Actual behavior**
+
+```yaml
+cat: meow
+```
+
+**Expected behavior**
+
+```yaml
+this: should really work
+but: it strangely didn't
+```
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,36 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+If we have data1.yml like:
+
+```yaml
+country: Australia
+```
+
+And we run a command:
+
+```bash
+yq predictWeather data1.yml
+```
+
+it could output
+
+```yaml
+temp: 32
+```
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/workflows/go.yml

@@ -0,0 +1,34 @@
+name: Build
+on: [push]
+jobs:
+
+  build:
+    name: Build
+    runs-on: ubuntu-latest
+    steps:
+
+    - name: Set up Go 1.14
+      uses: actions/setup-go@v1
+      with:
+        go-version: 1.14
+      id: go
+
+    - name: Check out code into the Go module directory
+      uses: actions/checkout@v2
+
+    - name: Get dependencies
+      run: |
+        go get -v -t -d ./...
+        if [ -f Gopkg.toml ]; then
+            curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
+            dep ensure
+        fi
+
+    - name: Download deps
+      run: |
+        export PATH=${PATH}:`go env GOPATH`/bin
+        scripts/devtools.sh
+    - name: Build
+      run: |
+        export PATH=${PATH}:`go env GOPATH`/bin
+        make local build

.travis.yml

@@ -1,6 +0,0 @@
-language: go
-go:
-  - 1.13.x
-script:
-  - scripts/devtools.sh
-  - make local build

CODE_OF_CONDUCT.md

@@ -0,0 +1,76 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+ advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+ address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+ professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at mikefarah@gmail.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq

CONTRIBUTING.md

@@ -0,0 +1,8 @@
+1. Install (golang)[https://golang.org/]
+1. Run  `scripts/devtools.sh` to install the required devtools
+2. Run `make [local] vendor` to install the vendor dependencies
+2. Run `make [local] test` to ensure you can run the existing tests
+3. Write unit tests - (see existing examples). Changes will not be accepted without corresponding unit tests.
+4. Make the code changes.
+5. `make [local] test` to lint code and run tests
+6. Profit! ok no profit, but raise a PR and get kudos :)

Dockerfile

@@ -1,4 +1,4 @@
-FROM golang:1.13 as builder
+FROM golang:1.14 as builder
 
 WORKDIR /go/src/mikefarah/yq
 

Dockerfile.dev

@@ -1,4 +1,4 @@
-FROM golang:1.13
+FROM golang:1.14
 
 COPY scripts/devtools.sh /opt/devtools.sh
 
@@ -9,15 +9,15 @@ RUN set -e -x \
 RUN set -ex \
     && buildDeps=' \
         build-essential \
-        python-dev \
+        python3-dev \
     ' \
     && apt-get update && apt-get install -y --no-install-recommends \
         $buildDeps \
-        python2.7 \
-        python-setuptools \
-        python-wheel \
-        python-pip \
-    && pip install --upgrade \
+        python3 \
+        python3-setuptools \
+        python3-wheel \
+        python3-pip \
+    && pip3 install --upgrade \
         pip \
         'Markdown>=2.6.9' \
         'mkdocs>=0.16.3' \

README.md

@@ -1,125 +1,36 @@
-# yq
+---
+description: yq is a lightweight and portable command-line YAML processor
+---
 
-[![Build Status](https://api.travis-ci.com/mikefarah/yq.svg?branch=master)](https://travis-ci.com/mikefarah/yq/)  ![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) ![Github Releases (by Release)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) ![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq)
+# Yq
 
+### Install[¶]() <a id="install"></a>
 
-a lightweight and portable command-line YAML processor
+On MacOS:
 
-The aim of the project is to be the [jq](https://github.com/stedolan/jq) or sed of yaml files.
-
-## New version!
-V3 is officially out - if you've been using v2 and want/need to upgrade, checkout the [upgrade guide](https://mikefarah.gitbook.io/yq/upgrading-from-v2).
-
-## Install
-
-### On MacOS:
-```
+```text
 brew install yq
 ```
-### On Ubuntu and other Linux distros supporting `snap` packages:
-```
+
+On Ubuntu and other Linux distros supporting `snap` packages:
+
+```text
 snap install yq
 ```
 
-#### Snap notes
-`yq` installs with with [_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, this means it doesn't have direct access to root files. To read root files you can:
+On Ubuntu 16.04 or higher from Debian package:
 
-```
-sudo cat /etc/myfile | yq r - a.path
-```
-
-And to write to a root file you can either use [sponge](https://linux.die.net/man/1/sponge):
-```
-sudo cat /etc/myfile | yq w - a.path value | sudo sponge /etc/myfile
-```
-or write to a temporary file:
-```
-sudo cat /etc/myfile | yq w - a.path value | sudo tee /etc/myfile.tmp
-sudo mv /etc/myfile.tmp /etc/myfile
-rm /etc/myfile.tmp
-```
-
-### On Ubuntu 16.04 or higher from Debian package:
-```
+```text
 sudo add-apt-repository ppa:rmescandon/yq
 sudo apt update
 sudo apt install yq -y
 ```
-### or, [Download latest binary](https://github.com/mikefarah/yq/releases/latest) or alternatively:
-```
-GO111MODULE=on go get github.com/mikefarah/yq/v3
+
+or, [Download latest binary](https://github.com/mikefarah/yq/releases/latest) or alternatively:
+
+```text
+GO111MODULE=on go get github.com/mikefarah/yq/v2
 ```
 
-## Run with Docker
+[View on GitHub](https://github.com/mikefarah/yq)
 
-Oneshot use:
-
-```bash
-docker run --rm -v ${PWD}:/workdir mikefarah/yq yq [flags] <command> FILE...
-```
-
-Run commands interactively:
-
-```bash
-docker run --rm -it -v ${PWD}:/workdir mikefarah/yq sh
-```
-
-It can be useful to have a bash function to avoid typing the whole docker command:
-
-```bash
-yq() {
-  docker run --rm -i -v ${PWD}:/workdir mikefarah/yq yq $@
-}
-```
-
-## Features
-- Written in portable go, so you can download a lovely dependency free binary
-- [Deep read a yaml file with a given path expression](https://mikefarah.gitbook.io/yq/commands/read#basic)
-- [List matching paths of a given path expression](https://mikefarah.gitbook.io/yq/commands/read#path-only)
-- Update a yaml file given a [path expression](https://mikefarah.gitbook.io/yq/commands/write-update#basic) or [script file](https://mikefarah.gitbook.io/yq/commands/write-update#basic)
-- Update creates any missing entries in the path on the fly
-- Keeps yaml formatting and comments when updating
-- Create a yaml file given a [deep path and value](https://mikefarah.gitbook.io/yq/commands/create#creating-a-simple-yaml-file) or a [script file](https://mikefarah.gitbook.io/yq/commands/create#creating-using-a-create-script)
-- [Prefix a path to a yaml file](https://mikefarah.gitbook.io/yq/commands/prefix)
-- [Convert to/from json to yaml](https://mikefarah.gitbook.io/yq/usage/convert)
-- [Pipe data in by using '-'](https://mikefarah.gitbook.io/yq/commands/read#from-stdin)
-- [Merge](https://mikefarah.gitbook.io/yq/commands/merge) multiple yaml files with various options for [overriding](https://mikefarah.gitbook.io/yq/commands/merge#overwrite-values) and [appending](https://mikefarah.gitbook.io/yq/commands/merge#append-values-with-arrays)
-- Supports multiple documents in a single yaml file for [reading](https://mikefarah.gitbook.io/yq/commands/read#multiple-documents), [writing](https://mikefarah.gitbook.io/yq/commands/write-update#multiple-documents) and [merging](https://mikefarah.gitbook.io/yq/commands/merge#multiple-documents)
-
-## [Usage](https://mikefarah.gitbook.io/yq/)
-
-Check out the [documentation](https://mikefarah.gitbook.io/yq/) for more detailed and advanced usage.
-
-```
-Usage:
-  yq [flags]
-  yq [command]
-
-Available Commands:
-  delete      yq d [--inplace/-i] [--doc/-d index] sample.yaml 'b.e(name==fred)'
-  help        Help about any command
-  merge       yq m [--inplace/-i] [--doc/-d index] [--overwrite/-x] [--append/-a] sample.yaml sample2.yaml
-  new         yq n [--script/-s script_file] a.b.c newValue
-  prefix      yq p [--inplace/-i] [--doc/-d index] sample.yaml a.b.c
-  read        yq r [--printMode/-p pv] sample.yaml 'b.e(name==fr*).value'
-  write       yq w [--inplace/-i] [--script/-s script_file] [--doc/-d index] sample.yaml 'b.e(name==fr*).value' newValue
-
-Flags:
-  -h, --help      help for yq
-  -j, --tojson    output as json
-  -v, --verbose   verbose mode
-  -V, --version   Print version information and quit
-
-Use "yq [command] --help" for more information about a command.
-```
-
-## Contribute
-
-**Note: v3 is currently in progress - for the moment I won't be accepting new feature PRs until v3 is ready :)**
-
-1. `scripts/devtools.sh`
-2. `make [local] vendor`
-3. add unit tests
-4. apply changes to go.mod
-5. `make [local] build`
-7. profit

SUMMARY.md

@@ -0,0 +1,11 @@
+# Table of contents
+
+* [Yq](README.md)
+* [Write](write-update.md)
+* [Read](read.md)
+* [Prefix](prefix.md)
+* [Delete](delete.md)
+* [Create](create.md)
+* [Convert](convert.md)
+* [Merge](merge.md)
+

Upgrade Notes

@@ -1,74 +0,0 @@
-
-Major release! Upgraded underlying yaml parser, re-written majority of yq. This has brought on a number of features that have been in demand for a while (see below).
-
-This is in beta and needs some community feedback and testing :)
-
-# New Features
-  - Keeps yaml comments and formatting, can specify yaml tags when updating. https://github.com/mikefarah/yq/issues/19, https://github.com/mikefarah/yq/issues/169, https://github.com/mikefarah/yq/issues/107, https://github.com/mikefarah/yq/issues/171, https://github.com/mikefarah/yq/issues/245, https://github.com/mikefarah/yq/issues/303,https://github.com/mikefarah/yq/issues/308,https://github.com/mikefarah/yq/issues/314
-  - Handles anchors!  https://github.com/mikefarah/yq/issues/310, https://github.com/mikefarah/yq/issues/178
-  - Can print out matching paths and values when splatting https://github.com/mikefarah/yq/issues/20
-  - JSON output works for all commands! Yaml files with multiple documents are printed out as one JSON document per line. 
-  - Deep splat (**) to match arbitrary paths
-
-
-# Breaking changes
-
-## Update scripts file format has changed to be more powerful. 
-Comments can be added, and delete commands have been introduced.
-
-Before:
-```yaml
-b.e[+].name: Mike Farah
-```
-
-After:
-```yaml
-- command: update 
-  path: b.e[+].thing
-  value:
-    #great 
-    things: frog # wow!
-- command: delete
-  path: b.d
-```
-
-https://github.com/mikefarah/yq/issues/305
-
-## Reading and splatting, matching results are printed once per line.
-  e.g:
-
-```json
-parent:
-  childA: 
-    no: matches here
-  childB:
-    there: matches
-    hi: no match
-    there2: also matches
-```
-
-```bash
-yq r sample.yaml 'parent.*.there*'
-```
-
-old
-```yaml
-- null
-- - matches
-  - also matches
-```
-
-new
-```yaml
-matches
-also matches
-```
-
-and you can print the matching paths:
-
-yq r --printMode pv sample.yaml 'parent.*.there*'
-
-```yaml
-parent.childB.there: matches
-parent.childB.there2: also matches
-```

action.yml

@@ -0,0 +1,13 @@
+name: 'yq - portable yaml processor'
+description: 'create, read, update, delete, merge, validate and do more with yaml'
+icon: command
+color: gray-dark
+inputs:
+  cmd:
+    description: 'The Command which should be run'
+    required: true
+runs:
+  using: 'docker'
+  image: 'github-action/Dockerfile'
+  args:
+    - ${{ inputs.cmd }}

cmd/compare.go

@@ -0,0 +1,89 @@
+package cmd
+
+import (
+	"bufio"
+	"bytes"
+	"os"
+	"strings"
+
+	"github.com/kylelemons/godebug/diff"
+	"github.com/mikefarah/yq/v3/pkg/yqlib"
+	errors "github.com/pkg/errors"
+	"github.com/spf13/cobra"
+)
+
+// turn off for unit tests :(
+var forceOsExit = true
+
+func createCompareCmd() *cobra.Command {
+	var cmdCompare = &cobra.Command{
+		Use:     "compare [yaml_file_a] [yaml_file_b]",
+		Aliases: []string{"x"},
+		Short:   "yq x [--prettyPrint/-P] dataA.yaml dataB.yaml 'b.e(name==fr*).value'",
+		Example: `
+yq x - data2.yml # reads from stdin
+yq x -pp dataA.yaml dataB.yaml '**' # compare paths
+yq x -d1 dataA.yaml dataB.yaml 'a.b.c'
+`,
+		Long: "Deeply compares two yaml files, prints the difference. Use with prettyPrint flag to ignore formatting differences.",
+		RunE: compareDocuments,
+	}
+	cmdCompare.PersistentFlags().StringVarP(&docIndex, "doc", "d", "0", "process document index number (0 based, * for all documents)")
+	cmdCompare.PersistentFlags().StringVarP(&printMode, "printMode", "p", "v", "print mode (v (values, default), p (paths), pv (path and value pairs)")
+	cmdCompare.PersistentFlags().StringVarP(&defaultValue, "defaultValue", "D", "", "default value printed when there are no results")
+	cmdCompare.PersistentFlags().BoolVarP(&stripComments, "stripComments", "", false, "strip comments out before comparing")
+	cmdCompare.PersistentFlags().BoolVarP(&explodeAnchors, "explodeAnchors", "X", false, "explode anchors")
+	return cmdCompare
+}
+
+func compareDocuments(cmd *cobra.Command, args []string) error {
+	var path = ""
+
+	if len(args) < 2 {
+		return errors.New("Must provide at 2 yaml files")
+	} else if len(args) > 2 {
+		path = args[2]
+	}
+
+	var updateAll, docIndexInt, errorParsingDocIndex = parseDocumentIndex()
+	if errorParsingDocIndex != nil {
+		return errorParsingDocIndex
+	}
+
+	var matchingNodesA []*yqlib.NodeContext
+	var matchingNodesB []*yqlib.NodeContext
+	var errorDoingThings error
+
+	matchingNodesA, errorDoingThings = readYamlFile(args[0], path, updateAll, docIndexInt)
+
+	if errorDoingThings != nil {
+		return errorDoingThings
+	}
+
+	matchingNodesB, errorDoingThings = readYamlFile(args[1], path, updateAll, docIndexInt)
+	if errorDoingThings != nil {
+		return errorDoingThings
+	}
+
+	var dataBufferA bytes.Buffer
+	var dataBufferB bytes.Buffer
+	errorDoingThings = printResults(matchingNodesA, bufio.NewWriter(&dataBufferA))
+	if errorDoingThings != nil {
+		return errorDoingThings
+	}
+	errorDoingThings = printResults(matchingNodesB, bufio.NewWriter(&dataBufferB))
+	if errorDoingThings != nil {
+		return errorDoingThings
+	}
+
+	diffString := diff.Diff(strings.TrimSuffix(dataBufferA.String(), "\n"), strings.TrimSuffix(dataBufferB.String(), "\n"))
+
+	if len(diffString) > 1 {
+		cmd.Print(diffString)
+		cmd.Print("\n")
+		if forceOsExit {
+			os.Exit(1)
+		}
+	}
+	return nil
+}

cmd/compare_test.go

@@ -0,0 +1,115 @@
+package cmd
+
+import (
+	"testing"
+
+	"github.com/mikefarah/yq/v3/test"
+)
+
+func TestCompareSameCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare ../examples/data1.yaml ../examples/data1.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := ``
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestCompareIgnoreCommentsCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare --stripComments ../examples/data1.yaml ../examples/data1-no-comments.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := ``
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestCompareDontIgnoreCommentsCmd(t *testing.T) {
+	forceOsExit = false
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare ../examples/data1.yaml ../examples/data1-no-comments.yaml")
+
+	expectedOutput := `-a: simple # just the best
++a: simple
+ b: [1, 2]
+ c:
+   test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestCompareExplodeAnchorsCommentsCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare --explodeAnchors ../examples/simple-anchor.yaml ../examples/simple-anchor-exploded.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := ``
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestCompareDontExplodeAnchorsCmd(t *testing.T) {
+	forceOsExit = false
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare ../examples/simple-anchor.yaml ../examples/simple-anchor-exploded.yaml")
+
+	expectedOutput := `-foo: &foo
++foo:
+   a: 1
+ foobar:
+-  !!merge <<: *foo
++  a: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestCompareDifferentCmd(t *testing.T) {
+	forceOsExit = false
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare ../examples/data1.yaml ../examples/data3.yaml")
+
+	expectedOutput := `-a: simple # just the best
+-b: [1, 2]
++a: "simple" # just the best
++b: [1, 3]
+ c:
+   test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestComparePrettyCmd(t *testing.T) {
+	forceOsExit = false
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare -P ../examples/data1.yaml ../examples/data3.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := ` a: simple # just the best
+ b:
+   - 1
+-  - 2
++  - 3
+ c:
+   test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestComparePathsCmd(t *testing.T) {
+	forceOsExit = false
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare -P -ppv ../examples/data1.yaml ../examples/data3.yaml **")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := ` a: simple # just the best
+ b.[0]: 1
+-b.[1]: 2
++b.[1]: 3
+ c.test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}

cmd/constant.go

@@ -7,12 +7,25 @@ import (
 
 var customTag = ""
 var printMode = "v"
+var printLength = false
+var unwrapScalar = true
+var customStyle = ""
+var anchorName = ""
+var makeAlias = false
+var stripComments = false
+var collectIntoArray = false
 var writeInplace = false
 var writeScript = ""
+var sourceYamlFile = ""
 var outputToJSON = false
+var exitStatus = false
+var prettyPrint = false
+var explodeAnchors = false
+var colorsEnabled = false
+var defaultValue = ""
+var indent = 2
 var overwriteFlag = false
 var autoCreateFlag = true
-var allowEmptyFlag = false
 var appendFlag = false
 var verbose = false
 var version = false

cmd/delete_test.go

@@ -0,0 +1,246 @@
+package cmd
+
+import (
+	"fmt"
+	"strings"
+	"testing"
+
+	"github.com/mikefarah/yq/v3/test"
+)
+
+func TestDeleteYamlCmd(t *testing.T) {
+	content := `a: 2
+b:
+  c: things
+  d: something else
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete %s b.c", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `a: 2
+b:
+  d: something else
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestDeleteDeepDoesNotExistCmd(t *testing.T) {
+	content := `a: 2`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete %s b.c", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `a: 2
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestDeleteSplatYaml(t *testing.T) {
+	content := `a: other
+b: [3, 4]
+c:
+  toast: leave
+  test: 1
+  tell: 1
+  tasty.taco: cool
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete %s c.te*", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `a: other
+b: [3, 4]
+c:
+  toast: leave
+  tasty.taco: cool
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestDeleteSplatArrayYaml(t *testing.T) {
+	content := `a: 2
+b:
+ hi:
+  - thing: item1
+    name: fred
+  - thing: item2
+    name: sam
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete %s b.hi[*].thing", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `a: 2
+b:
+  hi:
+    - name: fred
+    - name: sam
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestDeleteDeepSplatArrayYaml(t *testing.T) {
+	content := `thing: 123
+b:
+ hi:
+  - thing: item1
+    name: fred
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete %s **.thing", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `b:
+  hi:
+    - name: fred
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestDeleteSplatPrefixYaml(t *testing.T) {
+	content := `a: 2
+b:
+ hi:
+   c: things
+   d: something else
+ there:
+   c: more things
+   d: more something else
+ there2:
+   c: more things also
+   d: more something else also
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete %s b.there*.c", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `a: 2
+b:
+  hi:
+    c: things
+    d: something else
+  there:
+    d: more something else
+  there2:
+    d: more something else also
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestDeleteYamlArrayCmd(t *testing.T) {
+	content := `- 1
+- 2
+- 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete %s [1]", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `- 1
+- 3
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestDeleteYamlArrayExpressionCmd(t *testing.T) {
+	content := `- name: fred
+- name: cat
+- name: thing
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete %s (name==cat)", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `- name: fred
+- name: thing
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestDeleteYamlMulti(t *testing.T) {
+	content := `apples: great
+---
+- 1
+- 2
+- 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete -d 1 %s [1]", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `apples: great
+---
+- 1
+- 3
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestDeleteYamlMultiAllCmd(t *testing.T) {
+	content := `b:
+  c: 3
+apples: great
+---
+apples: great
+something: else
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete %s -d * apples", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 3
+---
+something: else`
+	test.AssertResult(t, expectedOutput, strings.Trim(result.Output, "\n "))
+}

cmd/merge.go

@@ -1,11 +1,10 @@
 package cmd
 
 import (
-	"strings"
-
 	"github.com/mikefarah/yq/v3/pkg/yqlib"
 	errors "github.com/pkg/errors"
 	"github.com/spf13/cobra"
+	yaml "gopkg.in/yaml.v3"
 )
 
 func createMergeCmd() *cobra.Command {
@@ -34,27 +33,40 @@ If append flag is set then existing arrays will be merged with the arrays from e
 	cmdMerge.PersistentFlags().BoolVarP(&overwriteFlag, "overwrite", "x", false, "update the yaml file by overwriting existing values")
 	cmdMerge.PersistentFlags().BoolVarP(&autoCreateFlag, "autocreate", "c", true, "automatically create any missing entries")
 	cmdMerge.PersistentFlags().BoolVarP(&appendFlag, "append", "a", false, "update the yaml file by appending array values")
-	cmdMerge.PersistentFlags().BoolVarP(&allowEmptyFlag, "allow-empty", "e", false, "allow empty yaml files")
 	cmdMerge.PersistentFlags().StringVarP(&docIndex, "doc", "d", "0", "process document index number (0 based, * for all documents)")
 	return cmdMerge
 }
 
-func mergeProperties(cmd *cobra.Command, args []string) error {
-	if len(args) < 2 {
-		return errors.New("Must provide at least 2 yaml files")
+/*
+* We don't deeply traverse arrays when appending a merge, instead we want to
+* append the entire array element.
+ */
+func createReadFunctionForMerge() func(*yaml.Node) ([]*yqlib.NodeContext, error) {
+	return func(dataBucket *yaml.Node) ([]*yqlib.NodeContext, error) {
+		return lib.Get(dataBucket, "**", !appendFlag)
 	}
-	// first generate update commands from the file
-	var filesToMerge = args[1:]
+}
+
+func mergeProperties(cmd *cobra.Command, args []string) error {
 	var updateCommands []yqlib.UpdateCommand = make([]yqlib.UpdateCommand, 0)
 
-	for _, fileToMerge := range filesToMerge {
-		matchingNodes, errorProcessingFile := readYamlFile(fileToMerge, "**", false, 0)
-		if errorProcessingFile != nil && (!allowEmptyFlag || !strings.HasPrefix(errorProcessingFile.Error(), "Could not process document index")) {
-			return errorProcessingFile
-		}
-		for _, matchingNode := range matchingNodes {
-			mergePath := lib.MergePathStackToString(matchingNode.PathStack, appendFlag)
-			updateCommands = append(updateCommands, yqlib.UpdateCommand{Command: "update", Path: mergePath, Value: matchingNode.Node, Overwrite: overwriteFlag})
+	if len(args) < 1 {
+		return errors.New("Must provide at least 1 yaml file")
+	}
+
+	if len(args) > 1 {
+		// first generate update commands from the file
+		var filesToMerge = args[1:]
+
+		for _, fileToMerge := range filesToMerge {
+			matchingNodes, errorProcessingFile := doReadYamlFile(fileToMerge, createReadFunctionForMerge(), false, 0)
+			if errorProcessingFile != nil {
+				return errorProcessingFile
+			}
+			for _, matchingNode := range matchingNodes {
+				mergePath := lib.MergePathStackToString(matchingNode.PathStack, appendFlag)
+				updateCommands = append(updateCommands, yqlib.UpdateCommand{Command: "update", Path: mergePath, Value: matchingNode.Node, Overwrite: overwriteFlag})
+			}
 		}
 	}
 

cmd/merge_test.go

@@ -0,0 +1,337 @@
+package cmd
+
+import (
+	"fmt"
+	"os"
+	"runtime"
+	"testing"
+
+	"github.com/mikefarah/yq/v3/test"
+)
+
+func TestMergeCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge ../examples/data1.yaml ../examples/data2.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `a: simple # just the best
+b: [1, 2]
+c:
+  test: 1
+  toast: leave
+  tell: 1
+  tasty.taco: cool
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeOneFileCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge ../examples/data1.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `a: simple # just the best
+b: [1, 2]
+c:
+  test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeNoAutoCreateCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge -c=false ../examples/data1.yaml ../examples/data2.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `a: simple # just the best
+b: [1, 2]
+c:
+  test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeOverwriteCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge -c=false --overwrite ../examples/data1.yaml ../examples/data2.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `a: other # better than the original
+b: [3, 4]
+c:
+  test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeAppendCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge --autocreate=false --append ../examples/data1.yaml ../examples/data2.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `a: simple # just the best
+b: [1, 2, 3, 4]
+c:
+  test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeAppendArraysCmd(t *testing.T) {
+	content := `people:
+  - name: Barry
+    age: 21`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	mergeContent := `people:
+  - name: Roger
+    age: 44`
+	mergeFilename := test.WriteTempYamlFile(mergeContent)
+	defer test.RemoveTempYamlFile(mergeFilename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("merge --append -d* %s %s", filename, mergeFilename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `people:
+  - name: Barry
+    age: 21
+  - name: Roger
+    age: 44
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeOverwriteAndAppendCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge --autocreate=false --append --overwrite ../examples/data1.yaml ../examples/data2.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `a: other # better than the original
+b: [1, 2, 3, 4]
+c:
+  test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeArraysCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge --append ../examples/sample_array.yaml ../examples/sample_array_2.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `[1, 2, 3, 4, 5]
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeCmd_Multi(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge -d1 ../examples/multiple_docs_small.yaml ../examples/data1.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `a: Easy! as one two three
+---
+another:
+  document: here
+a: simple # just the best
+b:
+  - 1
+  - 2
+c:
+  test: 1
+---
+- 1
+- 2
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeYamlMultiAllCmd(t *testing.T) {
+	content := `b:
+  c: 3
+apples: green
+---
+something: else`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	mergeContent := `apples: red
+something: good`
+	mergeFilename := test.WriteTempYamlFile(mergeContent)
+	defer test.RemoveTempYamlFile(mergeFilename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("merge -d* %s %s", filename, mergeFilename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 3
+apples: green
+something: good
+---
+something: else
+apples: red
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeSpecialCharacterKeysCmd(t *testing.T) {
+	content := ``
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	mergeContent := `key[bracket]: value
+key.bracket: value
+key"value": value
+key'value': value
+`
+	mergeFilename := test.WriteTempYamlFile(mergeContent)
+	defer test.RemoveTempYamlFile(mergeFilename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("merge %s %s", filename, mergeFilename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	test.AssertResult(t, mergeContent, result.Output)
+}
+
+func TestMergeYamlMultiAllOverwriteCmd(t *testing.T) {
+	content := `b:
+  c: 3
+apples: green
+---
+something: else`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	mergeContent := `apples: red
+something: good`
+	mergeFilename := test.WriteTempYamlFile(mergeContent)
+	defer test.RemoveTempYamlFile(mergeFilename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("merge --overwrite -d* %s %s", filename, mergeFilename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 3
+apples: red
+something: good
+---
+something: good
+apples: red
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeYamlNullMapCmd(t *testing.T) {
+	content := `b:`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	mergeContent := `b:
+  thing: a frog
+`
+	mergeFilename := test.WriteTempYamlFile(mergeContent)
+	defer test.RemoveTempYamlFile(mergeFilename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("merge %s %s", filename, mergeFilename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	test.AssertResult(t, mergeContent, result.Output)
+}
+
+func TestMergeCmd_Error(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge")
+	if result.Error == nil {
+		t.Error("Expected command to fail due to missing arg")
+	}
+	expectedOutput := `Must provide at least 1 yaml file`
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+
+func TestMergeCmd_ErrorUnreadableFile(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge ../examples/data1.yaml fake-unknown")
+	if result.Error == nil {
+		t.Error("Expected command to fail due to unknown file")
+	}
+	var expectedOutput string
+	if runtime.GOOS == "windows" {
+		expectedOutput = `open fake-unknown: The system cannot find the file specified.`
+	} else {
+		expectedOutput = `open fake-unknown: no such file or directory`
+	}
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+
+func TestMergeCmd_Inplace(t *testing.T) {
+	filename := test.WriteTempYamlFile(test.ReadTempYamlFile("../examples/data1.yaml"))
+	err := os.Chmod(filename, os.FileMode(int(0666)))
+	if err != nil {
+		t.Error(err)
+	}
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("merge -i %s ../examples/data2.yaml", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	info, _ := os.Stat(filename)
+	gotOutput := test.ReadTempYamlFile(filename)
+	expectedOutput := `a: simple # just the best
+b: [1, 2]
+c:
+  test: 1
+  toast: leave
+  tell: 1
+  tasty.taco: cool
+`
+	test.AssertResult(t, expectedOutput, gotOutput)
+	test.AssertResult(t, os.FileMode(int(0666)), info.Mode())
+}
+
+func TestMergeAllowEmptyTargetCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge ../examples/empty.yaml ../examples/data1.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `a: simple # just the best
+b:
+  - 1
+  - 2
+c:
+  test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestMergeAllowEmptyMergeCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "merge ../examples/data1.yaml ../examples/empty.yaml")
+	expectedOutput := `a: simple # just the best
+b: [1, 2]
+c:
+  test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}

cmd/new.go

@@ -1,8 +1,9 @@
 package cmd
 
 import (
+	"github.com/mikefarah/yq/v3/pkg/yqlib"
+	errors "github.com/pkg/errors"
 	"github.com/spf13/cobra"
-	yaml "gopkg.in/yaml.v3"
 )
 
 func createNewCmd() *cobra.Command {
@@ -27,11 +28,19 @@ Note that you can give a create script to perform more sophisticated yaml. This
 	}
 	cmdNew.PersistentFlags().StringVarP(&writeScript, "script", "s", "", "yaml script for creating yaml")
 	cmdNew.PersistentFlags().StringVarP(&customTag, "tag", "t", "", "set yaml tag (e.g. !!int)")
+	cmdNew.PersistentFlags().StringVarP(&customStyle, "style", "", "", "formatting style of the value: single, double, folded, flow, literal, tagged")
+	cmdNew.PersistentFlags().StringVarP(&anchorName, "anchorName", "", "", "anchor name")
+	cmdNew.PersistentFlags().BoolVarP(&makeAlias, "makeAlias", "", false, "create an alias using the value as the anchor name")
 	return cmdNew
 }
 
 func newProperty(cmd *cobra.Command, args []string) error {
-	var updateCommands, updateCommandsError = readUpdateCommands(args, 2, "Must provide <path_to_update> <value>")
+	var badArgsMessage = "Must provide <path_to_update> <value>"
+	if len(args) != 2 {
+		return errors.New(badArgsMessage)
+	}
+
+	var updateCommands, updateCommandsError = readUpdateCommands(args, 2, badArgsMessage)
 	if updateCommandsError != nil {
 		return updateCommandsError
 	}
@@ -46,9 +55,6 @@ func newProperty(cmd *cobra.Command, args []string) error {
 		}
 	}
 
-	var encoder = yaml.NewEncoder(cmd.OutOrStdout())
-	encoder.SetIndent(2)
-	errorEncoding := encoder.Encode(&newNode)
-	encoder.Close()
-	return errorEncoding
+	var encoder = yqlib.NewYamlEncoder(cmd.OutOrStdout(), indent, colorsEnabled)
+	return encoder.Encode(&newNode)
 }

cmd/new_test.go

@@ -0,0 +1,101 @@
+package cmd
+
+import (
+	"testing"
+
+	"github.com/mikefarah/yq/v3/test"
+)
+
+func TestNewCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "new b.c 3")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 3
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestNewAnchorCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "new b.c 3 --anchorName=fred")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: &fred 3
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestNewAliasCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "new b.c foo --makeAlias")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: *foo
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestNewArrayCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "new b[0] 3")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  - 3
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestNewCmd_Error(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "new b.c")
+	if result.Error == nil {
+		t.Error("Expected command to fail due to missing arg")
+	}
+	expectedOutput := `Must provide <path_to_update> <value>`
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+
+func TestNewWithTaggedStyleCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "new b.c cat --tag=!!str --style=tagged")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: !!str cat
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestNewWithDoubleQuotedStyleCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "new b.c cat --style=double")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: "cat"
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestNewWithSingleQuotedStyleCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "new b.c cat --style=single")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 'cat'
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}

cmd/prefix_test.go

@@ -0,0 +1,189 @@
+package cmd
+
+import (
+	"fmt"
+	"runtime"
+	"strings"
+	"testing"
+
+	"github.com/mikefarah/yq/v3/test"
+)
+
+func TestPrefixCmd(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("prefix %s d", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `d:
+  b:
+    c: 3
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestPrefixCmdArray(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("prefix %s [+].d.[+]", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `- d:
+    - b:
+        c: 3
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestPrefixCmd_MultiLayer(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("prefix %s d.e.f", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `d:
+  e:
+    f:
+      b:
+        c: 3
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestPrefixMultiCmd(t *testing.T) {
+	content := `b:
+  c: 3
+---
+apples: great
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("prefix %s -d 1 d", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 3
+---
+d:
+  apples: great
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+func TestPrefixInvalidDocumentIndexCmd(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("prefix %s -df d", filename))
+	if result.Error == nil {
+		t.Error("Expected command to fail due to invalid path")
+	}
+	expectedOutput := `Document index f is not a integer or *: strconv.ParseInt: parsing "f": invalid syntax`
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+
+func TestPrefixBadDocumentIndexCmd(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("prefix %s -d 1 d", filename))
+	if result.Error == nil {
+		t.Error("Expected command to fail due to invalid path")
+	}
+	expectedOutput := `asked to process document index 1 but there are only 1 document(s)`
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+func TestPrefixMultiAllCmd(t *testing.T) {
+	content := `b:
+  c: 3
+---
+apples: great
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("prefix %s -d * d", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `d:
+  b:
+    c: 3
+---
+d:
+  apples: great`
+	test.AssertResult(t, expectedOutput, strings.Trim(result.Output, "\n "))
+}
+
+func TestPrefixCmd_Error(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "prefix")
+	if result.Error == nil {
+		t.Error("Expected command to fail due to missing arg")
+	}
+	expectedOutput := `Must provide <filename> <prefixed_path>`
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+
+func TestPrefixCmd_ErrorUnreadableFile(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "prefix fake-unknown a.b")
+	if result.Error == nil {
+		t.Error("Expected command to fail due to unknown file")
+	}
+	var expectedOutput string
+	if runtime.GOOS == "windows" {
+		expectedOutput = `open fake-unknown: The system cannot find the file specified.`
+	} else {
+		expectedOutput = `open fake-unknown: no such file or directory`
+	}
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+
+func TestPrefixCmd_Inplace(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("prefix -i %s d", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	gotOutput := test.ReadTempYamlFile(filename)
+	expectedOutput := `d:
+  b:
+    c: 3`
+	test.AssertResult(t, expectedOutput, strings.Trim(gotOutput, "\n "))
+}

cmd/read.go

@@ -25,6 +25,13 @@ yq r -- things.yaml '--key-starting-with-dashes.blah'
 	}
 	cmdRead.PersistentFlags().StringVarP(&docIndex, "doc", "d", "0", "process document index number (0 based, * for all documents)")
 	cmdRead.PersistentFlags().StringVarP(&printMode, "printMode", "p", "v", "print mode (v (values, default), p (paths), pv (path and value pairs)")
+	cmdRead.PersistentFlags().StringVarP(&defaultValue, "defaultValue", "D", "", "default value printed when there are no results")
+	cmdRead.PersistentFlags().BoolVarP(&printLength, "length", "l", false, "print length of results")
+	cmdRead.PersistentFlags().BoolVarP(&collectIntoArray, "collect", "c", false, "collect results into array")
+	cmdRead.PersistentFlags().BoolVarP(&unwrapScalar, "unwrapScalar", "", true, "unwrap scalar, print the value with no quotes, colors or comments")
+	cmdRead.PersistentFlags().BoolVarP(&stripComments, "stripComments", "", false, "print yaml without any comments")
+	cmdRead.PersistentFlags().BoolVarP(&explodeAnchors, "explodeAnchors", "X", false, "explode anchors")
+	cmdRead.PersistentFlags().BoolVarP(&exitStatus, "exitStatus", "e", false, "set exit status if no matches are found")
 	return cmdRead
 }
 
@@ -44,9 +51,16 @@ func readProperty(cmd *cobra.Command, args []string) error {
 
 	matchingNodes, errorReadingStream := readYamlFile(args[0], path, updateAll, docIndexInt)
 
-	if errorReadingStream != nil {
-		return errorReadingStream
+	if exitStatus && len(matchingNodes) == 0 {
+		cmd.SilenceUsage = true
+		return errors.New("No matches found")
 	}
 
-	return printResults(matchingNodes, cmd)
+	if errorReadingStream != nil {
+		cmd.SilenceUsage = true
+		return errorReadingStream
+	}
+	out := cmd.OutOrStdout()
+
+	return printResults(matchingNodes, out)
 }

cmd/root.go

@@ -39,19 +39,22 @@ func New() *cobra.Command {
 	}
 
 	rootCmd.PersistentFlags().BoolVarP(&verbose, "verbose", "v", false, "verbose mode")
-	rootCmd.PersistentFlags().BoolVarP(&outputToJSON, "tojson", "j", false, "output as json")
+	rootCmd.PersistentFlags().BoolVarP(&outputToJSON, "tojson", "j", false, "output as json. By default it prints a json document in one line, use the prettyPrint flag to print a formatted doc.")
+	rootCmd.PersistentFlags().BoolVarP(&prettyPrint, "prettyPrint", "P", false, "pretty print")
+	rootCmd.PersistentFlags().IntVarP(&indent, "indent", "I", 2, "sets indent level for output")
 	rootCmd.Flags().BoolVarP(&version, "version", "V", false, "Print version information and quit")
+	rootCmd.PersistentFlags().BoolVarP(&colorsEnabled, "colors", "C", false, "print with colors")
 
 	rootCmd.AddCommand(
 		createReadCmd(),
+		createCompareCmd(),
 		createValidateCmd(),
 		createWriteCmd(),
 		createPrefixCmd(),
 		createDeleteCmd(),
 		createNewCmd(),
 		createMergeCmd(),
+		createBashCompletionCmd(rootCmd),
 	)
-	rootCmd.SetOutput(os.Stdout)
-
 	return rootCmd
 }

cmd/shell_completion.go

@@ -0,0 +1,57 @@
+package cmd
+
+import (
+	"fmt"
+	"os"
+
+	"github.com/spf13/cobra"
+)
+
+var shellVariant = "bash"
+
+func createBashCompletionCmd(rootCmd *cobra.Command) *cobra.Command {
+	var completionCmd = &cobra.Command{
+		Use:   "shell-completion",
+		Short: "Generates shell completion scripts",
+		Long: `To load completion for:
+bash:
+	Run	
+	. <(yq shell-completion)
+	
+	To configure your bash shell to load completions for each session add to
+	your bashrc
+	
+	# ~/.bashrc or ~/.profile
+	. <(yq shell-completion)
+
+zsh:
+	The generated completion script should be put somewhere in your $fpath named _yq
+
+powershell:
+	Users need PowerShell version 5.0 or above, which comes with Windows 10 and 
+	can be downloaded separately for Windows 7 or 8.1. They can then write the 
+	completions to a file and source this file from their PowerShell profile, 
+	which is referenced by the $Profile environment variable.
+
+fish:
+	Save the output to a fish file and add it to your completions directory.
+
+	`,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			switch shellVariant {
+			case "bash", "":
+				return rootCmd.GenBashCompletion(os.Stdout)
+			case "zsh":
+				return rootCmd.GenZshCompletion(os.Stdout)
+			case "fish":
+				return rootCmd.GenFishCompletion(os.Stdout, true)
+			case "powershell":
+				return rootCmd.GenPowerShellCompletion(os.Stdout)
+			default:
+				return fmt.Errorf("Unknown variant %v", shellVariant)
+			}
+		},
+	}
+	completionCmd.PersistentFlags().StringVarP(&shellVariant, "variation", "V", "", "shell variation: bash (default), zsh, fish, powershell")
+	return completionCmd
+}

cmd/utils.go

@@ -10,11 +10,22 @@ import (
 
 	"github.com/mikefarah/yq/v3/pkg/yqlib"
 	errors "github.com/pkg/errors"
-	"github.com/spf13/cobra"
 	yaml "gopkg.in/yaml.v3"
 )
 
+type readDataFn func(dataBucket *yaml.Node) ([]*yqlib.NodeContext, error)
+
+func createReadFunction(path string) func(*yaml.Node) ([]*yqlib.NodeContext, error) {
+	return func(dataBucket *yaml.Node) ([]*yqlib.NodeContext, error) {
+		return lib.Get(dataBucket, path, true)
+	}
+}
+
 func readYamlFile(filename string, path string, updateAll bool, docIndexInt int) ([]*yqlib.NodeContext, error) {
+	return doReadYamlFile(filename, createReadFunction(path), updateAll, docIndexInt)
+}
+
+func doReadYamlFile(filename string, readFn readDataFn, updateAll bool, docIndexInt int) ([]*yqlib.NodeContext, error) {
 	var matchingNodes []*yqlib.NodeContext
 
 	var currentIndex = 0
@@ -30,7 +41,7 @@ func readYamlFile(filename string, path string, updateAll bool, docIndexInt int)
 			}
 
 			var errorParsing error
-			matchingNodes, errorParsing = appendDocument(matchingNodes, dataBucket, path, updateAll, docIndexInt, currentIndex)
+			matchingNodes, errorParsing = appendDocument(matchingNodes, dataBucket, readFn, updateAll, docIndexInt, currentIndex)
 			if errorParsing != nil {
 				return errorParsing
 			}
@@ -46,78 +57,247 @@ func readYamlFile(filename string, path string, updateAll bool, docIndexInt int)
 
 func handleEOF(updateAll bool, docIndexInt int, currentIndex int) error {
 	log.Debugf("done %v / %v", currentIndex, docIndexInt)
-	if !updateAll && currentIndex <= docIndexInt {
+	if !updateAll && currentIndex <= docIndexInt && docIndexInt != 0 {
 		return fmt.Errorf("Could not process document index %v as there are only %v document(s)", docIndex, currentIndex)
 	}
 	return nil
 }
 
-func appendDocument(originalMatchingNodes []*yqlib.NodeContext, dataBucket yaml.Node, path string, updateAll bool, docIndexInt int, currentIndex int) ([]*yqlib.NodeContext, error) {
+func appendDocument(originalMatchingNodes []*yqlib.NodeContext, dataBucket yaml.Node, readFn readDataFn, updateAll bool, docIndexInt int, currentIndex int) ([]*yqlib.NodeContext, error) {
 	log.Debugf("processing document %v - requested index %v", currentIndex, docIndexInt)
 	yqlib.DebugNode(&dataBucket)
 	if !updateAll && currentIndex != docIndexInt {
 		return originalMatchingNodes, nil
 	}
-	log.Debugf("reading %v in document %v", path, currentIndex)
-	matchingNodes, errorParsing := lib.Get(&dataBucket, path)
+	log.Debugf("reading in document %v", currentIndex)
+	matchingNodes, errorParsing := readFn(&dataBucket)
 	if errorParsing != nil {
 		return nil, errors.Wrapf(errorParsing, "Error reading path in document index %v", currentIndex)
 	}
 	return append(originalMatchingNodes, matchingNodes...), nil
 }
 
-func printValue(node *yaml.Node, cmd *cobra.Command) error {
-	if node.Kind == yaml.ScalarNode {
-		cmd.Print(node.Value)
+func lengthOf(node *yaml.Node) int {
+	kindToCheck := node.Kind
+	if node.Kind == yaml.DocumentNode && len(node.Content) == 1 {
+		log.Debugf("length of document node, calculating length of child")
+		kindToCheck = node.Content[0].Kind
+	}
+	switch kindToCheck {
+	case yaml.ScalarNode:
+		return len(node.Value)
+	case yaml.MappingNode:
+		return len(node.Content) / 2
+	default:
+		return len(node.Content)
+	}
+}
+
+// transforms node before printing, if required
+func transformNode(node *yaml.Node) *yaml.Node {
+	if printLength {
+		return &yaml.Node{Kind: yaml.ScalarNode, Value: fmt.Sprintf("%v", lengthOf(node))}
+	}
+	return node
+}
+
+func printNode(node *yaml.Node, writer io.Writer) error {
+	var encoder yqlib.Encoder
+	if node.Kind == yaml.ScalarNode && unwrapScalar && !outputToJSON {
+		return writeString(writer, node.Value+"\n")
+	}
+	if outputToJSON {
+		encoder = yqlib.NewJsonEncoder(writer, prettyPrint, indent)
+	} else {
+		encoder = yqlib.NewYamlEncoder(writer, indent, colorsEnabled)
+	}
+	return encoder.Encode(node)
+}
+
+func removeComments(matchingNodes []*yqlib.NodeContext) {
+	for _, nodeContext := range matchingNodes {
+		removeCommentOfNode(nodeContext.Node)
+	}
+}
+
+func removeCommentOfNode(node *yaml.Node) {
+	node.HeadComment = ""
+	node.LineComment = ""
+	node.FootComment = ""
+
+	for _, child := range node.Content {
+		removeCommentOfNode(child)
+	}
+}
+
+func setStyle(matchingNodes []*yqlib.NodeContext, style yaml.Style) {
+	for _, nodeContext := range matchingNodes {
+		updateStyleOfNode(nodeContext.Node, style)
+	}
+}
+
+func updateStyleOfNode(node *yaml.Node, style yaml.Style) {
+	node.Style = style
+
+	for _, child := range node.Content {
+		updateStyleOfNode(child, style)
+	}
+}
+
+func writeString(writer io.Writer, txt string) error {
+	_, errorWriting := writer.Write([]byte(txt))
+	return errorWriting
+}
+
+func setIfNotThere(node *yaml.Node, key string, value *yaml.Node) {
+	for index := 0; index < len(node.Content); index = index + 2 {
+		keyNode := node.Content[index]
+		if keyNode.Value == key {
+			return
+		}
+	}
+	// need to add it to the map
+	mapEntryKey := yaml.Node{Value: key, Kind: yaml.ScalarNode}
+	node.Content = append(node.Content, &mapEntryKey)
+	node.Content = append(node.Content, value)
+}
+
+func applyAlias(node *yaml.Node, alias *yaml.Node) {
+	for index := 0; index < len(alias.Content); index = index + 2 {
+		keyNode := alias.Content[index]
+		log.Debugf("applying alias key %v", keyNode.Value)
+		valueNode := alias.Content[index+1]
+		setIfNotThere(node, keyNode.Value, valueNode)
+	}
+}
+
+func explodeNode(node *yaml.Node) error {
+	node.Anchor = ""
+	switch node.Kind {
+	case yaml.SequenceNode, yaml.DocumentNode:
+		for index, contentNode := range node.Content {
+			log.Debugf("exploding index %v", index)
+			errorInContent := explodeNode(contentNode)
+			if errorInContent != nil {
+				return errorInContent
+			}
+		}
+		return nil
+	case yaml.AliasNode:
+		log.Debugf("its an alias!")
+		node.Kind = node.Alias.Kind
+		node.Style = node.Alias.Style
+		node.Tag = node.Alias.Tag
+		node.Content = node.Alias.Content
+		node.Value = node.Alias.Value
+		node.Alias = nil
+		return nil
+	case yaml.MappingNode:
+		for index := 0; index < len(node.Content); index = index + 2 {
+			keyNode := node.Content[index]
+			valueNode := node.Content[index+1]
+			log.Debugf("traversing %v", keyNode.Value)
+			if keyNode.Value != "<<" {
+				errorInContent := explodeNode(valueNode)
+				if errorInContent != nil {
+					return errorInContent
+				}
+			} else {
+				if valueNode.Kind == yaml.SequenceNode {
+					log.Debugf("an alias merge list!")
+					for index := len(valueNode.Content) - 1; index >= 0; index = index - 1 {
+						aliasNode := valueNode.Content[index]
+						applyAlias(node, aliasNode.Alias)
+					}
+				} else {
+					log.Debugf("an alias merge!")
+					applyAlias(node, valueNode.Alias)
+				}
+				node.Content = append(node.Content[:index], node.Content[index+2:]...)
+				//replay that index, since the array is shorter now.
+				index = index - 2
+			}
+		}
+
+		return nil
+	default:
 		return nil
 	}
+}
 
-	bufferedWriter := bufio.NewWriter(cmd.OutOrStdout())
-	defer safelyFlush(bufferedWriter)
-
-	var encoder yqlib.Encoder
-	if outputToJSON {
-		encoder = yqlib.NewJsonEncoder(bufferedWriter)
-	} else {
-		encoder = yqlib.NewYamlEncoder(bufferedWriter)
-	}
-	if err := encoder.Encode(node); err != nil {
-		return err
+func explode(matchingNodes []*yqlib.NodeContext) error {
+	log.Debug("exploding nodes")
+	for _, nodeContext := range matchingNodes {
+		log.Debugf("exploding %v", nodeContext.Head)
+		errorExplodingNode := explodeNode(nodeContext.Node)
+		if errorExplodingNode != nil {
+			return errorExplodingNode
+		}
 	}
 	return nil
 }
 
-func printResults(matchingNodes []*yqlib.NodeContext, cmd *cobra.Command) error {
-	if len(matchingNodes) == 0 {
-		log.Debug("no matching results, nothing to print")
-		return nil
+func printResults(matchingNodes []*yqlib.NodeContext, writer io.Writer) error {
+	if prettyPrint {
+		setStyle(matchingNodes, 0)
 	}
 
-	for index, mappedDoc := range matchingNodes {
+	if stripComments {
+		removeComments(matchingNodes)
+	}
+
+	//always explode anchors when printing json
+	if explodeAnchors || outputToJSON {
+		errorExploding := explode(matchingNodes)
+		if errorExploding != nil {
+			return errorExploding
+		}
+	}
+
+	bufferedWriter := bufio.NewWriter(writer)
+	defer safelyFlush(bufferedWriter)
+
+	if len(matchingNodes) == 0 {
+		log.Debug("no matching results, nothing to print")
+		if defaultValue != "" {
+			return writeString(bufferedWriter, defaultValue)
+		}
+		return nil
+	}
+	var errorWriting error
+
+	var arrayCollection = yaml.Node{Kind: yaml.SequenceNode}
+
+	for _, mappedDoc := range matchingNodes {
 		switch printMode {
 		case "p":
-			cmd.Print(lib.PathStackToString(mappedDoc.PathStack))
-			if index < len(matchingNodes)-1 {
-				cmd.Print("\n")
+			errorWriting = writeString(bufferedWriter, lib.PathStackToString(mappedDoc.PathStack)+"\n")
+			if errorWriting != nil {
+				return errorWriting
 			}
 		case "pv", "vp":
 			// put it into a node and print that.
 			var parentNode = yaml.Node{Kind: yaml.MappingNode}
 			parentNode.Content = make([]*yaml.Node, 2)
 			parentNode.Content[0] = &yaml.Node{Kind: yaml.ScalarNode, Value: lib.PathStackToString(mappedDoc.PathStack)}
-			parentNode.Content[1] = mappedDoc.Node
-			if err := printValue(&parentNode, cmd); err != nil {
+			parentNode.Content[1] = transformNode(mappedDoc.Node)
+			if collectIntoArray {
+				arrayCollection.Content = append(arrayCollection.Content, &parentNode)
+			} else if err := printNode(&parentNode, bufferedWriter); err != nil {
 				return err
 			}
 		default:
-			if err := printValue(mappedDoc.Node, cmd); err != nil {
+			if collectIntoArray {
+				arrayCollection.Content = append(arrayCollection.Content, mappedDoc.Node)
+			} else if err := printNode(transformNode(mappedDoc.Node), bufferedWriter); err != nil {
 				return err
 			}
-			// Printing our Scalars does not print a new line at the end
-			// we only want to do that if there are more values (so users can easily script extraction of values in the yaml)
-			if index < len(matchingNodes)-1 && mappedDoc.Node.Kind == yaml.ScalarNode {
-				cmd.Print("\n")
-			}
+		}
+	}
+
+	if collectIntoArray {
+		if err := printNode(transformNode(&arrayCollection), bufferedWriter); err != nil {
+			return err
 		}
 	}
 
@@ -137,6 +317,11 @@ func parseDocumentIndex() (bool, int, error) {
 
 type updateDataFn func(dataBucket *yaml.Node, currentIndex int) error
 
+func isNullDocument(dataBucket *yaml.Node) bool {
+	return dataBucket.Kind == yaml.DocumentNode && (len(dataBucket.Content) == 0 ||
+		dataBucket.Content[0].Kind == yaml.ScalarNode && dataBucket.Content[0].Tag == "!!null")
+}
+
 func mapYamlDecoder(updateData updateDataFn, encoder yqlib.Encoder) yamlDecoderFn {
 	return func(decoder *yaml.Decoder) error {
 		var dataBucket yaml.Node
@@ -154,7 +339,15 @@ func mapYamlDecoder(updateData updateDataFn, encoder yqlib.Encoder) yamlDecoderF
 			log.Debugf("Read doc %v", currentIndex)
 			errorReading = decoder.Decode(&dataBucket)
 
-			if errorReading == io.EOF {
+			if errorReading == io.EOF && docIndexInt == 0 && currentIndex == 0 {
+				//empty document, lets just make one
+				dataBucket = yaml.Node{Kind: yaml.DocumentNode, Content: make([]*yaml.Node, 1)}
+				child := yaml.Node{Kind: yaml.MappingNode}
+				dataBucket.Content[0] = &child
+			} else if isNullDocument(&dataBucket) && (updateAll || docIndexInt == currentIndex) {
+				child := yaml.Node{Kind: yaml.MappingNode}
+				dataBucket.Content[0] = &child
+			} else if errorReading == io.EOF {
 				if !updateAll && currentIndex <= docIndexInt {
 					return fmt.Errorf("asked to process document index %v but there are only %v document(s)", docIndex, currentIndex)
 				}
@@ -167,6 +360,10 @@ func mapYamlDecoder(updateData updateDataFn, encoder yqlib.Encoder) yamlDecoderF
 				return errors.Wrapf(errorUpdating, "Error updating document at index %v", currentIndex)
 			}
 
+			if prettyPrint {
+				updateStyleOfNode(&dataBucket, 0)
+			}
+
 			errorWriting = encoder.Encode(&dataBucket)
 
 			if errorWriting != nil {
@@ -220,11 +417,22 @@ func updateDoc(inputFile string, updateCommands []yqlib.UpdateCommand, writer io
 func readAndUpdate(stdOut io.Writer, inputFile string, updateData updateDataFn) error {
 	var destination io.Writer
 	var destinationName string
+	var completedSuccessfully = false
 	if writeInplace {
 		info, err := os.Stat(inputFile)
 		if err != nil {
 			return err
 		}
+		// mkdir temp dir as some docker images does not have temp dir
+		_, err = os.Stat(os.TempDir())
+		if os.IsNotExist(err) {
+			err = os.Mkdir(os.TempDir(), 0700)
+			if err != nil {
+				return err
+			}
+		} else if err != nil {
+			return err
+		}
 		tempFile, err := ioutil.TempFile("", "temp")
 		if err != nil {
 			return err
@@ -237,7 +445,9 @@ func readAndUpdate(stdOut io.Writer, inputFile string, updateData updateDataFn)
 		destination = tempFile
 		defer func() {
 			safelyCloseFile(tempFile)
-			safelyRenameFile(tempFile.Name(), inputFile)
+			if completedSuccessfully {
+				safelyRenameFile(tempFile.Name(), inputFile)
+			}
 		}()
 	} else {
 		destination = stdOut
@@ -251,11 +461,14 @@ func readAndUpdate(stdOut io.Writer, inputFile string, updateData updateDataFn)
 
 	var encoder yqlib.Encoder
 	if outputToJSON {
-		encoder = yqlib.NewJsonEncoder(bufferedWriter)
+		encoder = yqlib.NewJsonEncoder(bufferedWriter, prettyPrint, indent)
 	} else {
-		encoder = yqlib.NewYamlEncoder(bufferedWriter)
+		encoder = yqlib.NewYamlEncoder(bufferedWriter, indent, colorsEnabled)
 	}
-	return readStream(inputFile, mapYamlDecoder(updateData, encoder))
+
+	var errorProcessing = readStream(inputFile, mapYamlDecoder(updateData, encoder))
+	completedSuccessfully = errorProcessing == nil
+	return errorProcessing
 }
 
 type updateCommandParsed struct {
@@ -283,14 +496,30 @@ func readUpdateCommands(args []string, expectedArgs int, badArgsMessage string)
 		}
 
 		log.Debugf("Read write commands file '%v'", updateCommands)
-	} else if len(args) < expectedArgs {
-		return nil, errors.New(badArgsMessage)
-	} else {
+	} else if sourceYamlFile != "" && len(args) == expectedArgs-1 {
+		log.Debugf("Reading value from %v", sourceYamlFile)
+		var value yaml.Node
+		err := readData(sourceYamlFile, 0, &value)
+		if err != nil && err != io.EOF {
+			return nil, err
+		}
+		log.Debug("args %v", args[expectedArgs-2])
+		updateCommands = make([]yqlib.UpdateCommand, 1)
+		updateCommands[0] = yqlib.UpdateCommand{Command: "update", Path: args[expectedArgs-2], Value: value.Content[0], Overwrite: true}
+	} else if len(args) == expectedArgs {
 		updateCommands = make([]yqlib.UpdateCommand, 1)
 		log.Debug("args %v", args)
 		log.Debug("path %v", args[expectedArgs-2])
 		log.Debug("Value %v", args[expectedArgs-1])
-		updateCommands[0] = yqlib.UpdateCommand{Command: "update", Path: args[expectedArgs-2], Value: valueParser.Parse(args[expectedArgs-1], customTag), Overwrite: true}
+		updateCommands[0] = yqlib.UpdateCommand{Command: "update", Path: args[expectedArgs-2], Value: valueParser.Parse(args[expectedArgs-1], customTag, customStyle, anchorName, makeAlias), Overwrite: true}
+	} else if len(args) == expectedArgs-1 {
+		// don't update the value
+		updateCommands = make([]yqlib.UpdateCommand, 1)
+		log.Debug("args %v", args)
+		log.Debug("path %v", args[expectedArgs-2])
+		updateCommands[0] = yqlib.UpdateCommand{Command: "update", Path: args[expectedArgs-2], Value: valueParser.Parse("", customTag, customStyle, anchorName, makeAlias), Overwrite: true, DontUpdateNodeValue: true}
+	} else {
+		return nil, errors.New(badArgsMessage)
 	}
 	return updateCommands, nil
 }

cmd/validate_test.go

@@ -0,0 +1,31 @@
+package cmd
+
+import (
+	"fmt"
+	"testing"
+
+	"github.com/mikefarah/yq/v3/test"
+)
+
+func TestValidateCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "validate ../examples/sample.yaml b.c")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	test.AssertResult(t, "", result.Output)
+}
+
+func TestValidateBadDataCmd(t *testing.T) {
+	content := `[!Whatever]`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("validate %s", filename))
+	if result.Error == nil {
+		t.Error("Expected command to fail")
+	}
+	expectedOutput := `yaml: line 1: did not find expected ',' or ']'`
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}

cmd/version.go

@@ -11,7 +11,7 @@ var (
 	GitDescribe string
 
 	// Version is main version number that is being run at the moment.
-	Version = "3.0.1"
+	Version = "3.3.2"
 
 	// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
 	// then it means that it is a final release. Otherwise, this is a pre-release

cmd/write.go

@@ -43,8 +43,12 @@ format is list of update commands (update or delete) like so:
 	}
 	cmdWrite.PersistentFlags().BoolVarP(&writeInplace, "inplace", "i", false, "update the yaml file inplace")
 	cmdWrite.PersistentFlags().StringVarP(&writeScript, "script", "s", "", "yaml script for updating yaml")
+	cmdWrite.PersistentFlags().StringVarP(&sourceYamlFile, "from", "f", "", "yaml file for updating yaml (as-is)")
 	cmdWrite.PersistentFlags().StringVarP(&customTag, "tag", "t", "", "set yaml tag (e.g. !!int)")
 	cmdWrite.PersistentFlags().StringVarP(&docIndex, "doc", "d", "0", "process document index number (0 based, * for all documents)")
+	cmdWrite.PersistentFlags().StringVarP(&customStyle, "style", "", "", "formatting style of the value: single, double, folded, flow, literal, tagged")
+	cmdWrite.PersistentFlags().StringVarP(&anchorName, "anchorName", "", "", "anchor name")
+	cmdWrite.PersistentFlags().BoolVarP(&makeAlias, "makeAlias", "", false, "create an alias using the value as the anchor name")
 	return cmdWrite
 }
 

cmd/write_test.go

@@ -0,0 +1,593 @@
+package cmd
+
+import (
+	"fmt"
+	"runtime"
+	"strings"
+	"testing"
+
+	"github.com/mikefarah/yq/v3/test"
+)
+
+func TestWriteCmd(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.c 7", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 7
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteWithTaggedStyleCmd(t *testing.T) {
+	content := `b:
+  c: dog
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.c cat --tag=!!str --style=tagged", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: !!str cat
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteWithDoubleQuotedStyleCmd(t *testing.T) {
+	content := `b:
+  c: dog
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.c cat --style=double", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: "cat"
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteUpdateStyleOnlyCmd(t *testing.T) {
+	content := `b:
+  c: dog
+  d: things
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.* --style=single", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 'dog'
+  d: 'things'
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteUpdateTagOnlyCmd(t *testing.T) {
+	content := `b:
+  c: true
+  d: false
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.* --tag=!!str", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: "true"
+  d: "false"
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteWithSingleQuotedStyleCmd(t *testing.T) {
+	content := `b:
+  c: dog
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.c cat --style=single", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 'cat'
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteWithLiteralStyleCmd(t *testing.T) {
+	content := `b:
+  c: dog
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.c cat --style=literal", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: |-
+    cat
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteWithFoldedStyleCmd(t *testing.T) {
+	content := `b:
+  c: dog
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.c cat --style=folded", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: >-
+    cat
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteEmptyMultiDocCmd(t *testing.T) {
+	content := `# this is empty
+---
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s c 7", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `c: 7
+
+# this is empty
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteSurroundingEmptyMultiDocCmd(t *testing.T) {
+	content := `---
+# empty
+---
+cat: frog
+---
+
+# empty
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s -d1 c 7", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `
+
+# empty
+---
+cat: frog
+c: 7
+---
+
+
+# empty
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteFromFileCmd(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	source := `kittens: are cute # sure are!`
+	fromFilename := test.WriteTempYamlFile(source)
+	defer test.RemoveTempYamlFile(fromFilename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.c -f %s", filename, fromFilename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c:
+    kittens: are cute # sure are!
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteEmptyCmd(t *testing.T) {
+	content := ``
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.c 7", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 7
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteAutoCreateCmd(t *testing.T) {
+	content := `applications:
+  - name: app
+    env:`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s applications[0].env.hello world", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `applications:
+  - name: app
+    env:
+      hello: world
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteCmdScript(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	updateScript := `- command: update
+  path: b.c
+  value: 7`
+	scriptFilename := test.WriteTempYamlFile(updateScript)
+	defer test.RemoveTempYamlFile(scriptFilename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write --script %s %s", scriptFilename, filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 7
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteCmdEmptyScript(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	updateScript := ``
+	scriptFilename := test.WriteTempYamlFile(updateScript)
+	defer test.RemoveTempYamlFile(scriptFilename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write --script %s %s", scriptFilename, filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 3
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteMultiCmd(t *testing.T) {
+	content := `b:
+  c: 3
+---
+apples: great
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s -d 1 apples ok", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 3
+---
+apples: ok
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+func TestWriteInvalidDocumentIndexCmd(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s -df apples ok", filename))
+	if result.Error == nil {
+		t.Error("Expected command to fail due to invalid path")
+	}
+	expectedOutput := `Document index f is not a integer or *: strconv.ParseInt: parsing "f": invalid syntax`
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+
+func TestWriteBadDocumentIndexCmd(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s -d 1 apples ok", filename))
+	if result.Error == nil {
+		t.Error("Expected command to fail due to invalid path")
+	}
+	expectedOutput := `asked to process document index 1 but there are only 1 document(s)`
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+func TestWriteMultiAllCmd(t *testing.T) {
+	content := `b:
+  c: 3
+---
+apples: great
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s -d * apples ok", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: 3
+apples: ok
+---
+apples: ok`
+	test.AssertResult(t, expectedOutput, strings.Trim(result.Output, "\n "))
+}
+
+func TestWriteCmd_EmptyArray(t *testing.T) {
+	content := `b: 3`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s a []", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b: 3
+a: []
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteCmd_Error(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "write")
+	if result.Error == nil {
+		t.Error("Expected command to fail due to missing arg")
+	}
+	expectedOutput := `Must provide <filename> <path_to_update> <value>`
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+
+func TestWriteCmd_ErrorUnreadableFile(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "write fake-unknown a.b 3")
+	if result.Error == nil {
+		t.Error("Expected command to fail due to unknown file")
+	}
+	var expectedOutput string
+	if runtime.GOOS == "windows" {
+		expectedOutput = `open fake-unknown: The system cannot find the file specified.`
+	} else {
+		expectedOutput = `open fake-unknown: no such file or directory`
+	}
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+
+func TestWriteCmd_Inplace(t *testing.T) {
+	content := `b:
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write -i %s b.c 7", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	gotOutput := test.ReadTempYamlFile(filename)
+	expectedOutput := `b:
+  c: 7`
+	test.AssertResult(t, expectedOutput, strings.Trim(gotOutput, "\n "))
+}
+
+func TestWriteCmd_InplaceError(t *testing.T) {
+	content := `b: cat
+  c: 3
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write -i %s b.c 7", filename))
+	if result.Error == nil {
+		t.Error("Expected Error to occur!")
+	}
+	gotOutput := test.ReadTempYamlFile(filename)
+	test.AssertResult(t, content, gotOutput)
+}
+
+func TestWriteCmd_Append(t *testing.T) {
+	content := `b:
+  - foo
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b[+] 7", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  - foo
+  - 7
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteCmd_AppendInline(t *testing.T) {
+	content := `b: [foo]`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b[+] 7", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b: [foo, 7]
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteCmd_AppendInlinePretty(t *testing.T) {
+	content := `b: [foo]`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s -P b[+] 7", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  - foo
+  - 7
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteCmd_AppendEmptyArray(t *testing.T) {
+	content := `a: 2
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b[+] v", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `a: 2
+b:
+  - v
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteCmd_SplatArray(t *testing.T) {
+	content := `b:
+- c: thing
+- c: another thing
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b[*].c new", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  - c: new
+  - c: new
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteCmd_SplatMap(t *testing.T) {
+	content := `b:
+  c: thing
+  d: another thing
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.* new", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: new
+  d: new
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestWriteCmd_SplatMapEmpty(t *testing.T) {
+	content := `b:
+  c: thing
+  d: another thing
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("write %s b.c.* new", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: {}
+  d: another thing
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}

convert.md

@@ -0,0 +1,52 @@
+# Convert
+
+### Yaml to Json[¶](convert.md#yaml-to-json) <a id="yaml-to-json"></a>
+
+To convert output to json, use the --tojson \(or -j\) flag. This can only be used with the read command.
+
+Given a sample.yaml file of:
+
+```text
+b:
+  c: 2
+```
+
+then
+
+```text
+yq r -j sample.yaml b.c
+```
+
+will output
+
+```text
+{"b":{"c":2}}
+```
+
+### Json to Yaml[¶](convert.md#json-to-yaml) <a id="json-to-yaml"></a>
+
+To read in json, just pass in a json file instead of yaml, it will just work :\)
+
+e.g given a json file
+
+```text
+{"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}
+```
+
+then
+
+```text
+yq r sample.json
+```
+
+will output
+
+```text
+a: Easy! as one two three
+b:
+  c: 2
+  d:
+  - 3
+  - 4
+```
+

create.md

@@ -0,0 +1,88 @@
+# Create
+
+ Yaml files can be created using the 'new' command. This works in the same way as the write command, but you don't pass in an existing Yaml file. Currently this does not support creating multiple documents in a single yaml file.
+
+```text
+yq n <path> <new value>
+```
+
+### Creating a simple yaml file[¶](create.md#creating-a-simple-yaml-file) <a id="creating-a-simple-yaml-file"></a>
+
+```text
+yq n b.c cat
+```
+
+will output:
+
+```text
+b:
+  c: cat
+```
+
+### Creating using a create script[¶](create.md#creating-using-a-create-script) <a id="creating-using-a-create-script"></a>
+
+Create scripts follow the same format as the update scripts.
+
+Given a script create\_instructions.yaml of:
+
+```text
+b.c: 3
+b.e[+].name: Howdy Partner
+```
+
+then
+
+```text
+yq n -s create_instructions.yaml
+```
+
+will output:
+
+```text
+b:
+  c: 3
+  e:
+    - name: Howdy Partner
+```
+
+You can also pipe the instructions in:
+
+```text
+cat create_instructions.yaml | yq n -s -
+```
+
+### Keys with dots[¶](create.md#keys-with-dots) <a id="keys-with-dots"></a>
+
+When specifying a key that has a dot use key lookup indicator.
+
+```text
+b:
+  foo.bar: 7
+```
+
+```text
+yaml r sample.yaml 'b[foo.bar]'
+```
+
+```text
+yaml w sample.yaml 'b[foo.bar]' 9
+```
+
+Any valid yaml key can be specified as part of a key lookup.
+
+Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
+
+### Keys \(and values\) with leading dashes[¶](create.md#keys-and-values-with-leading-dashes) <a id="keys-and-values-with-leading-dashes"></a>
+
+If a key or value has leading dashes, yq won't know that you are passing a value as opposed to a flag \(and you will get a 'bad flag syntax' error\).
+
+To fix that, you will need to tell it to stop processing flags by adding '--' after the last flag like so:
+
+```text
+yq n -t -- --key --value
+```
+
+Will result in
+
+``` --key: --value``
+

debian/changelog

@@ -1,3 +1,41 @@
+yq (3.3-0) focal; urgency=medium
+
+  * You can control string styles (quotes) using the new --style flag
+  * String values now always have quotes when outputting to json
+  * Negative array indices now traverse the array backwards
+  * Added a --stripComments flag to print yaml without any comments
+  * Bumped go to version 1.14
+
+ -- Roberto Mier Escandon <rmescandon@gmail.com>  Thu, 30 Apr 2020 20:45:44 +0200
+
+yq (3.1-2) eoan; urgency=medium
+
+  * Bug fix: yq 3 was removing empty inline-style objects and arrays (#355)
+  * Bug fix: Merge option returned different output when switching order of 
+    merging files(#347)
+  * Bug fix: Add new object to existing array object was failing in 3.1.1 (#361)
+  * Bug fix: yq 3 empty keys did not allow merging of values (#356)
+  * Bug fix: keys quoted during merge (#363)
+  * Bug fix: Correct length with wc -l (#362)
+  * Bug fix: Write to empty document removed path (#359)
+
+ -- Roberto Mier Escandon <rmescandon@gmail.com>  Mon, 24 Feb 2020 20:31:58 +0100
+
+yq (3.1-1) eoan; urgency=medium
+
+  * Keeps yaml comments and formatting, can specify yaml tags when updating.
+  * Handles anchors
+  * Can print out matching paths and values when splatting
+  * JSON output works for all commands
+  * Yaml files with multiple documents are printed out as one JSON 
+    document per line.
+  * Deep splat (**) to match arbitrary paths
+  * Update scripts file format has changed to be more powerful
+  * Reading and splatting, matching results are printed once per line
+  * Bugfixing
+
+ -- Roberto Mier Escandon <rmescandon@gmail.com>  Tue, 11 Feb 2020 22:18:24 +0100
+
 yq (2.2-1) bionic; urgency=medium
 
   * Added Windows support for the "--inplace" command flag

debian/compat

@@ -1 +1 @@
-9
+10

debian/control

@@ -1,22 +1,22 @@
 Source: yq
 Section: devel
-Priority: extra
+Priority: optional
 Maintainer: Roberto Mier Escandón <rmescandon@gmail.com>
-Build-Depends: debhelper (>= 9),
-               dh-golang,
-               golang-1.10-go,
+Build-Depends: debhelper (>=10),
+               dh-golang (>=1.34),
+               golang-1.13,
                rsync
-Standards-Version: 3.9.6
+Standards-Version: 4.1.4
 Homepage: https://github.com/mikefarah/yq.git
 Vcs-Browser: https://github.com/mikefarah/yq.git
 Vcs-Git: https://github.com/mikefarah/yq.git
+XS-Go-Import-Path: github.com/mikefarah/yq
+XSBC-Original-Maintainer: Roberto Mier Escandón <rmescandon@gmail.com>
 
 Package: yq
 Architecture: any
-Built-Using: ${misc:Built-Using}
-Depends: ${shlibs:Depends},
-         ${misc:Depends}
-Description:
- a lightweight and portable command-line YAML processor
+Depends: ${shlibs:Depends}, ${misc:Depends}
+Description: lightweight and portable command-line YAML processor
  .
- The aim of the project is to be the [jq](https://github.com/stedolan/jq) or sed of yaml files.
+ The aim of the project is to be the 
+ [jq](https://github.com/stedolan/jq) or sed of yaml files.

debian/copyright

@@ -3,5 +3,22 @@ Upstream-Name: yq
 Source: https://github.com/mikefarah/yq.git
 
 Files: *
-Copyright: 2017 Mike Farah Ltd. All rights reserved
-License: Proprietary
+Copyright: 2017 Mike Farah
+License: Expat
+    Permission is hereby granted, free of charge, to any person obtaining a copy
+    of this software and associated documentation files (the "Software"), to deal
+    in the Software without restriction, including without limitation the rights
+    to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+    copies of the Software, and to permit persons to whom the Software is
+    furnished to do so, subject to the following conditions:
+    .
+    The above copyright notice and this permission notice shall be included in all
+    copies or substantial portions of the Software.
+    .
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+    IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+    AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+    OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+    SOFTWARE.

debian/rules

@@ -14,46 +14,44 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-
 PROJECT := yq
 OWNER := mikefarah
 REPO := github.com
-GOVERSION := 1.10
+GOVERSION := 1.13
+
 export DH_OPTIONS
 export DH_GOPKG := ${REPO}/${OWNER}/${PROJECT}
 export GOROOT := /usr/lib/go-${GOVERSION}
 export GOPATH := ${CURDIR}/_build
 export GOBIN := ${GOPATH}/bin
 export PATH := ${GOROOT}/bin:${GOBIN}:${PATH}
-BLDPATH := $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE)
-SRCDIR := ${CURDIR}/_build/src/${DH_GOPKG}
+export GOCACHE := /tmp/gocache
+export GOFLAGS := -mod=vendor
+
+SRCDIR := ${GOPATH}/src/${DH_GOPKG}
 DESTDIR := ${CURDIR}/debian/${PROJECT}
 BINDIR := /usr/bin
 ASSETSDIR := /usr/share/${PROJECT}
 
 %:
-	dh $@ --buildsystem=golang --with=golang
+	dh $@ --builddirectory=${GOPATH} --buildsystem=golang
 
 override_dh_auto_build:
 	mkdir -p ${SRCDIR}
 	mkdir -p ${GOBIN}
 	# copy project to local srcdir to build from there
-	rsync -avz --progress --exclude=obj-${BLDPATH} --exclude=debian . $(SRCDIR)
+	rsync -avz --progress --exclude=_build --exclude=debian --exclude=tmp. --exclude=go.mod --exclude=docs . $(SRCDIR)
 	# build go code
-	(cd ${SRCDIR} && go install ./...)
-
+	(cd ${SRCDIR} && go install -buildmode=pie ./...)
+	
 override_dh_auto_test:
 	(cd ${SRCDIR} && go test -v ./...)
 
 override_dh_auto_install:
-	mkdir -p ${DESTDIR}/${BINDIR}
-	mkdir -p ${DESTDIR}/${ASSETSDIR}
-	cp ${CURDIR}/_build/bin/yq ${DESTDIR}/${BINDIR}
-	cp -rf ${SRCDIR}/LICENSE ${DESTDIR}/${ASSETSDIR}
-	cp -rf ${SRCDIR}/README.md ${DESTDIR}/${PLUGINSDIR}
+	cp ${GOBIN}/yq ${DESTDIR}/${BINDIR}
+	cp -f ${SRCDIR}/LICENSE ${DESTDIR}/${ASSETSDIR}
 	chmod a+x ${DESTDIR}/${BINDIR}/yq
 
 override_dh_auto_clean:
 	dh_clean
-	rm -rf ${CURDIR}/obj-${BLDPATH}
 	rm -rf ${CURDIR}/_build

debian/yq.dirs

@@ -0,0 +1,3 @@
+usr/bin
+usr/share/yq
+usr/share/man/man1

delete.md

@@ -0,0 +1,280 @@
+# Delete
+
+```text
+yq d <yaml_file> <path_to_delete>
+```
+
+### To Stdout[¶](delete.md#to-stdout) <a id="to-stdout"></a>
+
+Given a sample.yaml file of:
+
+```text
+b:
+  c: 2
+  apples: green
+```
+
+then
+
+```text
+yq d sample.yaml b.c
+```
+
+will output:
+
+```text
+b:
+  apples: green
+```
+
+### From STDIN[¶](delete.md#from-stdin) <a id="from-stdin"></a>
+
+```text
+cat sample.yaml | yq d - b.c
+```
+
+### Deleting array elements[¶](delete.md#deleting-array-elements) <a id="deleting-array-elements"></a>
+
+Given a sample.yaml file of:
+
+```text
+b:
+  c: 
+    - 1
+    - 2
+    - 3
+```
+
+then
+
+```text
+yq d sample.yaml 'b.c[1]'
+```
+
+will output:
+
+```text
+b:
+  c:
+  - 1
+  - 3
+```
+
+### Deleting nodes in-place[¶](delete.md#deleting-nodes-in-place) <a id="deleting-nodes-in-place"></a>
+
+Given a sample.yaml file of:
+
+```text
+b:
+  c: 2
+  apples: green
+```
+
+then
+
+```text
+yq d -i sample.yaml b.c
+```
+
+will update the sample.yaml file so that the 'c' node is deleted
+
+### Splat[¶](delete.md#splat) <a id="splat"></a>
+
+Given a sample.yaml file of:
+
+```text
+---
+bob:
+  item1:
+    cats: bananas
+    dogs: woof
+  item2:
+    cats: apples
+    dogs: woof2
+  thing:
+    cats: oranges
+    dogs: woof3
+```
+
+then
+
+```text
+yq d sample.yaml bob.*.cats
+```
+
+will output:
+
+```text
+---
+bob:
+  item1:
+    dogs: woof
+  item2:
+    dogs: woof2
+  thing:
+    dogs: woof3
+```
+
+### Prefix Splat[¶](delete.md#prefix-splat) <a id="prefix-splat"></a>
+
+Given a sample.yaml file of:
+
+```text
+---
+bob:
+  item1:
+    cats: bananas
+    dogs: woof
+  item2:
+    cats: apples
+    dogs: woof2
+  thing:
+    cats: oranges
+    dogs: woof3
+```
+
+then
+
+```text
+yq d sample.yaml bob.item*.cats
+```
+
+will output:
+
+```text
+---
+bob:
+  item1:
+    dogs: woof
+  item2:
+    dogs: woof2
+  thing:
+    cats: oranges
+    dogs: woof3
+```
+
+### Array Splat[¶](delete.md#array-splat) <a id="array-splat"></a>
+
+Given a sample.yaml file of:
+
+```text
+---
+bob:
+- cats: bananas
+  dogs: woof
+- cats: apples
+  dogs: woof2
+- cats: oranges
+  dogs: woof3
+```
+
+then
+
+```text
+yq d sample.yaml bob.[*].cats
+```
+
+will output:
+
+```text
+---
+bob:
+- dogs: woof
+- dogs: woof2
+- dogs: woof3
+```
+
+### Multiple Documents - delete from single document[¶](delete.md#multiple-documents-delete-from-single-document) <a id="multiple-documents-delete-from-single-document"></a>
+
+Given a sample.yaml file of:
+
+```text
+something: else
+field: leaveMe
+---
+b:
+  c: 2
+field: deleteMe
+```
+
+then
+
+```text
+yq w -d1 sample.yaml field
+```
+
+will output:
+
+```text
+something: else
+field: leaveMe
+---
+b:
+  c: 2
+```
+
+### Multiple Documents - delete from all documents[¶](delete.md#multiple-documents-delete-from-all-documents) <a id="multiple-documents-delete-from-all-documents"></a>
+
+Given a sample.yaml file of:
+
+```text
+something: else
+field: deleteMe
+---
+b:
+  c: 2
+field: deleteMeToo
+```
+
+then
+
+```text
+yq w -d'*' sample.yaml field
+```
+
+will output:
+
+```text
+something: else
+---
+b:
+  c: 2
+```
+
+Note that '\*' is in quotes to avoid being interpreted by your shell.
+
+### Keys with dots[¶](delete.md#keys-with-dots) <a id="keys-with-dots"></a>
+
+When specifying a key that has a dot use key lookup indicator.
+
+```text
+b:
+  foo.bar: 7
+```
+
+```text
+yaml r sample.yaml 'b[foo.bar]'
+```
+
+```text
+yaml w sample.yaml 'b[foo.bar]' 9
+```
+
+Any valid yaml key can be specified as part of a key lookup.
+
+Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
+
+### Keys \(and values\) with leading dashes[¶](delete.md#keys-and-values-with-leading-dashes) <a id="keys-and-values-with-leading-dashes"></a>
+
+If a key or value has leading dashes, yq won't know that you are passing a value as opposed to a flag \(and you will get a 'bad flag syntax' error\).
+
+To fix that, you will need to tell it to stop processing flags by adding '--' after the last flag like so:
+
+```text
+yq n -t -- --key --value
+```
+
+Will result in
+
+``` --key: --value``
+

examples/data1-no-comments.yaml

@@ -0,0 +1,4 @@
+a: simple
+b: [1, 2]
+c:
+  test: 1

examples/data2.yaml

@@ -4,4 +4,4 @@ c:
   toast: leave
   test: 1
   tell: 1
-  taco: cool
+  tasty.taco: cool

examples/data3.yaml

@@ -1,14 +1,4 @@
-deep1:
-   hostA:
-      value: 1234
-   notRelevant:
-      value: bananas
-   hostB:
-      value: 5678
-deep2:
-   hostC:
-      value: 1234
-   notRelevant:
-      value: bananas
-   hostD:
-      value: 5678
+a: "simple" # just the best
+b: [1, 3]
+c:
+  test: 1

examples/sample2.yaml

@@ -1,2 +0,0 @@
-b:
-  c: things

examples/simple-anchor-exploded.yaml

@@ -0,0 +1,5 @@
+foo:
+  a: 1
+
+foobar: 
+  a: 1

examples/simple-anchor.yaml

@@ -1,4 +1,5 @@
 foo: &foo
   a: 1
 
-foobar: *foo
+foobar: 
+  <<: *foo

github-action/Dockerfile

@@ -0,0 +1,5 @@
+FROM mikefarah/yq:3
+
+COPY entrypoint.sh /entrypoint.sh
+
+ENTRYPOINT ["/entrypoint.sh"]

github-action/entrypoint.sh

@@ -0,0 +1,4 @@
+#!/bin/sh -l
+
+echo "$1"
+eval $1

go.mod

@@ -1,12 +1,17 @@
 module github.com/mikefarah/yq/v3
 
 require (
-	github.com/pkg/errors v0.8.1
-	github.com/spf13/cobra v0.0.5
-	golang.org/x/tools v0.0.0-20191213221258-04c2e8eff935 // indirect
-	gopkg.in/imdario/mergo.v0 v0.3.7 // indirect
+	github.com/fatih/color v1.9.0
+	github.com/goccy/go-yaml v1.7.5
+	github.com/kylelemons/godebug v1.1.0
+	github.com/mattn/go-colorable v0.1.6 // indirect
+	github.com/pkg/errors v0.9.1
+	github.com/spf13/cobra v1.0.0
+	github.com/spf13/pflag v1.0.5 // indirect
+	golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980 // indirect
+	golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 // indirect
 	gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473
-	gopkg.in/yaml.v3 v3.0.0-20191120175047-4206685974f2
+	gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c
 )
 
-go 1.13
+go 1.14

go.sum

@@ -1,62 +1,180 @@
+cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
 github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
+github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
+github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
+github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
 github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
+github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
+github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
+github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
+github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
+github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
 github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
-github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
 github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
-github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
+github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
+github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
+github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
+github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
+github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
+github.com/fatih/color v1.9.0 h1:8xPHl4/q1VyqGIPif1F+1V3Y3lSmrq01EabUW3CoW5s=
+github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL+zU=
 github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
+github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
+github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
+github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
+github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
+github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
+github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
+github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
+github.com/goccy/go-yaml v1.7.5 h1:dWvj+p3BG11S/GlUzwzt1WZz0lhBTzTIDtmXT/ZOaPY=
+github.com/goccy/go-yaml v1.7.5/go.mod h1:wS4gNoLalDSJxo/SpngzPQ2BN4uuZVLCmbM4S3vd4+Y=
+github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
+github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
+github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
+github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
+github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
+github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
+github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
+github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
+github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
 github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
 github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
 github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
+github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
+github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
+github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
+github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
+github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
+github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
+github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
+github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
+github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
+github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
+github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
+github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
+github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII=
 github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
-github.com/mikefarah/yaml v2.1.0+incompatible h1:nu2cqmzk4WlWJNgnevY88faMcdrDzYGcsUjYFxEpB7Y=
-github.com/mikefarah/yaml/v2 v2.4.0 h1:eYqfooY0BnvKTJxr7+ABJs13n3dg9n347GScDaU2Lww=
-github.com/mikefarah/yaml/v2 v2.4.0/go.mod h1:ahVqZF4n1W4NqwvVnZzC4es67xsW9uR/RRf2RRxieJU=
-github.com/mikefarah/yq v2.4.0+incompatible h1:oBxbWy8R9hI3BIUUxEf0CzikWa2AgnGrGhvGQt5jgjk=
-github.com/mikefarah/yq/v2 v2.4.1 h1:tajDonaFK6WqitSZExB6fKlWQy/yCkptqxh2AXEe3N4=
+github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA=
+github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
+github.com/mattn/go-colorable v0.1.6 h1:6Su7aK7lXmJ/U79bYtBjLNaha4Fs1Rg9plHpcH+vvnE=
+github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
+github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
+github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84=
+github.com/mattn/go-isatty v0.0.11 h1:FxPOTFNqGkuDUGi3H/qkUbQO4ZiBa2brKq5r0l8TGeM=
+github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
+github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY=
+github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
+github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
 github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
 github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
+github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
+github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
 github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
-github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
-github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
+github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
-github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
+github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
+github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso=
+github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
+github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
+github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
+github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
+github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
+github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
+github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
+github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
+github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
+github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
+github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
+github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
 github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
 github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
-github.com/spf13/cobra v0.0.5 h1:f0B+LkLX6DtmRH1isoNA9VTtNUK9K8xYd28JNNfOv/s=
-github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=
+github.com/spf13/cobra v1.0.0 h1:6m/oheQuQ13N9ks4hubMG6BnvwOeaJrqSPLahSnczz8=
+github.com/spf13/cobra v1.0.0/go.mod h1:/6GTrnGXV9HjY+aR4k0oJ5tcvakLuG6EuKReYlHNrgE=
 github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
 github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
 github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
-github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
+github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
+github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
+github.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
-github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
+github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
+github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
+github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
+github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
 github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
-golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
+go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
+go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
+go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
+golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
-golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
-golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
-golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
-golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
-golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
+golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
+golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
-golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191010194322-b09406accb47/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=
+golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980 h1:OjiUf46hAmXblsZdnoSXsEUSKU8r1UEzcL5RVZ4gO9Y=
+golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
-golang.org/x/tools v0.0.0-20191030203535-5e247c9ad0a0 h1:s5lp4ug7qHzUccgyFdjsX7OZDzHXRaePrF3B3vmUiuM=
-golang.org/x/tools v0.0.0-20191030203535-5e247c9ad0a0/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
-golang.org/x/tools v0.0.0-20191213221258-04c2e8eff935 h1:kJQZhwFzSwJS2BxboKjdZzWczQOZx8VuH7Y8hhuGUtM=
-golang.org/x/tools v0.0.0-20191213221258-04c2e8eff935/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
-golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
+golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
+golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
+google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
+google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
+google.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
+gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
-gopkg.in/imdario/mergo.v0 v0.3.7 h1:QDotlIZtaO/p+Um0ok18HRTpq5i5/SAk/qprsor+9c8=
-gopkg.in/imdario/mergo.v0 v0.3.7/go.mod h1:9qPP6AGrlC1G2PTNXko614FwGZvorN7MiBU0Eppok+U=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/go-playground/assert.v1 v1.2.1/go.mod h1:9RXL0bg/zibRAgZUYszZSwO/z8Y/a8bDuhia5mkpMnE=
+gopkg.in/go-playground/validator.v9 v9.30.0/go.mod h1:+c9/zcJMFNgbLvly1L1V+PpxWdVbfP1avr/N00E2vyQ=
 gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473 h1:6D+BvnJ/j6e222UW8s2qTSe3wGBtvo0MbVQG/c5k8RE=
 gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473/go.mod h1:N1eN2tsCx0Ydtgjl4cqmbRCsY4/+z4cYDeqwZTk6zog=
+gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
+gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
+gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
 gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v3 v3.0.0-20191120175047-4206685974f2 h1:XZx7nhd5GMaZpmDaEHFVafUZC7ya0fuo7cSJ3UCKYmM=
-gopkg.in/yaml.v3 v3.0.0-20191120175047-4206685974f2/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c h1:grhR+C34yXImVGp7EzNk+DTIk+323eIUWOmEevy6bDo=
+gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=

merge.md

@@ -0,0 +1,245 @@
+# Merge
+
+ Yaml files can be merged using the 'merge' command. Each additional file merged with the first file will set values for any key not existing already or where the key has no value.
+
+```text
+yq m <yaml_file> <path>...
+```
+
+### To Stdout[¶](merge.md#to-stdout) <a id="to-stdout"></a>
+
+Given a data1.yaml file of:
+
+```text
+a: simple
+b: [1, 2]
+```
+
+and data2.yaml file of:
+
+```text
+a: other
+c:
+  test: 1
+```
+
+then
+
+```text
+yq m data1.yaml data2.yaml
+```
+
+will output:
+
+```text
+a: simple
+b: [1, 2]
+c:
+  test: 1
+```
+
+### Updating files in-place[¶](merge.md#updating-files-in-place) <a id="updating-files-in-place"></a>
+
+Given a data1.yaml file of:
+
+```text
+a: simple
+b: [1, 2]
+```
+
+and data2.yaml file of:
+
+```text
+a: other
+c:
+  test: 1
+```
+
+then
+
+```text
+yq m -i data1.yaml data2.yaml
+```
+
+will update the data1.yaml file so that the value of 'c' is 'test: 1'.
+
+### Overwrite values[¶](merge.md#overwrite-values) <a id="overwrite-values"></a>
+
+Given a data1.yaml file of:
+
+```text
+a: simple
+b: [1, 2]
+```
+
+and data2.yaml file of:
+
+```text
+a: other
+c:
+  test: 1
+```
+
+then
+
+```text
+yq m -x data1.yaml data2.yaml
+```
+
+will output:
+
+```text
+a: other
+b: [1, 2]
+c:
+  test: 1
+```
+
+### Overwrite values with arrays[¶](merge.md#overwrite-values-with-arrays) <a id="overwrite-values-with-arrays"></a>
+
+Given a data1.yaml file of:
+
+```text
+a: simple
+b: [1, 2]
+```
+
+and data3.yaml file of:
+
+```text
+b: [3, 4]
+c:
+  test: 2
+  other: true
+d: false
+```
+
+then
+
+```text
+yq m -x data1.yaml data3.yaml
+```
+
+will output:
+
+```text
+a: simple
+b: [3, 4]
+c:
+  test: 2
+  other: true
+d: false
+```
+
+Notice that 'b' does not result in the merging of the values within an array.
+
+### Append values with arrays[¶](merge.md#append-values-with-arrays) <a id="append-values-with-arrays"></a>
+
+Given a data1.yaml file of:
+
+```text
+a: simple
+b: [1, 2]
+d: hi
+```
+
+and data3.yaml file of:
+
+```text
+a: something
+b: [3, 4]
+c:
+  test: 2
+  other: true
+```
+
+then
+
+```text
+yq m -a data1.yaml data3.yaml
+```
+
+will output:
+
+```text
+a: simple
+b: [1, 2, 3, 4]
+c:
+  test: 2
+  other: true
+d: hi
+```
+
+Note that the 'b' array has concatenated the values from the second data file. Also note that other map keys are not overridden \(field a\).
+
+Append cannot be used with overwrite, if both flags are given then append is ignored.
+
+### Multiple Documents - merge into single document[¶](merge.md#multiple-documents-merge-into-single-document) <a id="multiple-documents-merge-into-single-document"></a>
+
+Currently yq only has multi-document support for the _first_ document being merged into. The remaining yaml files will have their first document selected.
+
+Given a data1.yaml file of:
+
+```text
+something: else
+---
+a: simple
+b: cat
+```
+
+and data3.yaml file of:
+
+```text
+b: dog
+```
+
+then
+
+```text
+yq m -x -d1 data1.yaml data3.yaml
+```
+
+will output:
+
+```text
+something: else
+---
+a: simple
+b: dog
+```
+
+### Multiple Documents - merge into all documents[¶](merge.md#multiple-documents-merge-into-all-documents) <a id="multiple-documents-merge-into-all-documents"></a>
+
+Currently yq only has multi-document support for the _first_ document being merged into. The remaining yaml files will have their first document selected.
+
+Given a data1.yaml file of:
+
+```text
+something: else
+---
+a: simple
+b: cat
+```
+
+and data3.yaml file of:
+
+```text
+b: dog
+```
+
+then
+
+```text
+yq m -x -d'*' data1.yaml data3.yaml
+```
+
+will output:
+
+```text
+b: dog
+something: else
+---
+a: simple
+b: dog
+```
+

pkg/yqlib/color_print.go

@@ -0,0 +1,61 @@
+package yqlib
+
+import (
+	"fmt"
+	"io"
+
+	"github.com/fatih/color"
+	"github.com/goccy/go-yaml/lexer"
+	"github.com/goccy/go-yaml/printer"
+)
+
+// Thanks @risentveber!
+
+const escape = "\x1b"
+
+func format(attr color.Attribute) string {
+	return fmt.Sprintf("%s[%dm", escape, attr)
+}
+
+func ColorizeAndPrint(bytes []byte, writer io.Writer) error {
+	tokens := lexer.Tokenize(string(bytes))
+	var p printer.Printer
+	p.Bool = func() *printer.Property {
+		return &printer.Property{
+			Prefix: format(color.FgHiMagenta),
+			Suffix: format(color.Reset),
+		}
+	}
+	p.Number = func() *printer.Property {
+		return &printer.Property{
+			Prefix: format(color.FgHiMagenta),
+			Suffix: format(color.Reset),
+		}
+	}
+	p.MapKey = func() *printer.Property {
+		return &printer.Property{
+			Prefix: format(color.FgCyan),
+			Suffix: format(color.Reset),
+		}
+	}
+	p.Anchor = func() *printer.Property {
+		return &printer.Property{
+			Prefix: format(color.FgHiYellow),
+			Suffix: format(color.Reset),
+		}
+	}
+	p.Alias = func() *printer.Property {
+		return &printer.Property{
+			Prefix: format(color.FgHiYellow),
+			Suffix: format(color.Reset),
+		}
+	}
+	p.String = func() *printer.Property {
+		return &printer.Property{
+			Prefix: format(color.FgGreen),
+			Suffix: format(color.Reset),
+		}
+	}
+	_, err := writer.Write([]byte(p.PrintTokens(tokens) + "\n"))
+	return err
+}

pkg/yqlib/data_navigator.go

@@ -1,13 +1,14 @@
 package yqlib
 
 import (
+	"fmt"
 	"strconv"
 
 	yaml "gopkg.in/yaml.v3"
 )
 
 type DataNavigator interface {
-	Traverse(value *yaml.Node, path []string) error
+	Traverse(value *yaml.Node, path []interface{}) error
 }
 
 type navigator struct {
@@ -20,9 +21,10 @@ func NewDataNavigator(NavigationStrategy NavigationStrategy) DataNavigator {
 	}
 }
 
-func (n *navigator) Traverse(value *yaml.Node, path []string) error {
+func (n *navigator) Traverse(value *yaml.Node, path []interface{}) error {
 	realValue := value
 	emptyArray := make([]interface{}, 0)
+	log.Debugf("Traversing path %v", pathStackToString(path))
 	if realValue.Kind == yaml.DocumentNode {
 		log.Debugf("its a document! returning the first child")
 		return n.doTraverse(value.Content[0], "", path, emptyArray)
@@ -30,12 +32,17 @@ func (n *navigator) Traverse(value *yaml.Node, path []string) error {
 	return n.doTraverse(value, "", path, emptyArray)
 }
 
-func (n *navigator) doTraverse(value *yaml.Node, head string, tail []string, pathStack []interface{}) error {
+func (n *navigator) doTraverse(value *yaml.Node, head interface{}, tail []interface{}, pathStack []interface{}) error {
+
 	log.Debug("head %v", head)
 	DebugNode(value)
+	var nodeContext = NewNodeContext(value, head, tail, pathStack)
+
 	var errorDeepSplatting error
-	if head == "**" && value.Kind != yaml.ScalarNode {
-		errorDeepSplatting = n.recurse(value, head, tail, pathStack)
+	if head == "**" && value.Kind != yaml.ScalarNode && n.navigationStrategy.ShouldDeeplyTraverse(nodeContext) {
+		if len(pathStack) == 0 || pathStack[len(pathStack)-1] != "<<" {
+			errorDeepSplatting = n.recurse(value, head, tail, pathStack)
+		}
 		// ignore errors here, we are deep splatting so we may accidently give a string key
 		// to an array sequence
 		if len(tail) > 0 {
@@ -44,66 +51,80 @@ func (n *navigator) doTraverse(value *yaml.Node, head string, tail []string, pat
 		return errorDeepSplatting
 	}
 
-	if len(tail) > 0 {
+	if len(tail) > 0 && value.Kind != yaml.ScalarNode {
 		log.Debugf("diving into %v", tail[0])
 		DebugNode(value)
 		return n.recurse(value, tail[0], tail[1:], pathStack)
 	}
-	return n.navigationStrategy.Visit(NewNodeContext(value, head, tail, pathStack))
+	return n.navigationStrategy.Visit(nodeContext)
 }
 
 func (n *navigator) getOrReplace(original *yaml.Node, expectedKind yaml.Kind) *yaml.Node {
 	if original.Kind != expectedKind {
-		log.Debug("wanted %v but it was %v, overriding", expectedKind, original.Kind)
+		log.Debug("wanted %v but it was %v, overriding", KindString(expectedKind), KindString(original.Kind))
 		return &yaml.Node{Kind: expectedKind}
 	}
 	return original
 }
 
-func (n *navigator) recurse(value *yaml.Node, head string, tail []string, pathStack []interface{}) error {
-	log.Debug("recursing, processing %v", head)
+func (n *navigator) recurse(value *yaml.Node, head interface{}, tail []interface{}, pathStack []interface{}) error {
+	log.Debug("recursing, processing %v, pathStack %v", head, pathStackToString(pathStack))
+
+	nodeContext := NewNodeContext(value, head, tail, pathStack)
+
+	if head == "**" && !n.navigationStrategy.ShouldOnlyDeeplyVisitLeaves(nodeContext) {
+		errorVisitingDeeply := n.navigationStrategy.Visit(nodeContext)
+		if errorVisitingDeeply != nil {
+			return errorVisitingDeeply
+		}
+	}
+
 	switch value.Kind {
 	case yaml.MappingNode:
 		log.Debug("its a map with %v entries", len(value.Content)/2)
-		return n.recurseMap(value, head, tail, pathStack)
+		headString := fmt.Sprintf("%v", head)
+		return n.recurseMap(value, headString, tail, pathStack)
+
 	case yaml.SequenceNode:
 		log.Debug("its a sequence of %v things!", len(value.Content))
 
-		var index, errorParsingIndex = strconv.ParseInt(head, 10, 64) // nolint
-		if errorParsingIndex == nil {
-			return n.recurseArray(value, index, head, tail, pathStack)
-		} else if head == "+" {
-			return n.appendArray(value, head, tail, pathStack)
-		}
-		return n.splatArray(value, head, tail, pathStack)
+		switch head := head.(type) {
+		case int64:
+			return n.recurseArray(value, head, head, tail, pathStack)
+		default:
 
+			if head == "+" {
+				return n.appendArray(value, head, tail, pathStack)
+			} else if len(value.Content) == 0 && head == "**" {
+				return n.navigationStrategy.Visit(nodeContext)
+			}
+			return n.splatArray(value, head, tail, pathStack)
+		}
 	case yaml.AliasNode:
 		log.Debug("its an alias!")
 		DebugNode(value.Alias)
-		if n.navigationStrategy.FollowAlias(NewNodeContext(value, head, tail, pathStack)) {
+		if n.navigationStrategy.FollowAlias(nodeContext) {
 			log.Debug("following the alias")
 			return n.recurse(value.Alias, head, tail, pathStack)
 		}
 		return nil
 	default:
-		return nil
+		return n.navigationStrategy.Visit(nodeContext)
 	}
 }
 
-func (n *navigator) recurseMap(value *yaml.Node, head string, tail []string, pathStack []interface{}) error {
+func (n *navigator) recurseMap(value *yaml.Node, head string, tail []interface{}, pathStack []interface{}) error {
 	traversedEntry := false
 	errorVisiting := n.visitMatchingEntries(value, head, tail, pathStack, func(contents []*yaml.Node, indexInMap int) error {
-		log.Debug("recurseMap: visitMatchingEntries")
+		log.Debug("recurseMap: visitMatchingEntries for %v", contents[indexInMap].Value)
 		n.navigationStrategy.DebugVisitedNodes()
 		newPathStack := append(pathStack, contents[indexInMap].Value)
-		log.Debug("appended %v", contents[indexInMap].Value)
-		n.navigationStrategy.DebugVisitedNodes()
 		log.Debug("should I traverse? head: %v, path: %v", head, pathStackToString(newPathStack))
 		DebugNode(value)
 		if n.navigationStrategy.ShouldTraverse(NewNodeContext(contents[indexInMap+1], head, tail, newPathStack), contents[indexInMap].Value) {
 			log.Debug("recurseMap: Going to traverse")
 			traversedEntry = true
-			// contents[indexInMap+1] = n.getOrReplace(contents[indexInMap+1], guessKind(head, tail, contents[indexInMap+1].Kind))
+			contents[indexInMap+1] = n.getOrReplace(contents[indexInMap+1], guessKind(head, tail, contents[indexInMap+1].Kind))
 			errorTraversing := n.doTraverse(contents[indexInMap+1], head, tail, newPathStack)
 			log.Debug("recurseMap: Finished traversing")
 			n.navigationStrategy.DebugVisitedNodes()
@@ -118,22 +139,33 @@ func (n *navigator) recurseMap(value *yaml.Node, head string, tail []string, pat
 		return errorVisiting
 	}
 
-	if traversedEntry || n.navigationStrategy.GetPathParser().IsPathExpression(head) || !n.navigationStrategy.AutoCreateMap(NewNodeContext(value, head, tail, pathStack)) {
+	if len(value.Content) == 0 && head == "**" {
+		return n.navigationStrategy.Visit(NewNodeContext(value, head, tail, pathStack))
+	} else if traversedEntry || n.navigationStrategy.GetPathParser().IsPathExpression(head) || !n.navigationStrategy.AutoCreateMap(NewNodeContext(value, head, tail, pathStack)) {
 		return nil
 	}
 
+	_, errorParsingInt := strconv.ParseInt(head, 10, 64)
+
 	mapEntryKey := yaml.Node{Value: head, Kind: yaml.ScalarNode}
+
+	if errorParsingInt == nil {
+		// fixes a json encoding problem where keys that look like numbers
+		// get treated as numbers and cannot be used in a json map
+		mapEntryKey.Style = yaml.LiteralStyle
+	}
+
 	value.Content = append(value.Content, &mapEntryKey)
 	mapEntryValue := yaml.Node{Kind: guessKind(head, tail, 0)}
 	value.Content = append(value.Content, &mapEntryValue)
-	log.Debug("adding new node %v", head)
+	log.Debug("adding a new node %v - def a string", head)
 	return n.doTraverse(&mapEntryValue, head, tail, append(pathStack, head))
 }
 
 // need to pass the node in, as it may be aliased
 type mapVisitorFn func(contents []*yaml.Node, index int) error
 
-func (n *navigator) visitDirectMatchingEntries(node *yaml.Node, head string, tail []string, pathStack []interface{}, visit mapVisitorFn) error {
+func (n *navigator) visitDirectMatchingEntries(node *yaml.Node, head string, tail []interface{}, pathStack []interface{}, visit mapVisitorFn) error {
 	var contents = node.Content
 	for index := 0; index < len(contents); index = index + 2 {
 		content := contents[index]
@@ -148,7 +180,7 @@ func (n *navigator) visitDirectMatchingEntries(node *yaml.Node, head string, tai
 	return nil
 }
 
-func (n *navigator) visitMatchingEntries(node *yaml.Node, head string, tail []string, pathStack []interface{}, visit mapVisitorFn) error {
+func (n *navigator) visitMatchingEntries(node *yaml.Node, head string, tail []interface{}, pathStack []interface{}, visit mapVisitorFn) error {
 	var contents = node.Content
 	log.Debug("visitMatchingEntries %v", head)
 	DebugNode(node)
@@ -164,17 +196,17 @@ func (n *navigator) visitMatchingEntries(node *yaml.Node, head string, tail []st
 	return n.visitAliases(contents, head, tail, pathStack, visit)
 }
 
-func (n *navigator) visitAliases(contents []*yaml.Node, head string, tail []string, pathStack []interface{}, visit mapVisitorFn) error {
+func (n *navigator) visitAliases(contents []*yaml.Node, head string, tail []interface{}, pathStack []interface{}, visit mapVisitorFn) error {
 	// merge aliases are defined first, but we only want to traverse them
 	// if we don't find a match on this node first.
 	// traverse them backwards so that the last alias overrides the preceding.
 	// a node can either be
 	// an alias to one other node (e.g. <<: *blah)
 	// or a sequence of aliases   (e.g. <<: [*blah, *foo])
-	log.Debug("checking for aliases")
+	log.Debug("checking for aliases, head: %v, pathstack: %v", head, pathStackToString(pathStack))
 	for index := len(contents) - 2; index >= 0; index = index - 2 {
 
-		if contents[index+1].Kind == yaml.AliasNode {
+		if contents[index+1].Kind == yaml.AliasNode && contents[index].Value == "<<" {
 			valueNode := contents[index+1]
 			log.Debug("found an alias")
 			DebugNode(contents[index])
@@ -195,7 +227,7 @@ func (n *navigator) visitAliases(contents []*yaml.Node, head string, tail []stri
 	return nil
 }
 
-func (n *navigator) visitAliasSequence(possibleAliasArray []*yaml.Node, head string, tail []string, pathStack []interface{}, visit mapVisitorFn) error {
+func (n *navigator) visitAliasSequence(possibleAliasArray []*yaml.Node, head string, tail []interface{}, pathStack []interface{}, visit mapVisitorFn) error {
 	// need to search this backwards too, so that aliases defined last override the preceding.
 	for aliasIndex := len(possibleAliasArray) - 1; aliasIndex >= 0; aliasIndex = aliasIndex - 1 {
 		child := possibleAliasArray[aliasIndex]
@@ -211,7 +243,7 @@ func (n *navigator) visitAliasSequence(possibleAliasArray []*yaml.Node, head str
 	return nil
 }
 
-func (n *navigator) splatArray(value *yaml.Node, head string, tail []string, pathStack []interface{}) error {
+func (n *navigator) splatArray(value *yaml.Node, head interface{}, tail []interface{}, pathStack []interface{}) error {
 	for index, childValue := range value.Content {
 		log.Debug("processing")
 		DebugNode(childValue)
@@ -219,6 +251,9 @@ func (n *navigator) splatArray(value *yaml.Node, head string, tail []string, pat
 
 		newPathStack := append(pathStack, index)
 		if n.navigationStrategy.ShouldTraverse(NewNodeContext(childValue, head, tail, newPathStack), childValue.Value) {
+			// here we should not deeply traverse the array if we are appending..not sure how to do that.
+			// need to visit instead...
+			// easiest way is to pop off the head and pass the rest of the tail in.
 			var err = n.doTraverse(childValue, head, tail, newPathStack)
 			if err != nil {
 				return err
@@ -228,19 +263,30 @@ func (n *navigator) splatArray(value *yaml.Node, head string, tail []string, pat
 	return nil
 }
 
-func (n *navigator) appendArray(value *yaml.Node, head string, tail []string, pathStack []interface{}) error {
+func (n *navigator) appendArray(value *yaml.Node, head interface{}, tail []interface{}, pathStack []interface{}) error {
 	var newNode = yaml.Node{Kind: guessKind(head, tail, 0)}
 	value.Content = append(value.Content, &newNode)
 	log.Debug("appending a new node, %v", value.Content)
 	return n.doTraverse(&newNode, head, tail, append(pathStack, len(value.Content)-1))
 }
 
-func (n *navigator) recurseArray(value *yaml.Node, index int64, head string, tail []string, pathStack []interface{}) error {
-	for int64(len(value.Content)) <= index {
+func (n *navigator) recurseArray(value *yaml.Node, index int64, head interface{}, tail []interface{}, pathStack []interface{}) error {
+	var contentLength = int64(len(value.Content))
+	for contentLength <= index {
 		value.Content = append(value.Content, &yaml.Node{Kind: guessKind(head, tail, 0)})
+		contentLength = int64(len(value.Content))
+	}
+	var indexToUse = index
+
+	if indexToUse < 0 {
+		indexToUse = contentLength + indexToUse
 	}
 
-	value.Content[index] = n.getOrReplace(value.Content[index], guessKind(head, tail, value.Content[index].Kind))
+	if indexToUse < 0 {
+		return fmt.Errorf("Index [%v] out of range, array size is %v", index, contentLength)
+	}
 
-	return n.doTraverse(value.Content[index], head, tail, append(pathStack, index))
+	value.Content[indexToUse] = n.getOrReplace(value.Content[indexToUse], guessKind(head, tail, value.Content[indexToUse].Kind))
+
+	return n.doTraverse(value.Content[indexToUse], head, tail, append(pathStack, index))
 }

pkg/yqlib/delete_navigation_strategy.go

@@ -1,12 +1,10 @@
 package yqlib
 
 import (
-	"strconv"
-
 	yaml "gopkg.in/yaml.v3"
 )
 
-func DeleteNavigationStrategy(pathElementToDelete string) NavigationStrategy {
+func DeleteNavigationStrategy(pathElementToDelete interface{}) NavigationStrategy {
 	parser := NewPathParser()
 	return &NavigationStrategyImpl{
 		visitedNodes: []*NodeContext{},
@@ -14,12 +12,12 @@ func DeleteNavigationStrategy(pathElementToDelete string) NavigationStrategy {
 		followAlias: func(nodeContext NodeContext) bool {
 			return false
 		},
-		autoCreateMap: func(nodeContext NodeContext) bool {
-			return true
+		shouldOnlyDeeplyVisitLeaves: func(nodeContext NodeContext) bool {
+			return false
 		},
 		visit: func(nodeContext NodeContext) error {
 			node := nodeContext.Node
-			log.Debug("need to find and delete %v in here", pathElementToDelete)
+			log.Debug("need to find and delete %v in here (%v)", pathElementToDelete, pathStackToString(nodeContext.PathStack))
 			DebugNode(node)
 			if node.Kind == yaml.SequenceNode {
 				newContent := deleteFromArray(parser, node.Content, nodeContext.PathStack, pathElementToDelete)
@@ -31,12 +29,12 @@ func DeleteNavigationStrategy(pathElementToDelete string) NavigationStrategy {
 		},
 	}
 }
-func deleteFromMap(pathParser PathParser, contents []*yaml.Node, pathStack []interface{}, pathElementToDelete string) []*yaml.Node {
+func deleteFromMap(pathParser PathParser, contents []*yaml.Node, pathStack []interface{}, pathElementToDelete interface{}) []*yaml.Node {
 	newContents := make([]*yaml.Node, 0)
 	for index := 0; index < len(contents); index = index + 2 {
 		keyNode := contents[index]
 		valueNode := contents[index+1]
-		if !pathParser.MatchesNextPathElement(NewNodeContext(keyNode, pathElementToDelete, []string{}, pathStack), keyNode.Value) {
+		if !pathParser.MatchesNextPathElement(NewNodeContext(keyNode, pathElementToDelete, make([]interface{}, 0), pathStack), keyNode.Value) {
 			log.Debug("adding node %v", keyNode.Value)
 			newContents = append(newContents, keyNode, valueNode)
 		} else {
@@ -46,21 +44,23 @@ func deleteFromMap(pathParser PathParser, contents []*yaml.Node, pathStack []int
 	return newContents
 }
 
-func deleteFromArray(pathParser PathParser, content []*yaml.Node, pathStack []interface{}, pathElementToDelete string) []*yaml.Node {
+func deleteFromArray(pathParser PathParser, content []*yaml.Node, pathStack []interface{}, pathElementToDelete interface{}) []*yaml.Node {
 
-	var indexToDelete, err = strconv.ParseInt(pathElementToDelete, 10, 64) // nolint
-	if err == nil {
-		return deleteIndexInArray(content, indexToDelete)
-	}
-	log.Debug("%v is not a numeric index, finding matching patterns", pathElementToDelete)
-	var newArray = make([]*yaml.Node, 0)
+	switch pathElementToDelete := pathElementToDelete.(type) {
+	case int64:
+		return deleteIndexInArray(content, pathElementToDelete)
+	default:
+		log.Debug("%v is not a numeric index, finding matching patterns", pathElementToDelete)
+		var newArray = make([]*yaml.Node, 0)
 
-	for _, childValue := range content {
-		if !pathParser.MatchesNextPathElement(NewNodeContext(childValue, pathElementToDelete, []string{}, pathStack), childValue.Value) {
-			newArray = append(newArray, childValue)
+		for _, childValue := range content {
+			if !pathParser.MatchesNextPathElement(NewNodeContext(childValue, pathElementToDelete, make([]interface{}, 0), pathStack), childValue.Value) {
+				newArray = append(newArray, childValue)
+			}
 		}
+		return newArray
 	}
-	return newArray
+
 }
 
 func deleteIndexInArray(content []*yaml.Node, index int64) []*yaml.Node {

pkg/yqlib/encoder.go

@@ -1,6 +1,7 @@
 package yqlib
 
 import (
+	"bytes"
 	"encoding/json"
 	"io"
 
@@ -12,25 +13,61 @@ type Encoder interface {
 }
 
 type yamlEncoder struct {
-	encoder *yaml.Encoder
+	destination io.Writer
+	indent      int
+	colorise    bool
+	firstDoc    bool
 }
 
-func NewYamlEncoder(destination io.Writer) Encoder {
-	var encoder = yaml.NewEncoder(destination)
-	encoder.SetIndent(2)
-	return &yamlEncoder{encoder}
+func NewYamlEncoder(destination io.Writer, indent int, colorise bool) Encoder {
+	if indent < 0 {
+		indent = 0
+	}
+	return &yamlEncoder{destination, indent, colorise, true}
 }
 
 func (ye *yamlEncoder) Encode(node *yaml.Node) error {
-	return ye.encoder.Encode(node)
+
+	destination := ye.destination
+	tempBuffer := bytes.NewBuffer(nil)
+	if ye.colorise {
+		destination = tempBuffer
+	}
+
+	var encoder = yaml.NewEncoder(destination)
+
+	encoder.SetIndent(ye.indent)
+	// TODO: work out if the first doc had a separator or not.
+	if ye.firstDoc {
+		ye.firstDoc = false
+	} else if _, err := destination.Write([]byte("---\n")); err != nil {
+		return err
+	}
+
+	if err := encoder.Encode(node); err != nil {
+		return err
+	}
+
+	if ye.colorise {
+		return ColorizeAndPrint(tempBuffer.Bytes(), ye.destination)
+	}
+	return nil
 }
 
 type jsonEncoder struct {
 	encoder *json.Encoder
 }
 
-func NewJsonEncoder(destination io.Writer) Encoder {
+func NewJsonEncoder(destination io.Writer, prettyPrint bool, indent int) Encoder {
 	var encoder = json.NewEncoder(destination)
+	var indentString = ""
+
+	for index := 0; index < indent; index++ {
+		indentString = indentString + " "
+	}
+	if prettyPrint {
+		encoder.SetIndent("", indentString)
+	}
 	return &jsonEncoder{encoder}
 }
 

pkg/yqlib/filter_matching_node_navigation_strategy.go

@@ -4,12 +4,6 @@ func FilterMatchingNodesNavigationStrategy(value string) NavigationStrategy {
 	return &NavigationStrategyImpl{
 		visitedNodes: []*NodeContext{},
 		pathParser:   NewPathParser(),
-		followAlias: func(nodeContext NodeContext) bool {
-			return true
-		},
-		autoCreateMap: func(nodeContext NodeContext) bool {
-			return false
-		},
 		visit: func(nodeContext NodeContext) error {
 			return nil
 		},

pkg/yqlib/lib.go

@@ -13,10 +13,28 @@ import (
 var log = logging.MustGetLogger("yq")
 
 type UpdateCommand struct {
-	Command   string
-	Path      string
-	Value     *yaml.Node
-	Overwrite bool
+	Command             string
+	Path                string
+	Value               *yaml.Node
+	Overwrite           bool
+	DontUpdateNodeValue bool
+}
+
+func KindString(kind yaml.Kind) string {
+	switch kind {
+	case yaml.ScalarNode:
+		return "ScalarNode"
+	case yaml.SequenceNode:
+		return "SequenceNode"
+	case yaml.MappingNode:
+		return "MappingNode"
+	case yaml.DocumentNode:
+		return "DocumentNode"
+	case yaml.AliasNode:
+		return "AliasNode"
+	default:
+		return "unknown!"
+	}
 }
 
 func DebugNode(value *yaml.Node) {
@@ -30,7 +48,7 @@ func DebugNode(value *yaml.Node) {
 			log.Error("Error debugging node, %v", errorEncoding.Error())
 		}
 		encoder.Close()
-		log.Debug("Tag: %v", value.Tag)
+		log.Debug("Tag: %v, Kind: %v, Anchor: %v", value.Tag, KindString(value.Kind), value.Anchor)
 		log.Debug("%v", buf.String())
 	}
 }
@@ -43,7 +61,7 @@ func mergePathStackToString(pathStack []interface{}, appendArrays bool) string {
 	var sb strings.Builder
 	for index, path := range pathStack {
 		switch path.(type) {
-		case int:
+		case int, int64:
 			if appendArrays {
 				sb.WriteString("[+]")
 			} else {
@@ -51,45 +69,71 @@ func mergePathStackToString(pathStack []interface{}, appendArrays bool) string {
 			}
 
 		default:
-			sb.WriteString(fmt.Sprintf("%v", path))
+			s := fmt.Sprintf("%v", path)
+			var _, errParsingInt = strconv.ParseInt(s, 10, 64) // nolint
+
+			hasSpecial := strings.Contains(s, ".") || strings.Contains(s, "[") || strings.Contains(s, "]") || strings.Contains(s, "\"")
+			hasDoubleQuotes := strings.Contains(s, "\"")
+			wrappingCharacterStart := "\""
+			wrappingCharacterEnd := "\""
+			if hasDoubleQuotes {
+				wrappingCharacterStart = "("
+				wrappingCharacterEnd = ")"
+			}
+			if hasSpecial || errParsingInt == nil {
+				sb.WriteString(wrappingCharacterStart)
+			}
+			sb.WriteString(s)
+			if hasSpecial || errParsingInt == nil {
+				sb.WriteString(wrappingCharacterEnd)
+			}
 		}
 
 		if index < len(pathStack)-1 {
 			sb.WriteString(".")
 		}
 	}
-	return sb.String()
+	var pathString = sb.String()
+	log.Debug("got a path string: %v", pathString)
+	return pathString
 }
 
-func guessKind(head string, tail []string, guess yaml.Kind) yaml.Kind {
-	log.Debug("tail %v", tail)
+func guessKind(head interface{}, tail []interface{}, guess yaml.Kind) yaml.Kind {
+	log.Debug("guessKind: tail %v", tail)
 	if len(tail) == 0 && guess == 0 {
 		log.Debug("end of path, must be a scalar")
 		return yaml.ScalarNode
 	} else if len(tail) == 0 {
 		return guess
 	}
-
-	var _, errorParsingInt = strconv.ParseInt(tail[0], 10, 64)
-	if tail[0] == "+" || errorParsingInt == nil {
+	var next = tail[0]
+	switch next.(type) {
+	case int64:
 		return yaml.SequenceNode
+	default:
+		var nextString = fmt.Sprintf("%v", next)
+		if nextString == "+" {
+			return yaml.SequenceNode
+		}
+		pathParser := NewPathParser()
+		if pathParser.IsPathExpression(nextString) && (guess == yaml.SequenceNode || guess == yaml.MappingNode) {
+			return guess
+		} else if guess == yaml.AliasNode {
+			log.Debug("guess was an alias, okey doke.")
+			return guess
+		} else if head == "**" {
+			log.Debug("deep wildcard, go with the guess")
+			return guess
+		}
+		log.Debug("forcing a mapping node")
+		log.Debug("yaml.SequenceNode %v", guess == yaml.SequenceNode)
+		log.Debug("yaml.ScalarNode %v", guess == yaml.ScalarNode)
+		return yaml.MappingNode
 	}
-	pathParser := NewPathParser()
-	if (pathParser.IsPathExpression(tail[0]) || head == "**") && (guess == yaml.SequenceNode || guess == yaml.MappingNode) {
-		return guess
-	}
-	if guess == yaml.AliasNode {
-		log.Debug("guess was an alias, okey doke.")
-		return guess
-	}
-	log.Debug("forcing a mapping node")
-	log.Debug("yaml.SequenceNode %v", guess == yaml.SequenceNode)
-	log.Debug("yaml.ScalarNode %v", guess == yaml.ScalarNode)
-	return yaml.MappingNode
 }
 
 type YqLib interface {
-	Get(rootNode *yaml.Node, path string) ([]*NodeContext, error)
+	Get(rootNode *yaml.Node, path string, deeplyTraverseArrays bool) ([]*NodeContext, error)
 	Update(rootNode *yaml.Node, updateCommand UpdateCommand, autoCreate bool) error
 	New(path string) yaml.Node
 
@@ -107,9 +151,9 @@ func NewYqLib() YqLib {
 	}
 }
 
-func (l *lib) Get(rootNode *yaml.Node, path string) ([]*NodeContext, error) {
+func (l *lib) Get(rootNode *yaml.Node, path string, deeplyTraverseArrays bool) ([]*NodeContext, error) {
 	var paths = l.parser.ParsePath(path)
-	navigationStrategy := ReadNavigationStrategy()
+	navigationStrategy := ReadNavigationStrategy(deeplyTraverseArrays)
 	navigator := NewDataNavigator(navigationStrategy)
 	error := navigator.Traverse(rootNode, paths)
 	return navigationStrategy.GetVisitedNodes(), error

pkg/yqlib/navigation_strategy.go

@@ -8,13 +8,13 @@ import (
 
 type NodeContext struct {
 	Node      *yaml.Node
-	Head      string
-	Tail      []string
+	Head      interface{}
+	Tail      []interface{}
 	PathStack []interface{}
 }
 
-func NewNodeContext(node *yaml.Node, head string, tail []string, pathStack []interface{}) NodeContext {
-	newTail := make([]string, len(tail))
+func NewNodeContext(node *yaml.Node, head interface{}, tail []interface{}, pathStack []interface{}) NodeContext {
+	newTail := make([]interface{}, len(tail))
 	copy(newTail, tail)
 
 	newPathStack := make([]interface{}, len(pathStack))
@@ -34,18 +34,23 @@ type NavigationStrategy interface {
 	// node key is the string value of the last element in the path stack
 	// we use it to match against the pathExpression in head.
 	ShouldTraverse(nodeContext NodeContext, nodeKey string) bool
+	ShouldDeeplyTraverse(nodeContext NodeContext) bool
+	// when deeply traversing, should we visit all matching nodes, or just leaves?
+	ShouldOnlyDeeplyVisitLeaves(NodeContext) bool
 	GetVisitedNodes() []*NodeContext
 	DebugVisitedNodes()
 	GetPathParser() PathParser
 }
 
 type NavigationStrategyImpl struct {
-	followAlias        func(nodeContext NodeContext) bool
-	autoCreateMap      func(nodeContext NodeContext) bool
-	visit              func(nodeContext NodeContext) error
-	shouldVisitExtraFn func(nodeContext NodeContext) bool
-	visitedNodes       []*NodeContext
-	pathParser         PathParser
+	followAlias                 func(nodeContext NodeContext) bool
+	autoCreateMap               func(nodeContext NodeContext) bool
+	visit                       func(nodeContext NodeContext) error
+	shouldVisitExtraFn          func(nodeContext NodeContext) bool
+	shouldDeeplyTraverse        func(nodeContext NodeContext) bool
+	shouldOnlyDeeplyVisitLeaves func(nodeContext NodeContext) bool
+	visitedNodes                []*NodeContext
+	pathParser                  PathParser
 }
 
 func (ns *NavigationStrategyImpl) GetPathParser() PathParser {
@@ -57,11 +62,32 @@ func (ns *NavigationStrategyImpl) GetVisitedNodes() []*NodeContext {
 }
 
 func (ns *NavigationStrategyImpl) FollowAlias(nodeContext NodeContext) bool {
-	return ns.followAlias(nodeContext)
+	if ns.followAlias != nil {
+		return ns.followAlias(nodeContext)
+	}
+	return true
 }
 
 func (ns *NavigationStrategyImpl) AutoCreateMap(nodeContext NodeContext) bool {
-	return ns.autoCreateMap(nodeContext)
+	if ns.autoCreateMap != nil {
+		return ns.autoCreateMap(nodeContext)
+	}
+	return false
+}
+
+func (ns *NavigationStrategyImpl) ShouldDeeplyTraverse(nodeContext NodeContext) bool {
+	if ns.shouldDeeplyTraverse != nil {
+		return ns.shouldDeeplyTraverse(nodeContext)
+	}
+	return true
+}
+
+func (ns *NavigationStrategyImpl) ShouldOnlyDeeplyVisitLeaves(nodeContext NodeContext) bool {
+	if ns.shouldOnlyDeeplyVisitLeaves != nil {
+		return ns.shouldOnlyDeeplyVisitLeaves(nodeContext)
+	}
+	return true
+
 }
 
 func (ns *NavigationStrategyImpl) ShouldTraverse(nodeContext NodeContext, nodeKey string) bool {
@@ -84,7 +110,6 @@ func (ns *NavigationStrategyImpl) shouldVisit(nodeContext NodeContext) bool {
 		return true
 	}
 	log.Debug("tail len %v", len(nodeContext.Tail))
-	// SOMETHING HERE!
 
 	if ns.alreadyVisited(pathStack) || len(nodeContext.Tail) != 0 {
 		return false

pkg/yqlib/path_parser.go

@@ -1,12 +1,15 @@
 package yqlib
 
 import (
+	"fmt"
 	"strconv"
 	"strings"
+
+	yaml "gopkg.in/yaml.v3"
 )
 
 type PathParser interface {
-	ParsePath(path string) []string
+	ParsePath(path string) []interface{}
 	MatchesNextPathElement(nodeContext NodeContext, nodeKey string) bool
 	IsPathExpression(pathElement string) bool
 }
@@ -42,9 +45,11 @@ func (p *pathParser) MatchesNextPathElement(nodeContext NodeContext, nodeKey str
 	if head == "**" || head == "*" {
 		return true
 	}
-	if strings.Contains(head, "==") {
+	var headString = fmt.Sprintf("%v", head)
+
+	if strings.Contains(headString, "==") && nodeContext.Node.Kind != yaml.ScalarNode {
 		log.Debug("ooh deep recursion time")
-		result := strings.SplitN(head, "==", 2)
+		result := strings.SplitN(headString, "==", 2)
 		path := strings.TrimSpace(result[0])
 		value := strings.TrimSpace(result[1])
 		log.Debug("path %v", path)
@@ -60,6 +65,14 @@ func (p *pathParser) MatchesNextPathElement(nodeContext NodeContext, nodeKey str
 		}
 		log.Debug("done deep recursing, found %v matches", len(navigationStrategy.GetVisitedNodes()))
 		return len(navigationStrategy.GetVisitedNodes()) > 0
+	} else if strings.Contains(headString, "==") && nodeContext.Node.Kind == yaml.ScalarNode {
+		result := strings.SplitN(headString, "==", 2)
+		path := strings.TrimSpace(result[0])
+		value := strings.TrimSpace(result[1])
+		if path == "." {
+			log.Debug("need to match scalar")
+			return matchesString(value, nodeContext.Node.Value)
+		}
 	}
 
 	if head == "+" {
@@ -70,17 +83,18 @@ func (p *pathParser) MatchesNextPathElement(nodeContext NodeContext, nodeKey str
 		}
 	}
 
-	return matchesString(head, nodeKey)
+	return matchesString(headString, nodeKey)
 }
 
-func (p *pathParser) ParsePath(path string) []string {
+func (p *pathParser) ParsePath(path string) []interface{} {
+	var paths = make([]interface{}, 0)
 	if path == "" {
-		return []string{}
+		return paths
 	}
-	return p.parsePathAccum([]string{}, path)
+	return p.parsePathAccum(paths, path)
 }
 
-func (p *pathParser) parsePathAccum(paths []string, remaining string) []string {
+func (p *pathParser) parsePathAccum(paths []interface{}, remaining string) []interface{} {
 	head, tail := p.nextYamlPath(remaining)
 	if tail == "" {
 		return append(paths, head)
@@ -88,11 +102,16 @@ func (p *pathParser) parsePathAccum(paths []string, remaining string) []string {
 	return p.parsePathAccum(append(paths, head), tail)
 }
 
-func (p *pathParser) nextYamlPath(path string) (pathElement string, remaining string) {
+func (p *pathParser) nextYamlPath(path string) (pathElement interface{}, remaining string) {
 	switch path[0] {
 	case '[':
 		// e.g [0].blah.cat -> we need to return "0" and "blah.cat"
-		return p.search(path[1:], []uint8{']'}, true)
+		var value, remainingBit = p.search(path[1:], []uint8{']'}, true)
+		var number, errParsingInt = strconv.ParseInt(value, 10, 64) // nolint
+		if errParsingInt == nil {
+			return number, remainingBit
+		}
+		return value, remainingBit
 	case '"':
 		// e.g "a.b".blah.cat -> we need to return "a.b" and "blah.cat"
 		return p.search(path[1:], []uint8{'"'}, true)

pkg/yqlib/path_parser_test.go

@@ -10,20 +10,21 @@ var parser = NewPathParser()
 
 var parsePathsTests = []struct {
 	path          string
-	expectedPaths []string
+	expectedPaths []interface{}
 }{
-	{"a.b", []string{"a", "b"}},
-	{"a.b.**", []string{"a", "b", "**"}},
-	{"a.b.*", []string{"a", "b", "*"}},
-	{"a.b[0]", []string{"a", "b", "0"}},
-	{"a.b.d[+]", []string{"a", "b", "d", "+"}},
-	{"a", []string{"a"}},
-	{"a.b.c", []string{"a", "b", "c"}},
-	{"\"a.b\".c", []string{"a.b", "c"}},
-	{"a.\"b.c\".d", []string{"a", "b.c", "d"}},
-	{"[1].a.d", []string{"1", "a", "d"}},
-	{"a[0].c", []string{"a", "0", "c"}},
-	{"[0]", []string{"0"}},
+	{"a.b", append(make([]interface{}, 0), "a", "b")},
+	{"a.b.**", append(make([]interface{}, 0), "a", "b", "**")},
+	{"a.b.*", append(make([]interface{}, 0), "a", "b", "*")},
+	{"a.b[0]", append(make([]interface{}, 0), "a", "b", int64(0))},
+	{"a.b.0", append(make([]interface{}, 0), "a", "b", "0")},
+	{"a.b.d[+]", append(make([]interface{}, 0), "a", "b", "d", "+")},
+	{"a", append(make([]interface{}, 0), "a")},
+	{"a.b.c", append(make([]interface{}, 0), "a", "b", "c")},
+	{"\"a.b\".c", append(make([]interface{}, 0), "a.b", "c")},
+	{"a.\"b.c\".d", append(make([]interface{}, 0), "a", "b.c", "d")},
+	{"[1].a.d", append(make([]interface{}, 0), int64(1), "a", "d")},
+	{"a[0].c", append(make([]interface{}, 0), "a", int64(0), "c")},
+	{"[0]", append(make([]interface{}, 0), int64(0))},
 }
 
 func TestPathParserParsePath(t *testing.T) {

pkg/yqlib/read_navigation_strategy.go

@@ -1,17 +1,24 @@
 package yqlib
 
-func ReadNavigationStrategy() NavigationStrategy {
+func ReadNavigationStrategy(deeplyTraverseArrays bool) NavigationStrategy {
 	return &NavigationStrategyImpl{
 		visitedNodes: []*NodeContext{},
 		pathParser:   NewPathParser(),
-		followAlias: func(nodeContext NodeContext) bool {
-			return true
-		},
-		autoCreateMap: func(nodeContext NodeContext) bool {
-			return false
-		},
 		visit: func(nodeContext NodeContext) error {
 			return nil
 		},
+		shouldDeeplyTraverse: func(nodeContext NodeContext) bool {
+			var isInArray = false
+			if len(nodeContext.PathStack) > 0 {
+				var lastElement = nodeContext.PathStack[len(nodeContext.PathStack)-1]
+				switch lastElement.(type) {
+				case int:
+					isInArray = true
+				default:
+					isInArray = false
+				}
+			}
+			return deeplyTraverseArrays || !isInArray
+		},
 	}
 }

pkg/yqlib/update_navigation_strategy.go

@@ -18,11 +18,15 @@ func UpdateNavigationStrategy(updateCommand UpdateCommand, autoCreate bool) Navi
 				DebugNode(node)
 				log.Debug("with")
 				DebugNode(changesToApply)
-				node.Value = changesToApply.Value
+				if !updateCommand.DontUpdateNodeValue {
+					node.Value = changesToApply.Value
+				}
 				node.Tag = changesToApply.Tag
 				node.Kind = changesToApply.Kind
 				node.Style = changesToApply.Style
 				node.Content = changesToApply.Content
+				node.Anchor = changesToApply.Anchor
+				node.Alias = changesToApply.Alias
 				node.HeadComment = changesToApply.HeadComment
 				node.LineComment = changesToApply.LineComment
 				node.FootComment = changesToApply.FootComment

pkg/yqlib/value_parser.go

@@ -5,7 +5,7 @@ import (
 )
 
 type ValueParser interface {
-	Parse(argument string, customTag string) *yaml.Node
+	Parse(argument string, customTag string, customStyle string, anchorName string, createAlias bool) *yaml.Node
 }
 
 type valueParser struct {
@@ -15,9 +15,32 @@ func NewValueParser() ValueParser {
 	return &valueParser{}
 }
 
-func (v *valueParser) Parse(argument string, customTag string) *yaml.Node {
-	if argument == "[]" {
-		return &yaml.Node{Tag: "!!seq", Kind: yaml.SequenceNode}
+func (v *valueParser) Parse(argument string, customTag string, customStyle string, anchorName string, createAlias bool) *yaml.Node {
+	var style yaml.Style
+	if customStyle == "tagged" {
+		style = yaml.TaggedStyle
+	} else if customStyle == "double" {
+		style = yaml.DoubleQuotedStyle
+	} else if customStyle == "single" {
+		style = yaml.SingleQuotedStyle
+	} else if customStyle == "literal" {
+		style = yaml.LiteralStyle
+	} else if customStyle == "folded" {
+		style = yaml.FoldedStyle
+	} else if customStyle == "flow" {
+		style = yaml.FlowStyle
+	} else if customStyle != "" {
+		log.Error("Unknown style %v, ignoring", customStyle)
 	}
-	return &yaml.Node{Value: argument, Tag: customTag, Kind: yaml.ScalarNode}
+	if argument == "[]" {
+		return &yaml.Node{Tag: "!!seq", Kind: yaml.SequenceNode, Style: style}
+	}
+
+	kind := yaml.ScalarNode
+
+	if createAlias {
+		kind = yaml.AliasNode
+	}
+
+	return &yaml.Node{Value: argument, Tag: customTag, Kind: kind, Style: style, Anchor: anchorName}
 }

pkg/yqlib/value_parser_test.go

@@ -7,6 +7,26 @@ import (
 	yaml "gopkg.in/yaml.v3"
 )
 
+var parseStyleTests = []struct {
+	customStyle   string
+	expectedStyle yaml.Style
+}{
+	{"", 0},
+	{"tagged", yaml.TaggedStyle},
+	{"double", yaml.DoubleQuotedStyle},
+	{"single", yaml.SingleQuotedStyle},
+	{"folded", yaml.FoldedStyle},
+	{"flow", yaml.FlowStyle},
+	{"literal", yaml.LiteralStyle},
+}
+
+func TestValueParserStyleTag(t *testing.T) {
+	for _, tt := range parseStyleTests {
+		actual := NewValueParser().Parse("cat", "", tt.customStyle, "", false)
+		test.AssertResultWithContext(t, tt.expectedStyle, actual.Style, tt.customStyle)
+	}
+}
+
 var parseValueTests = []struct {
 	argument        string
 	customTag       string
@@ -20,7 +40,7 @@ var parseValueTests = []struct {
 
 func TestValueParserParse(t *testing.T) {
 	for _, tt := range parseValueTests {
-		actual := NewValueParser().Parse(tt.argument, tt.customTag)
+		actual := NewValueParser().Parse(tt.argument, tt.customTag, "", "", false)
 		test.AssertResultWithContext(t, tt.argument, actual.Value, tt.testDescription)
 		test.AssertResultWithContext(t, tt.expectedTag, actual.Tag, tt.testDescription)
 		test.AssertResult(t, yaml.ScalarNode, actual.Kind)
@@ -28,7 +48,18 @@ func TestValueParserParse(t *testing.T) {
 }
 
 func TestValueParserParseEmptyArray(t *testing.T) {
-	actual := NewValueParser().Parse("[]", "")
+	actual := NewValueParser().Parse("[]", "", "", "", false)
 	test.AssertResult(t, "!!seq", actual.Tag)
 	test.AssertResult(t, yaml.SequenceNode, actual.Kind)
 }
+
+func TestValueParserParseAlias(t *testing.T) {
+	actual := NewValueParser().Parse("bob", "", "", "", true)
+	test.AssertResult(t, "bob", actual.Value)
+	test.AssertResult(t, yaml.AliasNode, actual.Kind)
+}
+
+func TestValueParserAnchorname(t *testing.T) {
+	actual := NewValueParser().Parse("caterpillar", "", "", "foo", false)
+	test.AssertResult(t, "foo", actual.Anchor)
+}

prefix.md

@@ -0,0 +1,127 @@
+# Prefix
+
+ Paths can be prefixed using the 'prefix' command. The complete yaml content will be nested inside the new prefix path.
+
+```text
+yq p <yaml_file> <path>
+```
+
+### To Stdout[¶](prefix.md#to-stdout) <a id="to-stdout"></a>
+
+Given a data1.yaml file of:
+
+```text
+a: simple
+b: [1, 2]
+```
+
+then
+
+```text
+yq p data1.yaml c
+```
+
+will output:
+
+```text
+c:
+  a: simple
+  b: [1, 2]
+```
+
+### Arbitrary depth[¶](prefix.md#arbitrary-depth) <a id="arbitrary-depth"></a>
+
+Given a data1.yaml file of:
+
+```text
+a:
+  b: [1, 2]
+```
+
+then
+
+```text
+yq p data1.yaml c.d
+```
+
+will output:
+
+```text
+c:
+  d:
+    a:
+      b: [1, 2]
+```
+
+### Updating files in-place[¶](prefix.md#updating-files-in-place) <a id="updating-files-in-place"></a>
+
+Given a data1.yaml file of:
+
+```text
+a: simple
+b: [1, 2]
+```
+
+then
+
+```text
+yq p -i data1.yaml c
+```
+
+will update the data1.yaml file so that the path 'c' is prefixed to all other paths.
+
+### Multiple Documents - prefix a single document[¶](prefix.md#multiple-documents-prefix-a-single-document) <a id="multiple-documents-prefix-a-single-document"></a>
+
+Given a data1.yaml file of:
+
+```text
+something: else
+---
+a: simple
+b: cat
+```
+
+then
+
+```text
+yq p -d1 data1.yaml c
+```
+
+will output:
+
+```text
+something: else
+---
+c:
+  a: simple
+  b: cat
+```
+
+### Multiple Documents - prefix all documents[¶](prefix.md#multiple-documents-prefix-all-documents) <a id="multiple-documents-prefix-all-documents"></a>
+
+Given a data1.yaml file of:
+
+```text
+something: else
+---
+a: simple
+b: cat
+```
+
+then
+
+```text
+yq p -d'*' data1.yaml c
+```
+
+will output:
+
+```text
+c:
+  something: else
+---
+c:
+  a: simple
+  b: cat
+```
+

read.md

@@ -0,0 +1,228 @@
+# Read
+
+```text
+yq r <yaml_file|json_file> <path>
+```
+
+This command can take a json file as input too, and will output yaml unless specified to export as json \(-j\)
+
+### Basic[¶](read.md#basic) <a id="basic"></a>
+
+Given a sample.yaml file of:
+
+```text
+b:
+  c: 2
+```
+
+then
+
+```text
+yq r sample.yaml b.c
+```
+
+will output the value of '2'.
+
+### From Stdin[¶](read.md#from-stdin) <a id="from-stdin"></a>
+
+Given a sample.yaml file of:
+
+```text
+cat sample.yaml | yq r - b.c
+```
+
+will output the value of '2'.
+
+### Splat[¶](read.md#splat) <a id="splat"></a>
+
+Given a sample.yaml file of:
+
+```text
+---
+bob:
+  item1:
+    cats: bananas
+  item2:
+    cats: apples
+  thing:
+    cats: oranges
+```
+
+then
+
+```text
+yq r sample.yaml bob.*.cats
+```
+
+will output
+
+```text
+- bananas
+- apples
+- oranges
+```
+
+### Prefix Splat[¶](read.md#prefix-splat) <a id="prefix-splat"></a>
+
+Given a sample.yaml file of:
+
+```text
+---
+bob:
+  item1:
+    cats: bananas
+  item2:
+    cats: apples
+  thing:
+    cats: oranges
+```
+
+then
+
+```text
+yq r sample.yaml bob.item*.cats
+```
+
+will output
+
+```text
+- bananas
+- apples
+```
+
+### Multiple Documents - specify a single document[¶](read.md#multiple-documents-specify-a-single-document) <a id="multiple-documents-specify-a-single-document"></a>
+
+Given a sample.yaml file of:
+
+```text
+something: else
+---
+b:
+  c: 2
+```
+
+then
+
+```text
+yq r -d1 sample.yaml b.c
+```
+
+will output the value of '2'.
+
+### Multiple Documents - read all documents[¶](read.md#multiple-documents-read-all-documents) <a id="multiple-documents-read-all-documents"></a>
+
+Reading all documents will return the result as an array. This can be converted to json using the '-j' flag if desired.
+
+Given a sample.yaml file of:
+
+```text
+name: Fred
+age: 22
+---
+name: Stella
+age: 23
+---
+name: Android
+age: 232
+```
+
+then
+
+```text
+yq r -d'*' sample.yaml name
+```
+
+will output:
+
+```text
+- Fred
+- Stella
+- Android
+```
+
+### Arrays[¶](read.md#arrays) <a id="arrays"></a>
+
+You can give an index to access a specific element: e.g.: given a sample file of
+
+```text
+b:
+  e:
+    - name: fred
+      value: 3
+    - name: sam
+      value: 4
+```
+
+then
+
+```text
+yq r sample.yaml 'b.e[1].name'
+```
+
+will output 'sam'
+
+Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
+
+### Array Splat[¶](read.md#array-splat) <a id="array-splat"></a>
+
+e.g.: given a sample file of
+
+```text
+b:
+  e:
+    - name: fred
+      value: 3
+    - name: sam
+      value: 4
+```
+
+then
+
+```text
+yq r sample.yaml 'b.e[*].name'
+```
+
+will output:
+
+```text
+- fred
+- sam
+```
+
+Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
+
+### Keys with dots[¶](read.md#keys-with-dots) <a id="keys-with-dots"></a>
+
+When specifying a key that has a dot use key lookup indicator.
+
+```text
+b:
+  foo.bar: 7
+```
+
+```text
+yaml r sample.yaml 'b[foo.bar]'
+```
+
+```text
+yaml w sample.yaml 'b[foo.bar]' 9
+```
+
+Any valid yaml key can be specified as part of a key lookup.
+
+Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
+
+### Keys \(and values\) with leading dashes[¶](read.md#keys-and-values-with-leading-dashes) <a id="keys-and-values-with-leading-dashes"></a>
+
+If a key or value has leading dashes, yq won't know that you are passing a value as opposed to a flag \(and you will get a 'bad flag syntax' error\).
+
+To fix that, you will need to tell it to stop processing flags by adding '--' after the last flag like so:
+
+```text
+yq n -t -- --key --value
+```
+
+Will result in
+
+``` --key: --value``
+

release_instructions.txt

@@ -33,6 +33,8 @@
   - docker build .  -t mikefarah/yq:latest -t mikefarah/yq:VERSION 
 
 - debian package
+  - ensure you get all vendor dependencies before packaging
+	```go mod vendor```
   - execute 
     ```dch -i```
   - fill debian/changelog with changes from last version
@@ -42,4 +44,4 @@
   - put to PPA
     ```dput ppa:<REPOSITORY> ../yq_<VERSION>_source.changes```
     (current distro repository is ppa:rmescandon/yq. In case that a new version
-    is released, please contact rmescandon@gmail.com to bump debian package)
+    is released, please contact rmescandon@gmail.com to bump debian package)

scripts/devtools.sh

@@ -1,4 +1,4 @@
 #!/bin/sh
-set -e
-wget -O- -nv https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s v1.23.1
-go get golang.org/x/tools/cmd/goimports
+set -ex
+go get golang.org/x/tools/cmd/goimports
+wget -O- -nv https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s v1.24.0

scripts/format.sh

@@ -1,3 +1,4 @@
 #!/bin/bash
 
 find . \( -path ./vendor \) -prune -o -name "*.go" -exec goimports -w {} \;
+go mod tidy

scripts/publish.sh

@@ -32,5 +32,5 @@ upload() {
     done < <(find ./build -mindepth 1 -maxdepth 1 -print0)
 }
 
-# release
+release
 upload

scripts/xcompile.sh

@@ -7,3 +7,6 @@ gox -ldflags "${LDFLAGS}" -output="build/yq_{{.OS}}_{{.Arch}}"
 # include non-default linux builds too
 gox -ldflags "${LDFLAGS}" -os=linux  -output="build/yq_{{.OS}}_{{.Arch}}"
 
+cd build
+rhash -r -a . -P -o checksums
+

snap/snapcraft.yaml

@@ -1,9 +1,10 @@
 name: yq
-version: '3.0.1'
+version: '3.3.2'
 summary: A lightweight and portable command-line YAML processor
 description: |
   The aim of the project is to be the jq or sed of yaml files.
 
+base: core18
 grade: stable # devel|stable. must be 'stable' to release into candidate/stable channels
 confinement: strict
 
@@ -15,8 +16,7 @@ apps:
 parts:
   yq:
     plugin: go
+    go-channel: 1.14/stable
     source: .
+    source-type: git
     go-importpath: github.com/mikefarah/yq
-    after: [go]
-  go:
-    source-tag: go1.11

test.yml

@@ -1,3 +0,0 @@
-b:
-  c: thing
-  d: another thing

write-update.md

@@ -0,0 +1,349 @@
+# Write
+
+```text
+yq w <yaml_file> <path> <new value>
+```
+
+### To Stdout[¶](write-update.md#to-stdout) <a id="to-stdout"></a>
+
+Given a sample.yaml file of:
+
+```text
+b:
+  c: 2
+```
+
+then
+
+```text
+yq w sample.yaml b.c cat
+```
+
+will output:
+
+```text
+b:
+  c: cat
+```
+
+### From STDIN[¶](write-update.md#from-stdin) <a id="from-stdin"></a>
+
+```text
+cat sample.yaml | yq w - b.c blah
+```
+
+### Adding new fields[¶](write-update.md#adding-new-fields) <a id="adding-new-fields"></a>
+
+Any missing fields in the path will be created on the fly.
+
+Given a sample.yaml file of:
+
+```text
+b:
+  c: 2
+```
+
+then
+
+```text
+yq w sample.yaml b.d[+] "new thing"
+```
+
+will output:
+
+```text
+b:
+  c: cat
+  d:
+    - new thing
+```
+
+### Splat[¶](write-update.md#splat) <a id="splat"></a>
+
+Given a sample.yaml file of:
+
+```text
+---
+bob:
+  item1:
+    cats: bananas
+  item2:
+    cats: apples
+  thing:
+    cats: oranges
+```
+
+then
+
+```text
+yq w sample.yaml bob.*.cats meow
+```
+
+will output:
+
+```text
+---
+bob:
+  item1:
+    cats: meow
+  item2:
+    cats: meow
+  thing:
+    cats: meow
+```
+
+### Prefix Splat[¶](write-update.md#prefix-splat) <a id="prefix-splat"></a>
+
+Given a sample.yaml file of:
+
+```text
+---
+bob:
+  item1:
+    cats: bananas
+  item2:
+    cats: apples
+  thing:
+    cats: oranges
+```
+
+then
+
+```text
+yq w sample.yaml bob.item*.cats meow
+```
+
+will output:
+
+```text
+---
+bob:
+  item1:
+    cats: meow
+  item2:
+    cats: meow
+  thing:
+    cats: oranges
+```
+
+### Array Splat[¶](write-update.md#array-splat) <a id="array-splat"></a>
+
+Given a sample.yaml file of:
+
+```text
+---
+bob:
+- cats: bananas
+- cats: apples
+- cats: oranges
+```
+
+then
+
+```text
+yq w sample.yaml bob[*].cats meow
+```
+
+will output:
+
+```text
+---
+bob:
+- cats: meow
+- cats: meow
+- cats: meow
+```
+
+### Appending value to an array field[¶](write-update.md#appending-value-to-an-array-field) <a id="appending-value-to-an-array-field"></a>
+
+Given a sample.yaml file of:
+
+```text
+b:
+  c: 2
+  d:
+    - new thing
+    - foo thing
+```
+
+then
+
+```text
+yq w sample.yaml "b.d[+]" "bar thing"
+```
+
+will output:
+
+```text
+b:
+  c: cat
+  d:
+    - new thing
+    - foo thing
+    - bar thing
+```
+
+Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
+
+### Multiple Documents - update a single document[¶](write-update.md#multiple-documents-update-a-single-document) <a id="multiple-documents-update-a-single-document"></a>
+
+Given a sample.yaml file of:
+
+```text
+something: else
+---
+b:
+  c: 2
+```
+
+then
+
+```text
+yq w -d1 sample.yaml b.c 5
+```
+
+will output:
+
+```text
+something: else
+---
+b:
+  c: 5
+```
+
+### Multiple Documents - update all documents[¶](write-update.md#multiple-documents-update-all-documents) <a id="multiple-documents-update-all-documents"></a>
+
+Given a sample.yaml file of:
+
+```text
+something: else
+---
+b:
+  c: 2
+```
+
+then
+
+```text
+yq w -d'*' sample.yaml b.c 5
+```
+
+will output:
+
+```text
+something: else
+b:
+  c: 5
+---
+b:
+  c: 5
+```
+
+Note that '\*' is in quotes to avoid being interpreted by your shell.
+
+### Updating files in-place[¶](write-update.md#updating-files-in-place) <a id="updating-files-in-place"></a>
+
+Given a sample.yaml file of:
+
+```text
+b:
+  c: 2
+```
+
+then
+
+```text
+yq w -i sample.yaml b.c cat
+```
+
+will update the sample.yaml file so that the value of 'c' is cat.
+
+### Updating multiple values with a script[¶](write-update.md#updating-multiple-values-with-a-script) <a id="updating-multiple-values-with-a-script"></a>
+
+Given a sample.yaml file of:
+
+```text
+b:
+  c: 2
+  e:
+    - name: Billy Bob
+```
+
+and a script update\_instructions.yaml of:
+
+```text
+b.c: 3
+b.e[+].name: Howdy Partner
+```
+
+then
+
+```text
+yq w -s update_instructions.yaml sample.yaml
+```
+
+will output:
+
+```text
+b:
+  c: 3
+  e:
+    - name: Howdy Partner
+```
+
+And, of course, you can pipe the instructions in using '-':
+
+```text
+cat update_instructions.yaml | yq w -s - sample.yaml
+```
+
+### Values starting with a hyphen \(or dash\)[¶](write-update.md#values-starting-with-a-hyphen-or-dash) <a id="values-starting-with-a-hyphen-or-dash"></a>
+
+The flag terminator needs to be used to stop the app from attempting to parse the subsequent arguments as flags:
+
+```text
+yq w -- my.path -3
+```
+
+will output
+
+```text
+my:
+  path: -3
+```
+
+### Keys with dots[¶](write-update.md#keys-with-dots) <a id="keys-with-dots"></a>
+
+When specifying a key that has a dot use key lookup indicator.
+
+```text
+b:
+  foo.bar: 7
+```
+
+```text
+yaml r sample.yaml 'b[foo.bar]'
+```
+
+```text
+yaml w sample.yaml 'b[foo.bar]' 9
+```
+
+Any valid yaml key can be specified as part of a key lookup.
+
+Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
+
+### Keys \(and values\) with leading dashes[¶](write-update.md#keys-and-values-with-leading-dashes) <a id="keys-and-values-with-leading-dashes"></a>
+
+If a key or value has leading dashes, yq won't know that you are passing a value as opposed to a flag \(and you will get a 'bad flag syntax' error\).
+
+To fix that, you will need to tell it to stop processing flags by adding '--' after the last flag like so:
+
+```text
+yq n -t -- --key --value
+```
+
+Will result in
+
+``` --key: --value``
+

yq.go

@@ -4,14 +4,11 @@ import (
 	"os"
 
 	command "github.com/mikefarah/yq/v3/cmd"
-	logging "gopkg.in/op/go-logging.v1"
 )
 
 func main() {
 	cmd := command.New()
-	log := logging.MustGetLogger("yq")
 	if err := cmd.Execute(); err != nil {
-		log.Error(err.Error())
 		os.Exit(1)
 	}
 }