GitBook: [v3.x] 17 pages modified

removed repeated word

.gitbook/assets/convert.md

@@ -0,0 +1,60 @@
+## Yaml to Json
+To convert output to json, use the --tojson (or -j) flag. This is supported by all commands.
+
+Each matching yaml node will be converted to json and printed out on a separate line.
+
+Given a sample.yaml file of:
+```yaml
+b:
+  c: 2
+```
+then
+```bash
+yq r -j sample.yaml
+```
+
+will output
+```json
+{"b":{"c":2}}
+```
+
+Given a sample.yaml file of:
+```yaml
+bob:
+  c: 2
+bab:
+  c: 5
+```
+then
+```bash
+yq r -j sample.yaml b*
+```
+
+will output
+```json
+{"c":2}
+{"c":5}
+```
+
+## Json to Yaml
+To read in json, just pass in a json file instead of yaml, it will just work :)
+
+e.g given a json file
+
+```json
+{"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}
+```
+then
+```bash
+yq r sample.json
+```
+will output
+```yaml
+a: Easy! as one two three
+b:
+  c: 2
+  d:
+  - 3
+  - 4
+```
+

.github/ISSUE_TEMPLATE/bug_report.md

@@ -10,6 +10,9 @@ assignees: ''
 **Describe the bug**
 A clear and concise description of what the bug is.
 
+version of yq: 
+operating system:
+
 **Input Yaml**
 Concise yaml document(s) (as simple as possible to show the bug)
 data1.yml:

README.md

@@ -1,61 +1,74 @@
+---
+description: yq is a lightweight and portable command-line YAML processor
+---
+
 # yq
 
-![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg)  ![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) ![Github Releases (by Release)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) ![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq)
-
-
-a lightweight and portable command-line YAML processor
-
-The aim of the project is to be the [jq](https://github.com/stedolan/jq) or sed of yaml files.
-
-## New version!
-V3 is officially out - if you've been using v2 and want/need to upgrade, checkout the [upgrade guide](https://mikefarah.gitbook.io/yq/upgrading-from-v2).
+ ![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg) ![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) ![Github Releases \(by Release\)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) ![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq)
 
 ## Install
 
-### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest)
+`yq` has pre-built binaries for most platforms - checkout the [releases page](https://github.com/mikefarah/yq/releases) for the latest build. Alternatively - you can use one of the methods below:
 
-### MacOS:
-```
+### On MacOS:
+
+```bash
 brew install yq
 ```
-### Ubuntu and other Linux distros supporting `snap` packages:
+
+### On Windows:
+
+```bash
+choco install yq
 ```
+
+Kindly maintained by @chillum \([https://github.com/chillum/choco-packages/tree/master/yq](https://github.com/chillum/choco-packages/tree/master/yq)\)
+
+### On Ubuntu and other Linux distributions supporting `snap` packages:
+
+```bash
 snap install yq
 ```
 
 #### Snap notes
+
 `yq` installs with with [_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, this means it doesn't have direct access to root files. To read root files you can:
 
-```
-sudo cat /etc/myfile | yq r - a.path
+```bash
+sudo cat /etc/myfile | yq -r - somecommand
 ```
 
 And to write to a root file you can either use [sponge](https://linux.die.net/man/1/sponge):
+
+```bash
+sudo cat /etc/myfile | yq -r - somecommand | sudo sponge /etc/myfile
 ```
-sudo cat /etc/myfile | yq w - a.path value | sudo sponge /etc/myfile
-```
+
 or write to a temporary file:
-```
-sudo cat /etc/myfile | yq w - a.path value | sudo tee /etc/myfile.tmp
+
+```bash
+sudo cat /etc/myfile | yq -r - somecommand | sudo tee /etc/myfile.tmp
 sudo mv /etc/myfile.tmp /etc/myfile
 rm /etc/myfile.tmp
 ```
 
 ### On Ubuntu 16.04 or higher from Debian package:
-```sh
-sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CC86BB64
+
+```bash
 sudo add-apt-repository ppa:rmescandon/yq
 sudo apt update
 sudo apt install yq -y
 ```
-Supported by @rmescandon
 
-### Go Get:
-```
+Kindly maintained by @rmescandon
+
+### go get:
+
+```text
 GO111MODULE=on go get github.com/mikefarah/yq/v3
 ```
 
-## Run with Docker
+## Docker
 
 Oneshot use:
 
@@ -77,52 +90,3 @@ yq() {
 }
 ```
 
-## Features
-- Written in portable go, so you can download a lovely dependency free binary
-- [Colorize the output](https://mikefarah.gitbook.io/yq/usage/output-format#colorize-output)
-- [Deep read a yaml file with a given path expression](https://mikefarah.gitbook.io/yq/commands/read#basic)
-- [List matching paths of a given path expression](https://mikefarah.gitbook.io/yq/commands/read#path-only)
-- [Return the lengths of arrays/object/scalars](https://mikefarah.gitbook.io/yq/commands/read#printing-length-of-the-results)
-- Update a yaml file given a [path expression](https://mikefarah.gitbook.io/yq/commands/write-update#basic) or [script file](https://mikefarah.gitbook.io/yq/commands/write-update#basic)
-- Update creates any missing entries in the path on the fly
-- Deeply [compare](https://mikefarah.gitbook.io/yq/commands/compare) yaml files
-- Keeps yaml formatting and comments when updating
-- [Validate a yaml file](https://mikefarah.gitbook.io/yq/commands/validate)
-- Create a yaml file given a [deep path and value](https://mikefarah.gitbook.io/yq/commands/create#creating-a-simple-yaml-file) or a [script file](https://mikefarah.gitbook.io/yq/commands/create#creating-using-a-create-script)
-- [Prefix a path to a yaml file](https://mikefarah.gitbook.io/yq/commands/prefix)
-- [Convert to/from json to yaml](https://mikefarah.gitbook.io/yq/usage/convert)
-- [Pipe data in by using '-'](https://mikefarah.gitbook.io/yq/commands/read#from-stdin)
-- [Merge](https://mikefarah.gitbook.io/yq/commands/merge) multiple yaml files with various options for [overriding](https://mikefarah.gitbook.io/yq/commands/merge#overwrite-values) and [appending](https://mikefarah.gitbook.io/yq/commands/merge#append-values-with-arrays)
-- Supports multiple documents in a single yaml file for [reading](https://mikefarah.gitbook.io/yq/commands/read#multiple-documents), [writing](https://mikefarah.gitbook.io/yq/commands/write-update#multiple-documents) and [merging](https://mikefarah.gitbook.io/yq/commands/merge#multiple-documents)
-
-## [Usage](https://mikefarah.gitbook.io/yq/)
-
-Check out the [documentation](https://mikefarah.gitbook.io/yq/) for more detailed and advanced usage.
-
-```
-Usage:
-  yq [flags]
-  yq [command]
-
-Available Commands:
-  compare     yq x [--prettyPrint/-P] dataA.yaml dataB.yaml 'b.e(name==fr*).value'
-  delete      yq d [--inplace/-i] [--doc/-d index] sample.yaml 'b.e(name==fred)'
-  help        Help about any command
-  merge       yq m [--inplace/-i] [--doc/-d index] [--overwrite/-x] [--append/-a] sample.yaml sample2.yaml
-  new         yq n [--script/-s script_file] a.b.c newValue
-  prefix      yq p [--inplace/-i] [--doc/-d index] sample.yaml a.b.c
-  read        yq r [--printMode/-p pv] sample.yaml 'b.e(name==fr*).value'
-  validate    yq v sample.yaml
-  write       yq w [--inplace/-i] [--script/-s script_file] [--doc/-d index] sample.yaml 'b.e(name==fr*).value' newValue
-
-Flags:
-  -C, --colors        print using colors
-  -h, --help          help for yq
-  -I, --indent int    sets indent level for output (default 2)
-  -P, --prettyPrint   pretty print
-  -j, --tojson        output as json. By default it prints a json document in one line, use the prettyPrint flag to print a formatted doc.
-  -v, --verbose       verbose mode
-  -V, --version       Print version information and quit
-
-Use "yq [command] --help" for more information about a command.
-```

SUMMARY.md

@@ -0,0 +1,25 @@
+# Table of contents
+
+* [yq](README.md)
+* [Upgrading from V2](upgrading-from-v2.md)
+
+## Commands
+
+* [Read](commands/read.md)
+* [Validate](commands/validate.md)
+* [Compare](commands/compare.md)
+* [Write](commands/write-update.md)
+* [Create](commands/create.md)
+* [Delete](commands/delete.md)
+* [Merge](commands/merge.md)
+* [Prefix](commands/prefix.md)
+* [Shell Completion](commands/shell-completion.md)
+
+## Usage
+
+* [Output format](usage/output-format.md)
+* [Path Expressions](usage/path-expressions.md)
+* [Value Parsing](usage/value-parsing.md)
+* [Working with JSON](usage/convert.md)
+* [Github Page](https://github.com/mikefarah/yq)
+

cmd/compare.go

@@ -31,6 +31,8 @@ yq x -d1 dataA.yaml dataB.yaml 'a.b.c'
 	cmdCompare.PersistentFlags().StringVarP(&docIndex, "doc", "d", "0", "process document index number (0 based, * for all documents)")
 	cmdCompare.PersistentFlags().StringVarP(&printMode, "printMode", "p", "v", "print mode (v (values, default), p (paths), pv (path and value pairs)")
 	cmdCompare.PersistentFlags().StringVarP(&defaultValue, "defaultValue", "D", "", "default value printed when there are no results")
+	cmdCompare.PersistentFlags().BoolVarP(&stripComments, "stripComments", "", false, "strip comments out before comparing")
+	cmdCompare.PersistentFlags().BoolVarP(&explodeAnchors, "explodeAnchors", "X", false, "explode anchors")
 	return cmdCompare
 }
 

cmd/compare_test.go

@@ -16,6 +16,55 @@ func TestCompareSameCmd(t *testing.T) {
 	test.AssertResult(t, expectedOutput, result.Output)
 }
 
+func TestCompareIgnoreCommentsCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare --stripComments ../examples/data1.yaml ../examples/data1-no-comments.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := ``
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestCompareDontIgnoreCommentsCmd(t *testing.T) {
+	forceOsExit = false
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare ../examples/data1.yaml ../examples/data1-no-comments.yaml")
+
+	expectedOutput := `-a: simple # just the best
++a: simple
+ b: [1, 2]
+ c:
+   test: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestCompareExplodeAnchorsCommentsCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare --explodeAnchors ../examples/simple-anchor.yaml ../examples/simple-anchor-exploded.yaml")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := ``
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestCompareDontExplodeAnchorsCmd(t *testing.T) {
+	forceOsExit = false
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "compare ../examples/simple-anchor.yaml ../examples/simple-anchor-exploded.yaml")
+
+	expectedOutput := `-foo: &foo
++foo:
+   a: 1
+ foobar:
+-  !!merge <<: *foo
++  a: 1
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
 func TestCompareDifferentCmd(t *testing.T) {
 	forceOsExit = false
 	cmd := getRootCommand()
@@ -40,9 +89,9 @@ func TestComparePrettyCmd(t *testing.T) {
 	}
 	expectedOutput := ` a: simple # just the best
  b:
- - 1
--- 2
-+- 3
+   - 1
+-  - 2
++  - 3
  c:
    test: 1
 `

cmd/constant.go

@@ -10,12 +10,15 @@ var printMode = "v"
 var printLength = false
 var unwrapScalar = true
 var customStyle = ""
+var anchorName = ""
+var makeAlias = false
 var stripComments = false
 var collectIntoArray = false
 var writeInplace = false
 var writeScript = ""
 var sourceYamlFile = ""
 var outputToJSON = false
+var exitStatus = false
 var prettyPrint = false
 var explodeAnchors = false
 var colorsEnabled = false

cmd/delete_test.go

@@ -94,8 +94,31 @@ b:
 	expectedOutput := `a: 2
 b:
   hi:
-  - name: fred
-  - name: sam
+    - name: fred
+    - name: sam
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestDeleteDeepSplatArrayYaml(t *testing.T) {
+	content := `thing: 123
+b:
+ hi:
+  - thing: item1
+    name: fred
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("delete %s **.thing", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `b:
+  hi:
+    - name: fred
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }

cmd/merge_test.go

@@ -101,10 +101,10 @@ func TestMergeAppendArraysCmd(t *testing.T) {
 		t.Error(result.Error)
 	}
 	expectedOutput := `people:
-- name: Barry
-  age: 21
-- name: Roger
-  age: 44
+  - name: Barry
+    age: 21
+  - name: Roger
+    age: 44
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }
@@ -146,8 +146,8 @@ another:
   document: here
 a: simple # just the best
 b:
-- 1
-- 2
+  - 1
+  - 2
 c:
   test: 1
 ---
@@ -317,8 +317,8 @@ func TestMergeAllowEmptyTargetCmd(t *testing.T) {
 	}
 	expectedOutput := `a: simple # just the best
 b:
-- 1
-- 2
+  - 1
+  - 2
 c:
   test: 1
 `

cmd/new.go

@@ -29,6 +29,8 @@ Note that you can give a create script to perform more sophisticated yaml. This
 	cmdNew.PersistentFlags().StringVarP(&writeScript, "script", "s", "", "yaml script for creating yaml")
 	cmdNew.PersistentFlags().StringVarP(&customTag, "tag", "t", "", "set yaml tag (e.g. !!int)")
 	cmdNew.PersistentFlags().StringVarP(&customStyle, "style", "", "", "formatting style of the value: single, double, folded, flow, literal, tagged")
+	cmdNew.PersistentFlags().StringVarP(&anchorName, "anchorName", "", "", "anchor name")
+	cmdNew.PersistentFlags().BoolVarP(&makeAlias, "makeAlias", "", false, "create an alias using the value as the anchor name")
 	return cmdNew
 }
 

cmd/new_test.go

@@ -18,6 +18,30 @@ func TestNewCmd(t *testing.T) {
 	test.AssertResult(t, expectedOutput, result.Output)
 }
 
+func TestNewAnchorCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "new b.c 3 --anchorName=fred")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: &fred 3
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
+func TestNewAliasCmd(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "new b.c foo --makeAlias")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	expectedOutput := `b:
+  c: *foo
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
 func TestNewArrayCmd(t *testing.T) {
 	cmd := getRootCommand()
 	result := test.RunCmd(cmd, "new b[0] 3")
@@ -25,7 +49,7 @@ func TestNewArrayCmd(t *testing.T) {
 		t.Error(result.Error)
 	}
 	expectedOutput := `b:
-- 3
+  - 3
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }

cmd/prefix_test.go

@@ -41,8 +41,8 @@ func TestPrefixCmdArray(t *testing.T) {
 		t.Error(result.Error)
 	}
 	expectedOutput := `- d:
-  - b:
-      c: 3
+    - b:
+        c: 3
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }

cmd/read.go

@@ -31,6 +31,7 @@ yq r -- things.yaml '--key-starting-with-dashes.blah'
 	cmdRead.PersistentFlags().BoolVarP(&unwrapScalar, "unwrapScalar", "", true, "unwrap scalar, print the value with no quotes, colors or comments")
 	cmdRead.PersistentFlags().BoolVarP(&stripComments, "stripComments", "", false, "print yaml without any comments")
 	cmdRead.PersistentFlags().BoolVarP(&explodeAnchors, "explodeAnchors", "X", false, "explode anchors")
+	cmdRead.PersistentFlags().BoolVarP(&exitStatus, "exitStatus", "e", false, "set exit status if no matches are found")
 	return cmdRead
 }
 
@@ -50,7 +51,13 @@ func readProperty(cmd *cobra.Command, args []string) error {
 
 	matchingNodes, errorReadingStream := readYamlFile(args[0], path, updateAll, docIndexInt)
 
+	if exitStatus && len(matchingNodes) == 0 {
+		cmd.SilenceUsage = true
+		return errors.New("No matches found")
+	}
+
 	if errorReadingStream != nil {
+		cmd.SilenceUsage = true
 		return errorReadingStream
 	}
 	out := cmd.OutOrStdout()

cmd/read_test.go

@@ -17,6 +17,30 @@ func TestReadCmd(t *testing.T) {
 	test.AssertResult(t, "2\n", result.Output)
 }
 
+func TestReadCmdWithExitStatus(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "read ../examples/sample.yaml b.c -e")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	test.AssertResult(t, "2\n", result.Output)
+}
+
+func TestReadCmdWithExitStatusNotExist(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "read ../examples/sample.yaml caterpillar -e")
+	test.AssertResult(t, "No matches found", result.Error.Error())
+}
+
+func TestReadCmdNotExist(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "read ../examples/sample.yaml caterpillar")
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+	test.AssertResult(t, "", result.Output)
+}
+
 func TestReadUnwrapCmd(t *testing.T) {
 
 	content := `b: 'frog' # my favourite`
@@ -871,17 +895,35 @@ func TestReadPrettyPrintCmd(t *testing.T) {
 b:
   c: 2
   d:
-  - 3
-  - 4
+    - 3
+    - 4
   e:
-  - name: fred
-    value: 3
-  - name: sam
-    value: 4
+    - name: fred
+      value: 3
+    - name: sam
+      value: 4
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }
 
+func TestReadNotFoundWithExitStatus(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "read ../examples/sample.yaml adsf -e")
+	if result.Error == nil {
+		t.Error("Expected command to fail")
+	}
+	expectedOutput := `No matches found`
+	test.AssertResult(t, expectedOutput, result.Error.Error())
+}
+
+func TestReadNotFoundWithoutExitStatus(t *testing.T) {
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, "read ../examples/sample.yaml adsf")
+	if result.Error != nil {
+		t.Error("Expected command to succeed!")
+	}
+}
+
 func TestReadPrettyPrintWithIndentCmd(t *testing.T) {
 	cmd := getRootCommand()
 	result := test.RunCmd(cmd, "read -P -I4 ../examples/sample.json")
@@ -892,13 +934,13 @@ func TestReadPrettyPrintWithIndentCmd(t *testing.T) {
 b:
     c: 2
     d:
-      - 3
-      - 4
+        - 3
+        - 4
     e:
-      - name: fred
-        value: 3
-      - name: sam
-        value: 4
+        - name: fred
+          value: 3
+        - name: sam
+          value: 4
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }
@@ -914,8 +956,8 @@ func TestReadCmd_ArrayYaml_NoPath(t *testing.T) {
   hosts: lalaland
   name: "Apply smth"
   roles:
-  - lala
-  - land
+    - lala
+    - land
   serial: 1
 - become: false
   gather_facts: true
@@ -934,8 +976,8 @@ gather_facts: false
 hosts: lalaland
 name: "Apply smth"
 roles:
-- lala
-- land
+  - lala
+  - land
 serial: 1
 `
 	test.AssertResult(t, expectedOutput, result.Output)
@@ -952,8 +994,8 @@ gather_facts: false
 hosts: lalaland
 name: "Apply smth"
 roles:
-- lala
-- land
+  - lala
+  - land
 serial: 1
 become: false
 gather_facts: true
@@ -973,8 +1015,8 @@ func TestReadCmd_ArrayYaml_SplatWithKeyAndValueCmd(t *testing.T) {
   hosts: lalaland
   name: "Apply smth"
   roles:
-  - lala
-  - land
+    - lala
+    - land
   serial: 1
 '[1]':
   become: false
@@ -1165,6 +1207,26 @@ func TestReadBadDataCmd(t *testing.T) {
 	test.AssertResult(t, expectedOutput, result.Error.Error())
 }
 
+func TestReadDeepFromRootCmd(t *testing.T) {
+	content := `state:
+  country: 
+    city: foo
+`
+	filename := test.WriteTempYamlFile(content)
+	defer test.RemoveTempYamlFile(filename)
+
+	cmd := getRootCommand()
+	result := test.RunCmd(cmd, fmt.Sprintf("read %s (**.city==foo)", filename))
+	if result.Error != nil {
+		t.Error(result.Error)
+	}
+
+	expectedOutput := `country:
+  city: foo
+`
+	test.AssertResult(t, expectedOutput, result.Output)
+}
+
 func TestReadSplatPrefixCmd(t *testing.T) {
 	content := `a: 2
 b:

cmd/root.go

@@ -54,8 +54,7 @@ func New() *cobra.Command {
 		createDeleteCmd(),
 		createNewCmd(),
 		createMergeCmd(),
+		createBashCompletionCmd(rootCmd),
 	)
-	rootCmd.SetOutput(os.Stdout)
-
 	return rootCmd
 }

cmd/shell_completion.go

@@ -0,0 +1,57 @@
+package cmd
+
+import (
+	"fmt"
+	"os"
+
+	"github.com/spf13/cobra"
+)
+
+var shellVariant = "bash"
+
+func createBashCompletionCmd(rootCmd *cobra.Command) *cobra.Command {
+	var completionCmd = &cobra.Command{
+		Use:   "shell-completion",
+		Short: "Generates shell completion scripts",
+		Long: `To load completion for:
+bash:
+	Run	
+	. <(yq shell-completion)
+	
+	To configure your bash shell to load completions for each session add to
+	your bashrc
+	
+	# ~/.bashrc or ~/.profile
+	. <(yq shell-completion)
+
+zsh:
+	The generated completion script should be put somewhere in your $fpath named _yq
+
+powershell:
+	Users need PowerShell version 5.0 or above, which comes with Windows 10 and 
+	can be downloaded separately for Windows 7 or 8.1. They can then write the 
+	completions to a file and source this file from their PowerShell profile, 
+	which is referenced by the $Profile environment variable.
+
+fish:
+	Save the output to a fish file and add it to your completions directory.
+
+	`,
+		RunE: func(cmd *cobra.Command, args []string) error {
+			switch shellVariant {
+			case "bash", "":
+				return rootCmd.GenBashCompletion(os.Stdout)
+			case "zsh":
+				return rootCmd.GenZshCompletion(os.Stdout)
+			case "fish":
+				return rootCmd.GenFishCompletion(os.Stdout, true)
+			case "powershell":
+				return rootCmd.GenPowerShellCompletion(os.Stdout)
+			default:
+				return fmt.Errorf("Unknown variant %v", shellVariant)
+			}
+		},
+	}
+	completionCmd.PersistentFlags().StringVarP(&shellVariant, "variation", "V", "", "shell variation: bash (default), zsh, fish, powershell")
+	return completionCmd
+}

cmd/utils.go

@@ -149,25 +149,91 @@ func writeString(writer io.Writer, txt string) error {
 	return errorWriting
 }
 
+func setIfNotThere(node *yaml.Node, key string, value *yaml.Node) {
+	for index := 0; index < len(node.Content); index = index + 2 {
+		keyNode := node.Content[index]
+		if keyNode.Value == key {
+			return
+		}
+	}
+	// need to add it to the map
+	mapEntryKey := yaml.Node{Value: key, Kind: yaml.ScalarNode}
+	node.Content = append(node.Content, &mapEntryKey)
+	node.Content = append(node.Content, value)
+}
+
+func applyAlias(node *yaml.Node, alias *yaml.Node) {
+	for index := 0; index < len(alias.Content); index = index + 2 {
+		keyNode := alias.Content[index]
+		log.Debugf("applying alias key %v", keyNode.Value)
+		valueNode := alias.Content[index+1]
+		setIfNotThere(node, keyNode.Value, valueNode)
+	}
+}
+
+func explodeNode(node *yaml.Node) error {
+	node.Anchor = ""
+	switch node.Kind {
+	case yaml.SequenceNode, yaml.DocumentNode:
+		for index, contentNode := range node.Content {
+			log.Debugf("exploding index %v", index)
+			errorInContent := explodeNode(contentNode)
+			if errorInContent != nil {
+				return errorInContent
+			}
+		}
+		return nil
+	case yaml.AliasNode:
+		log.Debugf("its an alias!")
+		node.Kind = node.Alias.Kind
+		node.Style = node.Alias.Style
+		node.Tag = node.Alias.Tag
+		node.Content = node.Alias.Content
+		node.Value = node.Alias.Value
+		node.Alias = nil
+		return nil
+	case yaml.MappingNode:
+		for index := 0; index < len(node.Content); index = index + 2 {
+			keyNode := node.Content[index]
+			valueNode := node.Content[index+1]
+			log.Debugf("traversing %v", keyNode.Value)
+			if keyNode.Value != "<<" {
+				errorInContent := explodeNode(valueNode)
+				if errorInContent != nil {
+					return errorInContent
+				}
+			} else {
+				if valueNode.Kind == yaml.SequenceNode {
+					log.Debugf("an alias merge list!")
+					for index := len(valueNode.Content) - 1; index >= 0; index = index - 1 {
+						aliasNode := valueNode.Content[index]
+						applyAlias(node, aliasNode.Alias)
+					}
+				} else {
+					log.Debugf("an alias merge!")
+					applyAlias(node, valueNode.Alias)
+				}
+				node.Content = append(node.Content[:index], node.Content[index+2:]...)
+				//replay that index, since the array is shorter now.
+				index = index - 2
+			}
+		}
+
+		return nil
+	default:
+		return nil
+	}
+}
+
 func explode(matchingNodes []*yqlib.NodeContext) error {
 	log.Debug("exploding nodes")
 	for _, nodeContext := range matchingNodes {
-		var targetNode = yaml.Node{Kind: nodeContext.Node.Kind}
-		explodedNodes, errorRetrieving := lib.Get(nodeContext.Node, "**", true)
-		if errorRetrieving != nil {
-			return errorRetrieving
+		log.Debugf("exploding %v", nodeContext.Head)
+		errorExplodingNode := explodeNode(nodeContext.Node)
+		if errorExplodingNode != nil {
+			return errorExplodingNode
 		}
-		for _, matchingNode := range explodedNodes {
-			mergePath := lib.MergePathStackToString(matchingNode.PathStack, appendFlag)
-			updateCommand := yqlib.UpdateCommand{Command: "update", Path: mergePath, Value: matchingNode.Node, Overwrite: overwriteFlag}
-			errorUpdating := lib.Update(&targetNode, updateCommand, true)
-			if errorUpdating != nil {
-				return errorUpdating
-			}
-		}
-		nodeContext.Node = &targetNode
 	}
-	log.Debug("done exploding nodes")
 	return nil
 }
 
@@ -357,6 +423,16 @@ func readAndUpdate(stdOut io.Writer, inputFile string, updateData updateDataFn)
 		if err != nil {
 			return err
 		}
+		// mkdir temp dir as some docker images does not have temp dir
+		_, err = os.Stat(os.TempDir())
+		if os.IsNotExist(err) {
+			err = os.Mkdir(os.TempDir(), 0700)
+			if err != nil {
+				return err
+			}
+		} else if err != nil {
+			return err
+		}
 		tempFile, err := ioutil.TempFile("", "temp")
 		if err != nil {
 			return err
@@ -435,13 +511,13 @@ func readUpdateCommands(args []string, expectedArgs int, badArgsMessage string)
 		log.Debug("args %v", args)
 		log.Debug("path %v", args[expectedArgs-2])
 		log.Debug("Value %v", args[expectedArgs-1])
-		updateCommands[0] = yqlib.UpdateCommand{Command: "update", Path: args[expectedArgs-2], Value: valueParser.Parse(args[expectedArgs-1], customTag, customStyle), Overwrite: true}
+		updateCommands[0] = yqlib.UpdateCommand{Command: "update", Path: args[expectedArgs-2], Value: valueParser.Parse(args[expectedArgs-1], customTag, customStyle, anchorName, makeAlias), Overwrite: true}
 	} else if len(args) == expectedArgs-1 {
 		// don't update the value
 		updateCommands = make([]yqlib.UpdateCommand, 1)
 		log.Debug("args %v", args)
 		log.Debug("path %v", args[expectedArgs-2])
-		updateCommands[0] = yqlib.UpdateCommand{Command: "update", Path: args[expectedArgs-2], Value: valueParser.Parse("", customTag, customStyle), Overwrite: true, DontUpdateNodeValue: true}
+		updateCommands[0] = yqlib.UpdateCommand{Command: "update", Path: args[expectedArgs-2], Value: valueParser.Parse("", customTag, customStyle, anchorName, makeAlias), Overwrite: true, DontUpdateNodeValue: true}
 	} else {
 		return nil, errors.New(badArgsMessage)
 	}

cmd/version.go

@@ -11,7 +11,7 @@ var (
 	GitDescribe string
 
 	// Version is main version number that is being run at the moment.
-	Version = "3.3.0"
+	Version = "3.3.2"
 
 	// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
 	// then it means that it is a final release. Otherwise, this is a pre-release

cmd/write.go

@@ -47,6 +47,8 @@ format is list of update commands (update or delete) like so:
 	cmdWrite.PersistentFlags().StringVarP(&customTag, "tag", "t", "", "set yaml tag (e.g. !!int)")
 	cmdWrite.PersistentFlags().StringVarP(&docIndex, "doc", "d", "0", "process document index number (0 based, * for all documents)")
 	cmdWrite.PersistentFlags().StringVarP(&customStyle, "style", "", "", "formatting style of the value: single, double, folded, flow, literal, tagged")
+	cmdWrite.PersistentFlags().StringVarP(&anchorName, "anchorName", "", "", "anchor name")
+	cmdWrite.PersistentFlags().BoolVarP(&makeAlias, "makeAlias", "", false, "create an alias using the value as the anchor name")
 	return cmdWrite
 }
 

cmd/write_test.go

@@ -261,9 +261,9 @@ func TestWriteAutoCreateCmd(t *testing.T) {
 		t.Error(result.Error)
 	}
 	expectedOutput := `applications:
-- name: app
-  env:
-    hello: world
+  - name: app
+    env:
+      hello: world
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }
@@ -476,8 +476,8 @@ func TestWriteCmd_Append(t *testing.T) {
 		t.Error(result.Error)
 	}
 	expectedOutput := `b:
-- foo
-- 7
+  - foo
+  - 7
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }
@@ -508,8 +508,8 @@ func TestWriteCmd_AppendInlinePretty(t *testing.T) {
 		t.Error(result.Error)
 	}
 	expectedOutput := `b:
-- foo
-- 7
+  - foo
+  - 7
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }
@@ -527,7 +527,7 @@ func TestWriteCmd_AppendEmptyArray(t *testing.T) {
 	}
 	expectedOutput := `a: 2
 b:
-- v
+  - v
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }
@@ -546,8 +546,8 @@ func TestWriteCmd_SplatArray(t *testing.T) {
 		t.Error(result.Error)
 	}
 	expectedOutput := `b:
-- c: new
-- c: new
+  - c: new
+  - c: new
 `
 	test.AssertResult(t, expectedOutput, result.Output)
 }

commands/compare.md

@@ -0,0 +1,103 @@
+---
+description: Deeply compare two yaml documents
+---
+
+# Compare
+
+```bash
+yq compare <yaml_file_1> <yaml_file_2> <path_expression>
+```
+
+Compares the matching yaml nodes at path expression in the two yaml documents. See [path expression](../usage/path-expressions.md) for more details. Difference calculated line by line, and is printed out line by line where the first character of each line is either:
+
+*  `` a space, indicating no change at this line
+* `-` a minus ,indicating the line is not present in the second document \(it's removed\)
+* `+` a plus, indicating that the line is not present in the first document \(it's added\)
+
+If there are differences then `yq` will print out the differences and exit with code 1. If there are no differences, then nothing will be printed and the exit code will be 0.
+
+## Example data
+
+Given data1.yaml
+
+```yaml
+"apples": are nice
+somethingElse: cool # this is nice
+favouriteNumbers: [1,2,3]
+noDifference: it's the same
+```
+
+and data2.yaml
+
+```yaml
+apples: are nice
+somethingElse: cool # yeah i like it
+favouriteNumbers:
+- 1
+- 3
+- 4
+noDifference: it's the same
+```
+
+## Basic
+
+Basic will compare the yaml documents 'as-is'
+
+```yaml
+yq compare data1.yaml data2.yaml
+```
+
+yields
+
+```text
+-"apples": are nice
+-somethingElse: cool # this is nice
+-favouriteNumbers: [1, 2, 3]
++apples: are nice
++somethingElse: cool # yeah i like it
++favouriteNumbers:
++- 1
++- 3
++- 4
+ noDifference: it's the same
+```
+
+## Formatted
+
+Most of the time, it will make sense to [format](../usage/output-format.md#pretty-print) the documents before comparing:
+
+```text
+yq compare --prettyPrint data1.yaml data2.yml
+```
+
+yields
+
+```text
+ apples: are nice
+-somethingElse: cool # this is nice
++somethingElse: cool # yeah i like it
+ favouriteNumbers:
+ - 1
+-- 2
+ - 3
++- 4
+ noDifference: it's the same
+```
+
+## Using path expressions
+
+Use [path expressions](../usage/path-expressions.md) to compare subsets of yaml documents
+
+```text
+yq compare -P data1.yaml data2.yml favouriteNumbers
+```
+
+yields
+
+```text
+ - 1
+-- 2
+ - 3
++- 4
+```
+

commands/create.md

@@ -0,0 +1,58 @@
+# Create
+
+```text
+yq n <path_expression> <new value>
+```
+
+This works in the same way as the write command, but you don't pass in an existing Yaml file. Currently this does not support creating multiple documents in a single yaml file.
+
+See docs for [path expression](../usage/path-expressions.md) and [value parsing](../usage/value-parsing.md) for more details, including controlling quotes and tags.
+
+## Creating a simple yaml file
+
+```bash
+yq n b.c cat
+```
+
+will output:
+
+```yaml
+b:
+  c: cat
+```
+
+## Creating using a create script
+
+Create scripts follow the same format as the update scripts.
+
+Given a script create\_instructions.yaml of:
+
+```yaml
+- command: update 
+  path: b.c
+  value:
+    #great 
+    things: frog # wow!
+```
+
+then
+
+```bash
+yq n -s create_instructions.yaml
+```
+
+will output:
+
+```yaml
+b:
+  c:
+    #great
+    things: frog # wow!
+```
+
+You can also pipe the instructions in:
+
+```bash
+cat create_instructions.yaml | yq n -s -
+```
+

commands/delete.md

@@ -0,0 +1,110 @@
+---
+description: Deletes all the matching nodes for the path expression in the given yaml input
+---
+
+# Delete
+
+```text
+yq delete <yaml_file|-> <path_expression>
+```
+
+See docs for [path expression](../usage/path-expressions.md) for more details.
+
+## Deleting from a simple document
+
+Given a sample.yaml file of:
+
+```yaml
+b:
+  c: 2
+  apples: green
+```
+
+then
+
+```bash
+yq d sample.yaml b.c
+```
+
+will output
+
+```yaml
+b:
+  apples: green
+```
+
+## From STDIN
+
+Use "-" \(without quotes\) in-place of a file name if you wish to pipe in input from STDIN.
+
+```bash
+cat sample.yaml | yq d - b.c
+```
+
+## Deleting in-place
+
+```bash
+yq d -i sample.yaml b.c
+```
+
+will update the sample.yaml file so that the 'c' node is deleted
+
+## Multiple Documents
+
+### Delete from single document
+
+Given a sample.yaml file of:
+
+```yaml
+something: else
+field: leaveMe
+---
+b:
+  c: 2
+field: deleteMe
+```
+
+then
+
+```bash
+yq w -d1 sample.yaml field
+```
+
+will output:
+
+```yaml
+something: else
+field: leaveMe
+---
+b:
+  c: 2
+```
+
+### Delete from all documents
+
+Given a sample.yaml file of:
+
+```yaml
+something: else
+field: deleteMe
+---
+b:
+  c: 2
+field: deleteMeToo
+```
+
+then
+
+```bash
+yq w -d'*' sample.yaml field
+```
+
+will output:
+
+```yaml
+something: else
+---
+b:
+  c: 2
+```
+

commands/merge.md

@@ -0,0 +1,232 @@
+---
+description: Merge multiple yaml files into a one
+---
+
+# Merge
+
+Yaml files can be merged using the 'merge' command. Each additional file merged with the first file will set values for any key not existing already or where the key has no value.
+
+```text
+yq m <yaml_file> <yaml_file2> <yaml_file3>...
+```
+
+## Merge example
+
+Given a data1.yaml file of:
+
+```yaml
+a: simple
+b: [1, 2]
+```
+
+and data2.yaml file of:
+
+```yaml
+a: other
+c:
+  test: 1
+```
+
+then
+
+```bash
+yq merge data1.yaml data2.yaml
+```
+
+will output:
+
+```yaml
+a: simple
+b: [1, 2]
+c:
+  test: 1
+```
+
+## Updating files in-place
+
+```bash
+yq m -i data1.yaml data2.yaml
+```
+
+will update the data1.yaml file with the merged result.
+
+## Overwrite values
+
+Given a data1.yaml file of:
+
+```yaml
+a: simple
+b: [1, 2]
+d: left alone
+```
+
+and data2.yaml file of:
+
+```yaml
+a: other
+b: [3, 4]
+c:
+  test: 1
+```
+
+then
+
+```bash
+yq m -x data1.yaml data2.yaml
+```
+
+will output:
+
+```yaml
+a: other
+b: [3, 4]
+c:
+  test: 1
+d: left alone
+```
+
+Notice that 'b' does not result in the merging of the values within an array.
+
+## Append values with arrays
+
+Given a data1.yaml file of:
+
+```yaml
+a: simple
+b: [1, 2]
+d: hi
+```
+
+and data2.yaml file of:
+
+```yaml
+a: something
+b: [3, 4]
+c:
+  test: 2
+  other: true
+```
+
+then
+
+```bash
+yq m -a data1.yaml data2.yaml
+```
+
+will output:
+
+```yaml
+a: simple
+b: [1, 2, 3, 4]
+c:
+  test: 2
+  other: true
+d: hi
+```
+
+Note that the 'b' array has concatenated the values from the second data file. Also note that other map keys are not overridden \(field a\).
+
+## Auto-create
+
+By default, `yq` will automatically create any missing entries in the target yaml file. This can be turned off so that only matching paths are merged in. When turned off - you will most likely want to use the [override flag](merge.md#overwrite-values).
+
+Given a data1.yml file of:
+
+```yaml
+a: thing
+b: something else
+```
+
+and data2.yml file of:
+
+```yaml
+b: new value
+d: not in original
+```
+
+Then
+
+```yaml
+yq m --overwrite --autocreate=false data1.yml data2.yml
+```
+
+will yield
+
+```yaml
+a: thing
+b: new value
+```
+
+
+
+## Multiple Documents
+
+### Merge into single document
+
+Currently yq only has multi-document support for the _first_ document being merged into. The remaining yaml files will have their first document selected.
+
+Given a data1.yaml file of:
+
+```yaml
+something: else
+---
+a: simple
+b: cat
+```
+
+and data3.yaml file of:
+
+```yaml
+b: dog
+```
+
+then
+
+```bash
+yq m -x -d1 data1.yaml data3.yaml
+```
+
+will output:
+
+```yaml
+something: else
+---
+a: simple
+b: dog
+```
+
+### Merge into all documents
+
+Currently yq only has multi-document support for the _first_ document being merged into. The remaining yaml files will have their first document selected.
+
+Given a data1.yaml file of:
+
+```yaml
+something: else
+---
+a: simple
+b: cat
+```
+
+and data3.yaml file of:
+
+```yaml
+b: dog
+```
+
+then
+
+```bash
+yq m -x -d'*' data1.yaml data3.yaml
+```
+
+will output:
+
+```yaml
+b: dog
+something: else
+---
+a: simple
+b: dog
+```
+

commands/prefix.md

@@ -0,0 +1,103 @@
+---
+description: >-
+  Prefixes a yaml document with the given path expression. The complete yaml
+  content will be nested inside the new prefix path.
+---
+
+# Prefix
+
+```text
+yq p <yaml_file> <path>
+```
+
+See docs for [path expression](../usage/path-expressions.md) for more details.
+
+## Prefix a document
+
+Given a data1.yaml file of:
+
+```yaml
+a:
+  b: [1, 2]
+```
+
+then
+
+```bash
+yq p data1.yaml c.d
+```
+
+will output:
+
+```yaml
+c:
+  d:
+    a:
+      b: [1, 2]
+```
+
+## Updating files in-place
+
+```bash
+yq p -i data1.yaml c
+```
+
+will update the data1.yaml file so that the path 'c' prefixes the document.
+
+## Multiple Documents
+
+### Prefix a single document
+
+Given a data1.yaml file of:
+
+```yaml
+something: else
+---
+a: simple
+b: cat
+```
+
+then
+
+```bash
+yq p -d1 data1.yaml c
+```
+
+will output:
+
+```yaml
+something: else
+---
+c:
+  a: simple
+  b: cat
+```
+
+### Prefix all documents
+
+Given a data1.yaml file of:
+
+```yaml
+something: else
+---
+a: simple
+b: cat
+```
+
+then
+
+```bash
+yq p -d'*' data1.yaml c
+```
+
+will output:
+
+```yaml
+c:
+  something: else
+---
+c:
+  a: simple
+  b: cat
+```
+

commands/read.md

@@ -0,0 +1,326 @@
+---
+description: Returns matching nodes/values of a path expression for a given yaml document
+---
+
+# Read
+
+```text
+yq r <yaml_file|json_file> <path_expression>
+```
+
+Returns the matching nodes of the path expression for the given yaml file \(or STDIN\).
+
+See docs for [path expression](../usage/path-expressions.md) for more details.
+
+## Basic
+
+Given a sample.yaml file of:
+
+```yaml
+b:
+  c: 2
+```
+
+then
+
+```bash
+yq r sample.yaml b.c
+```
+
+will output the value of '2'.
+
+## From Stdin
+
+Given a sample.yaml file of:
+
+```bash
+cat sample.yaml | yq r - b.c
+```
+
+will output the value of '2'.
+
+## Default values
+
+Using the `--defaultValue/-D` flag you can specify a default value to be printed when no matching nodes are found for an expression
+
+```text
+yq r sample.yaml --defaultValue frog path.not.there
+```
+
+will yield \(assuming `path.not.there` does not match any nodes\):
+
+```text
+frog
+```
+
+## Printing matching paths
+
+By default, yq will only print the value of the path expression for the yaml document. By specifying `--printMode` or `-p` you can print the matching paths.
+
+```yaml
+a:
+  thing_a: 
+    animal: cat
+  other: 
+    animal: frog
+  thing_b: 
+    vehicle: car
+```
+
+### Path Only
+
+```bash
+yq r --printMode p "a.thing*.*"
+```
+
+will print
+
+```text
+a.thing_a.animal
+a.thing_b.vehicle
+```
+
+### Path and Value
+
+```bash
+yq r --printMode pv "a.thing*.*"
+```
+
+will print
+
+```text
+a.thing_a.animal: cat
+a.thing_b.vehicle: car
+```
+
+## Collect results into an array
+
+By default, results are printed out line by line as independent matches. This is handy for both readability as well as piping into tools like `xargs`. However, if you would like to collect the matching results into an array then use the `--collect/-C` flag. This is particularly useful with the `length` flag described below.
+
+Given:
+
+```yaml
+a:
+  thing_a: 
+    animal: cat
+  other: 
+    animal: frog
+  thing_b: 
+    vehicle: car
+```
+
+```text
+yq r sample.yaml --collect a.*.animal
+```
+
+will print
+
+```text
+- cat
+- frog
+```
+
+## Printing length of the results
+
+Use the `--length/-L` flag to print the length of results. For arrays this will be the number of items, objects the number of entries and scalars the length of the value.
+
+Given
+
+```text
+animals:
+  - cats
+  - dog
+  - cheetah
+```
+
+```text
+yq r sample.yml --length animals
+```
+
+will print 
+
+```text
+3
+```
+
+### Length of filtered results
+
+By default, filtered results are printed _independently_ so you will get the length of each result printed on a separate line:
+
+```text
+yq r sample.yaml --length --printMode pv 'animals.(.==c*)'
+```
+
+```text
+animals.[0]: 4
+animals.[2]: 7
+```
+
+However, you'll often want to know the count of the filtered results - use the `--collect` flag to collect the results in the array. The length will then return the size of the array. 
+
+```text
+yq r sample.yaml --length --collect 'animals.(.==c*)' 
+```
+
+```text
+2
+```
+
+## Anchors and Aliases
+
+The read command will print out the anchors of a document and can also traverse them.
+
+Lets take a look at a simple example file:
+
+```yaml
+foo: &foo
+  a: 1
+
+foobar: *foo
+```
+
+### Printing anchors
+
+```bash
+yq r sample.yml foo
+```
+
+will print out
+
+```yaml
+&foo
+a: 1
+```
+
+Similarly,
+
+```text
+yq r sample.yml foobar
+```
+
+prints out
+
+```yaml
+*foo
+```
+
+### Traversing anchors
+
+To traverse an anchor, we need to either explicitly reference merged in values:
+
+```text
+yq r sample.yml foobar.a
+```
+
+to get
+
+```text
+1
+```
+
+or we can use deep splat to get all the values
+
+```bash
+yq r sample.yml -p pv "foobar.**"
+```
+
+prints
+
+```yaml
+foobar.a: 1
+```
+
+The same methods work for the `<<: [*blah, *thing]`anchors.
+
+### Exploding Anchors
+
+By default anchors are not exploded \(or expanded/de-referenced\) for viewing, and the yaml is shown as-is. Use the `--explodeAnchors/-X` flag to show the anchor values.
+
+Given sample.yml:
+
+```yaml
+foo: &foo
+  a: original
+  thing: coolasdf
+  thirsty: yep
+
+bar: &bar
+  b: 2
+  thing: coconut
+  c: oldbar
+
+foobarList:
+  <<: [*foo,*bar]
+  c: newbar
+```
+
+Then
+
+```text
+yq r -X sample.yml foobarList
+```
+
+yields
+
+```text
+c: newbar
+b: 2
+thing: coconut
+a: original
+thirsty: yep
+```
+
+Note that yq processes the merge anchor list in reverse order, to ensure that the last items in the list override the preceding.
+
+## Multiple Documents
+
+### Reading from a single document
+
+Given a sample.yaml file of:
+
+```yaml
+something: else
+---
+b:
+  c: 2
+```
+
+then
+
+```bash
+yq r -d1 sample.yaml b.c
+```
+
+will output the value of '2'.
+
+### Read from all documents
+
+Reading all documents will return the result as an array. This can be converted to json using the '-j' flag if desired.
+
+Given a sample.yaml file of:
+
+```yaml
+name: Fred
+age: 22
+---
+name: Stella
+age: 23
+---
+name: Android
+age: 232
+```
+
+then
+
+```bash
+yq r -d'*' sample.yaml name
+```
+
+will output:
+
+```text
+Fred
+Stella
+Android
+```
+

commands/shell-completion.md

@@ -0,0 +1,51 @@
+---
+description: >-
+  Generate a shell completion file for supported shells
+  (bash/fish/zsh/powershell)
+---
+
+# Shell Completion
+
+```bash
+yq shell-completion --variation=zsh
+```
+
+Prints to StdOut a shell completion script for zsh shell.
+
+### Bash \(default\)
+
+```bash
+yq shell-completion
+```
+
+To configure your bash shell to load completions for each session add to your bashrc
+
+```text
+# ~/.bashrc or ~/.profile
+. <(yq shell-completion)
+```
+
+### zsh
+
+```bash
+yq shell-completion --variation=zsh
+```
+
+The generated completion script should be put somewhere in your $fpath named \_yq
+
+### fish
+
+```bash
+yq shell-completion --variation=fish
+```
+
+Save the output to a '.fish' file and add it to your completions directory.
+
+### PowerShell
+
+```bash
+yq shell-completion --variation=powershell
+```
+
+Users need PowerShell version 5.0 or above, which comes with Windows 10 and can be downloaded separately for Windows 7 or 8.1. They can then write the completions to a file and source this file from their PowerShell profile, which is referenced by the $Profile environment variable.
+

commands/validate.md

@@ -0,0 +1,54 @@
+---
+description: Validate a given yaml file
+---
+
+# Validate
+
+```text
+yq v <yaml_file|->
+```
+
+Validates the given yaml file, does nothing if its valid, otherwise it will print errors to Stderr and exit with a non 0 exit code. This works like the [read command](read.md) - but does not take a path expression and does not print the yaml if it is valid.
+
+## Basic - valid
+
+```text
+yq v valid.yaml
+```
+
+This will not print anything, and finish with a successful \(0\) exit code.
+
+## Basic - invalid, from stdin
+
+```text
+echo '[1234' | yq v -
+```
+
+This will print the parsing error to stderr:
+
+```bash
+10:43:09 main [ERRO] yaml: line 1: did not find expected ',' or ']'
+```
+
+And return a error exit code \(1\)
+
+## Multiple documents
+
+Like other commands, by default the validate command will only run against the first document in the yaml file. Note that when running against other specific document indexes, _all previous documents will also be validated._
+
+### Validating a single document
+
+```bash
+yq v -d1 multidoc.yml
+```
+
+This will validate both document 0 and document 1 \(but not document 2\)
+
+### Validating all documents
+
+```bash
+yq v -d'*' multidoc.yml
+```
+
+This will validate all documents in the yaml file. Note that \* is quoted to avoid the CLI from processing the wildcard.
+

commands/write-update.md

@@ -0,0 +1,285 @@
+---
+description: >-
+  Updates all the matching nodes of path expression in a yaml file to the
+  supplied value.
+---
+
+# Write
+
+```bash
+yq w <yaml_file> <path_expression> <new value>
+```
+
+See docs for [path expression](../usage/path-expressions.md) and [value parsing](../usage/value-parsing.md) for more details, including controlling quotes and tags.
+
+## Basic
+
+Given a sample.yaml file of:
+
+```yaml
+b:
+  c: 2
+```
+
+then
+
+```bash
+yq w sample.yaml b.c cat
+```
+
+will output:
+
+```yaml
+b:
+  c: cat
+```
+
+### Updating files in-place
+
+```bash
+yq w -i sample.yaml b.c cat
+```
+
+will update the sample.yaml file so that the value of 'c' is cat.
+
+## From STDIN
+
+```bash
+cat sample.yaml | yq w - b.c blah
+```
+
+## Adding new fields
+
+Any missing fields in the path will be created on the fly.
+
+Given a sample.yaml file of:
+
+```yaml
+b:
+  c: 2
+```
+
+then
+
+```bash
+yq w sample.yaml b.d[+] "new thing"
+```
+
+will output:
+
+```yaml
+b:
+  c: cat
+  d:
+    - new thing
+```
+
+## Appending value to an array field
+
+Given a sample.yaml file of:
+
+```yaml
+b:
+  c: 2
+  d:
+    - new thing
+    - foo thing
+```
+
+then
+
+```bash
+yq w sample.yaml "b.d[+]" "bar thing"
+```
+
+will output:
+
+```yaml
+b:
+  c: cat
+  d:
+    - new thing
+    - foo thing
+    - bar thing
+```
+
+Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
+
+## Multiple Documents
+
+### Update a single document
+
+Given a sample.yaml file of:
+
+```yaml
+something: else
+---
+b:
+  c: 2
+```
+
+then
+
+```bash
+yq w -d1 sample.yaml b.c 5
+```
+
+will output:
+
+```yaml
+something: else
+---
+b:
+  c: 5
+```
+
+### Update all documents
+
+Given a sample.yaml file of:
+
+```yaml
+something: else
+---
+b:
+  c: 2
+```
+
+then
+
+```bash
+yq w -d'*' sample.yaml b.c 5
+```
+
+will output:
+
+```yaml
+something: else
+b:
+  c: 5
+---
+b:
+  c: 5
+```
+
+## Writing Anchors
+
+The `---anchorName` flag can be used to set the anchor name of a node
+
+Given a sample document of:
+
+```yaml
+commonStuff:
+    flavour: vanilla
+```
+
+Then:
+
+```bash
+yq write sample.yaml commonStuff --anchorName=commonBits
+```
+
+Will yield
+
+```yaml
+commonStuff: &commonBits
+    flavour: vanilla
+```
+
+## Writing Aliases
+
+The `--makeAlias` flag can create \(or update\) a node to be an alias to an anchor.
+
+Given a sample file of:
+
+```yaml
+commonStuff: &commonBits
+    flavour: vanilla
+```
+
+Then
+
+```bash
+yq write sample.yaml --makeAnchor foo commonBits
+```
+
+Will yield:
+
+```yaml
+commonStuff: &commonBits
+    flavour: vanilla
+foo: *commonBits
+```
+
+## Updating only styles/tags without affecting values
+
+You can use the write command to update the quoting style of nodes, or their tags, without re-specifying the values. This is done by omitting the value argument:
+
+Given a sample document:
+
+```yaml
+a:
+  c: things
+  d: other things
+```
+
+Then
+
+```bash
+yq write sample.yaml --style=single a.*
+```
+
+Will yield:
+
+```yaml
+a:
+  c: 'things'
+  d: 'other things'
+```
+
+## Using a script file to update
+
+Given a sample.yaml file of:
+
+```yaml
+b:
+  d: be gone
+  c: 2
+  e:
+    - name: Billy Bob # comment over here
+```
+
+and a script update\_instructions.yaml of:
+
+```yaml
+- command: update 
+  path: b.c
+  value:
+    #great 
+    things: frog # wow!
+- command: delete
+  path: b.d
+```
+
+then
+
+```bash
+yq w -s update_instructions.yaml sample.yaml
+```
+
+will output:
+
+```yaml
+b:
+  c:
+    #great
+    things: frog # wow!
+  e:
+  - name: Billy Bob # comment over here
+```
+
+And, of course, you can pipe the instructions in using '-':
+
+```bash
+cat update_instructions.yaml | yq w -s - sample.yaml
+```
+

debian/changelog

@@ -1,3 +1,13 @@
+yq (3.3-0) focal; urgency=medium
+
+  * You can control string styles (quotes) using the new --style flag
+  * String values now always have quotes when outputting to json
+  * Negative array indices now traverse the array backwards
+  * Added a --stripComments flag to print yaml without any comments
+  * Bumped go to version 1.14
+
+ -- Roberto Mier Escandon <rmescandon@gmail.com>  Thu, 30 Apr 2020 20:45:44 +0200
+
 yq (3.1-2) eoan; urgency=medium
 
   * Bug fix: yq 3 was removing empty inline-style objects and arrays (#355)

debian/files

@@ -1 +0,0 @@
-yq_3.1-2_source.buildinfo devel optional

examples/data1-no-comments.yaml

@@ -0,0 +1,4 @@
+a: simple
+b: [1, 2]
+c:
+  test: 1

examples/sample2.yaml

@@ -1,2 +0,0 @@
-b:
-  c: things

examples/simple-anchor-exploded.yaml

@@ -0,0 +1,5 @@
+foo:
+  a: 1
+
+foobar: 
+  a: 1

go.mod

@@ -2,16 +2,16 @@ module github.com/mikefarah/yq/v3
 
 require (
 	github.com/fatih/color v1.9.0
-	github.com/goccy/go-yaml v1.4.3
+	github.com/goccy/go-yaml v1.7.5
 	github.com/kylelemons/godebug v1.1.0
 	github.com/mattn/go-colorable v0.1.6 // indirect
 	github.com/pkg/errors v0.9.1
 	github.com/spf13/cobra v1.0.0
 	github.com/spf13/pflag v1.0.5 // indirect
-	golang.org/x/sys v0.0.0-20200413165638-669c56c373c4 // indirect
+	golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980 // indirect
 	golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 // indirect
 	gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473
-	gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c
+	gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c
 )
 
 go 1.14

go.sum

@@ -29,8 +29,8 @@ github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V
 github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
 github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
 github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
-github.com/goccy/go-yaml v1.4.3 h1:+1jK1ost1TBEfWjciIMU8rciBq0poxurgS7XvLgQInM=
-github.com/goccy/go-yaml v1.4.3/go.mod h1:PsEEJ29nIFZL07P/c8dv4P6rQkVFFXafQee85U+ERHA=
+github.com/goccy/go-yaml v1.7.5 h1:dWvj+p3BG11S/GlUzwzt1WZz0lhBTzTIDtmXT/ZOaPY=
+github.com/goccy/go-yaml v1.7.5/go.mod h1:wS4gNoLalDSJxo/SpngzPQ2BN4uuZVLCmbM4S3vd4+Y=
 github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
 github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
 github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
@@ -144,8 +144,8 @@ golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193
 golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200413165638-669c56c373c4 h1:opSr2sbRXk5X5/givKrrKj9HXxFpW2sdCiP8MJSKLQY=
-golang.org/x/sys v0.0.0-20200413165638-669c56c373c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980 h1:OjiUf46hAmXblsZdnoSXsEUSKU8r1UEzcL5RVZ4gO9Y=
+golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
 golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -175,7 +175,6 @@ gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bl
 gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
 gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
 gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
-gopkg.in/yaml.v3 v3.0.0-20191120175047-4206685974f2/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
-gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
-gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c h1:grhR+C34yXImVGp7EzNk+DTIk+323eIUWOmEevy6bDo=
+gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
 honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=

pkg/yqlib/data_navigator.go

@@ -24,6 +24,7 @@ func NewDataNavigator(NavigationStrategy NavigationStrategy) DataNavigator {
 func (n *navigator) Traverse(value *yaml.Node, path []interface{}) error {
 	realValue := value
 	emptyArray := make([]interface{}, 0)
+	log.Debugf("Traversing path %v", pathStackToString(path))
 	if realValue.Kind == yaml.DocumentNode {
 		log.Debugf("its a document! returning the first child")
 		return n.doTraverse(value.Content[0], "", path, emptyArray)
@@ -68,6 +69,16 @@ func (n *navigator) getOrReplace(original *yaml.Node, expectedKind yaml.Kind) *y
 
 func (n *navigator) recurse(value *yaml.Node, head interface{}, tail []interface{}, pathStack []interface{}) error {
 	log.Debug("recursing, processing %v, pathStack %v", head, pathStackToString(pathStack))
+
+	nodeContext := NewNodeContext(value, head, tail, pathStack)
+
+	if head == "**" && !n.navigationStrategy.ShouldOnlyDeeplyVisitLeaves(nodeContext) {
+		errorVisitingDeeply := n.navigationStrategy.Visit(nodeContext)
+		if errorVisitingDeeply != nil {
+			return errorVisitingDeeply
+		}
+	}
+
 	switch value.Kind {
 	case yaml.MappingNode:
 		log.Debug("its a map with %v entries", len(value.Content)/2)
@@ -85,20 +96,20 @@ func (n *navigator) recurse(value *yaml.Node, head interface{}, tail []interface
 			if head == "+" {
 				return n.appendArray(value, head, tail, pathStack)
 			} else if len(value.Content) == 0 && head == "**" {
-				return n.navigationStrategy.Visit(NewNodeContext(value, head, tail, pathStack))
+				return n.navigationStrategy.Visit(nodeContext)
 			}
 			return n.splatArray(value, head, tail, pathStack)
 		}
 	case yaml.AliasNode:
 		log.Debug("its an alias!")
 		DebugNode(value.Alias)
-		if n.navigationStrategy.FollowAlias(NewNodeContext(value, head, tail, pathStack)) {
+		if n.navigationStrategy.FollowAlias(nodeContext) {
 			log.Debug("following the alias")
 			return n.recurse(value.Alias, head, tail, pathStack)
 		}
 		return nil
 	default:
-		return n.navigationStrategy.Visit(NewNodeContext(value, head, tail, pathStack))
+		return n.navigationStrategy.Visit(nodeContext)
 	}
 }
 

pkg/yqlib/delete_navigation_strategy.go

@@ -12,15 +12,12 @@ func DeleteNavigationStrategy(pathElementToDelete interface{}) NavigationStrateg
 		followAlias: func(nodeContext NodeContext) bool {
 			return false
 		},
-		autoCreateMap: func(nodeContext NodeContext) bool {
+		shouldOnlyDeeplyVisitLeaves: func(nodeContext NodeContext) bool {
 			return false
 		},
-		shouldDeeplyTraverse: func(nodeContext NodeContext) bool {
-			return true
-		},
 		visit: func(nodeContext NodeContext) error {
 			node := nodeContext.Node
-			log.Debug("need to find and delete %v in here", pathElementToDelete)
+			log.Debug("need to find and delete %v in here (%v)", pathElementToDelete, pathStackToString(nodeContext.PathStack))
 			DebugNode(node)
 			if node.Kind == yaml.SequenceNode {
 				newContent := deleteFromArray(parser, node.Content, nodeContext.PathStack, pathElementToDelete)

pkg/yqlib/filter_matching_node_navigation_strategy.go

@@ -4,12 +4,6 @@ func FilterMatchingNodesNavigationStrategy(value string) NavigationStrategy {
 	return &NavigationStrategyImpl{
 		visitedNodes: []*NodeContext{},
 		pathParser:   NewPathParser(),
-		followAlias: func(nodeContext NodeContext) bool {
-			return true
-		},
-		autoCreateMap: func(nodeContext NodeContext) bool {
-			return false
-		},
 		visit: func(nodeContext NodeContext) error {
 			return nil
 		},

pkg/yqlib/lib.go

@@ -48,7 +48,7 @@ func DebugNode(value *yaml.Node) {
 			log.Error("Error debugging node, %v", errorEncoding.Error())
 		}
 		encoder.Close()
-		log.Debug("Tag: %v, Kind: %v", value.Tag, KindString(value.Kind))
+		log.Debug("Tag: %v, Kind: %v, Anchor: %v", value.Tag, KindString(value.Kind), value.Anchor)
 		log.Debug("%v", buf.String())
 	}
 }

pkg/yqlib/navigation_strategy.go

@@ -35,19 +35,22 @@ type NavigationStrategy interface {
 	// we use it to match against the pathExpression in head.
 	ShouldTraverse(nodeContext NodeContext, nodeKey string) bool
 	ShouldDeeplyTraverse(nodeContext NodeContext) bool
+	// when deeply traversing, should we visit all matching nodes, or just leaves?
+	ShouldOnlyDeeplyVisitLeaves(NodeContext) bool
 	GetVisitedNodes() []*NodeContext
 	DebugVisitedNodes()
 	GetPathParser() PathParser
 }
 
 type NavigationStrategyImpl struct {
-	followAlias          func(nodeContext NodeContext) bool
-	autoCreateMap        func(nodeContext NodeContext) bool
-	visit                func(nodeContext NodeContext) error
-	shouldVisitExtraFn   func(nodeContext NodeContext) bool
-	shouldDeeplyTraverse func(nodeContext NodeContext) bool
-	visitedNodes         []*NodeContext
-	pathParser           PathParser
+	followAlias                 func(nodeContext NodeContext) bool
+	autoCreateMap               func(nodeContext NodeContext) bool
+	visit                       func(nodeContext NodeContext) error
+	shouldVisitExtraFn          func(nodeContext NodeContext) bool
+	shouldDeeplyTraverse        func(nodeContext NodeContext) bool
+	shouldOnlyDeeplyVisitLeaves func(nodeContext NodeContext) bool
+	visitedNodes                []*NodeContext
+	pathParser                  PathParser
 }
 
 func (ns *NavigationStrategyImpl) GetPathParser() PathParser {
@@ -59,15 +62,32 @@ func (ns *NavigationStrategyImpl) GetVisitedNodes() []*NodeContext {
 }
 
 func (ns *NavigationStrategyImpl) FollowAlias(nodeContext NodeContext) bool {
-	return ns.followAlias(nodeContext)
+	if ns.followAlias != nil {
+		return ns.followAlias(nodeContext)
+	}
+	return true
 }
 
 func (ns *NavigationStrategyImpl) AutoCreateMap(nodeContext NodeContext) bool {
-	return ns.autoCreateMap(nodeContext)
+	if ns.autoCreateMap != nil {
+		return ns.autoCreateMap(nodeContext)
+	}
+	return false
 }
 
 func (ns *NavigationStrategyImpl) ShouldDeeplyTraverse(nodeContext NodeContext) bool {
-	return ns.shouldDeeplyTraverse(nodeContext)
+	if ns.shouldDeeplyTraverse != nil {
+		return ns.shouldDeeplyTraverse(nodeContext)
+	}
+	return true
+}
+
+func (ns *NavigationStrategyImpl) ShouldOnlyDeeplyVisitLeaves(nodeContext NodeContext) bool {
+	if ns.shouldOnlyDeeplyVisitLeaves != nil {
+		return ns.shouldOnlyDeeplyVisitLeaves(nodeContext)
+	}
+	return true
+
 }
 
 func (ns *NavigationStrategyImpl) ShouldTraverse(nodeContext NodeContext, nodeKey string) bool {

pkg/yqlib/read_navigation_strategy.go

@@ -4,12 +4,6 @@ func ReadNavigationStrategy(deeplyTraverseArrays bool) NavigationStrategy {
 	return &NavigationStrategyImpl{
 		visitedNodes: []*NodeContext{},
 		pathParser:   NewPathParser(),
-		followAlias: func(nodeContext NodeContext) bool {
-			return true
-		},
-		autoCreateMap: func(nodeContext NodeContext) bool {
-			return false
-		},
 		visit: func(nodeContext NodeContext) error {
 			return nil
 		},

pkg/yqlib/update_navigation_strategy.go

@@ -10,9 +10,6 @@ func UpdateNavigationStrategy(updateCommand UpdateCommand, autoCreate bool) Navi
 		autoCreateMap: func(nodeContext NodeContext) bool {
 			return autoCreate
 		},
-		shouldDeeplyTraverse: func(nodeContext NodeContext) bool {
-			return true
-		},
 		visit: func(nodeContext NodeContext) error {
 			node := nodeContext.Node
 			changesToApply := updateCommand.Value
@@ -28,6 +25,8 @@ func UpdateNavigationStrategy(updateCommand UpdateCommand, autoCreate bool) Navi
 				node.Kind = changesToApply.Kind
 				node.Style = changesToApply.Style
 				node.Content = changesToApply.Content
+				node.Anchor = changesToApply.Anchor
+				node.Alias = changesToApply.Alias
 				node.HeadComment = changesToApply.HeadComment
 				node.LineComment = changesToApply.LineComment
 				node.FootComment = changesToApply.FootComment

pkg/yqlib/value_parser.go

@@ -5,7 +5,7 @@ import (
 )
 
 type ValueParser interface {
-	Parse(argument string, customTag string, customStyle string) *yaml.Node
+	Parse(argument string, customTag string, customStyle string, anchorName string, createAlias bool) *yaml.Node
 }
 
 type valueParser struct {
@@ -15,7 +15,7 @@ func NewValueParser() ValueParser {
 	return &valueParser{}
 }
 
-func (v *valueParser) Parse(argument string, customTag string, customStyle string) *yaml.Node {
+func (v *valueParser) Parse(argument string, customTag string, customStyle string, anchorName string, createAlias bool) *yaml.Node {
 	var style yaml.Style
 	if customStyle == "tagged" {
 		style = yaml.TaggedStyle
@@ -27,12 +27,20 @@ func (v *valueParser) Parse(argument string, customTag string, customStyle strin
 		style = yaml.LiteralStyle
 	} else if customStyle == "folded" {
 		style = yaml.FoldedStyle
+	} else if customStyle == "flow" {
+		style = yaml.FlowStyle
 	} else if customStyle != "" {
 		log.Error("Unknown style %v, ignoring", customStyle)
 	}
-
 	if argument == "[]" {
 		return &yaml.Node{Tag: "!!seq", Kind: yaml.SequenceNode, Style: style}
 	}
-	return &yaml.Node{Value: argument, Tag: customTag, Kind: yaml.ScalarNode, Style: style}
+
+	kind := yaml.ScalarNode
+
+	if createAlias {
+		kind = yaml.AliasNode
+	}
+
+	return &yaml.Node{Value: argument, Tag: customTag, Kind: kind, Style: style, Anchor: anchorName}
 }

pkg/yqlib/value_parser_test.go

@@ -16,12 +16,13 @@ var parseStyleTests = []struct {
 	{"double", yaml.DoubleQuotedStyle},
 	{"single", yaml.SingleQuotedStyle},
 	{"folded", yaml.FoldedStyle},
+	{"flow", yaml.FlowStyle},
 	{"literal", yaml.LiteralStyle},
 }
 
 func TestValueParserStyleTag(t *testing.T) {
 	for _, tt := range parseStyleTests {
-		actual := NewValueParser().Parse("cat", "", tt.customStyle)
+		actual := NewValueParser().Parse("cat", "", tt.customStyle, "", false)
 		test.AssertResultWithContext(t, tt.expectedStyle, actual.Style, tt.customStyle)
 	}
 }
@@ -39,7 +40,7 @@ var parseValueTests = []struct {
 
 func TestValueParserParse(t *testing.T) {
 	for _, tt := range parseValueTests {
-		actual := NewValueParser().Parse(tt.argument, tt.customTag, "")
+		actual := NewValueParser().Parse(tt.argument, tt.customTag, "", "", false)
 		test.AssertResultWithContext(t, tt.argument, actual.Value, tt.testDescription)
 		test.AssertResultWithContext(t, tt.expectedTag, actual.Tag, tt.testDescription)
 		test.AssertResult(t, yaml.ScalarNode, actual.Kind)
@@ -47,7 +48,18 @@ func TestValueParserParse(t *testing.T) {
 }
 
 func TestValueParserParseEmptyArray(t *testing.T) {
-	actual := NewValueParser().Parse("[]", "", "")
+	actual := NewValueParser().Parse("[]", "", "", "", false)
 	test.AssertResult(t, "!!seq", actual.Tag)
 	test.AssertResult(t, yaml.SequenceNode, actual.Kind)
 }
+
+func TestValueParserParseAlias(t *testing.T) {
+	actual := NewValueParser().Parse("bob", "", "", "", true)
+	test.AssertResult(t, "bob", actual.Value)
+	test.AssertResult(t, yaml.AliasNode, actual.Kind)
+}
+
+func TestValueParserAnchorname(t *testing.T) {
+	actual := NewValueParser().Parse("caterpillar", "", "", "foo", false)
+	test.AssertResult(t, "foo", actual.Anchor)
+}

snap/snapcraft.yaml

@@ -1,5 +1,5 @@
 name: yq
-version: '3.3.0'
+version: '3.3.2'
 summary: A lightweight and portable command-line YAML processor
 description: |
   The aim of the project is to be the jq or sed of yaml files.

upgrading-from-v2.md

@@ -0,0 +1,107 @@
+---
+description: New features and breaking changes
+---
+
+# Upgrading from V2
+
+## New Features
+
+* Keeps yaml comments and formatting, can specify yaml [tags](usage/value-parsing.md#using-the-tag-field-to-override) when updating.
+* Handles anchors!
+* Can print out matching paths and values when splatting, more info [here](commands/read.md#printing-matching-paths).
+* JSON output works for all commands! Yaml files with multiple documents are printed out as one JSON document per line, more info [here](usage/convert.md)
+* Deep splat \(`**`\) to match arbitrary paths and match nodes by their children, more info [here](usage/path-expressions.md)
+
+## Breaking Changes
+
+### Parsing values from the CLI
+
+In V3 users are able to better control how values are treated when updating YAML by using a new `--tag` argument \(see more info [here](usage/value-parsing.md)\). A result of this however, is that quoting values, e.g. "true" will no longer have an effect on how the value is interpreted like it did in V2.
+
+For instance, to get the _string_ "true" into a yaml file:
+
+V2:
+
+```text
+yq n a.path '"true"'
+```
+
+V3
+
+```text
+yq n a.path --tag '!!str' true
+```
+
+### Reading paths that don't exist
+
+In V2 this would return null, V3 does not return anything.
+
+Similarly, reading null yaml values `null`, `~` and , V2 returns null whereas V3 returns the values as is.
+
+This is a result of taking effort not to format values coming in and out of the original YAML.
+
+
+
+### Update scripts file format has changed to be more powerful.
+
+Comments can be added, and delete commands have been introduced.
+
+V2
+
+```text
+b.e[+].name: Mike Farah
+```
+
+V3
+
+```yaml
+- command: update 
+  path: b.e[+].thing
+  value:
+    #great 
+    things: frog # wow!
+- command: delete
+  path: b.d
+```
+
+### Reading and splatting, matching results are printed once per line.
+
+e.g:
+
+```yaml
+parent:
+  childA: 
+    no: matches here
+  childB:
+    there: matches
+    hi: no match
+    there2: also matches
+```
+
+```text
+yq r sample.yaml 'parent.*.there*'
+```
+
+V2
+
+```text
+- null
+- - matches
+  - also matches
+```
+
+V3
+
+```text
+matches
+also matches
+```
+
+### Converting JSON to YAML
+
+As JSON is a subset of YAML, and `yq` now preserves the formatting of the passed in document, you will most likely need to use the `--prettyPrint` flag to format the JSON document as idiomatic YAML. See [Working with JSON](usage/convert.md#json-to-yaml) for more info.
+
+
+
+
+

usage/convert.md

@@ -0,0 +1,94 @@
+# Working with JSON
+
+## Yaml to Json
+
+To convert output to json, use the `--tojson` \(or `-j`\) flag. This is supported by all commands. You can change the json output format by using the [pretty print](output-format.md#pretty-print) or [indent](output-format.md#indent) flags. _Note that due to the implementation of the JSON marshaller in GO, object keys will be sorted on output \(_[_https://golang.org/pkg/encoding/json/\#Marshal_](https://golang.org/pkg/encoding/json/#Marshal)_\)._
+
+Given a sample.yaml file of:
+
+```yaml
+b:
+  c: 2
+```
+
+then
+
+```bash
+yq r -j sample.yaml
+```
+
+will output
+
+```javascript
+{"b":{"c":2}}
+```
+
+To format the json:
+
+```yaml
+yq r --prettyPrint -j sample.yaml
+```
+
+will yield
+
+```yaml
+{
+  "b": {
+    "c": 2
+  }
+}
+```
+
+### Multiple matches
+
+Each matching yaml node will be converted to json and printed out on a separate line. The [prettyPrint](output-format.md#pretty-print) and [indent](output-format.md#indent) flags will still work too. ****
+
+Given a sample.yaml file of:
+
+```yaml
+bob:
+  c: 2
+bab:
+  c: 5
+```
+
+then
+
+```bash
+yq r -j sample.yaml b*
+```
+
+will output
+
+```javascript
+{"c":2}
+{"c":5}
+```
+
+## Json to Yaml
+
+To read in json, just pass in a json file instead of yaml, it will just work - as json is a subset of yaml. However, you will probably want to [pretty print the output](output-format.md#pretty-print) to look more like an idiomatic yaml document.
+
+e.g given a json file
+
+```javascript
+{"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}
+```
+
+then
+
+```bash
+yq r --prettyPrint sample.json
+```
+
+will output
+
+```yaml
+a: Easy! as one two three
+b:
+  c: 2
+  d:
+  - 3
+  - 4
+```
+

usage/output-format.md

@@ -0,0 +1,161 @@
+---
+description: Flags to control yaml and json output format
+---
+
+# Output format
+
+These flags are available for all `yq` commands. 
+
+## Colorize Output
+
+Use the `--colors/-C`flag to print out yaml with colors. This does not work when outputing in JSON format.
+
+## Pretty Print
+
+Use the `--prettyPrint/-P` flag to enforce a formatting style for yaml documents. This is particularly useful when reading a json file \(which is a subset of yaml\) and wanting to format it in a more conventional yaml format.
+
+Given:
+
+```text
+{
+  "apples": [
+    {
+      "are": "great"
+    }
+  ]
+}
+```
+
+Then:
+
+```text
+yq r --prettyPrint sample.json
+```
+
+Will print out:
+
+```text
+apples:
+- are: great
+```
+
+This works in the same manner for yaml files:
+
+```text
+"apples": [are: great]
+```
+
+will format to:
+
+```text
+apples:
+- are: great
+```
+
+## Indent
+
+Use the indent flag `--indent/-I` to control the number of spaces used for indentation. This also works for JSON output. The default value is 2. 
+
+Note that lists are indented at the same level as the map key at indent level 2, but are more deeply indented at indent level 4 and greater. This is \(currently\) a quirk of the underlying [yaml parser](https://github.com/go-yaml/yaml/tree/v3).
+
+Given:
+
+```text
+apples:
+  collection:
+  - name: Green
+  - name: Blue
+  favourite: Pink Lady
+```
+
+Then:
+
+```text
+yq r -I4 sample.yaml
+```
+
+Will print out:
+
+```text
+apples:
+    collection:
+      - name: Green
+      - name: Blue
+    favourite: Pink Lady
+```
+
+With json, you must also specify the `--prettyPrint/-P` flag
+
+```text
+yq r -j -P -I4 sample.yaml
+```
+
+yields
+
+```text
+{
+    "apples": {
+        "collection": [
+            {
+                "name": "Green"
+            },
+            {
+                "name": "Blue"
+            }
+        ],
+        "favourite": "Pink Lady"
+    }
+}
+```
+
+## Unwrap scalars
+
+By default scalar values are 'unwrapped', that is only their value is printed \(except when outputting as JSON\). To print out the node as-is, with the original formatting an any comments pass in `--unwrapScalar=false`
+
+Given data.yml:
+
+```yaml
+a: "Things" # cool stuff
+```
+
+Then:
+
+`yq r --unwrapScalar=false data.yml a`
+
+Will yield:
+
+```yaml
+"Things" # cool stuff
+```
+
+where as without setting the flag to false you would get:
+
+```yaml
+Things
+```
+
+## Strip comments
+
+Use the `--stripComments` flag to print out the yaml file without any of the original comments.
+
+Given data.yml of:
+
+```yaml
+a:
+  b: # there is where the good stuff is
+    c: hi
+```
+
+Then
+
+```yaml
+yq r data.yml a --stripComments
+```
+
+Will yield:
+
+```yaml
+b:
+  c: hi
+```
+

usage/path-expressions.md

@@ -0,0 +1,251 @@
+---
+description: Path expressions are used to deeply navigate and match particular yaml nodes.
+---
+
+# Path Expressions
+
+_As a general rule, you should wrap paths in quotes to prevent your CLI from processing `*`, `[]` and other special characters._
+
+## Simple expressions
+
+### Maps
+
+`'a.b.c'`
+
+```yaml
+a:
+  b:
+    c: thing # MATCHES
+```
+
+### Arrays
+
+`'a.b[1].c'`
+
+```yaml
+a:
+  b:
+  - c: thing0 
+  - c: thing1 # MATCHES
+  - c: thing2
+```
+
+#### Appending to arrays
+
+\(e.g. when using the write command\)
+
+`'a.b[+].c'`
+
+```yaml
+a:
+  b:
+  - c: thing0
+```
+
+Will add a new entry:
+
+```yaml
+a:
+  b:
+  - c: thing0 
+  - c: thing1 # NEW entry from [+] on B array.
+```
+
+#### Negative Array indexes
+
+Negative array indexes can be used to traverse the array in reverse
+
+`'a.b[-1].c'`
+
+Will access the last element in the `b` array and yield:
+
+```yaml
+thing2
+```
+
+## Splat
+
+### Maps
+
+`'a.*.c'`
+
+```yaml
+a:
+  b1:
+    c: thing # MATCHES
+    d: whatever
+  b2:
+    c: thing # MATCHES
+    f: something irrelevant
+```
+
+#### Prefix splat
+
+`'bob.item*.cats'`
+
+```yaml
+bob:
+  item:
+    cats: bananas # MATCHES
+  something:
+    cats: lemons
+  itemThing:
+    cats: more bananas # MATCHES
+  item2:
+    cats: apples # MATCHES
+  thing:
+    cats: oranges
+```
+
+### Arrays
+
+`'a.b[*].c'`
+
+```yaml
+a:
+  b:
+  - c: thing0 # MATCHES
+    d: what..ever
+  - c: thing1 # MATCHES
+    d: blarh
+  - c: thing2 # MATCHES
+    f: thingamabob
+```
+
+## Deep Splat
+
+`**` will match arbitrary nodes for both maps and arrays:
+
+`'a.**.c'`
+
+```yaml
+a:
+  b1:
+    c: thing1 # MATCHES
+    d: cat cat
+  b2:
+    c: thing2 # MATCHES
+    d: dog dog
+  b3:
+    d:
+    - f:
+        c: thing3 # MATCHES
+        d: beep
+    - f:
+        g:
+          c: thing4 # MATCHES
+          d: boop
+    - d: mooo
+```
+
+## Search by children nodes
+
+You can search children by nodes - note that this will return the _parent_ of the matching expression, in the example below the parent\(s\) will be the matching indices of the 'a' array. We can then navigate down to get 'b.c' of each matching indice.
+
+`'a.(b.d==cat).b.c'`
+
+```yaml
+a:
+  - b:
+      c: thing0
+      d: leopard
+    ba: fast
+  - b:
+      c: thing1 # MATCHES
+      d: cat
+    ba: meowy
+  - b:
+      c: thing2
+      d: caterpillar
+    ba: icky
+  - b:
+      c: thing3 # MATCHES
+      d: cat
+    ba: also meowy
+```
+
+### With prefixes
+
+`'a.(b.d==cat*).c'`
+
+```yaml
+a:
+  - b:
+      c: thing0
+      d: leopard
+    ba: fast
+  - b:
+      c: thing1 # MATCHES
+      d: cat
+    ba: meowy
+  - b:
+      c: thing2 # MATCHES
+      d: caterpillar
+    ba: icky
+  - b:
+      c: thing3 # MATCHES
+      d: cat
+    ba: also meowy
+```
+
+### Matching children values
+
+`'animals(.==cat)'`
+
+```yaml
+animals:
+  - dog
+  - cat # MATCHES  
+  - rat
+```
+
+this also works in maps, and with prefixes
+
+`'animals(.==c*)'`
+
+```yaml
+animals:
+  friendliest: cow # MATCHES
+  favourite: cat # MATCHES
+  smallest: rat
+```
+
+## Special Characters
+
+When your yaml field has special characters that overlap with `yq` path expression characters, you will need to escape them in order for the command to work.
+
+### Keys with dots
+
+When specifying a key that has a dot use key lookup indicator.
+
+```yaml
+b:
+  foo.bar: 7
+```
+
+```bash
+yaml r sample.yaml 'b."foo.bar"'
+```
+
+```bash
+yaml w sample.yaml 'b."foo.bar"' 9
+```
+
+Any valid yaml key can be specified as part of a key lookup.
+
+Note that the path is in single quotes to avoid the double quotes being interpreted by your shell.
+
+### Keys \(and values\) with leading dashes
+
+The flag terminator needs to be used to stop the app from attempting to parse the subsequent arguments as flags, if they start if a dash.
+
+```bash
+yq n -j -- --key --value
+```
+
+Will result in
+
+```text
+--key: --value
+```
+

usage/value-parsing.md

@@ -0,0 +1,288 @@
+---
+description: >-
+  How values are parsed from the CLI to commands that create/update yaml (e.g.
+  new/write).
+---
+
+# Value Parsing
+
+`yq` attempts to parse values intelligently, e.g. when a number is passed it - it will assume it's a number as opposed to a string. `yq` will not alter the representation of what you give. So if you pass '03.0' in, it will assume it's a number and keep the value formatted as it was passed in, that is '03.0'.
+
+The `--tag` flag can be used to override the tag type to force particular tags.
+
+## Default behavior
+
+### Integers
+
+_Given_
+
+```bash
+yq new key 3
+```
+
+results in
+
+```yaml
+key: 3
+```
+
+_Given a formatted number_
+
+```bash
+yq new key 03
+```
+
+results in
+
+```yaml
+key: 03
+```
+
+`yq` keeps the number formatted as it was passed in.
+
+### Float
+
+_Given_
+
+```bash
+yq new key "3.1"
+```
+
+results in
+
+```yaml
+key: 3.1
+```
+
+Note that quoting the number does not make a difference.
+
+_Given a formatted decimal number_
+
+```bash
+yq new key 03.0
+```
+
+results in
+
+```yaml
+key: 03.0
+```
+
+`yq` keeps the number formatted as it was passed in
+
+### Booleans
+
+```bash
+yq new key true
+```
+
+results in
+
+```yaml
+key: true
+```
+
+### Nulls
+
+```bash
+yq new key null
+```
+
+results in
+
+```yaml
+key: null
+```
+
+```bash
+yq new key '~'
+```
+
+results in
+
+```yaml
+key: ~
+```
+
+```bash
+yq new key ''
+```
+
+results in
+
+```yaml
+key:
+```
+
+### Strings
+
+```bash
+yq new key whatever
+```
+
+results in
+
+```yaml
+key: whatever
+```
+
+```bash
+yq new key ' whatever '
+```
+
+results in
+
+```yaml
+key: ' whatever '
+```
+
+## Using the tag flag to cast
+
+Previous versions of yq required double quoting to force values to be strings, this no longer works - instead use the --tag flag.
+
+### Casting booleans
+
+```bash
+yq new --tag '!!str' key true
+```
+
+results in
+
+```yaml
+key: "true"
+```
+
+### Casting nulls
+
+```bash
+yq new --tag '!!str' key null
+```
+
+results in
+
+```yaml
+key: "null"
+```
+
+### Custom types
+
+```bash
+yq new --tag '!!farah' key gold
+```
+
+results in
+
+```yaml
+key: !!farah gold
+```
+
+## The style flag
+
+The `--style` flag can be used to specify the quote or block style of the node value. Valid values are
+
+* single
+* double
+* folded
+* flow
+* literal
+* tagged
+
+For example, given:
+
+```bash
+MULTILINE=$(cat <<END
+    This is line one.
+    This is line two.
+END
+)
+
+SINGLE="only one line"
+```
+
+### Single
+
+```yaml
+yq n --style single things "$MULTILINE"
+```
+
+```yaml
+things: 'This is line one.
+
+  This is line two.'
+```
+
+### Double
+
+```yaml
+things: "This is line one.\nThis is line two."
+```
+
+### Folded:
+
+```yaml
+things: >-
+  This is line one.
+
+  This is line two.
+
+```
+
+#### Folded single line:
+
+```yaml
+things: >-
+  only one line
+```
+
+### Flow:
+
+```yaml
+things: |-
+  This is line one.
+  This is line two.
+
+```
+
+#### Flow single line:
+
+```yaml
+things: only one line
+```
+
+### Literal
+
+```yaml
+things: |-
+  This is line one.
+  This is line two.
+
+```
+
+#### Literal single line
+
+```yaml
+things: |-
+  only one line
+```
+
+### Tagged
+
+Always show the tag, note - you must also pass in `--tag='!!str'`
+
+```yaml
+things: !!str |-
+  This is line one.
+  This is line two.
+
+```
+
+#### Tagged single line
+
+```yaml
+things: !!str only one line
+```
+
+
+
+
+

yq.go

@@ -4,14 +4,11 @@ import (
 	"os"
 
 	command "github.com/mikefarah/yq/v3/cmd"
-	logging "gopkg.in/op/go-logging.v1"
 )
 
 func main() {
 	cmd := command.New()
-	log := logging.MustGetLogger("yq")
 	if err := cmd.Execute(); err != nil {
-		log.Error(err.Error())
 		os.Exit(1)
 	}
 }