fixing xcompile for git action

alias, anchor and explode ops are now all documented together

.github/ISSUE_TEMPLATE/bug_report_v3.md

@@ -1,8 +1,8 @@
 ---
-name: Bug report
+name: Bug report - V3
 about: Create a report to help us improve
 title: ''
-labels: bug
+labels: bug, v3
 assignees: ''
 
 ---
@@ -10,11 +10,12 @@ assignees: ''
 **Describe the bug**
 A clear and concise description of what the bug is.
 
-version of yq: 
+Version of yq: 3.X.X
-operating system:
+Operating system: mac/linux/windows/....
+Installed via: docker/binary release/homebrew/snap/...
 
 **Input Yaml**
-Concise yaml document(s) (as simple as possible to show the bug)
+Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)
 data1.yml:
 ```yaml
 this: should really work

.github/ISSUE_TEMPLATE/bug_report_v4.md

@@ -0,0 +1,49 @@
+---
+name: Bug report - V4
+about: Create a report to help us improve
+title: ''
+labels: bug, v4
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+Version of yq: 4.X.X
+Operating system: mac/linux/windows/....
+Installed via: docker/binary release/homebrew/snap/...
+
+**Input Yaml**
+Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)
+data1.yml:
+```yaml
+this: should really work
+```
+
+data2.yml:
+```yaml
+but: it strangely didn't
+```
+
+**Command**
+The command you ran:
+```
+yq eval-all 'select(fileIndex==0) | .a.b.c' data1.yml data2.yml
+```
+
+**Actual behavior**
+
+```yaml
+cat: meow
+```
+
+**Expected behavior**
+
+```yaml
+this: should really work
+but: it strangely didn't
+```
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,17 +1,21 @@
 ---
-name: Feature request
+name: Feature request - V4
 about: Suggest an idea for this project
 title: ''
-labels: enhancement
+labels: enhancement, v4
 assignees: ''
 
 ---
 
-**Is your feature request related to a problem? Please describe.**
+**Please describe your feature request.**
-A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+A clear and concise description of what the request is and what it would solve. 
+Ex. I wish I could use yq to [...]
+
+Please note that V3 will no longer have any enhancements.
 
 **Describe the solution you'd like**
 If we have data1.yml like:
+(please keep to around 10 lines )
 
 ```yaml
 country: Australia
@@ -20,7 +24,7 @@ country: Australia
 And we run a command:
 
 ```bash
-yq predictWeather data1.yml
+yq eval 'predictWeatherOf(.country)'
 ```
 
 it could output

.github/workflows/go.yml

@@ -28,7 +28,4 @@ jobs:
       run: |
         export PATH=${PATH}:`go env GOPATH`/bin
         scripts/devtools.sh
-    - name: Build
-      run: |
-        export PATH=${PATH}:`go env GOPATH`/bin
         make local build

.github/workflows/release.yml

@@ -0,0 +1,79 @@
+name: Release YQ
+on:
+  push:
+    tags:
+      - 'v*'
+
+jobs:
+  publishGitRelease:
+    environment: gitrelease
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-go@v2
+        with:
+          go-version: '^1.15'
+      - name: Cross compile
+        run: |
+          sudo apt-get install rhash -y
+          go get github.com/mitchellh/gox
+          ./scripts/xcompile.sh
+
+      - name: Create Release
+        id: create_release
+        uses: actions/create-release@v1.0.0
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        with:
+          tag_name: ${{ github.ref }}
+          release_name: ${{ github.ref }}
+          draft: true
+          prerelease: false
+
+      - uses: shogo82148/actions-upload-release-asset@v1
+        with:
+          upload_url: ${{ steps.create_release.outputs.upload_url }}
+          asset_path: build/*
+
+  publishDocker:
+    environment: dockerhub
+    env:
+      IMAGE_NAME: mikefarah/yq
+    runs-on: ubuntu-latest
+    steps:
+      - name: Get latest release tag      
+        uses: oprypin/find-latest-tag@v1
+        with:
+          repository: mikefarah/yq  # The repository to scan.
+          releases-only: true  # We know that all relevant tags have a GitHub release for them.
+        id: yq  
+        
+      - name: Clone source code
+        uses: actions/checkout@v2
+        with:
+          ref: ${{ steps.yq.outputs.tag }}
+      
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v1
+        with:
+          platforms: all
+  
+      - name: Set up Docker Buildx
+        id: buildx
+        uses: docker/setup-buildx-action@v1
+        with:
+          version: latest
+
+      - name: Available platforms
+        run: echo ${{ steps.buildx.outputs.platforms }} && docker version
+ 
+      - name: Build and push image
+        run: |
+          IMAGE_VERSION="$(git describe --tags --abbrev=0)"
+          SHORT_SHA1=$(git rev-parse --short HEAD)
+          PLATFORMS="linux/amd64,linux/ppc64le,linux/arm64"
+          echo "Building and pushing version ${IMAGE_VERSION} of image ${IMAGE_NAME}"
+          echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
+          docker buildx build --platform "${PLATFORMS}" -t "${IMAGE_NAME}:${IMAGE_VERSION}"  -t "${IMAGE_NAME}:latest" -t "${IMAGE_NAME}:4" \
+            --push .
+    

Dockerfile

@@ -12,7 +12,7 @@ RUN CGO_ENABLED=0 make local build
 
 # Choose alpine as a base image to make this useful for CI, as many
 # CI tools expect an interactive shell inside the container
-FROM alpine:3.12 as production
+FROM alpine:3.12.3 as production
 
 COPY --from=builder /go/src/mikefarah/yq/yq /usr/bin/yq
 RUN chmod +x /usr/bin/yq

README.md

@@ -7,11 +7,17 @@ a lightweight and portable command-line YAML processor
 
 The aim of the project is to be the [jq](https://github.com/stedolan/jq) or sed of yaml files.
 
+## V4 released!
+V4 is now officially released, it's quite different from V3 (sorry for the migration), however it is much more similar to ```jq```, using a similar expression syntax and therefore support much more complex functionality! 
+
+If you've been using v3 and want/need to upgrade, checkout the [upgrade guide](https://mikefarah.gitbook.io/yq/v/v4.x/upgrading-from-v3).
+
 ## Install
 
 ### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest)
 
 ### MacOS:
+Using [Homebrew](https://brew.sh/)
 ```
 brew install yq
 ```
@@ -44,13 +50,12 @@ rm /etc/myfile.tmp
 Use wget to download the pre-compiled binaries:
 
 ```bash
-wget https://github.com/mikefarah/yq/releases/download/{VERSION}/{BINARY} -O /usr/bin/yq &&\
+wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq &&\
     chmod +x /usr/bin/yq
 ```
 
 For instance, VERSION=4.0.0 and BINARY=yq_linux_amd64
 
-
 ### Run with Docker
 
 #### Oneshot use:
@@ -88,6 +93,14 @@ choco install yq
 ```
 Supported by @chillum (https://chocolatey.org/packages/yq)
 
+### Mac:
+Using [MacPorts](https://www.macports.org/)
+```
+sudo port selfupdate
+sudo port install yq
+```
+Supported by @herbygillot (https://ports.macports.org/maintainer/github/herbygillot)
+
 ### Alpine Linux
 - Enable edge/community repo by adding ```$MIRROR/alpine/edge/community``` to ```/etc/apk/repositories```
 - Update database index with ```apk update```
@@ -108,21 +121,18 @@ Supported by @rmescandon (https://launchpad.net/~rmescandon/+archive/ubuntu/yq)
 
 ## Features
 - Written in portable go, so you can download a lovely dependency free binary
+- Uses similar syntax as `jq` but works with YAML and JSON files
+- Fully supports multi document yaml files
 - Colorized yaml output
-- [Deep read a yaml file with a given path expression](https://mikefarah.gitbook.io/yq/v/v4.x/traverse)
+- [Deeply traverse yaml](https://mikefarah.gitbook.io/yq/v/v4.x/traverse)
-- [Return the lengths of arrays/object/scalars](https://mikefarah.gitbook.io/yq/commands/read#printing-length-of-the-results)
+- [Sort yaml by keys](https://mikefarah.gitbook.io/yq/v/v4.x/sort-keys)
-- Update a yaml file given a [path expression](https://mikefarah.gitbook.io/yq/commands/write-update#basic) or [script file](https://mikefarah.gitbook.io/yq/commands/write-update#basic)
+- Manipulate yaml [comments](https://app.gitbook.com/@mikefarah/s/yq/v/v4.x/comment-operators), [styling](https://app.gitbook.com/@mikefarah/s/yq/v/v4.x/style), [tags](https://app.gitbook.com/@mikefarah/s/yq/v/v4.x/tag) and [anchors](https://app.gitbook.com/@mikefarah/s/yq/v/v4.x/explode).
-- Update creates any missing entries in the path on the fly
+- [Update yaml inplace](https://mikefarah.gitbook.io/yq/v/v4.x/commands/evaluate#flags)
-- Deeply [compare](https://mikefarah.gitbook.io/yq/commands/compare) yaml files
+- [Complex expressions to select and update](https://mikefarah.gitbook.io/yq/v/v4.x/select#select-and-update-matching-values-in-map)
-- Keeps yaml formatting and comments when updating
+- Keeps yaml formatting and comments when updating (though there are issues with whitespace)
-- [Validate a yaml file](https://mikefarah.gitbook.io/yq/commands/validate)
+- [Convert to/from json to yaml](https://mikefarah.gitbook.io/yq/v/v4.x/usage/convert)
-- Create a yaml file given a [deep path and value](https://mikefarah.gitbook.io/yq/commands/create#creating-a-simple-yaml-file) or a [script file](https://mikefarah.gitbook.io/yq/commands/create#creating-using-a-create-script)
+- [Pipe data in by using '-'](https://mikefarah.gitbook.io/yq/v/v4.x/commands/evaluate)
-- [Prefix a path to a yaml file](https://mikefarah.gitbook.io/yq/commands/prefix)
+- [General shell completion scripts (bash/zsh/fish/powershell)](https://mikefarah.gitbook.io/yq/v/v4.x/commands/shell-completion)
-- [Convert to/from json to yaml](https://mikefarah.gitbook.io/yq/usage/convert)
-- [Pipe data in by using '-'](https://mikefarah.gitbook.io/yq/commands/read#from-stdin)
-- [Merge](https://mikefarah.gitbook.io/yq/commands/merge) multiple yaml files with various options for [overriding](https://mikefarah.gitbook.io/yq/commands/merge#overwrite-values) and [appending](https://mikefarah.gitbook.io/yq/commands/merge#append-values-with-arrays)
-- Supports multiple documents in a single yaml file for [reading](https://mikefarah.gitbook.io/yq/commands/read#multiple-documents), [writing](https://mikefarah.gitbook.io/yq/commands/write-update#multiple-documents) and [merging](https://mikefarah.gitbook.io/yq/commands/merge#multiple-documents)
-- General shell completion scripts (bash/zsh/fish/powershell) (https://mikefarah.gitbook.io/yq/commands/shell-completion)
 
 ## [Usage](https://mikefarah.gitbook.io/yq/)
 
@@ -134,32 +144,32 @@ Usage:
   yq [command]
 
 Available Commands:
-  compare          yq x [--prettyPrint/-P] dataA.yaml dataB.yaml 'b.e(name==fr*).value'
+  eval             Apply expression to each document in each yaml file given in sequence
-  delete           yq d [--inplace/-i] [--doc/-d index] sample.yaml 'b.e(name==fred)'
+  eval-all         Loads _all_ yaml documents of _all_ yaml files and runs expression once
   help             Help about any command
-  merge            yq m [--inplace/-i] [--doc/-d index] [--overwrite/-x] [--append/-a] sample.yaml sample2.yaml
+  shell-completion Generate completion script
-  new              yq n [--script/-s script_file] a.b.c newValue
-  prefix           yq p [--inplace/-i] [--doc/-d index] sample.yaml a.b.c
-  read             yq r [--printMode/-p pv] sample.yaml 'b.e(name==fr*).value'
-  shell-completion Generates shell completion scripts
-  validate         yq v sample.yaml
-  write            yq w [--inplace/-i] [--script/-s script_file] [--doc/-d index] sample.yaml 'b.e(name==fr*).value' newValue
 
 Flags:
-  -C, --colors        print with colors
+  -C, --colors        force print with colors
+  -e, --exit-status   set exit status if there are no matches or null or false is returned
   -h, --help          help for yq
   -I, --indent int    sets indent level for output (default 2)
-  -P, --prettyPrint   pretty print
+  -i, --inplace       update the yaml file inplace of first yaml file given.
-  -j, --tojson        output as json. By default it prints a json document in one line, use the prettyPrint flag to print a formatted doc.
+  -M, --no-colors     force print with no colors
+  -N, --no-doc        Don't print document separators (---)
+  -n, --null-input    Don't read input, simply evaluate the expression given. Useful for creating yaml docs from scratch.
+  -j, --tojson        output as json. Set indent to 0 to print json in one line.
   -v, --verbose       verbose mode
   -V, --version       Print version information and quit
 
 Use "yq [command] --help" for more information about a command.
 ```
 
-## Upgrade from V2
+Simple Example:
-If you've been using v2 and want/need to upgrade, checkout the [upgrade guide](https://mikefarah.gitbook.io/yq/upgrading-from-v2).
+
+```bash
+yq e '.a.b | length' f1.yml f2.yml 
+```
 
 ## Known Issues / Missing Features
 - `yq` attempts to preserve comment positions and whitespace as much as possible, but it does not handle all scenarios (see https://github.com/go-yaml/yaml/tree/v3 for details)
-- You cannot (yet) select multiple paths/keys from the yaml to be printed out (https://github.com/mikefarah/yq/issues/287)

cmd/root.go

@@ -47,6 +47,7 @@ func New() *cobra.Command {
 	rootCmd.PersistentFlags().IntVarP(&indent, "indent", "I", 2, "sets indent level for output")
 	rootCmd.Flags().BoolVarP(&version, "version", "V", false, "Print version information and quit")
 	rootCmd.PersistentFlags().BoolVarP(&writeInplace, "inplace", "i", false, "update the yaml file inplace of first yaml file given.")
+	rootCmd.PersistentFlags().BoolVarP(&unwrapScalar, "unwrapScalar", "", true, "unwrap scalar, print the value with no quotes, colors or comments")
 	rootCmd.PersistentFlags().BoolVarP(&exitStatus, "exit-status", "e", false, "set exit status if there are no matches or null or false is returned")
 
 	rootCmd.PersistentFlags().BoolVarP(&forceColor, "colors", "C", false, "force print with colors")

cmd/version.go

@@ -11,7 +11,7 @@ var (
 	GitDescribe string
 
 	// Version is main version number that is being run at the moment.
-	Version = "4.0.0-beta1"
+	Version = "4.0.0"
 
 	// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
 	// then it means that it is a final release. Otherwise, this is a pre-release

go.sum

@@ -263,6 +263,7 @@ golang.org/x/sys v0.0.0-20191010194322-b09406accb47/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=
 golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae h1:/WDfKMnPU+m5M4xB+6x4kaepxRw6jWvR5iDRdvjHgy8=
 golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f h1:+Nyd8tzPX9R7BWHguqsrbFdRx3WQ/1ib8I44HXV5yTA=
 golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=

pkg/yqlib/doc/Alternative (Default value).md

@@ -0,0 +1,73 @@
+This operator is used to provide alternative (or default) values when a particular expression is either null or false.
+
+## LHS is defined
+Given a sample.yml file of:
+```yaml
+a: bridge
+```
+then
+```bash
+yq eval '.a // "hello"' sample.yml
+```
+will output
+```yaml
+bridge
+```
+
+## LHS is not defined
+Given a sample.yml file of:
+```yaml
+{}
+```
+then
+```bash
+yq eval '.a // "hello"' sample.yml
+```
+will output
+```yaml
+hello
+```
+
+## LHS is null
+Given a sample.yml file of:
+```yaml
+a: ~
+```
+then
+```bash
+yq eval '.a // "hello"' sample.yml
+```
+will output
+```yaml
+hello
+```
+
+## LHS is false
+Given a sample.yml file of:
+```yaml
+a: false
+```
+then
+```bash
+yq eval '.a // "hello"' sample.yml
+```
+will output
+```yaml
+hello
+```
+
+## RHS is an expression
+Given a sample.yml file of:
+```yaml
+a: false
+b: cat
+```
+then
+```bash
+yq eval '.a // .b' sample.yml
+```
+will output
+```yaml
+cat
+```
+

pkg/yqlib/doc/Anchor and Aliases Operators.md

@@ -1,4 +1,67 @@
-Explodes (or dereferences) aliases and anchors.
+Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference aliases and remove anchor names).
+
+`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag.
+
+
+## Get anchor
+Given a sample.yml file of:
+```yaml
+a: &billyBob cat
+```
+then
+```bash
+yq eval '.a | anchor' sample.yml
+```
+will output
+```yaml
+billyBob
+```
+
+## Set anchor
+Given a sample.yml file of:
+```yaml
+a: cat
+```
+then
+```bash
+yq eval '.a anchor = "foobar"' sample.yml
+```
+will output
+```yaml
+a: &foobar cat
+```
+
+## Get alias
+Given a sample.yml file of:
+```yaml
+b: &billyBob meow
+a: *billyBob
+```
+then
+```bash
+yq eval '.a | alias' sample.yml
+```
+will output
+```yaml
+billyBob
+```
+
+## Set alias
+Given a sample.yml file of:
+```yaml
+b: &meow purr
+a: cat
+```
+then
+```bash
+yq eval '.a alias = "meow"' sample.yml
+```
+will output
+```yaml
+b: &meow purr
+a: *meow
+```
+
 ## Explode alias and anchor
 Given a sample.yml file of:
 ```yaml

pkg/yqlib/doc/Delete.md

@@ -14,6 +14,23 @@ will output
 a: cat
 ```
 
+## Delete nested entry in map
+Given a sample.yml file of:
+```yaml
+a:
+  a1: fred
+  a2: frood
+```
+then
+```bash
+yq eval 'del(.a.a1)' sample.yml
+```
+will output
+```yaml
+a:
+  a2: frood
+```
+
 ## Delete entry in array
 Given a sample.yml file of:
 ```yaml
@@ -31,6 +48,21 @@ will output
 - 3
 ```
 
+## Delete nested entry in array
+Given a sample.yml file of:
+```yaml
+- a: cat
+  b: dog
+```
+then
+```bash
+yq eval 'del(.[0].a)' sample.yml
+```
+will output
+```yaml
+- b: dog
+```
+
 ## Delete no matches
 Given a sample.yml file of:
 ```yaml

pkg/yqlib/doc/headers/Alternative (Default value).md

@@ -0,0 +1 @@
+This operator is used to provide alternative (or default) values when a particular expression is either null or false.

pkg/yqlib/doc/headers/Anchor and Aliases Operators.md

@@ -0,0 +1,4 @@
+Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference aliases and remove anchor names).
+
+`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag.
+

pkg/yqlib/doc/headers/Explode.md

@@ -1 +0,0 @@
-Explodes (or dereferences) aliases and anchors.

pkg/yqlib/lib.go

@@ -36,9 +36,12 @@ var AssignAttributes = &OperationType{Type: "ASSIGN_ATTRIBUTES", NumArgs: 2, Pre
 var AssignStyle = &OperationType{Type: "ASSIGN_STYLE", NumArgs: 2, Precedence: 40, Handler: AssignStyleOperator}
 var AssignTag = &OperationType{Type: "ASSIGN_TAG", NumArgs: 2, Precedence: 40, Handler: AssignTagOperator}
 var AssignComment = &OperationType{Type: "ASSIGN_COMMENT", NumArgs: 2, Precedence: 40, Handler: AssignCommentsOperator}
+var AssignAnchor = &OperationType{Type: "ASSIGN_ANCHOR", NumArgs: 2, Precedence: 40, Handler: AssignAnchorOperator}
+var AssignAlias = &OperationType{Type: "ASSIGN_ALIAS", NumArgs: 2, Precedence: 40, Handler: AssignAliasOperator}
 
 var Multiply = &OperationType{Type: "MULTIPLY", NumArgs: 2, Precedence: 45, Handler: MultiplyOperator}
 var Add = &OperationType{Type: "ADD", NumArgs: 2, Precedence: 45, Handler: AddOperator}
+var Alternative = &OperationType{Type: "ALTERNATIVE", NumArgs: 2, Precedence: 45, Handler: AlternativeOperator}
 
 var Equals = &OperationType{Type: "EQUALS", NumArgs: 2, Precedence: 40, Handler: EqualsOperator}
 var CreateMap = &OperationType{Type: "CREATE_MAP", NumArgs: 2, Precedence: 40, Handler: CreateMapOperator}
@@ -50,6 +53,8 @@ var Collect = &OperationType{Type: "COLLECT", NumArgs: 0, Precedence: 50, Handle
 var GetStyle = &OperationType{Type: "GET_STYLE", NumArgs: 0, Precedence: 50, Handler: GetStyleOperator}
 var GetTag = &OperationType{Type: "GET_TAG", NumArgs: 0, Precedence: 50, Handler: GetTagOperator}
 var GetComment = &OperationType{Type: "GET_COMMENT", NumArgs: 0, Precedence: 50, Handler: GetCommentsOperator}
+var GetAnchor = &OperationType{Type: "GET_ANCHOR", NumArgs: 0, Precedence: 50, Handler: GetAnchorOperator}
+var GetAlias = &OperationType{Type: "GET_ALIAS", NumArgs: 0, Precedence: 50, Handler: GetAliasOperator}
 var GetDocumentIndex = &OperationType{Type: "GET_DOCUMENT_INDEX", NumArgs: 0, Precedence: 50, Handler: GetDocumentIndexOperator}
 var GetFilename = &OperationType{Type: "GET_FILENAME", NumArgs: 0, Precedence: 50, Handler: GetFilenameOperator}
 var GetFileIndex = &OperationType{Type: "GET_FILE_INDEX", NumArgs: 0, Precedence: 50, Handler: GetFileIndexOperator}
@@ -72,6 +77,7 @@ var RecursiveDescent = &OperationType{Type: "RECURSIVE_DESCENT", NumArgs: 0, Pre
 var Select = &OperationType{Type: "SELECT", NumArgs: 1, Precedence: 50, Handler: SelectOperator}
 var Has = &OperationType{Type: "HAS", NumArgs: 1, Precedence: 50, Handler: HasOperator}
 var DeleteChild = &OperationType{Type: "DELETE", NumArgs: 1, Precedence: 40, Handler: DeleteChildOperator}
+var DeleteImmediateChild = &OperationType{Type: "DELETE_IMMEDIATE_CHILD", NumArgs: 1, Precedence: 40, Handler: DeleteImmediateChildOperator}
 
 type Operation struct {
 	OperationType *OperationType

pkg/yqlib/operator_add.go

@@ -22,13 +22,7 @@ func AddAssignOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode
 	return d.GetMatchingNodes(matchingNodes, assignmentOpNode)
 }
 
-func toNodes(candidates *list.List) []*yaml.Node {
+func toNodes(candidate *CandidateNode) []*yaml.Node {
-
-	if candidates.Len() == 0 {
-		return []*yaml.Node{}
-	}
-	candidate := candidates.Front().Value.(*CandidateNode)
-
 	if candidate.Node.Tag == "!!null" {
 		return []*yaml.Node{}
 	}
@@ -44,40 +38,33 @@ func toNodes(candidates *list.List) []*yaml.Node {
 
 func AddOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
 	log.Debugf("Add operator")
-	var results = list.New()
-	lhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Lhs)
-	if err != nil {
-		return nil, err
-	}
-	rhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Rhs)
 
-	if err != nil {
+	return crossFunction(d, matchingNodes, pathNode, add)
-		return nil, err
+}
-	}
+
-
+func add(d *dataTreeNavigator, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
-	for el := lhs.Front(); el != nil; el = el.Next() {
+	lhs.Node = UnwrapDoc(lhs.Node)
-		lhsCandidate := el.Value.(*CandidateNode)
+	rhs.Node = UnwrapDoc(rhs.Node)
-		lhsNode := UnwrapDoc(lhsCandidate.Node)
+
-
+	target := &CandidateNode{
-		target := &CandidateNode{
+		Path:     lhs.Path,
-			Path:     lhsCandidate.Path,
+		Document: lhs.Document,
-			Document: lhsCandidate.Document,
+		Filename: lhs.Filename,
-			Filename: lhsCandidate.Filename,
+		Node:     &yaml.Node{},
-			Node:     &yaml.Node{},
+	}
-		}
+	lhsNode := lhs.Node
-
+
-		switch lhsNode.Kind {
+	switch lhsNode.Kind {
-		case yaml.MappingNode:
+	case yaml.MappingNode:
-			return nil, fmt.Errorf("Maps not yet supported for addition")
+		return nil, fmt.Errorf("Maps not yet supported for addition")
-		case yaml.SequenceNode:
+	case yaml.SequenceNode:
-			target.Node.Kind = yaml.SequenceNode
+		target.Node.Kind = yaml.SequenceNode
-			target.Node.Style = lhsNode.Style
+		target.Node.Style = lhsNode.Style
-			target.Node.Tag = "!!seq"
+		target.Node.Tag = "!!seq"
-			target.Node.Content = append(lhsNode.Content, toNodes(rhs)...)
+		target.Node.Content = append(lhsNode.Content, toNodes(rhs)...)
-			results.PushBack(target)
+	case yaml.ScalarNode:
-		case yaml.ScalarNode:
+		return nil, fmt.Errorf("Scalars not yet supported for addition")
-			return nil, fmt.Errorf("Scalars not yet supported for addition")
+	}
-		}
+
-	}
+	return target, nil
-	return results, nil
 }

pkg/yqlib/operator_add_test.go

@@ -21,6 +21,14 @@ var addOperatorScenarios = []expressionScenario{
 			"D0, P[a], (!!seq)::[1, 2, 3, 4]\n",
 		},
 	},
+	{
+		skipDoc:    true,
+		expression: `[1] + ([2], [3])`,
+		expected: []string{
+			"D0, P[], (!!seq)::- 1\n- 2\n",
+			"D0, P[], (!!seq)::- 1\n- 3\n",
+		},
+	},
 	{
 		description: "Concatenate null to array",
 		document:    `{a: [1,2]}`,

pkg/yqlib/operator_alternative.go

@@ -0,0 +1,28 @@
+package yqlib
+
+import (
+	"container/list"
+)
+
+// corssFunction no matches
+// can boolean use crossfunction
+
+func AlternativeOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
+	log.Debugf("-- alternative")
+	return crossFunction(d, matchingNodes, pathNode, alternativeFunc)
+}
+
+func alternativeFunc(d *dataTreeNavigator, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
+	lhs.Node = UnwrapDoc(lhs.Node)
+	rhs.Node = UnwrapDoc(rhs.Node)
+	log.Debugf("Alternative LHS: %v", lhs.Node.Tag)
+	log.Debugf("-          RHS: %v", rhs.Node.Tag)
+
+	isTrue, err := isTruthy(lhs)
+	if err != nil {
+		return nil, err
+	} else if isTrue {
+		return lhs, nil
+	}
+	return rhs, nil
+}

pkg/yqlib/operator_alternative_test.go

@@ -0,0 +1,62 @@
+package yqlib
+
+import (
+	"testing"
+)
+
+var alternativeOperatorScenarios = []expressionScenario{
+	{
+		description: "LHS is defined",
+		expression:  `.a // "hello"`,
+		document:    `{a: bridge}`,
+		expected: []string{
+			"D0, P[a], (!!str)::bridge\n",
+		},
+	},
+	{
+		description: "LHS is not defined",
+		expression:  `.a // "hello"`,
+		document:    `{}`,
+		expected: []string{
+			"D0, P[], (!!str)::hello\n",
+		},
+	},
+	{
+		description: "LHS is null",
+		expression:  `.a // "hello"`,
+		document:    `{a: ~}`,
+		expected: []string{
+			"D0, P[], (!!str)::hello\n",
+		},
+	},
+	{
+		description: "LHS is false",
+		expression:  `.a // "hello"`,
+		document:    `{a: false}`,
+		expected: []string{
+			"D0, P[], (!!str)::hello\n",
+		},
+	},
+	{
+		description: "RHS is an expression",
+		expression:  `.a // .b`,
+		document:    `{a: false, b: cat}`,
+		expected: []string{
+			"D0, P[b], (!!str)::cat\n",
+		},
+	},
+	{
+		skipDoc:    true,
+		expression: `false // true`,
+		expected: []string{
+			"D0, P[], (!!bool)::true\n",
+		},
+	},
+}
+
+func TestAlternativeOperatorScenarios(t *testing.T) {
+	for _, tt := range alternativeOperatorScenarios {
+		testScenario(t, &tt)
+	}
+	documentScenarios(t, "Alternative (Default value)", alternativeOperatorScenarios)
+}

pkg/yqlib/operator_anchors_aliases.go

@@ -6,6 +6,88 @@ import (
 	"gopkg.in/yaml.v3"
 )
 
+func AssignAliasOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
+
+	log.Debugf("AssignAlias operator!")
+
+	rhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Rhs)
+	if err != nil {
+		return nil, err
+	}
+	aliasName := ""
+	if rhs.Front() != nil {
+		aliasName = rhs.Front().Value.(*CandidateNode).Node.Value
+	}
+
+	lhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Lhs)
+
+	if err != nil {
+		return nil, err
+	}
+
+	for el := lhs.Front(); el != nil; el = el.Next() {
+		candidate := el.Value.(*CandidateNode)
+		log.Debugf("Setting aliasName : %v", candidate.GetKey())
+		candidate.Node.Kind = yaml.AliasNode
+		candidate.Node.Value = aliasName
+	}
+	return matchingNodes, nil
+}
+
+func GetAliasOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
+	log.Debugf("GetAlias operator!")
+	var results = list.New()
+
+	for el := matchingNodes.Front(); el != nil; el = el.Next() {
+		candidate := el.Value.(*CandidateNode)
+		node := &yaml.Node{Kind: yaml.ScalarNode, Value: candidate.Node.Value, Tag: "!!str"}
+		lengthCand := &CandidateNode{Node: node, Document: candidate.Document, Path: candidate.Path}
+		results.PushBack(lengthCand)
+	}
+	return results, nil
+}
+
+func AssignAnchorOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
+
+	log.Debugf("AssignAnchor operator!")
+
+	rhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Rhs)
+	if err != nil {
+		return nil, err
+	}
+	anchorName := ""
+	if rhs.Front() != nil {
+		anchorName = rhs.Front().Value.(*CandidateNode).Node.Value
+	}
+
+	lhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Lhs)
+
+	if err != nil {
+		return nil, err
+	}
+
+	for el := lhs.Front(); el != nil; el = el.Next() {
+		candidate := el.Value.(*CandidateNode)
+		log.Debugf("Setting anchorName of : %v", candidate.GetKey())
+		candidate.Node.Anchor = anchorName
+	}
+	return matchingNodes, nil
+}
+
+func GetAnchorOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
+	log.Debugf("GetAnchor operator!")
+	var results = list.New()
+
+	for el := matchingNodes.Front(); el != nil; el = el.Next() {
+		candidate := el.Value.(*CandidateNode)
+		anchor := candidate.Node.Anchor
+		node := &yaml.Node{Kind: yaml.ScalarNode, Value: anchor, Tag: "!!str"}
+		lengthCand := &CandidateNode{Node: node, Document: candidate.Document, Path: candidate.Path}
+		results.PushBack(lengthCand)
+	}
+	return results, nil
+}
+
 func ExplodeOperator(d *dataTreeNavigator, matchMap *list.List, pathNode *PathTreeNode) (*list.List, error) {
 	log.Debugf("-- ExplodeOperation")
 

pkg/yqlib/operator_anchors_aliases_test.go

@@ -4,7 +4,39 @@ import (
 	"testing"
 )
 
-var explodeTest = []expressionScenario{
+var anchorOperatorScenarios = []expressionScenario{
+	{
+		description: "Get anchor",
+		document:    `a: &billyBob cat`,
+		expression:  `.a | anchor`,
+		expected: []string{
+			"D0, P[a], (!!str)::billyBob\n",
+		},
+	},
+	{
+		description: "Set anchor",
+		document:    `a: cat`,
+		expression:  `.a anchor = "foobar"`,
+		expected: []string{
+			"D0, P[], (doc)::a: &foobar cat\n",
+		},
+	},
+	{
+		description: "Get alias",
+		document:    `{b: &billyBob meow, a: *billyBob}`,
+		expression:  `.a | alias`,
+		expected: []string{
+			"D0, P[a], (!!str)::billyBob\n",
+		},
+	},
+	{
+		description: "Set alias",
+		document:    `{b: &meow purr, a: cat}`,
+		expression:  `.a alias = "meow"`,
+		expected: []string{
+			"D0, P[], (doc)::{b: &meow purr, a: *meow}\n",
+		},
+	},
 	{
 		description: "Explode alias and anchor",
 		document:    `{f : {a: &a cat, b: *a}}`,
@@ -82,9 +114,9 @@ foobar:
 	},
 }
 
-func TestExplodeOperatorScenarios(t *testing.T) {
+func TestAnchorAliaseOperatorScenarios(t *testing.T) {
-	for _, tt := range explodeTest {
+	for _, tt := range anchorOperatorScenarios {
 		testScenario(t, &tt)
 	}
-	documentScenarios(t, "Explode", explodeTest)
+	documentScenarios(t, "Anchor and Aliases Operators", anchorOperatorScenarios)
 }

pkg/yqlib/operator_booleans.go

@@ -25,74 +25,39 @@ func isTruthy(c *CandidateNode) (bool, error) {
 
 type boolOp func(bool, bool) bool
 
-func performBoolOp(results *list.List, lhs *list.List, rhs *list.List, op boolOp) error {
+func performBoolOp(op boolOp) func(d *dataTreeNavigator, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
-	for lhsChild := lhs.Front(); lhsChild != nil; lhsChild = lhsChild.Next() {
+	return func(d *dataTreeNavigator, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
-		lhsCandidate := lhsChild.Value.(*CandidateNode)
+		lhs.Node = UnwrapDoc(lhs.Node)
-		lhsTrue, errDecoding := isTruthy(lhsCandidate)
+		rhs.Node = UnwrapDoc(rhs.Node)
+
+		lhsTrue, errDecoding := isTruthy(lhs)
 		if errDecoding != nil {
-			return errDecoding
+			return nil, errDecoding
 		}
 
-		for rhsChild := rhs.Front(); rhsChild != nil; rhsChild = rhsChild.Next() {
+		rhsTrue, errDecoding := isTruthy(rhs)
-			rhsCandidate := rhsChild.Value.(*CandidateNode)
+		if errDecoding != nil {
-			rhsTrue, errDecoding := isTruthy(rhsCandidate)
+			return nil, errDecoding
-			if errDecoding != nil {
-				return errDecoding
-			}
-			boolResult := createBooleanCandidate(lhsCandidate, op(lhsTrue, rhsTrue))
-			results.PushBack(boolResult)
 		}
+
+		return createBooleanCandidate(lhs, op(lhsTrue, rhsTrue)), nil
 	}
-	return nil
-}
-
-func booleanOp(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode, op boolOp) (*list.List, error) {
-	var results = list.New()
-
-	if matchingNodes.Len() == 0 {
-		lhs, err := d.GetMatchingNodes(list.New(), pathNode.Lhs)
-		if err != nil {
-			return nil, err
-		}
-		rhs, err := d.GetMatchingNodes(list.New(), pathNode.Rhs)
-		if err != nil {
-			return nil, err
-		}
-		return results, performBoolOp(results, lhs, rhs, op)
-	}
-
-	for el := matchingNodes.Front(); el != nil; el = el.Next() {
-		candidate := el.Value.(*CandidateNode)
-		lhs, err := d.GetMatchingNodes(nodeToMap(candidate), pathNode.Lhs)
-		if err != nil {
-			return nil, err
-		}
-		rhs, err := d.GetMatchingNodes(nodeToMap(candidate), pathNode.Rhs)
-		if err != nil {
-			return nil, err
-		}
-
-		err = performBoolOp(results, lhs, rhs, op)
-		if err != nil {
-			return nil, err
-		}
-
-	}
-	return results, nil
 }
 
 func OrOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
 	log.Debugf("-- orOp")
-	return booleanOp(d, matchingNodes, pathNode, func(b1 bool, b2 bool) bool {
+	return crossFunction(d, matchingNodes, pathNode, performBoolOp(
-		return b1 || b2
+		func(b1 bool, b2 bool) bool {
-	})
+			return b1 || b2
+		}))
 }
 
 func AndOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
 	log.Debugf("-- AndOp")
-	return booleanOp(d, matchingNodes, pathNode, func(b1 bool, b2 bool) bool {
+	return crossFunction(d, matchingNodes, pathNode, performBoolOp(
-		return b1 && b2
+		func(b1 bool, b2 bool) bool {
-	})
+			return b1 && b2
+		}))
 }
 
 func NotOperator(d *dataTreeNavigator, matchMap *list.List, pathNode *PathTreeNode) (*list.List, error) {

pkg/yqlib/operator_delete.go

@@ -2,39 +2,67 @@ package yqlib
 
 import (
 	"container/list"
+	"fmt"
 
 	yaml "gopkg.in/yaml.v3"
 )
 
 func DeleteChildOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
-	// for each lhs, splat the node,
-	// the intersect it against the rhs expression
-	// recreate the contents using only the intersection result.
 
-	for el := matchingNodes.Front(); el != nil; el = el.Next() {
+	nodesToDelete, err := d.GetMatchingNodes(matchingNodes, pathNode.Rhs)
+
+	if err != nil {
+		return nil, err
+	}
+
+	for el := nodesToDelete.Front(); el != nil; el = el.Next() {
 		candidate := el.Value.(*CandidateNode)
-		elMap := list.New()
+
-		elMap.PushBack(candidate)
+		deleteImmediateChildOp := &Operation{
-		nodesToDelete, err := d.GetMatchingNodes(elMap, pathNode.Rhs)
+			OperationType: DeleteImmediateChild,
-		log.Debug("nodesToDelete:\n%v", NodesToString(nodesToDelete))
+			Value:         candidate.Path[len(candidate.Path)-1],
-		if err != nil {
-			return nil, err
 		}
 
-		realNode := UnwrapDoc(candidate.Node)
+		deleteImmediateChildOpNode := &PathTreeNode{
+			Operation: deleteImmediateChildOp,
+			Rhs:       createTraversalTree(candidate.Path[0 : len(candidate.Path)-1]),
+		}
 
-		if realNode.Kind == yaml.SequenceNode {
+		_, err := d.GetMatchingNodes(matchingNodes, deleteImmediateChildOpNode)
-			deleteFromArray(candidate, nodesToDelete)
+		if err != nil {
-		} else if realNode.Kind == yaml.MappingNode {
+			return nil, err
-			deleteFromMap(candidate, nodesToDelete)
-		} else {
-			log.Debug("Cannot delete from node that's not a map or array %v", NodeToString(candidate))
 		}
 	}
 	return matchingNodes, nil
 }
 
-func deleteFromMap(candidate *CandidateNode, nodesToDelete *list.List) {
+func DeleteImmediateChildOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
+	parents, err := d.GetMatchingNodes(matchingNodes, pathNode.Rhs)
+
+	if err != nil {
+		return nil, err
+	}
+
+	childPath := pathNode.Operation.Value
+
+	log.Debug("childPath to remove %v", childPath)
+
+	for el := parents.Front(); el != nil; el = el.Next() {
+		parent := el.Value.(*CandidateNode)
+		parentNode := UnwrapDoc(parent.Node)
+		if parentNode.Kind == yaml.MappingNode {
+			deleteFromMap(parent, childPath)
+		} else if parentNode.Kind == yaml.SequenceNode {
+			deleteFromArray(parent, childPath)
+		} else {
+			return nil, fmt.Errorf("Cannot delete nodes from parent of tag %v", parentNode.Tag)
+		}
+
+	}
+	return matchingNodes, nil
+}
+
+func deleteFromMap(candidate *CandidateNode, childPath interface{}) {
 	log.Debug("deleteFromMap")
 	node := UnwrapDoc(candidate.Node)
 	contents := node.Content
@@ -50,12 +78,7 @@ func deleteFromMap(candidate *CandidateNode, nodesToDelete *list.List) {
 			Path:     append(candidate.Path, key.Value),
 		}
 
-		shouldDelete := false
+		shouldDelete := key.Value == childPath
-		for el := nodesToDelete.Front(); el != nil && !shouldDelete; el = el.Next() {
-			if el.Value.(*CandidateNode).GetKey() == childCandidate.GetKey() {
-				shouldDelete = true
-			}
-		}
 
 		log.Debugf("shouldDelete %v ? %v", childCandidate.GetKey(), shouldDelete)
 
@@ -66,7 +89,7 @@ func deleteFromMap(candidate *CandidateNode, nodesToDelete *list.List) {
 	node.Content = newContents
 }
 
-func deleteFromArray(candidate *CandidateNode, nodesToDelete *list.List) {
+func deleteFromArray(candidate *CandidateNode, childPath interface{}) {
 	log.Debug("deleteFromArray")
 	node := UnwrapDoc(candidate.Node)
 	contents := node.Content
@@ -75,18 +98,7 @@ func deleteFromArray(candidate *CandidateNode, nodesToDelete *list.List) {
 	for index := 0; index < len(contents); index = index + 1 {
 		value := contents[index]
 
-		childCandidate := &CandidateNode{
+		shouldDelete := fmt.Sprintf("%v", index) == fmt.Sprintf("%v", childPath)
-			Node:     value,
-			Document: candidate.Document,
-			Path:     append(candidate.Path, index),
-		}
-
-		shouldDelete := false
-		for el := nodesToDelete.Front(); el != nil && !shouldDelete; el = el.Next() {
-			if el.Value.(*CandidateNode).GetKey() == childCandidate.GetKey() {
-				shouldDelete = true
-			}
-		}
 
 		if !shouldDelete {
 			newContents = append(newContents, value)

pkg/yqlib/operator_delete_test.go

@@ -13,6 +13,22 @@ var deleteOperatorScenarios = []expressionScenario{
 			"D0, P[], (doc)::{a: cat}\n",
 		},
 	},
+	{
+		description: "Delete nested entry in map",
+		document:    `{a: {a1: fred, a2: frood}}`,
+		expression:  `del(.a.a1)`,
+		expected: []string{
+			"D0, P[], (doc)::{a: {a2: frood}}\n",
+		},
+	},
+	{
+		skipDoc:    true,
+		document:   `{a: {a1: fred, a2: frood}}`,
+		expression: `del(.. | select(.=="frood"))`,
+		expected: []string{
+			"D0, P[], (!!map)::{a: {a1: fred}}\n",
+		},
+	},
 	{
 		description: "Delete entry in array",
 		document:    `[1,2,3]`,
@@ -21,6 +37,14 @@ var deleteOperatorScenarios = []expressionScenario{
 			"D0, P[], (doc)::[1, 3]\n",
 		},
 	},
+	{
+		description: "Delete nested entry in array",
+		document:    `[{a: cat, b: dog}]`,
+		expression:  `del(.[0].a)`,
+		expected: []string{
+			"D0, P[], (doc)::[{b: dog}]\n",
+		},
+	},
 	{
 		description: "Delete no matches",
 		document:    `{a: cat, b: dog}`,

pkg/yqlib/operator_multiply.go

@@ -104,19 +104,6 @@ func mergeObjects(d *dataTreeNavigator, lhs *CandidateNode, rhs *CandidateNode,
 	return lhs, nil
 }
 
-func createTraversalTree(path []interface{}) *PathTreeNode {
-	if len(path) == 0 {
-		return &PathTreeNode{Operation: &Operation{OperationType: SelfReference}}
-	} else if len(path) == 1 {
-		return &PathTreeNode{Operation: &Operation{OperationType: TraversePath, Value: path[0], StringValue: fmt.Sprintf("%v", path[0])}}
-	}
-	return &PathTreeNode{
-		Operation: &Operation{OperationType: ShortPipe},
-		Lhs:       createTraversalTree(path[0:1]),
-		Rhs:       createTraversalTree(path[1:])}
-
-}
-
 func applyAssignment(d *dataTreeNavigator, pathIndexToStartFrom int, lhs *CandidateNode, rhs *CandidateNode, shouldAppendArrays bool) error {
 
 	log.Debugf("merge - applyAssignment lhs %v, rhs: %v", NodeToString(lhs), NodeToString(rhs))

pkg/yqlib/operator_value_test.go

@@ -36,13 +36,22 @@ var valueOperatorScenarios = []expressionScenario{
 		expected: []string{
 			"D0, P[], (!!float)::5e-10\n",
 		},
-	}, {
+	},
+	{
 		document:   ``,
 		expression: `"cat"`,
 		expected: []string{
 			"D0, P[], (!!str)::cat\n",
 		},
-	}, {
+	},
+	{
+		document:   ``,
+		expression: `"frog jumps"`,
+		expected: []string{
+			"D0, P[], (!!str)::frog jumps\n",
+		},
+	},
+	{
 		document:   ``,
 		expression: `"1.3"`,
 		expected: []string{

pkg/yqlib/operators.go

@@ -2,6 +2,7 @@ package yqlib
 
 import (
 	"container/list"
+	"fmt"
 
 	"gopkg.in/yaml.v3"
 )
@@ -33,3 +34,15 @@ func nodeToMap(candidate *CandidateNode) *list.List {
 	elMap.PushBack(candidate)
 	return elMap
 }
+
+func createTraversalTree(path []interface{}) *PathTreeNode {
+	if len(path) == 0 {
+		return &PathTreeNode{Operation: &Operation{OperationType: SelfReference}}
+	} else if len(path) == 1 {
+		return &PathTreeNode{Operation: &Operation{OperationType: TraversePath, Value: path[0], StringValue: fmt.Sprintf("%v", path[0])}}
+	}
+	return &PathTreeNode{
+		Operation: &Operation{OperationType: ShortPipe},
+		Lhs:       createTraversalTree(path[0:1]),
+		Rhs:       createTraversalTree(path[1:])}
+}

pkg/yqlib/path_tokeniser.go

@@ -1,6 +1,7 @@
 package yqlib
 
 import (
+	"fmt"
 	"strconv"
 
 	lex "github.com/timtadh/lexmachine"
@@ -200,12 +201,15 @@ func initLexer() (*lex.Lexer, error) {
 	lexer.Add([]byte(`or`), opToken(Or))
 	lexer.Add([]byte(`and`), opToken(And))
 	lexer.Add([]byte(`not`), opToken(Not))
+	lexer.Add([]byte(`\/\/`), opToken(Alternative))
 
 	lexer.Add([]byte(`documentIndex`), opToken(GetDocumentIndex))
 
 	lexer.Add([]byte(`style`), opAssignableToken(GetStyle, AssignStyle))
 
 	lexer.Add([]byte(`tag`), opAssignableToken(GetTag, AssignTag))
+	lexer.Add([]byte(`anchor`), opAssignableToken(GetAnchor, AssignAnchor))
+	lexer.Add([]byte(`alias`), opAssignableToken(GetAlias, AssignAlias))
 	lexer.Add([]byte(`filename`), opToken(GetFilename))
 	lexer.Add([]byte(`fileIndex`), opToken(GetFileIndex))
 	lexer.Add([]byte(`path`), opToken(GetPath))
@@ -248,7 +252,7 @@ func initLexer() (*lex.Lexer, error) {
 	lexer.Add([]byte(`[Nn][Uu][Ll][Ll]`), nullValue())
 	lexer.Add([]byte(`~`), nullValue())
 
-	lexer.Add([]byte(`"[^ "]*"`), stringValue(true))
+	lexer.Add([]byte(`"[^"]*"`), stringValue(true))
 
 	lexer.Add([]byte(`\[\]`), literalToken(SplatOrEmptyCollect, true))
 
@@ -288,7 +292,7 @@ func (p *pathTokeniser) Tokenise(path string) ([]*Token, error) {
 	scanner, err := p.lexer.Scanner([]byte(path))
 
 	if err != nil {
-		return nil, err
+		return nil, fmt.Errorf("Parsing expression: %v", err)
 	}
 	var tokens []*Token
 	for tok, err, eof := scanner.Next(); !eof; tok, err, eof = scanner.Next() {
@@ -299,7 +303,7 @@ func (p *pathTokeniser) Tokenise(path string) ([]*Token, error) {
 			tokens = append(tokens, token)
 		}
 		if err != nil {
-			return nil, err
+			return nil, fmt.Errorf("Parsing expression: %v", err)
 		}
 	}
 	var postProcessedTokens = make([]*Token, 0)

pkg/yqlib/path_tree.go

@@ -1,6 +1,9 @@
 package yqlib
 
-import "fmt"
+import (
+	"fmt"
+	"strings"
+)
 
 var myPathTokeniser = NewPathTokeniser()
 var myPathPostfixer = NewPathPostFixer()
@@ -49,10 +52,16 @@ func (p *pathTreeCreator) CreatePathTree(postFixPath []*Operation) (*PathTreeNod
 		if Operation.OperationType.NumArgs > 0 {
 			numArgs := Operation.OperationType.NumArgs
 			if numArgs == 1 {
+				if len(stack) < 1 {
+					return nil, fmt.Errorf("'%v' expects 1 arg but received none", strings.TrimSpace(Operation.StringValue))
+				}
 				remaining, rhs := stack[:len(stack)-1], stack[len(stack)-1]
 				newNode.Rhs = rhs
 				stack = remaining
 			} else if numArgs == 2 {
+				if len(stack) < 2 {
+					return nil, fmt.Errorf("'%v' expects 2 args but there is %v", strings.TrimSpace(Operation.StringValue), len(stack))
+				}
 				remaining, lhs, rhs := stack[:len(stack)-2], stack[len(stack)-2], stack[len(stack)-1]
 				newNode.Lhs = lhs
 				newNode.Rhs = rhs
@@ -62,7 +71,7 @@ func (p *pathTreeCreator) CreatePathTree(postFixPath []*Operation) (*PathTreeNod
 		stack = append(stack, &newNode)
 	}
 	if len(stack) != 1 {
-		return nil, fmt.Errorf("expected stack to have 1 thing but its %v", stack)
+		return nil, fmt.Errorf("expected end of expression but found '%v', please check expression syntax", strings.TrimSpace(stack[1].Operation.StringValue))
 	}
 	return stack[0], nil
 }

pkg/yqlib/path_tree_test.go

@@ -0,0 +1,42 @@
+package yqlib
+
+import (
+	"testing"
+
+	"github.com/mikefarah/yq/v4/test"
+)
+
+func TestPathTreeNoArgsForTwoArgOp(t *testing.T) {
+	_, err := treeCreator.ParsePath("=")
+	test.AssertResultComplex(t, "'=' expects 2 args but there is 0", err.Error())
+}
+
+func TestPathTreeOneLhsArgsForTwoArgOp(t *testing.T) {
+	_, err := treeCreator.ParsePath(".a =")
+	test.AssertResultComplex(t, "'=' expects 2 args but there is 1", err.Error())
+}
+
+func TestPathTreeOneRhsArgsForTwoArgOp(t *testing.T) {
+	_, err := treeCreator.ParsePath("= .a")
+	test.AssertResultComplex(t, "'=' expects 2 args but there is 1", err.Error())
+}
+
+func TestPathTreeTwoArgsForTwoArgOp(t *testing.T) {
+	_, err := treeCreator.ParsePath(".a = .b")
+	test.AssertResultComplex(t, nil, err)
+}
+
+func TestPathTreeNoArgsForOneArgOp(t *testing.T) {
+	_, err := treeCreator.ParsePath("explode")
+	test.AssertResultComplex(t, "'explode' expects 1 arg but received none", err.Error())
+}
+
+func TestPathTreeOneArgForOneArgOp(t *testing.T) {
+	_, err := treeCreator.ParsePath("explode(.)")
+	test.AssertResultComplex(t, nil, err)
+}
+
+func TestPathTreeExtraArgs(t *testing.T) {
+	_, err := treeCreator.ParsePath("sortKeys(.) explode(.)")
+	test.AssertResultComplex(t, "expected end of expression but found 'explode', please check expression syntax", err.Error())
+}

pkg/yqlib/printer.go

@@ -22,6 +22,7 @@ type resultsPrinter struct {
 	writer             io.Writer
 	firstTimePrinting  bool
 	previousDocIndex   uint
+	previousFileIndex  int
 	printedMatches     bool
 }
 
@@ -89,19 +90,20 @@ func (p *resultsPrinter) PrintResults(matchingNodes *list.List) error {
 		return nil
 	}
 	if p.firstTimePrinting {
-		p.previousDocIndex = matchingNodes.Front().Value.(*CandidateNode).Document
+		node := matchingNodes.Front().Value.(*CandidateNode)
+		p.previousDocIndex = node.Document
+		p.previousFileIndex = node.FileIndex
 		p.firstTimePrinting = false
 	}
 
 	for el := matchingNodes.Front(); el != nil; el = el.Next() {
 		mappedDoc := el.Value.(*CandidateNode)
 		log.Debug("-- print sep logic: p.firstTimePrinting: %v, previousDocIndex: %v, mappedDoc.Document: %v, printDocSeparators: %v", p.firstTimePrinting, p.previousDocIndex, mappedDoc.Document, p.printDocSeparators)
-		if (p.previousDocIndex != mappedDoc.Document) && p.printDocSeparators {
+		if (p.previousDocIndex != mappedDoc.Document || p.previousFileIndex != mappedDoc.FileIndex) && p.printDocSeparators {
 			log.Debug("-- writing doc sep")
 			if err := p.writeString(bufferedWriter, "---\n"); err != nil {
 				return err
 			}
-
 		}
 
 		if err := p.printNode(mappedDoc.Node, bufferedWriter); err != nil {

pkg/yqlib/printer_test.go

@@ -52,7 +52,53 @@ func TestPrinterMultipleDocsInSequence(t *testing.T) {
 
 	writer.Flush()
 	test.AssertResult(t, multiDocSample, output.String())
+}
 
+func TestPrinterMultipleFilesInSequence(t *testing.T) {
+	var output bytes.Buffer
+	var writer = bufio.NewWriter(&output)
+	printer := NewPrinter(writer, false, true, false, 2, true)
+
+	inputs, err := readDocuments(strings.NewReader(multiDocSample), "sample.yml", 0)
+	if err != nil {
+		panic(err)
+	}
+
+	el := inputs.Front()
+	elNode := el.Value.(*CandidateNode)
+	elNode.Document = 0
+	elNode.FileIndex = 0
+	sample1 := nodeToMap(elNode)
+
+	el = el.Next()
+	elNode = el.Value.(*CandidateNode)
+	elNode.Document = 0
+	elNode.FileIndex = 1
+	sample2 := nodeToMap(elNode)
+
+	el = el.Next()
+	elNode = el.Value.(*CandidateNode)
+	elNode.Document = 0
+	elNode.FileIndex = 2
+	sample3 := nodeToMap(elNode)
+
+	err = printer.PrintResults(sample1)
+	if err != nil {
+		panic(err)
+	}
+
+	err = printer.PrintResults(sample2)
+	if err != nil {
+		panic(err)
+	}
+
+	err = printer.PrintResults(sample3)
+	if err != nil {
+		panic(err)
+	}
+
+	writer.Flush()
+	test.AssertResult(t, multiDocSample, output.String())
 }
 
 func TestPrinterMultipleDocsInSinglePrint(t *testing.T) {

release_instructions.txt

@@ -4,6 +4,7 @@
 - tag git with same version number
 - make sure local build passes
 - push tag to git
+  - 3.4.0, v3
   - git push --tags
 - make local xcompile (builds binaries for all platforms)
 
@@ -30,7 +31,7 @@
 
 - docker
   - build and push latest and new version tag
-  - docker build .  -t mikefarah/yq:latest -t mikefarah/yq:VERSION 
+  - docker build .  -t mikefarah/yq:latest -t mikefarah/yq:3 -t mikefarah/yq:3.X
 
 - debian package
   - ensure you get all vendor dependencies before packaging

scripts/check.sh

@@ -3,7 +3,7 @@
 set -o errexit
 set -o pipefail
 
-./bin/golangci-lint run
+./bin/golangci-lint run --timeout=5m
 
 # ./bin/golangci-lint \
 #   --tests \

scripts/format.sh

@@ -2,3 +2,4 @@
 
 find . \( -path ./vendor \) -prune -o -name "*.go" -exec goimports -w {} \;
 go mod tidy
+go mod vendor

scripts/publish-docker.sh

@@ -6,4 +6,7 @@ docker build \
   --build-arg VERSION=${VERSION} \
   -t mikefarah/yq:latest \
   -t mikefarah/yq:${VERSION} \
-  .
+  -t mikefarah/yq:4 \
+  .
+
+trivy image mikefarah/yq:${VERSION}

scripts/xcompile.sh

@@ -1,14 +1,24 @@
 #!/bin/bash
-
+set -e
 # This assumes that gonative and gox is installed as per the 'one time setup' instructions
 # at https://github.com/inconshreveable/gonative
 
 
-CGO_ENABLED=0 gox -ldflags "${LDFLAGS}" -output="build/yq_{{.OS}}_{{.Arch}}"
+CGO_ENABLED=0 gox -ldflags "${LDFLAGS}" -output="build/yq_{{.OS}}_{{.Arch}}" --osarch="darwin/amd64 freebsd/386 freebsd/amd64 freebsd/arm linux/386 linux/amd64 linux/arm linux/arm64 linux/mips linux/mips64 linux/mips64le linux/mipsle linux/ppc64 linux/ppc64le linux/s390x netbsd/386 netbsd/amd64 netbsd/arm openbsd/386 openbsd/amd64 windows/386 windows/amd64"
-# include non-default linux builds too
-CGO_ENABLED=0 gox -ldflags "${LDFLAGS}" -os=linux  -output="build/yq_{{.OS}}_{{.Arch}}"
 
 cd build
-rhash -r -a . -P -o checksums
+rhash -r -a . -o checksums
 
-rhash --list-hashes > checksums_hashes_order
+rhash --list-hashes > checksums_hashes_order
+
+find . -executable -type f | xargs -I {} tar czvf {}.tar.gz {}
+
+# just in case find thinks this is executable...
+rm -f checksums_hashes_order.tar.gz 
+rm -f checksums.tar.gz
+
+rm yq_windows_386.exe.tar.gz
+rm yq_windows_amd64.exe.tar.gz
+
+zip yq_windows_386.zip yq_windows_386.exe
+zip yq_windows_amd64.zip yq_windows_amd64.exe

snap/snapcraft.yaml

@@ -1,5 +1,5 @@
 name: yq
-version: '4.0.0-beta1'
+version: '4.0.0'
 summary: A lightweight and portable command-line YAML processor
 description: |
   The aim of the project is to be the jq or sed of yaml files.