Bumping version to beta2

.github/ISSUE_TEMPLATE/bug_report.md

@@ -1,8 +1,8 @@
 ---
-name: Bug report - V3
+name: Bug report
 about: Create a report to help us improve
 title: ''
-labels: bug, v3
+labels: bug
 assignees: ''
 
 ---
@@ -10,12 +10,11 @@ assignees: ''
 **Describe the bug**
 A clear and concise description of what the bug is.
 
-Version of yq: 3.X.X
+version of yq: 
-Operating system: mac/linux/windows/....
+operating system:
-Installed via: docker/binary release/homebrew/snap/...
 
 **Input Yaml**
-Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)
+Concise yaml document(s) (as simple as possible to show the bug)
 data1.yml:
 ```yaml
 this: should really work

.github/ISSUE_TEMPLATE/bug_report_v4.md

@@ -1,49 +0,0 @@
----
-name: Bug report - V4
-about: Create a report to help us improve
-title: ''
-labels: bug, v4
-assignees: ''
-
----
-
-**Describe the bug**
-A clear and concise description of what the bug is.
-
-Version of yq: 4.X.X
-Operating system: mac/linux/windows/....
-Installed via: docker/binary release/homebrew/snap/...
-
-**Input Yaml**
-Concise yaml document(s) (as simple as possible to show the bug, please keep it to 10 lines or less)
-data1.yml:
-```yaml
-this: should really work
-```
-
-data2.yml:
-```yaml
-but: it strangely didn't
-```
-
-**Command**
-The command you ran:
-```
-yq eval-all 'select(fileIndex==0) | .a.b.c' data1.yml data2.yml
-```
-
-**Actual behavior**
-
-```yaml
-cat: meow
-```
-
-**Expected behavior**
-
-```yaml
-this: should really work
-but: it strangely didn't
-```
-
-**Additional context**
-Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,21 +1,17 @@
 ---
-name: Feature request - V4
+name: Feature request
 about: Suggest an idea for this project
 title: ''
-labels: enhancement, v4
+labels: enhancement
 assignees: ''
 
 ---
 
-**Please describe your feature request.**
+**Is your feature request related to a problem? Please describe.**
-A clear and concise description of what the request is and what it would solve. 
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
-Ex. I wish I could use yq to [...]
-
-Please note that V3 will no longer have any enhancements.
 
 **Describe the solution you'd like**
 If we have data1.yml like:
-(please keep to around 10 lines )
 
 ```yaml
 country: Australia
@@ -24,7 +20,7 @@ country: Australia
 And we run a command:
 
 ```bash
-yq eval 'predictWeatherOf(.country)'
+yq predictWeather data1.yml
 ```
 
 it could output

.github/workflows/go.yml

@@ -28,4 +28,7 @@ jobs:
       run: |
         export PATH=${PATH}:`go env GOPATH`/bin
         scripts/devtools.sh
+    - name: Build
+      run: |
+        export PATH=${PATH}:`go env GOPATH`/bin
         make local build

.github/workflows/release.yml

@@ -1,75 +0,0 @@
-name: Publish image to Dockerhub
-on:
-  push:
-    tags:
-      - 'v*'
-
-jobs:
-  publishGitRelease:
-    environment: gitrelease
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v2
-
-      # steps for building assets
-      - name: Cross compile
-        run: ./scripts/xcompile.sh
-
-      - name: Create Release
-        id: create_release
-        uses: actions/create-release@v1.0.0
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-        with:
-          tag_name: ${{ github.ref }}
-          release_name: ${{ github.ref }}
-          draft: true
-          prerelease: false
-
-      - uses: shogo82148/actions-upload-release-asset@v1
-        with:
-          upload_url: ${{ steps.create_release.outputs.upload_url }}
-          asset_path: build/*
-
-  publishDocker:
-    environment: dockerhub
-    env:
-      IMAGE_NAME: mikefarah/yq
-    runs-on: ubuntu-latest
-    steps:
-      - name: Get latest release tag      
-        uses: oprypin/find-latest-tag@v1
-        with:
-          repository: mikefarah/yq  # The repository to scan.
-          releases-only: true  # We know that all relevant tags have a GitHub release for them.
-        id: yq  
-        
-      - name: Clone source code
-        uses: actions/checkout@v2
-        with:
-          ref: ${{ steps.yq.outputs.tag }}
-      
-      - name: Set up QEMU
-        uses: docker/setup-qemu-action@v1
-        with:
-          platforms: all
-  
-      - name: Set up Docker Buildx
-        id: buildx
-        uses: docker/setup-buildx-action@v1
-        with:
-          version: latest
-
-      - name: Available platforms
-        run: echo ${{ steps.buildx.outputs.platforms }} && docker version
- 
-      - name: Build and push image
-        run: |
-          IMAGE_VERSION="$(git describe --tags --abbrev=0)"
-          SHORT_SHA1=$(git rev-parse --short HEAD)
-          PLATFORMS="linux/amd64,linux/ppc64le,linux/arm64"
-          echo "Building and pushing version ${IMAGE_VERSION} of image ${IMAGE_NAME}"
-          echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
-          docker buildx build --platform "${PLATFORMS}" -t "${IMAGE_NAME}:${IMAGE_VERSION}"  -t "${IMAGE_NAME}:latest" -t "${IMAGE_NAME}:4" \
-            --push .
-    

Dockerfile

@@ -12,7 +12,7 @@ RUN CGO_ENABLED=0 make local build
 
 # Choose alpine as a base image to make this useful for CI, as many
 # CI tools expect an interactive shell inside the container
-FROM alpine:3.12.3 as production
+FROM alpine:3.12 as production
 
 COPY --from=builder /go/src/mikefarah/yq/yq /usr/bin/yq
 RUN chmod +x /usr/bin/yq

README.md

@@ -7,17 +7,11 @@ a lightweight and portable command-line YAML processor
 
 The aim of the project is to be the [jq](https://github.com/stedolan/jq) or sed of yaml files.
 
-## V4 released!
-V4 is now officially released, it's quite different from V3 (sorry for the migration), however it is much more similar to ```jq```, using a similar expression syntax and therefore support much more complex functionality! 
-
-If you've been using v3 and want/need to upgrade, checkout the [upgrade guide](https://mikefarah.gitbook.io/yq/v/v4.x/upgrading-from-v3).
-
 ## Install
 
 ### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest)
 
 ### MacOS:
-Using [Homebrew](https://brew.sh/)
 ```
 brew install yq
 ```
@@ -50,12 +44,13 @@ rm /etc/myfile.tmp
 Use wget to download the pre-compiled binaries:
 
 ```bash
-wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq &&\
+wget https://github.com/mikefarah/yq/releases/download/{VERSION}/{BINARY} -O /usr/bin/yq &&\
     chmod +x /usr/bin/yq
 ```
 
 For instance, VERSION=4.0.0 and BINARY=yq_linux_amd64
 
+
 ### Run with Docker
 
 #### Oneshot use:
@@ -93,14 +88,6 @@ choco install yq
 ```
 Supported by @chillum (https://chocolatey.org/packages/yq)
 
-### Mac:
-Using [MacPorts](https://www.macports.org/)
-```
-sudo port selfupdate
-sudo port install yq
-```
-Supported by @herbygillot (https://ports.macports.org/maintainer/github/herbygillot)
-
 ### Alpine Linux
 - Enable edge/community repo by adding ```$MIRROR/alpine/edge/community``` to ```/etc/apk/repositories```
 - Update database index with ```apk update```
@@ -121,18 +108,21 @@ Supported by @rmescandon (https://launchpad.net/~rmescandon/+archive/ubuntu/yq)
 
 ## Features
 - Written in portable go, so you can download a lovely dependency free binary
-- Uses similar syntax as `jq` but works with YAML and JSON files
-- Fully supports multi document yaml files
 - Colorized yaml output
-- [Deeply traverse yaml](https://mikefarah.gitbook.io/yq/v/v4.x/traverse)
+- [Deep read a yaml file with a given path expression](https://mikefarah.gitbook.io/yq/v/v4.x/traverse)
-- [Sort yaml by keys](https://mikefarah.gitbook.io/yq/v/v4.x/sort-keys)
+- [Return the lengths of arrays/object/scalars](https://mikefarah.gitbook.io/yq/commands/read#printing-length-of-the-results)
-- Manipulate yaml [comments](https://app.gitbook.com/@mikefarah/s/yq/v/v4.x/comment-operators), [styling](https://app.gitbook.com/@mikefarah/s/yq/v/v4.x/style), [tags](https://app.gitbook.com/@mikefarah/s/yq/v/v4.x/tag) and [anchors](https://app.gitbook.com/@mikefarah/s/yq/v/v4.x/explode).
+- Update a yaml file given a [path expression](https://mikefarah.gitbook.io/yq/commands/write-update#basic) or [script file](https://mikefarah.gitbook.io/yq/commands/write-update#basic)
-- [Update yaml inplace](https://mikefarah.gitbook.io/yq/v/v4.x/commands/evaluate#flags)
+- Update creates any missing entries in the path on the fly
-- [Complex expressions to select and update](https://mikefarah.gitbook.io/yq/v/v4.x/select#select-and-update-matching-values-in-map)
+- Deeply [compare](https://mikefarah.gitbook.io/yq/commands/compare) yaml files
-- Keeps yaml formatting and comments when updating (though there are issues with whitespace)
+- Keeps yaml formatting and comments when updating
-- [Convert to/from json to yaml](https://mikefarah.gitbook.io/yq/v/v4.x/usage/convert)
+- [Validate a yaml file](https://mikefarah.gitbook.io/yq/commands/validate)
-- [Pipe data in by using '-'](https://mikefarah.gitbook.io/yq/v/v4.x/commands/evaluate)
+- Create a yaml file given a [deep path and value](https://mikefarah.gitbook.io/yq/commands/create#creating-a-simple-yaml-file) or a [script file](https://mikefarah.gitbook.io/yq/commands/create#creating-using-a-create-script)
-- [General shell completion scripts (bash/zsh/fish/powershell)](https://mikefarah.gitbook.io/yq/v/v4.x/commands/shell-completion)
+- [Prefix a path to a yaml file](https://mikefarah.gitbook.io/yq/commands/prefix)
+- [Convert to/from json to yaml](https://mikefarah.gitbook.io/yq/usage/convert)
+- [Pipe data in by using '-'](https://mikefarah.gitbook.io/yq/commands/read#from-stdin)
+- [Merge](https://mikefarah.gitbook.io/yq/commands/merge) multiple yaml files with various options for [overriding](https://mikefarah.gitbook.io/yq/commands/merge#overwrite-values) and [appending](https://mikefarah.gitbook.io/yq/commands/merge#append-values-with-arrays)
+- Supports multiple documents in a single yaml file for [reading](https://mikefarah.gitbook.io/yq/commands/read#multiple-documents), [writing](https://mikefarah.gitbook.io/yq/commands/write-update#multiple-documents) and [merging](https://mikefarah.gitbook.io/yq/commands/merge#multiple-documents)
+- General shell completion scripts (bash/zsh/fish/powershell) (https://mikefarah.gitbook.io/yq/commands/shell-completion)
 
 ## [Usage](https://mikefarah.gitbook.io/yq/)
 
@@ -144,32 +134,32 @@ Usage:
   yq [command]
 
 Available Commands:
-  eval             Apply expression to each document in each yaml file given in sequence
+  compare          yq x [--prettyPrint/-P] dataA.yaml dataB.yaml 'b.e(name==fr*).value'
-  eval-all         Loads _all_ yaml documents of _all_ yaml files and runs expression once
+  delete           yq d [--inplace/-i] [--doc/-d index] sample.yaml 'b.e(name==fred)'
   help             Help about any command
-  shell-completion Generate completion script
+  merge            yq m [--inplace/-i] [--doc/-d index] [--overwrite/-x] [--append/-a] sample.yaml sample2.yaml
+  new              yq n [--script/-s script_file] a.b.c newValue
+  prefix           yq p [--inplace/-i] [--doc/-d index] sample.yaml a.b.c
+  read             yq r [--printMode/-p pv] sample.yaml 'b.e(name==fr*).value'
+  shell-completion Generates shell completion scripts
+  validate         yq v sample.yaml
+  write            yq w [--inplace/-i] [--script/-s script_file] [--doc/-d index] sample.yaml 'b.e(name==fr*).value' newValue
 
 Flags:
-  -C, --colors        force print with colors
+  -C, --colors        print with colors
-  -e, --exit-status   set exit status if there are no matches or null or false is returned
   -h, --help          help for yq
   -I, --indent int    sets indent level for output (default 2)
-  -i, --inplace       update the yaml file inplace of first yaml file given.
+  -P, --prettyPrint   pretty print
-  -M, --no-colors     force print with no colors
+  -j, --tojson        output as json. By default it prints a json document in one line, use the prettyPrint flag to print a formatted doc.
-  -N, --no-doc        Don't print document separators (---)
-  -n, --null-input    Don't read input, simply evaluate the expression given. Useful for creating yaml docs from scratch.
-  -j, --tojson        output as json. Set indent to 0 to print json in one line.
   -v, --verbose       verbose mode
   -V, --version       Print version information and quit
 
 Use "yq [command] --help" for more information about a command.
 ```
 
-Simple Example:
+## Upgrade from V2
-
+If you've been using v2 and want/need to upgrade, checkout the [upgrade guide](https://mikefarah.gitbook.io/yq/upgrading-from-v2).
-```bash
-yq e '.a.b | length' f1.yml f2.yml 
-```
 
 ## Known Issues / Missing Features
 - `yq` attempts to preserve comment positions and whitespace as much as possible, but it does not handle all scenarios (see https://github.com/go-yaml/yaml/tree/v3 for details)
+- You cannot (yet) select multiple paths/keys from the yaml to be printed out (https://github.com/mikefarah/yq/issues/287)

cmd/root.go

@@ -47,7 +47,6 @@ func New() *cobra.Command {
 	rootCmd.PersistentFlags().IntVarP(&indent, "indent", "I", 2, "sets indent level for output")
 	rootCmd.Flags().BoolVarP(&version, "version", "V", false, "Print version information and quit")
 	rootCmd.PersistentFlags().BoolVarP(&writeInplace, "inplace", "i", false, "update the yaml file inplace of first yaml file given.")
-	rootCmd.PersistentFlags().BoolVarP(&unwrapScalar, "unwrapScalar", "", true, "unwrap scalar, print the value with no quotes, colors or comments")
 	rootCmd.PersistentFlags().BoolVarP(&exitStatus, "exit-status", "e", false, "set exit status if there are no matches or null or false is returned")
 
 	rootCmd.PersistentFlags().BoolVarP(&forceColor, "colors", "C", false, "force print with colors")

cmd/version.go

@@ -11,7 +11,7 @@ var (
 	GitDescribe string
 
 	// Version is main version number that is being run at the moment.
-	Version = "4.0.0"
+	Version = "4.0.0-beta2"
 
 	// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
 	// then it means that it is a final release. Otherwise, this is a pre-release

go.sum

@@ -263,7 +263,6 @@ golang.org/x/sys v0.0.0-20191010194322-b09406accb47/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=
 golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae h1:/WDfKMnPU+m5M4xB+6x4kaepxRw6jWvR5iDRdvjHgy8=
 golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f h1:+Nyd8tzPX9R7BWHguqsrbFdRx3WQ/1ib8I44HXV5yTA=
 golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=

pkg/yqlib/doc/Alternative (Default value).md

@@ -1,73 +0,0 @@
-This operator is used to provide alternative (or default) values when a particular expression is either null or false.
-
-## LHS is defined
-Given a sample.yml file of:
-```yaml
-a: bridge
-```
-then
-```bash
-yq eval '.a // "hello"' sample.yml
-```
-will output
-```yaml
-bridge
-```
-
-## LHS is not defined
-Given a sample.yml file of:
-```yaml
-{}
-```
-then
-```bash
-yq eval '.a // "hello"' sample.yml
-```
-will output
-```yaml
-hello
-```
-
-## LHS is null
-Given a sample.yml file of:
-```yaml
-a: ~
-```
-then
-```bash
-yq eval '.a // "hello"' sample.yml
-```
-will output
-```yaml
-hello
-```
-
-## LHS is false
-Given a sample.yml file of:
-```yaml
-a: false
-```
-then
-```bash
-yq eval '.a // "hello"' sample.yml
-```
-will output
-```yaml
-hello
-```
-
-## RHS is an expression
-Given a sample.yml file of:
-```yaml
-a: false
-b: cat
-```
-then
-```bash
-yq eval '.a // .b' sample.yml
-```
-will output
-```yaml
-cat
-```
-

pkg/yqlib/doc/Delete.md

@@ -14,23 +14,6 @@ will output
 a: cat
 ```
 
-## Delete nested entry in map
-Given a sample.yml file of:
-```yaml
-a:
-  a1: fred
-  a2: frood
-```
-then
-```bash
-yq eval 'del(.a.a1)' sample.yml
-```
-will output
-```yaml
-a:
-  a2: frood
-```
-
 ## Delete entry in array
 Given a sample.yml file of:
 ```yaml
@@ -48,21 +31,6 @@ will output
 - 3
 ```
 
-## Delete nested entry in array
-Given a sample.yml file of:
-```yaml
-- a: cat
-  b: dog
-```
-then
-```bash
-yq eval 'del(.[0].a)' sample.yml
-```
-will output
-```yaml
-- b: dog
-```
-
 ## Delete no matches
 Given a sample.yml file of:
 ```yaml

pkg/yqlib/doc/Explode.md

@@ -1,67 +1,4 @@
-Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference aliases and remove anchor names).
+Explodes (or dereferences) aliases and anchors.
-
-`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag.
-
-
-## Get anchor
-Given a sample.yml file of:
-```yaml
-a: &billyBob cat
-```
-then
-```bash
-yq eval '.a | anchor' sample.yml
-```
-will output
-```yaml
-billyBob
-```
-
-## Set anchor
-Given a sample.yml file of:
-```yaml
-a: cat
-```
-then
-```bash
-yq eval '.a anchor = "foobar"' sample.yml
-```
-will output
-```yaml
-a: &foobar cat
-```
-
-## Get alias
-Given a sample.yml file of:
-```yaml
-b: &billyBob meow
-a: *billyBob
-```
-then
-```bash
-yq eval '.a | alias' sample.yml
-```
-will output
-```yaml
-billyBob
-```
-
-## Set alias
-Given a sample.yml file of:
-```yaml
-b: &meow purr
-a: cat
-```
-then
-```bash
-yq eval '.a alias = "meow"' sample.yml
-```
-will output
-```yaml
-b: &meow purr
-a: *meow
-```
-
 ## Explode alias and anchor
 Given a sample.yml file of:
 ```yaml

pkg/yqlib/doc/headers/Alternative (Default value).md

@@ -1 +0,0 @@
-This operator is used to provide alternative (or default) values when a particular expression is either null or false.

pkg/yqlib/doc/headers/Anchor and Aliases Operators.md

@@ -1,4 +0,0 @@
-Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference aliases and remove anchor names).
-
-`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag.
-

pkg/yqlib/doc/headers/Explode.md

@@ -0,0 +1 @@
+Explodes (or dereferences) aliases and anchors.

pkg/yqlib/lib.go

@@ -36,12 +36,9 @@ var AssignAttributes = &OperationType{Type: "ASSIGN_ATTRIBUTES", NumArgs: 2, Pre
 var AssignStyle = &OperationType{Type: "ASSIGN_STYLE", NumArgs: 2, Precedence: 40, Handler: AssignStyleOperator}
 var AssignTag = &OperationType{Type: "ASSIGN_TAG", NumArgs: 2, Precedence: 40, Handler: AssignTagOperator}
 var AssignComment = &OperationType{Type: "ASSIGN_COMMENT", NumArgs: 2, Precedence: 40, Handler: AssignCommentsOperator}
-var AssignAnchor = &OperationType{Type: "ASSIGN_ANCHOR", NumArgs: 2, Precedence: 40, Handler: AssignAnchorOperator}
-var AssignAlias = &OperationType{Type: "ASSIGN_ALIAS", NumArgs: 2, Precedence: 40, Handler: AssignAliasOperator}
 
 var Multiply = &OperationType{Type: "MULTIPLY", NumArgs: 2, Precedence: 45, Handler: MultiplyOperator}
 var Add = &OperationType{Type: "ADD", NumArgs: 2, Precedence: 45, Handler: AddOperator}
-var Alternative = &OperationType{Type: "ALTERNATIVE", NumArgs: 2, Precedence: 45, Handler: AlternativeOperator}
 
 var Equals = &OperationType{Type: "EQUALS", NumArgs: 2, Precedence: 40, Handler: EqualsOperator}
 var CreateMap = &OperationType{Type: "CREATE_MAP", NumArgs: 2, Precedence: 40, Handler: CreateMapOperator}
@@ -53,8 +50,6 @@ var Collect = &OperationType{Type: "COLLECT", NumArgs: 0, Precedence: 50, Handle
 var GetStyle = &OperationType{Type: "GET_STYLE", NumArgs: 0, Precedence: 50, Handler: GetStyleOperator}
 var GetTag = &OperationType{Type: "GET_TAG", NumArgs: 0, Precedence: 50, Handler: GetTagOperator}
 var GetComment = &OperationType{Type: "GET_COMMENT", NumArgs: 0, Precedence: 50, Handler: GetCommentsOperator}
-var GetAnchor = &OperationType{Type: "GET_ANCHOR", NumArgs: 0, Precedence: 50, Handler: GetAnchorOperator}
-var GetAlias = &OperationType{Type: "GET_ALIAS", NumArgs: 0, Precedence: 50, Handler: GetAliasOperator}
 var GetDocumentIndex = &OperationType{Type: "GET_DOCUMENT_INDEX", NumArgs: 0, Precedence: 50, Handler: GetDocumentIndexOperator}
 var GetFilename = &OperationType{Type: "GET_FILENAME", NumArgs: 0, Precedence: 50, Handler: GetFilenameOperator}
 var GetFileIndex = &OperationType{Type: "GET_FILE_INDEX", NumArgs: 0, Precedence: 50, Handler: GetFileIndexOperator}
@@ -77,7 +72,6 @@ var RecursiveDescent = &OperationType{Type: "RECURSIVE_DESCENT", NumArgs: 0, Pre
 var Select = &OperationType{Type: "SELECT", NumArgs: 1, Precedence: 50, Handler: SelectOperator}
 var Has = &OperationType{Type: "HAS", NumArgs: 1, Precedence: 50, Handler: HasOperator}
 var DeleteChild = &OperationType{Type: "DELETE", NumArgs: 1, Precedence: 40, Handler: DeleteChildOperator}
-var DeleteImmediateChild = &OperationType{Type: "DELETE_IMMEDIATE_CHILD", NumArgs: 1, Precedence: 40, Handler: DeleteImmediateChildOperator}
 
 type Operation struct {
 	OperationType *OperationType

pkg/yqlib/operator_add.go

@@ -22,7 +22,13 @@ func AddAssignOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode
 	return d.GetMatchingNodes(matchingNodes, assignmentOpNode)
 }
 
-func toNodes(candidate *CandidateNode) []*yaml.Node {
+func toNodes(candidates *list.List) []*yaml.Node {
+
+	if candidates.Len() == 0 {
+		return []*yaml.Node{}
+	}
+	candidate := candidates.Front().Value.(*CandidateNode)
+
 	if candidate.Node.Tag == "!!null" {
 		return []*yaml.Node{}
 	}
@@ -38,33 +44,40 @@ func toNodes(candidate *CandidateNode) []*yaml.Node {
 
 func AddOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
 	log.Debugf("Add operator")
-
+	var results = list.New()
-	return crossFunction(d, matchingNodes, pathNode, add)
+	lhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Lhs)
-}
+	if err != nil {
-
+		return nil, err
-func add(d *dataTreeNavigator, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
-	lhs.Node = UnwrapDoc(lhs.Node)
-	rhs.Node = UnwrapDoc(rhs.Node)
-
-	target := &CandidateNode{
-		Path:     lhs.Path,
-		Document: lhs.Document,
-		Filename: lhs.Filename,
-		Node:     &yaml.Node{},
 	}
-	lhsNode := lhs.Node
+	rhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Rhs)
 
-	switch lhsNode.Kind {
+	if err != nil {
-	case yaml.MappingNode:
+		return nil, err
-		return nil, fmt.Errorf("Maps not yet supported for addition")
-	case yaml.SequenceNode:
-		target.Node.Kind = yaml.SequenceNode
-		target.Node.Style = lhsNode.Style
-		target.Node.Tag = "!!seq"
-		target.Node.Content = append(lhsNode.Content, toNodes(rhs)...)
-	case yaml.ScalarNode:
-		return nil, fmt.Errorf("Scalars not yet supported for addition")
 	}
 
-	return target, nil
+	for el := lhs.Front(); el != nil; el = el.Next() {
+		lhsCandidate := el.Value.(*CandidateNode)
+		lhsNode := UnwrapDoc(lhsCandidate.Node)
+
+		target := &CandidateNode{
+			Path:     lhsCandidate.Path,
+			Document: lhsCandidate.Document,
+			Filename: lhsCandidate.Filename,
+			Node:     &yaml.Node{},
+		}
+
+		switch lhsNode.Kind {
+		case yaml.MappingNode:
+			return nil, fmt.Errorf("Maps not yet supported for addition")
+		case yaml.SequenceNode:
+			target.Node.Kind = yaml.SequenceNode
+			target.Node.Style = lhsNode.Style
+			target.Node.Tag = "!!seq"
+			target.Node.Content = append(lhsNode.Content, toNodes(rhs)...)
+			results.PushBack(target)
+		case yaml.ScalarNode:
+			return nil, fmt.Errorf("Scalars not yet supported for addition")
+		}
+	}
+	return results, nil
 }

pkg/yqlib/operator_add_test.go

@@ -21,14 +21,6 @@ var addOperatorScenarios = []expressionScenario{
 			"D0, P[a], (!!seq)::[1, 2, 3, 4]\n",
 		},
 	},
-	{
-		skipDoc:    true,
-		expression: `[1] + ([2], [3])`,
-		expected: []string{
-			"D0, P[], (!!seq)::- 1\n- 2\n",
-			"D0, P[], (!!seq)::- 1\n- 3\n",
-		},
-	},
 	{
 		description: "Concatenate null to array",
 		document:    `{a: [1,2]}`,

pkg/yqlib/operator_alternative.go

@@ -1,28 +0,0 @@
-package yqlib
-
-import (
-	"container/list"
-)
-
-// corssFunction no matches
-// can boolean use crossfunction
-
-func AlternativeOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
-	log.Debugf("-- alternative")
-	return crossFunction(d, matchingNodes, pathNode, alternativeFunc)
-}
-
-func alternativeFunc(d *dataTreeNavigator, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
-	lhs.Node = UnwrapDoc(lhs.Node)
-	rhs.Node = UnwrapDoc(rhs.Node)
-	log.Debugf("Alternative LHS: %v", lhs.Node.Tag)
-	log.Debugf("-          RHS: %v", rhs.Node.Tag)
-
-	isTrue, err := isTruthy(lhs)
-	if err != nil {
-		return nil, err
-	} else if isTrue {
-		return lhs, nil
-	}
-	return rhs, nil
-}

pkg/yqlib/operator_alternative_test.go

@@ -1,62 +0,0 @@
-package yqlib
-
-import (
-	"testing"
-)
-
-var alternativeOperatorScenarios = []expressionScenario{
-	{
-		description: "LHS is defined",
-		expression:  `.a // "hello"`,
-		document:    `{a: bridge}`,
-		expected: []string{
-			"D0, P[a], (!!str)::bridge\n",
-		},
-	},
-	{
-		description: "LHS is not defined",
-		expression:  `.a // "hello"`,
-		document:    `{}`,
-		expected: []string{
-			"D0, P[], (!!str)::hello\n",
-		},
-	},
-	{
-		description: "LHS is null",
-		expression:  `.a // "hello"`,
-		document:    `{a: ~}`,
-		expected: []string{
-			"D0, P[], (!!str)::hello\n",
-		},
-	},
-	{
-		description: "LHS is false",
-		expression:  `.a // "hello"`,
-		document:    `{a: false}`,
-		expected: []string{
-			"D0, P[], (!!str)::hello\n",
-		},
-	},
-	{
-		description: "RHS is an expression",
-		expression:  `.a // .b`,
-		document:    `{a: false, b: cat}`,
-		expected: []string{
-			"D0, P[b], (!!str)::cat\n",
-		},
-	},
-	{
-		skipDoc:    true,
-		expression: `false // true`,
-		expected: []string{
-			"D0, P[], (!!bool)::true\n",
-		},
-	},
-}
-
-func TestAlternativeOperatorScenarios(t *testing.T) {
-	for _, tt := range alternativeOperatorScenarios {
-		testScenario(t, &tt)
-	}
-	documentScenarios(t, "Alternative (Default value)", alternativeOperatorScenarios)
-}

pkg/yqlib/operator_booleans.go

@@ -25,39 +25,74 @@ func isTruthy(c *CandidateNode) (bool, error) {
 
 type boolOp func(bool, bool) bool
 
-func performBoolOp(op boolOp) func(d *dataTreeNavigator, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
+func performBoolOp(results *list.List, lhs *list.List, rhs *list.List, op boolOp) error {
-	return func(d *dataTreeNavigator, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
+	for lhsChild := lhs.Front(); lhsChild != nil; lhsChild = lhsChild.Next() {
-		lhs.Node = UnwrapDoc(lhs.Node)
+		lhsCandidate := lhsChild.Value.(*CandidateNode)
-		rhs.Node = UnwrapDoc(rhs.Node)
+		lhsTrue, errDecoding := isTruthy(lhsCandidate)
-
-		lhsTrue, errDecoding := isTruthy(lhs)
 		if errDecoding != nil {
-			return nil, errDecoding
+			return errDecoding
 		}
 
-		rhsTrue, errDecoding := isTruthy(rhs)
+		for rhsChild := rhs.Front(); rhsChild != nil; rhsChild = rhsChild.Next() {
-		if errDecoding != nil {
+			rhsCandidate := rhsChild.Value.(*CandidateNode)
-			return nil, errDecoding
+			rhsTrue, errDecoding := isTruthy(rhsCandidate)
+			if errDecoding != nil {
+				return errDecoding
+			}
+			boolResult := createBooleanCandidate(lhsCandidate, op(lhsTrue, rhsTrue))
+			results.PushBack(boolResult)
 		}
-
-		return createBooleanCandidate(lhs, op(lhsTrue, rhsTrue)), nil
 	}
+	return nil
+}
+
+func booleanOp(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode, op boolOp) (*list.List, error) {
+	var results = list.New()
+
+	if matchingNodes.Len() == 0 {
+		lhs, err := d.GetMatchingNodes(list.New(), pathNode.Lhs)
+		if err != nil {
+			return nil, err
+		}
+		rhs, err := d.GetMatchingNodes(list.New(), pathNode.Rhs)
+		if err != nil {
+			return nil, err
+		}
+		return results, performBoolOp(results, lhs, rhs, op)
+	}
+
+	for el := matchingNodes.Front(); el != nil; el = el.Next() {
+		candidate := el.Value.(*CandidateNode)
+		lhs, err := d.GetMatchingNodes(nodeToMap(candidate), pathNode.Lhs)
+		if err != nil {
+			return nil, err
+		}
+		rhs, err := d.GetMatchingNodes(nodeToMap(candidate), pathNode.Rhs)
+		if err != nil {
+			return nil, err
+		}
+
+		err = performBoolOp(results, lhs, rhs, op)
+		if err != nil {
+			return nil, err
+		}
+
+	}
+	return results, nil
 }
 
 func OrOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
 	log.Debugf("-- orOp")
-	return crossFunction(d, matchingNodes, pathNode, performBoolOp(
+	return booleanOp(d, matchingNodes, pathNode, func(b1 bool, b2 bool) bool {
-		func(b1 bool, b2 bool) bool {
+		return b1 || b2
-			return b1 || b2
+	})
-		}))
 }
 
 func AndOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
 	log.Debugf("-- AndOp")
-	return crossFunction(d, matchingNodes, pathNode, performBoolOp(
+	return booleanOp(d, matchingNodes, pathNode, func(b1 bool, b2 bool) bool {
-		func(b1 bool, b2 bool) bool {
+		return b1 && b2
-			return b1 && b2
+	})
-		}))
 }
 
 func NotOperator(d *dataTreeNavigator, matchMap *list.List, pathNode *PathTreeNode) (*list.List, error) {

pkg/yqlib/operator_delete.go

@@ -2,67 +2,39 @@ package yqlib
 
 import (
 	"container/list"
-	"fmt"
 
 	yaml "gopkg.in/yaml.v3"
 )
 
 func DeleteChildOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
+	// for each lhs, splat the node,
+	// the intersect it against the rhs expression
+	// recreate the contents using only the intersection result.
 
-	nodesToDelete, err := d.GetMatchingNodes(matchingNodes, pathNode.Rhs)
+	for el := matchingNodes.Front(); el != nil; el = el.Next() {
-
-	if err != nil {
-		return nil, err
-	}
-
-	for el := nodesToDelete.Front(); el != nil; el = el.Next() {
 		candidate := el.Value.(*CandidateNode)
-
+		elMap := list.New()
-		deleteImmediateChildOp := &Operation{
+		elMap.PushBack(candidate)
-			OperationType: DeleteImmediateChild,
+		nodesToDelete, err := d.GetMatchingNodes(elMap, pathNode.Rhs)
-			Value:         candidate.Path[len(candidate.Path)-1],
+		log.Debug("nodesToDelete:\n%v", NodesToString(nodesToDelete))
-		}
-
-		deleteImmediateChildOpNode := &PathTreeNode{
-			Operation: deleteImmediateChildOp,
-			Rhs:       createTraversalTree(candidate.Path[0 : len(candidate.Path)-1]),
-		}
-
-		_, err := d.GetMatchingNodes(matchingNodes, deleteImmediateChildOpNode)
 		if err != nil {
 			return nil, err
 		}
-	}
-	return matchingNodes, nil
-}
 
-func DeleteImmediateChildOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
+		realNode := UnwrapDoc(candidate.Node)
-	parents, err := d.GetMatchingNodes(matchingNodes, pathNode.Rhs)
 
-	if err != nil {
+		if realNode.Kind == yaml.SequenceNode {
-		return nil, err
+			deleteFromArray(candidate, nodesToDelete)
-	}
+		} else if realNode.Kind == yaml.MappingNode {
-
+			deleteFromMap(candidate, nodesToDelete)
-	childPath := pathNode.Operation.Value
-
-	log.Debug("childPath to remove %v", childPath)
-
-	for el := parents.Front(); el != nil; el = el.Next() {
-		parent := el.Value.(*CandidateNode)
-		parentNode := UnwrapDoc(parent.Node)
-		if parentNode.Kind == yaml.MappingNode {
-			deleteFromMap(parent, childPath)
-		} else if parentNode.Kind == yaml.SequenceNode {
-			deleteFromArray(parent, childPath)
 		} else {
-			return nil, fmt.Errorf("Cannot delete nodes from parent of tag %v", parentNode.Tag)
+			log.Debug("Cannot delete from node that's not a map or array %v", NodeToString(candidate))
 		}
-
 	}
 	return matchingNodes, nil
 }
 
-func deleteFromMap(candidate *CandidateNode, childPath interface{}) {
+func deleteFromMap(candidate *CandidateNode, nodesToDelete *list.List) {
 	log.Debug("deleteFromMap")
 	node := UnwrapDoc(candidate.Node)
 	contents := node.Content
@@ -78,7 +50,12 @@ func deleteFromMap(candidate *CandidateNode, childPath interface{}) {
 			Path:     append(candidate.Path, key.Value),
 		}
 
-		shouldDelete := key.Value == childPath
+		shouldDelete := false
+		for el := nodesToDelete.Front(); el != nil && !shouldDelete; el = el.Next() {
+			if el.Value.(*CandidateNode).GetKey() == childCandidate.GetKey() {
+				shouldDelete = true
+			}
+		}
 
 		log.Debugf("shouldDelete %v ? %v", childCandidate.GetKey(), shouldDelete)
 
@@ -89,7 +66,7 @@ func deleteFromMap(candidate *CandidateNode, childPath interface{}) {
 	node.Content = newContents
 }
 
-func deleteFromArray(candidate *CandidateNode, childPath interface{}) {
+func deleteFromArray(candidate *CandidateNode, nodesToDelete *list.List) {
 	log.Debug("deleteFromArray")
 	node := UnwrapDoc(candidate.Node)
 	contents := node.Content
@@ -98,7 +75,18 @@ func deleteFromArray(candidate *CandidateNode, childPath interface{}) {
 	for index := 0; index < len(contents); index = index + 1 {
 		value := contents[index]
 
-		shouldDelete := fmt.Sprintf("%v", index) == fmt.Sprintf("%v", childPath)
+		childCandidate := &CandidateNode{
+			Node:     value,
+			Document: candidate.Document,
+			Path:     append(candidate.Path, index),
+		}
+
+		shouldDelete := false
+		for el := nodesToDelete.Front(); el != nil && !shouldDelete; el = el.Next() {
+			if el.Value.(*CandidateNode).GetKey() == childCandidate.GetKey() {
+				shouldDelete = true
+			}
+		}
 
 		if !shouldDelete {
 			newContents = append(newContents, value)

pkg/yqlib/operator_delete_test.go

@@ -13,22 +13,6 @@ var deleteOperatorScenarios = []expressionScenario{
 			"D0, P[], (doc)::{a: cat}\n",
 		},
 	},
-	{
-		description: "Delete nested entry in map",
-		document:    `{a: {a1: fred, a2: frood}}`,
-		expression:  `del(.a.a1)`,
-		expected: []string{
-			"D0, P[], (doc)::{a: {a2: frood}}\n",
-		},
-	},
-	{
-		skipDoc:    true,
-		document:   `{a: {a1: fred, a2: frood}}`,
-		expression: `del(.. | select(.=="frood"))`,
-		expected: []string{
-			"D0, P[], (!!map)::{a: {a1: fred}}\n",
-		},
-	},
 	{
 		description: "Delete entry in array",
 		document:    `[1,2,3]`,
@@ -37,14 +21,6 @@ var deleteOperatorScenarios = []expressionScenario{
 			"D0, P[], (doc)::[1, 3]\n",
 		},
 	},
-	{
-		description: "Delete nested entry in array",
-		document:    `[{a: cat, b: dog}]`,
-		expression:  `del(.[0].a)`,
-		expected: []string{
-			"D0, P[], (doc)::[{b: dog}]\n",
-		},
-	},
 	{
 		description: "Delete no matches",
 		document:    `{a: cat, b: dog}`,

pkg/yqlib/operator_explode.go

@@ -6,88 +6,6 @@ import (
 	"gopkg.in/yaml.v3"
 )
 
-func AssignAliasOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
-
-	log.Debugf("AssignAlias operator!")
-
-	rhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Rhs)
-	if err != nil {
-		return nil, err
-	}
-	aliasName := ""
-	if rhs.Front() != nil {
-		aliasName = rhs.Front().Value.(*CandidateNode).Node.Value
-	}
-
-	lhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Lhs)
-
-	if err != nil {
-		return nil, err
-	}
-
-	for el := lhs.Front(); el != nil; el = el.Next() {
-		candidate := el.Value.(*CandidateNode)
-		log.Debugf("Setting aliasName : %v", candidate.GetKey())
-		candidate.Node.Kind = yaml.AliasNode
-		candidate.Node.Value = aliasName
-	}
-	return matchingNodes, nil
-}
-
-func GetAliasOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
-	log.Debugf("GetAlias operator!")
-	var results = list.New()
-
-	for el := matchingNodes.Front(); el != nil; el = el.Next() {
-		candidate := el.Value.(*CandidateNode)
-		node := &yaml.Node{Kind: yaml.ScalarNode, Value: candidate.Node.Value, Tag: "!!str"}
-		lengthCand := &CandidateNode{Node: node, Document: candidate.Document, Path: candidate.Path}
-		results.PushBack(lengthCand)
-	}
-	return results, nil
-}
-
-func AssignAnchorOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
-
-	log.Debugf("AssignAnchor operator!")
-
-	rhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Rhs)
-	if err != nil {
-		return nil, err
-	}
-	anchorName := ""
-	if rhs.Front() != nil {
-		anchorName = rhs.Front().Value.(*CandidateNode).Node.Value
-	}
-
-	lhs, err := d.GetMatchingNodes(matchingNodes, pathNode.Lhs)
-
-	if err != nil {
-		return nil, err
-	}
-
-	for el := lhs.Front(); el != nil; el = el.Next() {
-		candidate := el.Value.(*CandidateNode)
-		log.Debugf("Setting anchorName of : %v", candidate.GetKey())
-		candidate.Node.Anchor = anchorName
-	}
-	return matchingNodes, nil
-}
-
-func GetAnchorOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNode *PathTreeNode) (*list.List, error) {
-	log.Debugf("GetAnchor operator!")
-	var results = list.New()
-
-	for el := matchingNodes.Front(); el != nil; el = el.Next() {
-		candidate := el.Value.(*CandidateNode)
-		anchor := candidate.Node.Anchor
-		node := &yaml.Node{Kind: yaml.ScalarNode, Value: anchor, Tag: "!!str"}
-		lengthCand := &CandidateNode{Node: node, Document: candidate.Document, Path: candidate.Path}
-		results.PushBack(lengthCand)
-	}
-	return results, nil
-}
-
 func ExplodeOperator(d *dataTreeNavigator, matchMap *list.List, pathNode *PathTreeNode) (*list.List, error) {
 	log.Debugf("-- ExplodeOperation")
 

pkg/yqlib/operator_explode_test.go

@@ -4,39 +4,7 @@ import (
 	"testing"
 )
 
-var anchorOperatorScenarios = []expressionScenario{
+var explodeTest = []expressionScenario{
-	{
-		description: "Get anchor",
-		document:    `a: &billyBob cat`,
-		expression:  `.a | anchor`,
-		expected: []string{
-			"D0, P[a], (!!str)::billyBob\n",
-		},
-	},
-	{
-		description: "Set anchor",
-		document:    `a: cat`,
-		expression:  `.a anchor = "foobar"`,
-		expected: []string{
-			"D0, P[], (doc)::a: &foobar cat\n",
-		},
-	},
-	{
-		description: "Get alias",
-		document:    `{b: &billyBob meow, a: *billyBob}`,
-		expression:  `.a | alias`,
-		expected: []string{
-			"D0, P[a], (!!str)::billyBob\n",
-		},
-	},
-	{
-		description: "Set alias",
-		document:    `{b: &meow purr, a: cat}`,
-		expression:  `.a alias = "meow"`,
-		expected: []string{
-			"D0, P[], (doc)::{b: &meow purr, a: *meow}\n",
-		},
-	},
 	{
 		description: "Explode alias and anchor",
 		document:    `{f : {a: &a cat, b: *a}}`,
@@ -114,9 +82,9 @@ foobar:
 	},
 }
 
-func TestAnchorAliaseOperatorScenarios(t *testing.T) {
+func TestExplodeOperatorScenarios(t *testing.T) {
-	for _, tt := range anchorOperatorScenarios {
+	for _, tt := range explodeTest {
 		testScenario(t, &tt)
 	}
-	documentScenarios(t, "Anchor and Aliases Operators", anchorOperatorScenarios)
+	documentScenarios(t, "Explode", explodeTest)
 }

pkg/yqlib/operator_multiply.go

@@ -104,6 +104,19 @@ func mergeObjects(d *dataTreeNavigator, lhs *CandidateNode, rhs *CandidateNode,
 	return lhs, nil
 }
 
+func createTraversalTree(path []interface{}) *PathTreeNode {
+	if len(path) == 0 {
+		return &PathTreeNode{Operation: &Operation{OperationType: SelfReference}}
+	} else if len(path) == 1 {
+		return &PathTreeNode{Operation: &Operation{OperationType: TraversePath, Value: path[0], StringValue: fmt.Sprintf("%v", path[0])}}
+	}
+	return &PathTreeNode{
+		Operation: &Operation{OperationType: ShortPipe},
+		Lhs:       createTraversalTree(path[0:1]),
+		Rhs:       createTraversalTree(path[1:])}
+
+}
+
 func applyAssignment(d *dataTreeNavigator, pathIndexToStartFrom int, lhs *CandidateNode, rhs *CandidateNode, shouldAppendArrays bool) error {
 
 	log.Debugf("merge - applyAssignment lhs %v, rhs: %v", NodeToString(lhs), NodeToString(rhs))

pkg/yqlib/operator_value_test.go

@@ -36,22 +36,13 @@ var valueOperatorScenarios = []expressionScenario{
 		expected: []string{
 			"D0, P[], (!!float)::5e-10\n",
 		},
-	},
+	}, {
-	{
 		document:   ``,
 		expression: `"cat"`,
 		expected: []string{
 			"D0, P[], (!!str)::cat\n",
 		},
-	},
+	}, {
-	{
-		document:   ``,
-		expression: `"frog jumps"`,
-		expected: []string{
-			"D0, P[], (!!str)::frog jumps\n",
-		},
-	},
-	{
 		document:   ``,
 		expression: `"1.3"`,
 		expected: []string{

pkg/yqlib/operators.go

@@ -2,7 +2,6 @@ package yqlib
 
 import (
 	"container/list"
-	"fmt"
 
 	"gopkg.in/yaml.v3"
 )
@@ -34,15 +33,3 @@ func nodeToMap(candidate *CandidateNode) *list.List {
 	elMap.PushBack(candidate)
 	return elMap
 }
-
-func createTraversalTree(path []interface{}) *PathTreeNode {
-	if len(path) == 0 {
-		return &PathTreeNode{Operation: &Operation{OperationType: SelfReference}}
-	} else if len(path) == 1 {
-		return &PathTreeNode{Operation: &Operation{OperationType: TraversePath, Value: path[0], StringValue: fmt.Sprintf("%v", path[0])}}
-	}
-	return &PathTreeNode{
-		Operation: &Operation{OperationType: ShortPipe},
-		Lhs:       createTraversalTree(path[0:1]),
-		Rhs:       createTraversalTree(path[1:])}
-}

pkg/yqlib/path_tokeniser.go

@@ -1,7 +1,6 @@
 package yqlib
 
 import (
-	"fmt"
 	"strconv"
 
 	lex "github.com/timtadh/lexmachine"
@@ -201,15 +200,12 @@ func initLexer() (*lex.Lexer, error) {
 	lexer.Add([]byte(`or`), opToken(Or))
 	lexer.Add([]byte(`and`), opToken(And))
 	lexer.Add([]byte(`not`), opToken(Not))
-	lexer.Add([]byte(`\/\/`), opToken(Alternative))
 
 	lexer.Add([]byte(`documentIndex`), opToken(GetDocumentIndex))
 
 	lexer.Add([]byte(`style`), opAssignableToken(GetStyle, AssignStyle))
 
 	lexer.Add([]byte(`tag`), opAssignableToken(GetTag, AssignTag))
-	lexer.Add([]byte(`anchor`), opAssignableToken(GetAnchor, AssignAnchor))
-	lexer.Add([]byte(`alias`), opAssignableToken(GetAlias, AssignAlias))
 	lexer.Add([]byte(`filename`), opToken(GetFilename))
 	lexer.Add([]byte(`fileIndex`), opToken(GetFileIndex))
 	lexer.Add([]byte(`path`), opToken(GetPath))
@@ -252,7 +248,7 @@ func initLexer() (*lex.Lexer, error) {
 	lexer.Add([]byte(`[Nn][Uu][Ll][Ll]`), nullValue())
 	lexer.Add([]byte(`~`), nullValue())
 
-	lexer.Add([]byte(`"[^"]*"`), stringValue(true))
+	lexer.Add([]byte(`"[^ "]*"`), stringValue(true))
 
 	lexer.Add([]byte(`\[\]`), literalToken(SplatOrEmptyCollect, true))
 
@@ -292,7 +288,7 @@ func (p *pathTokeniser) Tokenise(path string) ([]*Token, error) {
 	scanner, err := p.lexer.Scanner([]byte(path))
 
 	if err != nil {
-		return nil, fmt.Errorf("Parsing expression: %v", err)
+		return nil, err
 	}
 	var tokens []*Token
 	for tok, err, eof := scanner.Next(); !eof; tok, err, eof = scanner.Next() {
@@ -303,7 +299,7 @@ func (p *pathTokeniser) Tokenise(path string) ([]*Token, error) {
 			tokens = append(tokens, token)
 		}
 		if err != nil {
-			return nil, fmt.Errorf("Parsing expression: %v", err)
+			return nil, err
 		}
 	}
 	var postProcessedTokens = make([]*Token, 0)

pkg/yqlib/path_tree.go

@@ -1,9 +1,6 @@
 package yqlib
 
-import (
+import "fmt"
-	"fmt"
-	"strings"
-)
 
 var myPathTokeniser = NewPathTokeniser()
 var myPathPostfixer = NewPathPostFixer()
@@ -52,16 +49,10 @@ func (p *pathTreeCreator) CreatePathTree(postFixPath []*Operation) (*PathTreeNod
 		if Operation.OperationType.NumArgs > 0 {
 			numArgs := Operation.OperationType.NumArgs
 			if numArgs == 1 {
-				if len(stack) < 1 {
-					return nil, fmt.Errorf("'%v' expects 1 arg but received none", strings.TrimSpace(Operation.StringValue))
-				}
 				remaining, rhs := stack[:len(stack)-1], stack[len(stack)-1]
 				newNode.Rhs = rhs
 				stack = remaining
 			} else if numArgs == 2 {
-				if len(stack) < 2 {
-					return nil, fmt.Errorf("'%v' expects 2 args but there is %v", strings.TrimSpace(Operation.StringValue), len(stack))
-				}
 				remaining, lhs, rhs := stack[:len(stack)-2], stack[len(stack)-2], stack[len(stack)-1]
 				newNode.Lhs = lhs
 				newNode.Rhs = rhs
@@ -71,7 +62,7 @@ func (p *pathTreeCreator) CreatePathTree(postFixPath []*Operation) (*PathTreeNod
 		stack = append(stack, &newNode)
 	}
 	if len(stack) != 1 {
-		return nil, fmt.Errorf("expected end of expression but found '%v', please check expression syntax", strings.TrimSpace(stack[1].Operation.StringValue))
+		return nil, fmt.Errorf("expected stack to have 1 thing but its %v", stack)
 	}
 	return stack[0], nil
 }

pkg/yqlib/path_tree_test.go

@@ -1,42 +0,0 @@
-package yqlib
-
-import (
-	"testing"
-
-	"github.com/mikefarah/yq/v4/test"
-)
-
-func TestPathTreeNoArgsForTwoArgOp(t *testing.T) {
-	_, err := treeCreator.ParsePath("=")
-	test.AssertResultComplex(t, "'=' expects 2 args but there is 0", err.Error())
-}
-
-func TestPathTreeOneLhsArgsForTwoArgOp(t *testing.T) {
-	_, err := treeCreator.ParsePath(".a =")
-	test.AssertResultComplex(t, "'=' expects 2 args but there is 1", err.Error())
-}
-
-func TestPathTreeOneRhsArgsForTwoArgOp(t *testing.T) {
-	_, err := treeCreator.ParsePath("= .a")
-	test.AssertResultComplex(t, "'=' expects 2 args but there is 1", err.Error())
-}
-
-func TestPathTreeTwoArgsForTwoArgOp(t *testing.T) {
-	_, err := treeCreator.ParsePath(".a = .b")
-	test.AssertResultComplex(t, nil, err)
-}
-
-func TestPathTreeNoArgsForOneArgOp(t *testing.T) {
-	_, err := treeCreator.ParsePath("explode")
-	test.AssertResultComplex(t, "'explode' expects 1 arg but received none", err.Error())
-}
-
-func TestPathTreeOneArgForOneArgOp(t *testing.T) {
-	_, err := treeCreator.ParsePath("explode(.)")
-	test.AssertResultComplex(t, nil, err)
-}
-
-func TestPathTreeExtraArgs(t *testing.T) {
-	_, err := treeCreator.ParsePath("sortKeys(.) explode(.)")
-	test.AssertResultComplex(t, "expected end of expression but found 'explode', please check expression syntax", err.Error())
-}

pkg/yqlib/printer.go

@@ -22,7 +22,6 @@ type resultsPrinter struct {
 	writer             io.Writer
 	firstTimePrinting  bool
 	previousDocIndex   uint
-	previousFileIndex  int
 	printedMatches     bool
 }
 
@@ -90,20 +89,19 @@ func (p *resultsPrinter) PrintResults(matchingNodes *list.List) error {
 		return nil
 	}
 	if p.firstTimePrinting {
-		node := matchingNodes.Front().Value.(*CandidateNode)
+		p.previousDocIndex = matchingNodes.Front().Value.(*CandidateNode).Document
-		p.previousDocIndex = node.Document
-		p.previousFileIndex = node.FileIndex
 		p.firstTimePrinting = false
 	}
 
 	for el := matchingNodes.Front(); el != nil; el = el.Next() {
 		mappedDoc := el.Value.(*CandidateNode)
 		log.Debug("-- print sep logic: p.firstTimePrinting: %v, previousDocIndex: %v, mappedDoc.Document: %v, printDocSeparators: %v", p.firstTimePrinting, p.previousDocIndex, mappedDoc.Document, p.printDocSeparators)
-		if (p.previousDocIndex != mappedDoc.Document || p.previousFileIndex != mappedDoc.FileIndex) && p.printDocSeparators {
+		if (p.previousDocIndex != mappedDoc.Document) && p.printDocSeparators {
 			log.Debug("-- writing doc sep")
 			if err := p.writeString(bufferedWriter, "---\n"); err != nil {
 				return err
 			}
+
 		}
 
 		if err := p.printNode(mappedDoc.Node, bufferedWriter); err != nil {

pkg/yqlib/printer_test.go

@@ -52,53 +52,7 @@ func TestPrinterMultipleDocsInSequence(t *testing.T) {
 
 	writer.Flush()
 	test.AssertResult(t, multiDocSample, output.String())
-}
 
-func TestPrinterMultipleFilesInSequence(t *testing.T) {
-	var output bytes.Buffer
-	var writer = bufio.NewWriter(&output)
-	printer := NewPrinter(writer, false, true, false, 2, true)
-
-	inputs, err := readDocuments(strings.NewReader(multiDocSample), "sample.yml", 0)
-	if err != nil {
-		panic(err)
-	}
-
-	el := inputs.Front()
-	elNode := el.Value.(*CandidateNode)
-	elNode.Document = 0
-	elNode.FileIndex = 0
-	sample1 := nodeToMap(elNode)
-
-	el = el.Next()
-	elNode = el.Value.(*CandidateNode)
-	elNode.Document = 0
-	elNode.FileIndex = 1
-	sample2 := nodeToMap(elNode)
-
-	el = el.Next()
-	elNode = el.Value.(*CandidateNode)
-	elNode.Document = 0
-	elNode.FileIndex = 2
-	sample3 := nodeToMap(elNode)
-
-	err = printer.PrintResults(sample1)
-	if err != nil {
-		panic(err)
-	}
-
-	err = printer.PrintResults(sample2)
-	if err != nil {
-		panic(err)
-	}
-
-	err = printer.PrintResults(sample3)
-	if err != nil {
-		panic(err)
-	}
-
-	writer.Flush()
-	test.AssertResult(t, multiDocSample, output.String())
 }
 
 func TestPrinterMultipleDocsInSinglePrint(t *testing.T) {

release_instructions.txt

@@ -4,7 +4,6 @@
 - tag git with same version number
 - make sure local build passes
 - push tag to git
-  - 3.4.0, v3
   - git push --tags
 - make local xcompile (builds binaries for all platforms)
 
@@ -31,7 +30,7 @@
 
 - docker
   - build and push latest and new version tag
-  - docker build .  -t mikefarah/yq:latest -t mikefarah/yq:3 -t mikefarah/yq:3.X
+  - docker build .  -t mikefarah/yq:latest -t mikefarah/yq:VERSION 
 
 - debian package
   - ensure you get all vendor dependencies before packaging

scripts/check.sh

@@ -3,7 +3,7 @@
 set -o errexit
 set -o pipefail
 
-./bin/golangci-lint run --timeout=5m
+./bin/golangci-lint run
 
 # ./bin/golangci-lint \
 #   --tests \

scripts/format.sh

@@ -2,4 +2,3 @@
 
 find . \( -path ./vendor \) -prune -o -name "*.go" -exec goimports -w {} \;
 go mod tidy
-go mod vendor

scripts/publish-docker.sh

@@ -6,7 +6,4 @@ docker build \
   --build-arg VERSION=${VERSION} \
   -t mikefarah/yq:latest \
   -t mikefarah/yq:${VERSION} \
-  -t mikefarah/yq:4 \
   .
-
-trivy image mikefarah/yq:${VERSION}

scripts/xcompile.sh

@@ -12,13 +12,3 @@ cd build
 rhash -r -a . -P -o checksums
 
 rhash --list-hashes > checksums_hashes_order
-
-find . | xargs -I {} tar czvf {}.tar.gz {}
-
-rm checksums_hashes_order.tar.gz
-rm checksums.tar.gz
-rm yq_windows_386.exe.tar.gz
-rm yq_windows_amd64.exe.tar.gz
-
-zip yq_windows_386.zip yq_windows_386.exe
-zip yq_windows_amd64.zip yq_windows_amd64.exe

snap/snapcraft.yaml

@@ -1,5 +1,5 @@
 name: yq
-version: '4.0.0'
+version: '4.0.0-beta2'
 summary: A lightweight and portable command-line YAML processor
 description: |
   The aim of the project is to be the jq or sed of yaml files.