wip

.github/ISSUE_TEMPLATE/bug_report_v3.md

@@ -10,8 +10,6 @@ assignees: ''
 **Describe the bug**
 A clear and concise description of what the bug is.
 
-Note that any how to questions should be posted in the discussion board and not raised as an issue.
-
 Version of yq: 3.X.X
 Operating system: mac/linux/windows/....
 Installed via: docker/binary release/homebrew/snap/...

.github/ISSUE_TEMPLATE/bug_report_v4.md

@@ -10,8 +10,6 @@ assignees: ''
 **Describe the bug**
 A clear and concise description of what the bug is.
 
-Note that any how to questions should be posted in the discussion board and not raised as an issue.
-
 Version of yq: 4.X.X
 Operating system: mac/linux/windows/....
 Installed via: docker/binary release/homebrew/snap/...

.github/ISSUE_TEMPLATE/feature_request.md

@@ -9,11 +9,9 @@ assignees: ''
 
 **Please describe your feature request.**
 A clear and concise description of what the request is and what it would solve. 
-Eg. I wish I could use yq to [...]
+Ex. I wish I could use yq to [...]
 
-Note:
+Please note that V3 will no longer have any enhancements.
-- how to questions should be posted in the discussion board and not raised as an issue.
-- V3 will no longer have any enhancements.
 
 **Describe the solution you'd like**
 If we have data1.yml like:

.github/workflows/release.yml

@@ -68,9 +68,7 @@ jobs:
  
       - name: Build and push image
         run: |
-          IMAGE_V_VERSION="$(git describe --tags --abbrev=0)"
+          IMAGE_VERSION="$(git describe --tags --abbrev=0)"
-          IMAGE_VERSION=${IMAGE_V_VERSION:1}
-
           SHORT_SHA1=$(git rev-parse --short HEAD)
           PLATFORMS="linux/amd64,linux/ppc64le,linux/arm64"
           echo "Building and pushing version ${IMAGE_VERSION} of image ${IMAGE_NAME}"

Dockerfile.dev

@@ -4,7 +4,6 @@ COPY scripts/devtools.sh /opt/devtools.sh
 
 RUN set -e -x \
     && /opt/devtools.sh
-ENV PATH=/go/bin:$PATH
 
 # install mkdocs
 RUN set -ex \

README.md

@@ -3,9 +3,9 @@
 ![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg)  ![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) ![Github Releases (by Release)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) ![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq)
 
 
-a lightweight and portable command-line YAML processor. `yq` uses [jq](https://github.com/stedolan/jq) like syntax but works with yaml files as well as json. It doesn't yet support everything `jq` does - but it does support the most common operations and functions, and more is being added continuously.
+a lightweight and portable command-line YAML processor
 
-yq is written in go - so you can download a dependency free binary for your platform and you are good to go! If you prefer there are a variety of package managers that can be used as well as docker, all listed below.
+The aim of the project is to be the [jq](https://github.com/stedolan/jq) or sed of yaml files.
 
 ## V4 released!
 V4 is now officially released, it's quite different from V3 (sorry for the migration), however it is much more similar to ```jq```, using a similar expression syntax and therefore support much more complex functionality! 
@@ -16,31 +16,13 @@ If you've been using v3 and want/need to upgrade, checkout the [upgrade guide](h
 
 ### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest)
 
-### wget
+### MacOS:
-Use wget to download the pre-compiled binaries:
-
-#### Compressed via tar.gz
-```bash
-wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\
-  tar xz && mv ${BINARY} /usr/bin/yq
-```
-
-#### Plain binary
-
-```bash
-wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq &&\
-    chmod +x /usr/bin/yq
-```
-
-For instance, VERSION=v4.2.0 and BINARY=yq_linux_amd64
-
-### MacOS / Linux via Homebrew:
 Using [Homebrew](https://brew.sh/)
 ```
 brew install yq
 ```
 
-### Linux via snap:
+### Ubuntu and other Linux distros supporting `snap` packages:
 ```
 snap install yq
 ```
@@ -63,6 +45,17 @@ sudo mv /etc/myfile.tmp /etc/myfile
 rm /etc/myfile.tmp
 ```
 
+### wget
+
+Use wget to download the pre-compiled binaries:
+
+```bash
+wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq &&\
+    chmod +x /usr/bin/yq
+```
+
+For instance, VERSION=4.0.0 and BINARY=yq_linux_amd64
+
 ### Run with Docker
 
 #### Oneshot use:

cmd/version.go

@@ -11,7 +11,7 @@ var (
 	GitDescribe string
 
 	// Version is main version number that is being run at the moment.
-	Version = "4.2.1"
+	Version = "4.2.0"
 
 	// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
 	// then it means that it is a final release. Otherwise, this is a pre-release

github-action/Dockerfile

@@ -1,4 +1,4 @@
-FROM mikefarah/yq:4.2.1
+FROM mikefarah/yq:3
 
 COPY entrypoint.sh /entrypoint.sh
 

pkg/yqlib/candidate_node.go

@@ -40,7 +40,6 @@ func (n *CandidateNode) Copy() (*CandidateNode, error) {
 
 // updates this candidate from the given candidate node
 func (n *CandidateNode) UpdateFrom(other *CandidateNode) {
-
 	n.UpdateAttributesFrom(other)
 	n.Node.Content = other.Node.Content
 	n.Node.Value = other.Node.Value

pkg/yqlib/doc/Assign.md

@@ -34,27 +34,6 @@ a:
   g: foof
 ```
 
-## Update node from another file
-Note this will also work when the second file is a scalar (string/number)
-
-Given a sample.yml file of:
-```yaml
-a: apples
-```
-And another sample another.yml file of:
-```yaml
-b: bob
-```
-then
-```bash
-yq eval-all 'select(fileIndex==0).a = select(fileIndex==1) | select(fileIndex==0)' sample.yml another.yml
-```
-will output
-```yaml
-a:
-  b: bob
-```
-
 ## Update node to be the sibling value
 Given a sample.yml file of:
 ```yaml

pkg/yqlib/doc/Document Index.md

@@ -26,7 +26,7 @@ a: frog
 ```
 then
 ```bash
-yq eval 'select(documentIndex == 1)' sample.yml
+yq eval 'select(. | documentIndex == 1)' sample.yml
 ```
 will output
 ```yaml
@@ -42,7 +42,7 @@ a: frog
 ```
 then
 ```bash
-yq eval '.a | ({"match": ., "doc": documentIndex})' sample.yml
+yq eval '.a | ({"match": ., "doc": (. | documentIndex)})' sample.yml
 ```
 will output
 ```yaml

pkg/yqlib/doc/File Operators.md

@@ -1,11 +1,9 @@
 File operators are most often used with merge when needing to merge specific files together. Note that when doing this, you will need to use `eval-all` to ensure all yaml documents are loaded into memory before performing the merge (as opposed to `eval` which runs the expression once per document).
 
-Note that the `fileIndex` operator has a short alias of `fi`.
-
 ## Merging files
 Note the use of eval-all to ensure all documents are loaded into memory.
 ```bash
-yq eval-all 'select(fi == 0) * select(filename == "file2.yaml")' file1.yaml file2.yaml
+yq eval-all 'select(fileIndex == 0) * select(filename == "file2.yaml")' file1.yaml file2.yaml
 ```
 ## Get filename
 Given a sample.yml file of:
@@ -18,7 +16,7 @@ yq eval 'filename' sample.yml
 ```
 will output
 ```yaml
-sample.yml
+sample.yaml
 ```
 
 ## Get file index
@@ -35,17 +33,3 @@ will output
 0
 ```
 
-## Get file index alias
-Given a sample.yml file of:
-```yaml
-a: cat
-```
-then
-```bash
-yq eval 'fi' sample.yml
-```
-will output
-```yaml
-0
-```
-

pkg/yqlib/doc/Keys.md

@@ -0,0 +1,38 @@
+
+## Get Keys
+Given a sample.yml file of:
+```yaml
+a:
+  b: cat
+  c: dog
+  d: frog
+```
+then
+```bash
+yq eval '.a | keys' sample.yml
+```
+will output
+```yaml
+3
+```
+
+## Set key style
+Given a sample.yml file of:
+```yaml
+a:
+  b: cat
+  c: dog
+  d: frog
+```
+then
+```bash
+yq eval '(.a | keys) style = 'single'' sample.yml
+```
+will output
+```yaml
+a:
+  b: cat
+  c: dog
+  d: frog
+```
+

pkg/yqlib/doc/Style.md

@@ -146,7 +146,7 @@ will output
 ```
 
 ## Pretty print
-Set empty (default) quote style, note the usage of `...` to match keys too. Note that there is a `--prettyPrint/-P` short flag for this.
+Set empty (default) quote style, note the usage of `...` to match keys too.
 
 Given a sample.yml file of:
 ```yaml

pkg/yqlib/doc/headers/File Operators.md

@@ -1,9 +1,7 @@
 File operators are most often used with merge when needing to merge specific files together. Note that when doing this, you will need to use `eval-all` to ensure all yaml documents are loaded into memory before performing the merge (as opposed to `eval` which runs the expression once per document).
 
-Note that the `fileIndex` operator has a short alias of `fi`.
-
 ## Merging files
 Note the use of eval-all to ensure all documents are loaded into memory.
 ```bash
-yq eval-all 'select(fi == 0) * select(filename == "file2.yaml")' file1.yaml file2.yaml
+yq eval-all 'select(fileIndex == 0) * select(filename == "file2.yaml")' file1.yaml file2.yaml
 ```

pkg/yqlib/lib.go

@@ -49,6 +49,7 @@ var CreateMap = &OperationType{Type: "CREATE_MAP", NumArgs: 2, Precedence: 40, H
 var ShortPipe = &OperationType{Type: "SHORT_PIPE", NumArgs: 2, Precedence: 45, Handler: PipeOperator}
 
 var Length = &OperationType{Type: "LENGTH", NumArgs: 0, Precedence: 50, Handler: LengthOperator}
+var Keys = &OperationType{Type: "KEYS", NumArgs: 0, Precedence: 50, Handler: KeysOperator}
 var Collect = &OperationType{Type: "COLLECT", NumArgs: 0, Precedence: 50, Handler: CollectOperator}
 var GetStyle = &OperationType{Type: "GET_STYLE", NumArgs: 0, Precedence: 50, Handler: GetStyleOperator}
 var GetTag = &OperationType{Type: "GET_TAG", NumArgs: 0, Precedence: 50, Handler: GetTagOperator}

pkg/yqlib/operator_assign.go

@@ -33,9 +33,7 @@ func AssignUpdateOperator(d *dataTreeNavigator, matchingNodes *list.List, pathNo
 		first := rhs.Front()
 
 		if first != nil {
-			rhsCandidate := first.Value.(*CandidateNode)
+			candidate.UpdateFrom(first.Value.(*CandidateNode))
-			rhsCandidate.Node = UnwrapDoc(rhsCandidate.Node)
-			candidate.UpdateFrom(rhsCandidate)
 		}
 	}
 	// // if there was nothing given, perhaps we are creating a new yaml doc

pkg/yqlib/operator_assign_test.go

@@ -20,16 +20,6 @@ var assignOperatorScenarios = []expressionScenario{
 			"D0, P[], (doc)::{a: {g: foof}}\n",
 		},
 	},
-	{
-		description:    "Update node from another file",
-		subdescription: "Note this will also work when the second file is a scalar (string/number)",
-		document:       `{a: apples}`,
-		document2:      "{b: bob}",
-		expression:     `select(fileIndex==0).a = select(fileIndex==1) | select(fileIndex==0)`,
-		expected: []string{
-			"D0, P[], (doc)::{a: {b: bob}}\n",
-		},
-	},
 	{
 		description: "Update node to be the sibling value",
 		document:    `{a: {b: child}, b: sibling}`,

pkg/yqlib/operator_document_index_test.go

@@ -17,7 +17,7 @@ var documentIndexScenarios = []expressionScenario{
 	{
 		description: "Filter by document index",
 		document:    "a: cat\n---\na: frog\n",
-		expression:  `select(documentIndex == 1)`,
+		expression:  `select(. | documentIndex == 1)`,
 		expected: []string{
 			"D1, P[], (doc)::a: frog\n",
 		},
@@ -25,7 +25,7 @@ var documentIndexScenarios = []expressionScenario{
 	{
 		description: "Print Document Index with matches",
 		document:    "a: cat\n---\na: frog\n",
-		expression:  `.a | ({"match": ., "doc": documentIndex})`,
+		expression:  `.a | ({"match": ., "doc": (. | documentIndex)})`,
 		expected: []string{
 			"D0, P[], (!!map)::match: cat\ndoc: 0\n",
 			"D0, P[], (!!map)::match: frog\ndoc: 1\n",

pkg/yqlib/operator_file_test.go

@@ -21,14 +21,6 @@ var fileOperatorScenarios = []expressionScenario{
 			"D0, P[], (!!int)::0\n",
 		},
 	},
-	{
-		description: "Get file index alias",
-		document:    `{a: cat}`,
-		expression:  `fi`,
-		expected: []string{
-			"D0, P[], (!!int)::0\n",
-		},
-	},
 }
 
 func TestFileOperatorsScenarios(t *testing.T) {

pkg/yqlib/operator_keys.go

@@ -0,0 +1,34 @@
+package yqlib
+
+import (
+	"container/list"
+	"fmt"
+
+	yaml "gopkg.in/yaml.v3"
+)
+
+func KeysOperator(d *dataTreeNavigator, matchMap *list.List, pathNode *PathTreeNode) (*list.List, error) {
+	log.Debugf("-- KeyOperation")
+	var results = list.New()
+
+	for el := matchMap.Front(); el != nil; el = el.Next() {
+		candidate := el.Value.(*CandidateNode)
+		targetNode := UnwrapDoc(candidate.Node)
+
+		switch targetNode.Kind {
+		case yaml.MappingNode:
+			node := &yaml.Node{Kind: yaml.SequenceNode, Value: fmt.Sprintf("%v", length), Tag: "!!int"}
+
+		case yaml.SequenceNode:
+			length = len(targetNode.Content)
+		default:
+			length = 0
+		}
+
+		node := &yaml.Node{Kind: yaml.ScalarNode, Value: fmt.Sprintf("%v", length), Tag: "!!int"}
+		lengthCand := &CandidateNode{Node: node, Document: candidate.Document, Path: candidate.Path}
+		results.PushBack(lengthCand)
+	}
+
+	return results, nil
+}

pkg/yqlib/operator_keys_test.go

@@ -0,0 +1,88 @@
+package yqlib
+
+import (
+	"testing"
+)
+
+var keyOperatorScenarios = []expressionScenario{
+	{
+		description: "Get Keys of map",
+		document:    `{a: {b: cat, c: dog, d: frog}}`,
+		expression:  `.a | keys`,
+		expected: []string{
+			"D0, P[a], (!!seq)::- b\n- c\n- d\n",
+		},
+	},
+	{
+		skipDoc:    true,
+		document:   `{a1: {b: cat, c: dog, d: frog}, a2: {e: cat, f: dog, g: frog}}`,
+		expression: `(.a1, .a2) | keys`,
+		expected: []string{
+			"D0, P[a1], (!!seq)::- b\n- c\n- d\n",
+			"D0, P[a2], (!!seq)::- e\n- f\n- g\n",
+		},
+	},
+	{
+		description: "Get Keys of array",
+		document:    `{a: [0,1,2]}`,
+		expression:  `.a | keys`,
+		expected: []string{
+			"D0, P[a], (!!seq)::0\n",
+			"D0, P[a], (!!int)::1\n",
+			"D0, P[a], (!!int)::2\n",
+		},
+	},
+	{
+		description: "Set key style",
+		document:    `{a: {b: cat, c: dog, d: frog}}`,
+		expression:  `(.a | keys | ..) style = 'single'`,
+		expected: []string{
+			"D0, P[], (!!doc)::{a: {'b': cat, 'c': dog, 'd': frog}}\n",
+		},
+	},
+	{
+		description: "Set key alias",
+		document:    `{a: {b: cat, c: dog, d: frog}}`,
+		expression:  `(.a | keys | .. | select(.=="b")) alias = 'boo'`,
+		expected: []string{
+			"D0, P[], (!!doc)::{a: {*meow: cat, c: dog, d: frog}}\n",
+		},
+	},
+	{
+		description: "Set key alias",
+		document:    `{a: {b: cat, c: dog, d: frog}}`,
+		expression:  `(.a | key("b")) alias = 'boo'`,
+		expected: []string{
+			"D0, P[], (!!doc)::{a: {*meow: cat, c: dog, d: frog}}\n",
+		},
+	},
+	{
+		skipDoc:    true,
+		document:   `{a: [0,1,2]}`,
+		expression: `(.a | keys) style = 'single'`,
+		expected: []string{
+			"D0, P[a], (!!int)::0\n",
+			"D0, P[a], (!!int)::1\n",
+			"D0, P[a], (!!int)::2\n",
+		},
+	},
+	{
+		skipDoc:    true,
+		document:   `a: cat`,
+		expression: `.a | keys`,
+		expected:   []string{},
+	},
+	{
+		skipDoc:    true,
+		document:   `{}`,
+		expression: `.a | keys`,
+		expected:   []string{},
+	},
+}
+
+func TestKeyOperatorScenarios(t *testing.T) {
+	for _, tt := range keyOperatorScenarios {
+		testScenario(t, &tt)
+	}
+	documentScenarios(t, "Keys", keyOperatorScenarios)
+}

pkg/yqlib/operators_test.go

@@ -18,7 +18,6 @@ type expressionScenario struct {
 	description           string
 	subdescription        string
 	document              string
-	document2             string
 	expression            string
 	expected              []string
 	skipDoc               bool
@@ -42,14 +41,6 @@ func testScenario(t *testing.T, s *expressionScenario) {
 			t.Error(err, s.document, s.expression)
 			return
 		}
-		if s.document2 != "" {
-			moreInputs, err := readDocuments(strings.NewReader(s.document2), "another.yml", 1)
-			if err != nil {
-				t.Error(err, s.document, s.expression)
-				return
-			}
-			inputs.PushBackList(moreInputs)
-		}
 	} else {
 		candidateNode := &CandidateNode{
 			Document:  0,
@@ -101,7 +92,7 @@ func copyFromHeader(title string, out *os.File) error {
 	return err
 }
 
-func formatYaml(yaml string, filename string) string {
+func formatYaml(yaml string) string {
 	var output bytes.Buffer
 	printer := NewPrinter(bufio.NewWriter(&output), false, true, false, 2, true)
 
@@ -110,7 +101,7 @@ func formatYaml(yaml string, filename string) string {
 		panic(err)
 	}
 	streamEvaluator := NewStreamEvaluator()
-	err = streamEvaluator.Evaluate(filename, strings.NewReader(yaml), node, printer)
+	err = streamEvaluator.Evaluate("sample.yaml", strings.NewReader(yaml), node, printer)
 	if err != nil {
 		panic(err)
 	}
@@ -137,116 +128,63 @@ func documentScenarios(t *testing.T, title string, scenarios []expressionScenari
 
 	for _, s := range scenarios {
 		if !s.skipDoc {
-			documentScenario(t, w, s)
+
+			writeOrPanic(w, fmt.Sprintf("## %v\n", s.description))
+
+			if s.subdescription != "" {
+				writeOrPanic(w, s.subdescription)
+				writeOrPanic(w, "\n\n")
+			}
+			formattedDoc := ""
+			if s.document != "" {
+				if s.dontFormatInputForDoc {
+					formattedDoc = s.document + "\n"
+				} else {
+					formattedDoc = formatYaml(s.document)
+				}
+				//TODO: pretty here
+				writeOrPanic(w, "Given a sample.yml file of:\n")
+
+				writeOrPanic(w, fmt.Sprintf("```yaml\n%v```\n", formattedDoc))
+				writeOrPanic(w, "then\n")
+				if s.expression != "" {
+					writeOrPanic(w, fmt.Sprintf("```bash\nyq eval '%v' sample.yml\n```\n", s.expression))
+				} else {
+					writeOrPanic(w, "```bash\nyq eval sample.yml\n```\n")
+				}
+			} else {
+				writeOrPanic(w, "Running\n")
+				writeOrPanic(w, fmt.Sprintf("```bash\nyq eval --null-input '%v'\n```\n", s.expression))
+			}
+
+			writeOrPanic(w, "will output\n")
+
+			var output bytes.Buffer
+			var err error
+			printer := NewPrinter(bufio.NewWriter(&output), false, true, false, 2, true)
+			streamEvaluator := NewStreamEvaluator()
+
+			if s.document != "" {
+				node, err := treeCreator.ParsePath(s.expression)
+				if err != nil {
+					t.Error(err, s.expression)
+				}
+				err = streamEvaluator.Evaluate("sample.yaml", strings.NewReader(formattedDoc), node, printer)
+
+				if err != nil {
+					t.Error(err, s.expression)
+				}
+			} else {
+				err = streamEvaluator.EvaluateNew(s.expression, printer)
+				if err != nil {
+					t.Error(err, s.expression)
+				}
+			}
+
+			writeOrPanic(w, fmt.Sprintf("```yaml\n%v```\n\n", output.String()))
+
 		}
+
 	}
 	w.Flush()
 }
-
-func documentScenario(t *testing.T, w *bufio.Writer, s expressionScenario) {
-	writeOrPanic(w, fmt.Sprintf("## %v\n", s.description))
-
-	if s.subdescription != "" {
-		writeOrPanic(w, s.subdescription)
-		writeOrPanic(w, "\n\n")
-	}
-
-	formattedDoc, formattedDoc2 := documentInput(w, s)
-
-	writeOrPanic(w, "will output\n")
-
-	documentOutput(t, w, s, formattedDoc, formattedDoc2)
-}
-
-func documentInput(w *bufio.Writer, s expressionScenario) (string, string) {
-	formattedDoc := ""
-	formattedDoc2 := ""
-	command := "eval"
-	if s.document != "" {
-		if s.dontFormatInputForDoc {
-			formattedDoc = s.document + "\n"
-		} else {
-			formattedDoc = formatYaml(s.document, "sample.yml")
-		}
-
-		writeOrPanic(w, "Given a sample.yml file of:\n")
-		writeOrPanic(w, fmt.Sprintf("```yaml\n%v```\n", formattedDoc))
-
-		files := "sample.yml"
-
-		if s.document2 != "" {
-			if s.dontFormatInputForDoc {
-				formattedDoc2 = s.document2 + "\n"
-			} else {
-				formattedDoc2 = formatYaml(s.document2, "another.yml")
-			}
-
-			writeOrPanic(w, "And another sample another.yml file of:\n")
-			writeOrPanic(w, fmt.Sprintf("```yaml\n%v```\n", formattedDoc2))
-			files = "sample.yml another.yml"
-			command = "eval-all"
-		}
-
-		writeOrPanic(w, "then\n")
-		if s.expression != "" {
-			writeOrPanic(w, fmt.Sprintf("```bash\nyq %v '%v' %v\n```\n", command, s.expression, files))
-		} else {
-			writeOrPanic(w, fmt.Sprintf("```bash\nyq %v %v\n```\n", command, files))
-		}
-	} else {
-		writeOrPanic(w, "Running\n")
-		writeOrPanic(w, fmt.Sprintf("```bash\nyq %v --null-input '%v'\n```\n", command, s.expression))
-	}
-	return formattedDoc, formattedDoc2
-}
-
-func documentOutput(t *testing.T, w *bufio.Writer, s expressionScenario, formattedDoc string, formattedDoc2 string) {
-	var output bytes.Buffer
-	var err error
-	printer := NewPrinter(bufio.NewWriter(&output), false, true, false, 2, true)
-
-	node, err := treeCreator.ParsePath(s.expression)
-	if err != nil {
-		t.Error(fmt.Errorf("Error parsing expression %v of %v: %v", s.expression, s.description, err))
-		return
-	}
-
-	inputs := list.New()
-
-	if s.document != "" {
-		inputs, err = readDocuments(strings.NewReader(formattedDoc), "sample.yml", 0)
-		if err != nil {
-			t.Error(err, s.document, s.expression)
-			return
-		}
-		if s.document2 != "" {
-			moreInputs, err := readDocuments(strings.NewReader(formattedDoc2), "another.yml", 1)
-			if err != nil {
-				t.Error(err, s.document, s.expression)
-				return
-			}
-			inputs.PushBackList(moreInputs)
-		}
-	} else {
-		candidateNode := &CandidateNode{
-			Document:  0,
-			Filename:  "",
-			Node:      &yaml.Node{Tag: "!!null"},
-			FileIndex: 0,
-		}
-		inputs.PushBack(candidateNode)
-
-	}
-
-	results, err := treeNavigator.GetMatchingNodes(inputs, node)
-	if err != nil {
-		t.Error(err, s.expression)
-	}
-
-	err = printer.PrintResults(results)
-	if err != nil {
-		t.Error(err, s.expression)
-	}
-
-	writeOrPanic(w, fmt.Sprintf("```yaml\n%v```\n\n", output.String()))
-}

pkg/yqlib/path_tokeniser.go

@@ -182,6 +182,7 @@ func initLexer() (*lex.Lexer, error) {
 	lexer.Add([]byte(`,`), opToken(Union))
 	lexer.Add([]byte(`:\s*`), opToken(CreateMap))
 	lexer.Add([]byte(`length`), opToken(Length))
+	lexer.Add([]byte(`keys`), opToken(Keys))
 	lexer.Add([]byte(`sortKeys`), opToken(SortKeys))
 	lexer.Add([]byte(`select`), opToken(Select))
 	lexer.Add([]byte(`has`), opToken(Has))
@@ -200,7 +201,6 @@ func initLexer() (*lex.Lexer, error) {
 	lexer.Add([]byte(`alias`), opAssignableToken(GetAlias, AssignAlias))
 	lexer.Add([]byte(`filename`), opToken(GetFilename))
 	lexer.Add([]byte(`fileIndex`), opToken(GetFileIndex))
-	lexer.Add([]byte(`fi`), opToken(GetFileIndex))
 	lexer.Add([]byte(`path`), opToken(GetPath))
 
 	lexer.Add([]byte(`lineComment`), opTokenWithPrefs(GetComment, AssignComment, &CommentOpPreferences{LineComment: true}))

pkg/yqlib/test.yml

@@ -0,0 +1,10 @@
+{
+  "a1": {
+    "b": "cat",
+    "c": "dog"
+  },
+  "a2": {
+    "d": "cat",
+    "e": "dog"
+  }
+}

release_instructions.txt

@@ -1,11 +1,9 @@
 - increment version in version.go
 - increment version in snapcraft.yaml
-- increment version in github-action/Dockerfile
 - make sure local build passes
 - tag git with same version number
 - commit vX tag - this will trigger github actions
 - use github actions to publish docker and make github release
-- check github updated yq action in marketplace
 
 - snapcraft
   - will auto create a candidate, test it works then promote

scripts/check.sh

@@ -3,12 +3,7 @@
 set -o errexit
 set -o pipefail
 
-if command -v golangci-lint &> /dev/null
+./bin/golangci-lint run --timeout=5m
-then
-    golangci-lint run --timeout=5m
-else
-  ./bin/golangci-lint run --timeout=5m
-fi
 
 # ./bin/golangci-lint \
 #   --tests \

scripts/publish-docker.sh

@@ -0,0 +1,12 @@
+#!/bin/bash
+set -ex
+VERSION="$(git describe --tags --abbrev=0)"
+docker build \
+  --target production \
+  --build-arg VERSION=${VERSION} \
+  -t mikefarah/yq:latest \
+  -t mikefarah/yq:${VERSION} \
+  -t mikefarah/yq:4 \
+  .
+
+trivy image mikefarah/yq:${VERSION}

scripts/release.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+set -ex
+GITHUB_TOKEN="${GITHUB_TOKEN:?missing required input \'GITHUB_TOKEN\'}"
+
+CURRENT="$(git describe --tags --abbrev=0)"
+PREVIOUS="$(git describe --tags --abbrev=0 --always "${CURRENT}"^)"
+OWNER="mikefarah"
+REPO="yq"
+
+release() {
+    github-release release \
+        --user "$OWNER" \
+        --draft \
+        --repo "$REPO" \
+        --tag "$CURRENT"
+}
+
+
+
+release
+

scripts/upload.sh

@@ -0,0 +1,28 @@
+#!/bin/bash
+set -ex
+GITHUB_TOKEN="${GITHUB_TOKEN:?missing required input \'GITHUB_TOKEN\'}"
+
+CURRENT="$(git describe --tags --abbrev=0)"
+PREVIOUS="$(git describe --tags --abbrev=0 --always "${CURRENT}"^)"
+OWNER="mikefarah"
+REPO="yq"
+
+upload() {
+    mkdir -p ./build-done
+    while IFS=  read -r -d $'\0'; do
+        file=$REPLY
+        BINARY=$(basename "${file}")
+        echo "--> ${BINARY}"
+        github-release upload \
+            --replace \
+            --user "$OWNER" \
+            --repo "$REPO" \
+            --tag "$CURRENT" \
+            --name "${BINARY}" \
+            --file "$file"
+        mv "$file" "./build-done/${BINARY}"
+    done < <(find ./build -mindepth 1 -maxdepth 1 -print0)
+}
+
+
+upload

snap/snapcraft.yaml

@@ -1,5 +1,5 @@
 name: yq
-version: '4.2.1'
+version: '4.2.0'
 summary: A lightweight and portable command-line YAML processor
 description: |
   The aim of the project is to be the jq or sed of yaml files.