1
0
mirror of https://github.com/taigrr/yq synced 2025-01-18 04:53:17 -08:00

Compare commits

..

1 Commits
3.3.4 ... v2.x

Author SHA1 Message Date
Mike Farah
767208871b GitBook: [v2.x] 10 pages modified 2020-06-16 01:19:03 +00:00
26 changed files with 1498 additions and 200 deletions

View File

@@ -0,0 +1,60 @@
## Yaml to Json
To convert output to json, use the --tojson (or -j) flag. This is supported by all commands.
Each matching yaml node will be converted to json and printed out on a separate line.
Given a sample.yaml file of:
```yaml
b:
c: 2
```
then
```bash
yq r -j sample.yaml
```
will output
```json
{"b":{"c":2}}
```
Given a sample.yaml file of:
```yaml
bob:
c: 2
bab:
c: 5
```
then
```bash
yq r -j sample.yaml b*
```
will output
```json
{"c":2}
{"c":5}
```
## Json to Yaml
To read in json, just pass in a json file instead of yaml, it will just work :)
e.g given a json file
```json
{"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}
```
then
```bash
yq r sample.json
```
will output
```yaml
a: Easy! as one two three
b:
c: 2
d:
- 3
- 4
```

View File

@@ -7,10 +7,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.15
- name: Set up Go 1.14
uses: actions/setup-go@v1
with:
go-version: 1.15
go-version: 1.14
id: go
- name: Check out code into the Go module directory

View File

@@ -1,4 +1,4 @@
FROM golang:1.15 as builder
FROM golang:1.14 as builder
WORKDIR /go/src/mikefarah/yq

View File

@@ -1,4 +1,4 @@
FROM golang:1.15
FROM golang:1.14
COPY scripts/devtools.sh /opt/devtools.sh

136
README.md
View File

@@ -1,140 +1,36 @@
# yq
---
description: yq is a lightweight and portable command-line YAML processor
---
![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg) ![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) ![Github Releases (by Release)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) ![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq)
# Yq
### Install[¶]() <a id="install"></a>
a lightweight and portable command-line YAML processor
On MacOS:
The aim of the project is to be the [jq](https://github.com/stedolan/jq) or sed of yaml files.
## New version!
V3 is officially out - if you've been using v2 and want/need to upgrade, checkout the [upgrade guide](https://mikefarah.gitbook.io/yq/upgrading-from-v2).
## Install
### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest)
### MacOS:
```
```text
brew install yq
```
### Windows:
```
choco install yq
```
Supported by @chillum
On Ubuntu and other Linux distros supporting `snap` packages:
### Ubuntu and other Linux distros supporting `snap` packages:
```
```text
snap install yq
```
#### Snap notes
`yq` installs with [_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, this means it doesn't have direct access to root files. To read root files you can:
On Ubuntu 16.04 or higher from Debian package:
```
sudo cat /etc/myfile | yq r - a.path
```
And to write to a root file you can either use [sponge](https://linux.die.net/man/1/sponge):
```
sudo cat /etc/myfile | yq w - a.path value | sudo sponge /etc/myfile
```
or write to a temporary file:
```
sudo cat /etc/myfile | yq w - a.path value | sudo tee /etc/myfile.tmp
sudo mv /etc/myfile.tmp /etc/myfile
rm /etc/myfile.tmp
```
### On Ubuntu 16.04 or higher from Debian package:
```sh
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CC86BB64
```text
sudo add-apt-repository ppa:rmescandon/yq
sudo apt update
sudo apt install yq -y
```
Supported by @rmescandon
### Go Get:
```
GO111MODULE=on go get github.com/mikefarah/yq/v3
or, [Download latest binary](https://github.com/mikefarah/yq/releases/latest) or alternatively:
```text
GO111MODULE=on go get github.com/mikefarah/yq/v2
```
## Run with Docker
[View on GitHub](https://github.com/mikefarah/yq)
Oneshot use:
```bash
docker run --rm -v "${PWD}":/workdir mikefarah/yq yq [flags] <command> FILE...
```
Run commands interactively:
```bash
docker run --rm -it -v "${PWD}":/workdir mikefarah/yq sh
```
It can be useful to have a bash function to avoid typing the whole docker command:
```bash
yq() {
docker run --rm -i -v "${PWD}":/workdir mikefarah/yq yq "$@"
}
```
## Features
- Written in portable go, so you can download a lovely dependency free binary
- [Colorize the output](https://mikefarah.gitbook.io/yq/usage/output-format#colorize-output)
- [Deep read a yaml file with a given path expression](https://mikefarah.gitbook.io/yq/commands/read#basic)
- [List matching paths of a given path expression](https://mikefarah.gitbook.io/yq/commands/read#path-only)
- [Return the lengths of arrays/object/scalars](https://mikefarah.gitbook.io/yq/commands/read#printing-length-of-the-results)
- Update a yaml file given a [path expression](https://mikefarah.gitbook.io/yq/commands/write-update#basic) or [script file](https://mikefarah.gitbook.io/yq/commands/write-update#basic)
- Update creates any missing entries in the path on the fly
- Deeply [compare](https://mikefarah.gitbook.io/yq/commands/compare) yaml files
- Keeps yaml formatting and comments when updating
- [Validate a yaml file](https://mikefarah.gitbook.io/yq/commands/validate)
- Create a yaml file given a [deep path and value](https://mikefarah.gitbook.io/yq/commands/create#creating-a-simple-yaml-file) or a [script file](https://mikefarah.gitbook.io/yq/commands/create#creating-using-a-create-script)
- [Prefix a path to a yaml file](https://mikefarah.gitbook.io/yq/commands/prefix)
- [Convert to/from json to yaml](https://mikefarah.gitbook.io/yq/usage/convert)
- [Pipe data in by using '-'](https://mikefarah.gitbook.io/yq/commands/read#from-stdin)
- [Merge](https://mikefarah.gitbook.io/yq/commands/merge) multiple yaml files with various options for [overriding](https://mikefarah.gitbook.io/yq/commands/merge#overwrite-values) and [appending](https://mikefarah.gitbook.io/yq/commands/merge#append-values-with-arrays)
- Supports multiple documents in a single yaml file for [reading](https://mikefarah.gitbook.io/yq/commands/read#multiple-documents), [writing](https://mikefarah.gitbook.io/yq/commands/write-update#multiple-documents) and [merging](https://mikefarah.gitbook.io/yq/commands/merge#multiple-documents)
- General shell completion scripts (bash/zsh/fish/powershell) (https://mikefarah.gitbook.io/yq/commands/shell-completion)
## [Usage](https://mikefarah.gitbook.io/yq/)
Check out the [documentation](https://mikefarah.gitbook.io/yq/) for more detailed and advanced usage.
```
Usage:
yq [flags]
yq [command]
Available Commands:
compare yq x [--prettyPrint/-P] dataA.yaml dataB.yaml 'b.e(name==fr*).value'
delete yq d [--inplace/-i] [--doc/-d index] sample.yaml 'b.e(name==fred)'
help Help about any command
merge yq m [--inplace/-i] [--doc/-d index] [--overwrite/-x] [--append/-a] sample.yaml sample2.yaml
new yq n [--script/-s script_file] a.b.c newValue
prefix yq p [--inplace/-i] [--doc/-d index] sample.yaml a.b.c
read yq r [--printMode/-p pv] sample.yaml 'b.e(name==fr*).value'
shell-completion Generates shell completion scripts
validate yq v sample.yaml
write yq w [--inplace/-i] [--script/-s script_file] [--doc/-d index] sample.yaml 'b.e(name==fr*).value' newValue
Flags:
-C, --colors print with colors
-h, --help help for yq
-I, --indent int sets indent level for output (default 2)
-P, --prettyPrint pretty print
-j, --tojson output as json. By default it prints a json document in one line, use the prettyPrint flag to print a formatted doc.
-v, --verbose verbose mode
-V, --version Print version information and quit
Use "yq [command] --help" for more information about a command.
```
## Known Issues
- `yq` attempts to preserve comment positions and whitespace as much as possible, but it does not handle all scenarios (see https://github.com/go-yaml/yaml/tree/v3 for details)

11
SUMMARY.md Normal file
View File

@@ -0,0 +1,11 @@
# Table of contents
* [Yq](README.md)
* [Write](write-update.md)
* [Read](read.md)
* [Prefix](prefix.md)
* [Delete](delete.md)
* [Create](create.md)
* [Convert](convert.md)
* [Merge](merge.md)

View File

@@ -2,6 +2,7 @@ package cmd
import (
"github.com/mikefarah/yq/v3/pkg/yqlib"
errors "github.com/pkg/errors"
"github.com/spf13/cobra"
)
@@ -35,7 +36,11 @@ Note that you can give a create script to perform more sophisticated yaml. This
func newProperty(cmd *cobra.Command, args []string) error {
var badArgsMessage = "Must provide <path_to_update> <value>"
var updateCommands, updateCommandsError = readUpdateCommands(args, 2, badArgsMessage, false)
if len(args) != 2 {
return errors.New(badArgsMessage)
}
var updateCommands, updateCommandsError = readUpdateCommands(args, 2, badArgsMessage)
if updateCommandsError != nil {
return updateCommandsError
}

View File

@@ -1,7 +1,6 @@
package cmd
import (
"fmt"
"testing"
"github.com/mikefarah/yq/v3/test"
@@ -19,24 +18,6 @@ func TestNewCmd(t *testing.T) {
test.AssertResult(t, expectedOutput, result.Output)
}
func TestNewCmdScript(t *testing.T) {
updateScript := `- command: update
path: b.c
value: 7`
scriptFilename := test.WriteTempYamlFile(updateScript)
defer test.RemoveTempYamlFile(scriptFilename)
cmd := getRootCommand()
result := test.RunCmd(cmd, fmt.Sprintf("new --script %s", scriptFilename))
if result.Error != nil {
t.Error(result.Error)
}
expectedOutput := `b:
c: 7
`
test.AssertResult(t, expectedOutput, result.Output)
}
func TestNewAnchorCmd(t *testing.T) {
cmd := getRootCommand()
result := test.RunCmd(cmd, "new b.c 3 --anchorName=fred")

View File

@@ -477,7 +477,7 @@ type updateCommandParsed struct {
Value yaml.Node
}
func readUpdateCommands(args []string, expectedArgs int, badArgsMessage string, allowNoValue bool) ([]yqlib.UpdateCommand, error) {
func readUpdateCommands(args []string, expectedArgs int, badArgsMessage string) ([]yqlib.UpdateCommand, error) {
var updateCommands []yqlib.UpdateCommand = make([]yqlib.UpdateCommand, 0)
if writeScript != "" {
var parsedCommands = make([]updateCommandParsed, 0)
@@ -511,9 +511,8 @@ func readUpdateCommands(args []string, expectedArgs int, badArgsMessage string,
log.Debug("args %v", args)
log.Debug("path %v", args[expectedArgs-2])
log.Debug("Value %v", args[expectedArgs-1])
value := valueParser.Parse(args[expectedArgs-1], customTag, customStyle, anchorName, makeAlias)
updateCommands[0] = yqlib.UpdateCommand{Command: "update", Path: args[expectedArgs-2], Value: value, Overwrite: true, DontUpdateComments: true}
} else if len(args) == expectedArgs-1 && allowNoValue {
updateCommands[0] = yqlib.UpdateCommand{Command: "update", Path: args[expectedArgs-2], Value: valueParser.Parse(args[expectedArgs-1], customTag, customStyle, anchorName, makeAlias), Overwrite: true}
} else if len(args) == expectedArgs-1 {
// don't update the value
updateCommands = make([]yqlib.UpdateCommand, 1)
log.Debug("args %v", args)

View File

@@ -11,7 +11,7 @@ var (
GitDescribe string
// Version is main version number that is being run at the moment.
Version = "3.3.3"
Version = "3.3.2"
// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
// then it means that it is a final release. Otherwise, this is a pre-release

View File

@@ -53,7 +53,7 @@ format is list of update commands (update or delete) like so:
}
func writeProperty(cmd *cobra.Command, args []string) error {
var updateCommands, updateCommandsError = readUpdateCommands(args, 3, "Must provide <filename> <path_to_update> <value>", true)
var updateCommands, updateCommandsError = readUpdateCommands(args, 3, "Must provide <filename> <path_to_update> <value>")
if updateCommandsError != nil {
return updateCommandsError
}

View File

@@ -27,24 +27,6 @@ func TestWriteCmd(t *testing.T) {
test.AssertResult(t, expectedOutput, result.Output)
}
func TestWriteKeepCommentsCmd(t *testing.T) {
content := `b:
c: 3 # comment
`
filename := test.WriteTempYamlFile(content)
defer test.RemoveTempYamlFile(filename)
cmd := getRootCommand()
result := test.RunCmd(cmd, fmt.Sprintf("write %s b.c 7", filename))
if result.Error != nil {
t.Error(result.Error)
}
expectedOutput := `b:
c: 7 # comment
`
test.AssertResult(t, expectedOutput, result.Output)
}
func TestWriteWithTaggedStyleCmd(t *testing.T) {
content := `b:
c: dog

52
convert.md Normal file
View File

@@ -0,0 +1,52 @@
# Convert
### Yaml to Json[¶](convert.md#yaml-to-json) <a id="yaml-to-json"></a>
To convert output to json, use the --tojson \(or -j\) flag. This can only be used with the read command.
Given a sample.yaml file of:
```text
b:
c: 2
```
then
```text
yq r -j sample.yaml b.c
```
will output
```text
{"b":{"c":2}}
```
### Json to Yaml[¶](convert.md#json-to-yaml) <a id="json-to-yaml"></a>
To read in json, just pass in a json file instead of yaml, it will just work :\)
e.g given a json file
```text
{"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}
```
then
```text
yq r sample.json
```
will output
```text
a: Easy! as one two three
b:
c: 2
d:
- 3
- 4
```

88
create.md Normal file
View File

@@ -0,0 +1,88 @@
# Create
Yaml files can be created using the 'new' command. This works in the same way as the write command, but you don't pass in an existing Yaml file. Currently this does not support creating multiple documents in a single yaml file.
```text
yq n <path> <new value>
```
### Creating a simple yaml file[¶](create.md#creating-a-simple-yaml-file) <a id="creating-a-simple-yaml-file"></a>
```text
yq n b.c cat
```
will output:
```text
b:
c: cat
```
### Creating using a create script[¶](create.md#creating-using-a-create-script) <a id="creating-using-a-create-script"></a>
Create scripts follow the same format as the update scripts.
Given a script create\_instructions.yaml of:
```text
b.c: 3
b.e[+].name: Howdy Partner
```
then
```text
yq n -s create_instructions.yaml
```
will output:
```text
b:
c: 3
e:
- name: Howdy Partner
```
You can also pipe the instructions in:
```text
cat create_instructions.yaml | yq n -s -
```
### Keys with dots[¶](create.md#keys-with-dots) <a id="keys-with-dots"></a>
When specifying a key that has a dot use key lookup indicator.
```text
b:
foo.bar: 7
```
```text
yaml r sample.yaml 'b[foo.bar]'
```
```text
yaml w sample.yaml 'b[foo.bar]' 9
```
Any valid yaml key can be specified as part of a key lookup.
Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
### Keys \(and values\) with leading dashes[¶](create.md#keys-and-values-with-leading-dashes) <a id="keys-and-values-with-leading-dashes"></a>
If a key or value has leading dashes, yq won't know that you are passing a value as opposed to a flag \(and you will get a 'bad flag syntax' error\).
To fix that, you will need to tell it to stop processing flags by adding '--' after the last flag like so:
```text
yq n -t -- --key --value
```
Will result in
``` --key: --value``

7
debian/changelog vendored
View File

@@ -1,10 +1,3 @@
yq (3.3.2) focal; urgency=medium
* Bug fix: existStatus bug (#459)
* Automatically makes a os temp directory if it does not exist (#461)
-- Roberto Mier Escandon <rmescandon@gmail.com> Fri, 07 Aug 2020 18:53:01 +0200
yq (3.3-0) focal; urgency=medium
* You can control string styles (quotes) using the new --style flag

280
delete.md Normal file
View File

@@ -0,0 +1,280 @@
# Delete
```text
yq d <yaml_file> <path_to_delete>
```
### To Stdout[¶](delete.md#to-stdout) <a id="to-stdout"></a>
Given a sample.yaml file of:
```text
b:
c: 2
apples: green
```
then
```text
yq d sample.yaml b.c
```
will output:
```text
b:
apples: green
```
### From STDIN[¶](delete.md#from-stdin) <a id="from-stdin"></a>
```text
cat sample.yaml | yq d - b.c
```
### Deleting array elements[¶](delete.md#deleting-array-elements) <a id="deleting-array-elements"></a>
Given a sample.yaml file of:
```text
b:
c:
- 1
- 2
- 3
```
then
```text
yq d sample.yaml 'b.c[1]'
```
will output:
```text
b:
c:
- 1
- 3
```
### Deleting nodes in-place[¶](delete.md#deleting-nodes-in-place) <a id="deleting-nodes-in-place"></a>
Given a sample.yaml file of:
```text
b:
c: 2
apples: green
```
then
```text
yq d -i sample.yaml b.c
```
will update the sample.yaml file so that the 'c' node is deleted
### Splat[¶](delete.md#splat) <a id="splat"></a>
Given a sample.yaml file of:
```text
---
bob:
item1:
cats: bananas
dogs: woof
item2:
cats: apples
dogs: woof2
thing:
cats: oranges
dogs: woof3
```
then
```text
yq d sample.yaml bob.*.cats
```
will output:
```text
---
bob:
item1:
dogs: woof
item2:
dogs: woof2
thing:
dogs: woof3
```
### Prefix Splat[¶](delete.md#prefix-splat) <a id="prefix-splat"></a>
Given a sample.yaml file of:
```text
---
bob:
item1:
cats: bananas
dogs: woof
item2:
cats: apples
dogs: woof2
thing:
cats: oranges
dogs: woof3
```
then
```text
yq d sample.yaml bob.item*.cats
```
will output:
```text
---
bob:
item1:
dogs: woof
item2:
dogs: woof2
thing:
cats: oranges
dogs: woof3
```
### Array Splat[¶](delete.md#array-splat) <a id="array-splat"></a>
Given a sample.yaml file of:
```text
---
bob:
- cats: bananas
dogs: woof
- cats: apples
dogs: woof2
- cats: oranges
dogs: woof3
```
then
```text
yq d sample.yaml bob.[*].cats
```
will output:
```text
---
bob:
- dogs: woof
- dogs: woof2
- dogs: woof3
```
### Multiple Documents - delete from single document[¶](delete.md#multiple-documents-delete-from-single-document) <a id="multiple-documents-delete-from-single-document"></a>
Given a sample.yaml file of:
```text
something: else
field: leaveMe
---
b:
c: 2
field: deleteMe
```
then
```text
yq w -d1 sample.yaml field
```
will output:
```text
something: else
field: leaveMe
---
b:
c: 2
```
### Multiple Documents - delete from all documents[¶](delete.md#multiple-documents-delete-from-all-documents) <a id="multiple-documents-delete-from-all-documents"></a>
Given a sample.yaml file of:
```text
something: else
field: deleteMe
---
b:
c: 2
field: deleteMeToo
```
then
```text
yq w -d'*' sample.yaml field
```
will output:
```text
something: else
---
b:
c: 2
```
Note that '\*' is in quotes to avoid being interpreted by your shell.
### Keys with dots[¶](delete.md#keys-with-dots) <a id="keys-with-dots"></a>
When specifying a key that has a dot use key lookup indicator.
```text
b:
foo.bar: 7
```
```text
yaml r sample.yaml 'b[foo.bar]'
```
```text
yaml w sample.yaml 'b[foo.bar]' 9
```
Any valid yaml key can be specified as part of a key lookup.
Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
### Keys \(and values\) with leading dashes[¶](delete.md#keys-and-values-with-leading-dashes) <a id="keys-and-values-with-leading-dashes"></a>
If a key or value has leading dashes, yq won't know that you are passing a value as opposed to a flag \(and you will get a 'bad flag syntax' error\).
To fix that, you will need to tell it to stop processing flags by adding '--' after the last flag like so:
```text
yq n -t -- --key --value
```
Will result in
``` --key: --value``

12
go.mod
View File

@@ -2,16 +2,16 @@ module github.com/mikefarah/yq/v3
require (
github.com/fatih/color v1.9.0
github.com/goccy/go-yaml v1.8.1
github.com/goccy/go-yaml v1.7.5
github.com/kylelemons/godebug v1.1.0
github.com/mattn/go-colorable v0.1.7 // indirect
github.com/mattn/go-colorable v0.1.6 // indirect
github.com/pkg/errors v0.9.1
github.com/spf13/cobra v1.0.0
github.com/spf13/pflag v1.0.5 // indirect
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980 // indirect
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 // indirect
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776
gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c
)
go 1.15
go 1.14

21
go.sum
View File

@@ -29,8 +29,8 @@ github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V
github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/goccy/go-yaml v1.8.1 h1:JuZRFlqLM5cWF6A+waL8AKVuCcqvKOuhJtUQI+L3ez0=
github.com/goccy/go-yaml v1.8.1/go.mod h1:wS4gNoLalDSJxo/SpngzPQ2BN4uuZVLCmbM4S3vd4+Y=
github.com/goccy/go-yaml v1.7.5 h1:dWvj+p3BG11S/GlUzwzt1WZz0lhBTzTIDtmXT/ZOaPY=
github.com/goccy/go-yaml v1.7.5/go.mod h1:wS4gNoLalDSJxo/SpngzPQ2BN4uuZVLCmbM4S3vd4+Y=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
@@ -64,8 +64,8 @@ github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgx
github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-colorable v0.1.7 h1:bQGKb3vps/j0E9GfJQ03JyhRuxsvdAanXlT9BTw3mdw=
github.com/mattn/go-colorable v0.1.7/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.6 h1:6Su7aK7lXmJ/U79bYtBjLNaha4Fs1Rg9plHpcH+vvnE=
github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84=
github.com/mattn/go-isatty v0.0.11 h1:FxPOTFNqGkuDUGi3H/qkUbQO4ZiBa2brKq5r0l8TGeM=
@@ -144,20 +144,19 @@ golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f h1:Fqb3ao1hUmOR3GkUOg/Y+BadLwykBIzs5q8Ez2SbHyc=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980 h1:OjiUf46hAmXblsZdnoSXsEUSKU8r1UEzcL5RVZ4gO9Y=
golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd h1:/e+gpKk9r3dJobndpTytxS2gOy6m5uvpg+ISQoEcusQ=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
@@ -176,6 +175,6 @@ gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bl
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776 h1:tQIYjPdBoyREyB9XMu+nnTclpTYkz2zFM+lzLJFO4gQ=
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c h1:grhR+C34yXImVGp7EzNk+DTIk+323eIUWOmEevy6bDo=
gopkg.in/yaml.v3 v3.0.0-20200605160147-a5ece683394c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=

245
merge.md Normal file
View File

@@ -0,0 +1,245 @@
# Merge
Yaml files can be merged using the 'merge' command. Each additional file merged with the first file will set values for any key not existing already or where the key has no value.
```text
yq m <yaml_file> <path>...
```
### To Stdout[¶](merge.md#to-stdout) <a id="to-stdout"></a>
Given a data1.yaml file of:
```text
a: simple
b: [1, 2]
```
and data2.yaml file of:
```text
a: other
c:
test: 1
```
then
```text
yq m data1.yaml data2.yaml
```
will output:
```text
a: simple
b: [1, 2]
c:
test: 1
```
### Updating files in-place[¶](merge.md#updating-files-in-place) <a id="updating-files-in-place"></a>
Given a data1.yaml file of:
```text
a: simple
b: [1, 2]
```
and data2.yaml file of:
```text
a: other
c:
test: 1
```
then
```text
yq m -i data1.yaml data2.yaml
```
will update the data1.yaml file so that the value of 'c' is 'test: 1'.
### Overwrite values[¶](merge.md#overwrite-values) <a id="overwrite-values"></a>
Given a data1.yaml file of:
```text
a: simple
b: [1, 2]
```
and data2.yaml file of:
```text
a: other
c:
test: 1
```
then
```text
yq m -x data1.yaml data2.yaml
```
will output:
```text
a: other
b: [1, 2]
c:
test: 1
```
### Overwrite values with arrays[¶](merge.md#overwrite-values-with-arrays) <a id="overwrite-values-with-arrays"></a>
Given a data1.yaml file of:
```text
a: simple
b: [1, 2]
```
and data3.yaml file of:
```text
b: [3, 4]
c:
test: 2
other: true
d: false
```
then
```text
yq m -x data1.yaml data3.yaml
```
will output:
```text
a: simple
b: [3, 4]
c:
test: 2
other: true
d: false
```
Notice that 'b' does not result in the merging of the values within an array.
### Append values with arrays[¶](merge.md#append-values-with-arrays) <a id="append-values-with-arrays"></a>
Given a data1.yaml file of:
```text
a: simple
b: [1, 2]
d: hi
```
and data3.yaml file of:
```text
a: something
b: [3, 4]
c:
test: 2
other: true
```
then
```text
yq m -a data1.yaml data3.yaml
```
will output:
```text
a: simple
b: [1, 2, 3, 4]
c:
test: 2
other: true
d: hi
```
Note that the 'b' array has concatenated the values from the second data file. Also note that other map keys are not overridden \(field a\).
Append cannot be used with overwrite, if both flags are given then append is ignored.
### Multiple Documents - merge into single document[¶](merge.md#multiple-documents-merge-into-single-document) <a id="multiple-documents-merge-into-single-document"></a>
Currently yq only has multi-document support for the _first_ document being merged into. The remaining yaml files will have their first document selected.
Given a data1.yaml file of:
```text
something: else
---
a: simple
b: cat
```
and data3.yaml file of:
```text
b: dog
```
then
```text
yq m -x -d1 data1.yaml data3.yaml
```
will output:
```text
something: else
---
a: simple
b: dog
```
### Multiple Documents - merge into all documents[¶](merge.md#multiple-documents-merge-into-all-documents) <a id="multiple-documents-merge-into-all-documents"></a>
Currently yq only has multi-document support for the _first_ document being merged into. The remaining yaml files will have their first document selected.
Given a data1.yaml file of:
```text
something: else
---
a: simple
b: cat
```
and data3.yaml file of:
```text
b: dog
```
then
```text
yq m -x -d'*' data1.yaml data3.yaml
```
will output:
```text
b: dog
something: else
---
a: simple
b: dog
```

View File

@@ -18,7 +18,6 @@ type UpdateCommand struct {
Value *yaml.Node
Overwrite bool
DontUpdateNodeValue bool
DontUpdateComments bool
}
func KindString(kind yaml.Kind) string {

View File

@@ -27,11 +27,9 @@ func UpdateNavigationStrategy(updateCommand UpdateCommand, autoCreate bool) Navi
node.Content = changesToApply.Content
node.Anchor = changesToApply.Anchor
node.Alias = changesToApply.Alias
if !updateCommand.DontUpdateComments {
node.HeadComment = changesToApply.HeadComment
node.LineComment = changesToApply.LineComment
node.FootComment = changesToApply.FootComment
}
node.HeadComment = changesToApply.HeadComment
node.LineComment = changesToApply.LineComment
node.FootComment = changesToApply.FootComment
} else {
log.Debug("skipping update as node already has value %v and overwriteFlag is ", node.Value, updateCommand.Overwrite)
}

127
prefix.md Normal file
View File

@@ -0,0 +1,127 @@
# Prefix
Paths can be prefixed using the 'prefix' command. The complete yaml content will be nested inside the new prefix path.
```text
yq p <yaml_file> <path>
```
### To Stdout[¶](prefix.md#to-stdout) <a id="to-stdout"></a>
Given a data1.yaml file of:
```text
a: simple
b: [1, 2]
```
then
```text
yq p data1.yaml c
```
will output:
```text
c:
a: simple
b: [1, 2]
```
### Arbitrary depth[¶](prefix.md#arbitrary-depth) <a id="arbitrary-depth"></a>
Given a data1.yaml file of:
```text
a:
b: [1, 2]
```
then
```text
yq p data1.yaml c.d
```
will output:
```text
c:
d:
a:
b: [1, 2]
```
### Updating files in-place[¶](prefix.md#updating-files-in-place) <a id="updating-files-in-place"></a>
Given a data1.yaml file of:
```text
a: simple
b: [1, 2]
```
then
```text
yq p -i data1.yaml c
```
will update the data1.yaml file so that the path 'c' is prefixed to all other paths.
### Multiple Documents - prefix a single document[¶](prefix.md#multiple-documents-prefix-a-single-document) <a id="multiple-documents-prefix-a-single-document"></a>
Given a data1.yaml file of:
```text
something: else
---
a: simple
b: cat
```
then
```text
yq p -d1 data1.yaml c
```
will output:
```text
something: else
---
c:
a: simple
b: cat
```
### Multiple Documents - prefix all documents[¶](prefix.md#multiple-documents-prefix-all-documents) <a id="multiple-documents-prefix-all-documents"></a>
Given a data1.yaml file of:
```text
something: else
---
a: simple
b: cat
```
then
```text
yq p -d'*' data1.yaml c
```
will output:
```text
c:
something: else
---
c:
a: simple
b: cat
```

228
read.md Normal file
View File

@@ -0,0 +1,228 @@
# Read
```text
yq r <yaml_file|json_file> <path>
```
This command can take a json file as input too, and will output yaml unless specified to export as json \(-j\)
### Basic[¶](read.md#basic) <a id="basic"></a>
Given a sample.yaml file of:
```text
b:
c: 2
```
then
```text
yq r sample.yaml b.c
```
will output the value of '2'.
### From Stdin[¶](read.md#from-stdin) <a id="from-stdin"></a>
Given a sample.yaml file of:
```text
cat sample.yaml | yq r - b.c
```
will output the value of '2'.
### Splat[¶](read.md#splat) <a id="splat"></a>
Given a sample.yaml file of:
```text
---
bob:
item1:
cats: bananas
item2:
cats: apples
thing:
cats: oranges
```
then
```text
yq r sample.yaml bob.*.cats
```
will output
```text
- bananas
- apples
- oranges
```
### Prefix Splat[¶](read.md#prefix-splat) <a id="prefix-splat"></a>
Given a sample.yaml file of:
```text
---
bob:
item1:
cats: bananas
item2:
cats: apples
thing:
cats: oranges
```
then
```text
yq r sample.yaml bob.item*.cats
```
will output
```text
- bananas
- apples
```
### Multiple Documents - specify a single document[¶](read.md#multiple-documents-specify-a-single-document) <a id="multiple-documents-specify-a-single-document"></a>
Given a sample.yaml file of:
```text
something: else
---
b:
c: 2
```
then
```text
yq r -d1 sample.yaml b.c
```
will output the value of '2'.
### Multiple Documents - read all documents[¶](read.md#multiple-documents-read-all-documents) <a id="multiple-documents-read-all-documents"></a>
Reading all documents will return the result as an array. This can be converted to json using the '-j' flag if desired.
Given a sample.yaml file of:
```text
name: Fred
age: 22
---
name: Stella
age: 23
---
name: Android
age: 232
```
then
```text
yq r -d'*' sample.yaml name
```
will output:
```text
- Fred
- Stella
- Android
```
### Arrays[¶](read.md#arrays) <a id="arrays"></a>
You can give an index to access a specific element: e.g.: given a sample file of
```text
b:
e:
- name: fred
value: 3
- name: sam
value: 4
```
then
```text
yq r sample.yaml 'b.e[1].name'
```
will output 'sam'
Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
### Array Splat[¶](read.md#array-splat) <a id="array-splat"></a>
e.g.: given a sample file of
```text
b:
e:
- name: fred
value: 3
- name: sam
value: 4
```
then
```text
yq r sample.yaml 'b.e[*].name'
```
will output:
```text
- fred
- sam
```
Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
### Keys with dots[¶](read.md#keys-with-dots) <a id="keys-with-dots"></a>
When specifying a key that has a dot use key lookup indicator.
```text
b:
foo.bar: 7
```
```text
yaml r sample.yaml 'b[foo.bar]'
```
```text
yaml w sample.yaml 'b[foo.bar]' 9
```
Any valid yaml key can be specified as part of a key lookup.
Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
### Keys \(and values\) with leading dashes[¶](read.md#keys-and-values-with-leading-dashes) <a id="keys-and-values-with-leading-dashes"></a>
If a key or value has leading dashes, yq won't know that you are passing a value as opposed to a flag \(and you will get a 'bad flag syntax' error\).
To fix that, you will need to tell it to stop processing flags by adding '--' after the last flag like so:
```text
yq n -t -- --key --value
```
Will result in
``` --key: --value``

6
scripts/doctools.sh Executable file
View File

@@ -0,0 +1,6 @@
#!/bin/bash
brew install mkdocs libyaml
pip3 install markdown-include
pip3 install mkdocs-material

View File

@@ -1,5 +1,5 @@
name: yq
version: '3.3.3'
version: '3.3.2'
summary: A lightweight and portable command-line YAML processor
description: |
The aim of the project is to be the jq or sed of yaml files.
@@ -16,7 +16,7 @@ apps:
parts:
yq:
plugin: go
go-channel: 1.15/stable
go-channel: 1.14/stable
source: .
source-type: git
go-importpath: github.com/mikefarah/yq

349
write-update.md Normal file
View File

@@ -0,0 +1,349 @@
# Write
```text
yq w <yaml_file> <path> <new value>
```
### To Stdout[¶](write-update.md#to-stdout) <a id="to-stdout"></a>
Given a sample.yaml file of:
```text
b:
c: 2
```
then
```text
yq w sample.yaml b.c cat
```
will output:
```text
b:
c: cat
```
### From STDIN[¶](write-update.md#from-stdin) <a id="from-stdin"></a>
```text
cat sample.yaml | yq w - b.c blah
```
### Adding new fields[¶](write-update.md#adding-new-fields) <a id="adding-new-fields"></a>
Any missing fields in the path will be created on the fly.
Given a sample.yaml file of:
```text
b:
c: 2
```
then
```text
yq w sample.yaml b.d[+] "new thing"
```
will output:
```text
b:
c: cat
d:
- new thing
```
### Splat[¶](write-update.md#splat) <a id="splat"></a>
Given a sample.yaml file of:
```text
---
bob:
item1:
cats: bananas
item2:
cats: apples
thing:
cats: oranges
```
then
```text
yq w sample.yaml bob.*.cats meow
```
will output:
```text
---
bob:
item1:
cats: meow
item2:
cats: meow
thing:
cats: meow
```
### Prefix Splat[¶](write-update.md#prefix-splat) <a id="prefix-splat"></a>
Given a sample.yaml file of:
```text
---
bob:
item1:
cats: bananas
item2:
cats: apples
thing:
cats: oranges
```
then
```text
yq w sample.yaml bob.item*.cats meow
```
will output:
```text
---
bob:
item1:
cats: meow
item2:
cats: meow
thing:
cats: oranges
```
### Array Splat[¶](write-update.md#array-splat) <a id="array-splat"></a>
Given a sample.yaml file of:
```text
---
bob:
- cats: bananas
- cats: apples
- cats: oranges
```
then
```text
yq w sample.yaml bob[*].cats meow
```
will output:
```text
---
bob:
- cats: meow
- cats: meow
- cats: meow
```
### Appending value to an array field[¶](write-update.md#appending-value-to-an-array-field) <a id="appending-value-to-an-array-field"></a>
Given a sample.yaml file of:
```text
b:
c: 2
d:
- new thing
- foo thing
```
then
```text
yq w sample.yaml "b.d[+]" "bar thing"
```
will output:
```text
b:
c: cat
d:
- new thing
- foo thing
- bar thing
```
Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
### Multiple Documents - update a single document[¶](write-update.md#multiple-documents-update-a-single-document) <a id="multiple-documents-update-a-single-document"></a>
Given a sample.yaml file of:
```text
something: else
---
b:
c: 2
```
then
```text
yq w -d1 sample.yaml b.c 5
```
will output:
```text
something: else
---
b:
c: 5
```
### Multiple Documents - update all documents[¶](write-update.md#multiple-documents-update-all-documents) <a id="multiple-documents-update-all-documents"></a>
Given a sample.yaml file of:
```text
something: else
---
b:
c: 2
```
then
```text
yq w -d'*' sample.yaml b.c 5
```
will output:
```text
something: else
b:
c: 5
---
b:
c: 5
```
Note that '\*' is in quotes to avoid being interpreted by your shell.
### Updating files in-place[¶](write-update.md#updating-files-in-place) <a id="updating-files-in-place"></a>
Given a sample.yaml file of:
```text
b:
c: 2
```
then
```text
yq w -i sample.yaml b.c cat
```
will update the sample.yaml file so that the value of 'c' is cat.
### Updating multiple values with a script[¶](write-update.md#updating-multiple-values-with-a-script) <a id="updating-multiple-values-with-a-script"></a>
Given a sample.yaml file of:
```text
b:
c: 2
e:
- name: Billy Bob
```
and a script update\_instructions.yaml of:
```text
b.c: 3
b.e[+].name: Howdy Partner
```
then
```text
yq w -s update_instructions.yaml sample.yaml
```
will output:
```text
b:
c: 3
e:
- name: Howdy Partner
```
And, of course, you can pipe the instructions in using '-':
```text
cat update_instructions.yaml | yq w -s - sample.yaml
```
### Values starting with a hyphen \(or dash\)[¶](write-update.md#values-starting-with-a-hyphen-or-dash) <a id="values-starting-with-a-hyphen-or-dash"></a>
The flag terminator needs to be used to stop the app from attempting to parse the subsequent arguments as flags:
```text
yq w -- my.path -3
```
will output
```text
my:
path: -3
```
### Keys with dots[¶](write-update.md#keys-with-dots) <a id="keys-with-dots"></a>
When specifying a key that has a dot use key lookup indicator.
```text
b:
foo.bar: 7
```
```text
yaml r sample.yaml 'b[foo.bar]'
```
```text
yaml w sample.yaml 'b[foo.bar]' 9
```
Any valid yaml key can be specified as part of a key lookup.
Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
### Keys \(and values\) with leading dashes[¶](write-update.md#keys-and-values-with-leading-dashes) <a id="keys-and-values-with-leading-dashes"></a>
If a key or value has leading dashes, yq won't know that you are passing a value as opposed to a flag \(and you will get a 'bad flag syntax' error\).
To fix that, you will need to tell it to stop processing flags by adding '--' after the last flag like so:
```text
yq n -t -- --key --value
```
Will result in
``` --key: --value``