taigrr/yq
1
0
Fork 0
mirror of https://github.com/taigrr/yq synced 2025-01-18 04:53:17 -08:00
Code Issues Projects Releases Wiki Activity

GitBook: [v3.x] 17 pages modified

Browse Source
This commit is contained in:
Mike Farah 2020-06-16 01:19:04 +00:00 committed by gitbook-bot
parent 086f0ec6b9
commit 85bb1aec16
No known key found for this signature in database
GPG Key ID: 07D2180C7B12D0FF
17 changed files with 2347 additions and 84 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

60
.gitbook/assets/convert.md Normal file
View File

@ -0,0 +1,60 @@
## Yaml to Json
To convert output to json, use the --tojson (or -j) flag. This is supported by all commands.
Each matching yaml node will be converted to json and printed out on a separate line.
Given a sample.yaml file of:
```yaml
b:
c: 2
```
then
```bash
yq r -j sample.yaml
```
will output
```json
{"b":{"c":2}}
```
Given a sample.yaml file of:
```yaml
bob:
c: 2
bab:
c: 5
```
then
```bash
yq r -j sample.yaml b*
```
will output
```json
{"c":2}
{"c":5}
```
## Json to Yaml
To read in json, just pass in a json file instead of yaml, it will just work :)
e.g given a json file
```json
{"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}
```
then
```bash
yq r sample.json
```
will output
```yaml
a: Easy! as one two three
b:
c: 2
d:
- 3
- 4
```

123
README.md
View File

@ -1,137 +1,92 @@
---
description: yq is a lightweight and portable command-line YAML processor
---
# yq
![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg) ![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) ![Github Releases (by Release)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) ![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq)
a lightweight and portable command-line YAML processor
The aim of the project is to be the [jq](https://github.com/stedolan/jq) or sed of yaml files.
## New version!
V3 is officially out - if you've been using v2 and want/need to upgrade, checkout the [upgrade guide](https://mikefarah.gitbook.io/yq/upgrading-from-v2).
![Build](https://github.com/mikefarah/yq/workflows/Build/badge.svg) ![Docker Pulls](https://img.shields.io/docker/pulls/mikefarah/yq.svg) ![Github Releases \(by Release\)](https://img.shields.io/github/downloads/mikefarah/yq/total.svg) ![Go Report](https://goreportcard.com/badge/github.com/mikefarah/yq)
## Install
### [Download the latest binary](https://github.com/mikefarah/yq/releases/latest)
`yq` has pre-built binaries for most platforms - checkout the [releases page](https://github.com/mikefarah/yq/releases) for the latest build. Alternatively - you can use one of the methods below:
### MacOS:
```
### On MacOS:
```bash
brew install yq
```
### Windows:
```
### On Windows:
```bash
choco install yq
```
Supported by @chillum
### Ubuntu and other Linux distros supporting `snap` packages:
```
Kindly maintained by @chillum \([https://github.com/chillum/choco-packages/tree/master/yq](https://github.com/chillum/choco-packages/tree/master/yq)\)
### On Ubuntu and other Linux distributions supporting `snap` packages:
```bash
snap install yq
```
#### Snap notes
`yq` installs with [_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, this means it doesn't have direct access to root files. To read root files you can:
```
sudo cat /etc/myfile | yq r - a.path
`yq` installs with with [_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, this means it doesn't have direct access to root files. To read root files you can:
```bash
sudo cat /etc/myfile | yq -r - somecommand
```
And to write to a root file you can either use [sponge](https://linux.die.net/man/1/sponge):
```bash
sudo cat /etc/myfile | yq -r - somecommand | sudo sponge /etc/myfile
```
sudo cat /etc/myfile | yq w - a.path value | sudo sponge /etc/myfile
```
or write to a temporary file:
```
sudo cat /etc/myfile | yq w - a.path value | sudo tee /etc/myfile.tmp
```bash
sudo cat /etc/myfile | yq -r - somecommand | sudo tee /etc/myfile.tmp
sudo mv /etc/myfile.tmp /etc/myfile
rm /etc/myfile.tmp
```
### On Ubuntu 16.04 or higher from Debian package:
```sh
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys CC86BB64
```bash
sudo add-apt-repository ppa:rmescandon/yq
sudo apt update
sudo apt install yq -y
```
Supported by @rmescandon
### Go Get:
```
Kindly maintained by @rmescandon
### go get:
```text
GO111MODULE=on go get github.com/mikefarah/yq/v3
```
## Run with Docker
## Docker
Oneshot use:
```bash
docker run --rm -v "${PWD}":/workdir mikefarah/yq yq [flags] <command> FILE...
docker run --rm -v ${PWD}:/workdir mikefarah/yq yq [flags] <command> FILE...
```
Run commands interactively:
```bash
docker run --rm -it -v "${PWD}":/workdir mikefarah/yq sh
docker run --rm -it -v ${PWD}:/workdir mikefarah/yq sh
```
It can be useful to have a bash function to avoid typing the whole docker command:
```bash
yq() {
docker run --rm -i -v "${PWD}":/workdir mikefarah/yq yq "$@"
docker run --rm -i -v ${PWD}:/workdir mikefarah/yq yq $@
}
```
## Features
- Written in portable go, so you can download a lovely dependency free binary
- [Colorize the output](https://mikefarah.gitbook.io/yq/usage/output-format#colorize-output)
- [Deep read a yaml file with a given path expression](https://mikefarah.gitbook.io/yq/commands/read#basic)
- [List matching paths of a given path expression](https://mikefarah.gitbook.io/yq/commands/read#path-only)
- [Return the lengths of arrays/object/scalars](https://mikefarah.gitbook.io/yq/commands/read#printing-length-of-the-results)
- Update a yaml file given a [path expression](https://mikefarah.gitbook.io/yq/commands/write-update#basic) or [script file](https://mikefarah.gitbook.io/yq/commands/write-update#basic)
- Update creates any missing entries in the path on the fly
- Deeply [compare](https://mikefarah.gitbook.io/yq/commands/compare) yaml files
- Keeps yaml formatting and comments when updating
- [Validate a yaml file](https://mikefarah.gitbook.io/yq/commands/validate)
- Create a yaml file given a [deep path and value](https://mikefarah.gitbook.io/yq/commands/create#creating-a-simple-yaml-file) or a [script file](https://mikefarah.gitbook.io/yq/commands/create#creating-using-a-create-script)
- [Prefix a path to a yaml file](https://mikefarah.gitbook.io/yq/commands/prefix)
- [Convert to/from json to yaml](https://mikefarah.gitbook.io/yq/usage/convert)
- [Pipe data in by using '-'](https://mikefarah.gitbook.io/yq/commands/read#from-stdin)
- [Merge](https://mikefarah.gitbook.io/yq/commands/merge) multiple yaml files with various options for [overriding](https://mikefarah.gitbook.io/yq/commands/merge#overwrite-values) and [appending](https://mikefarah.gitbook.io/yq/commands/merge#append-values-with-arrays)
- Supports multiple documents in a single yaml file for [reading](https://mikefarah.gitbook.io/yq/commands/read#multiple-documents), [writing](https://mikefarah.gitbook.io/yq/commands/write-update#multiple-documents) and [merging](https://mikefarah.gitbook.io/yq/commands/merge#multiple-documents)
- General shell completion scripts (bash/zsh/fish/powershell) (https://mikefarah.gitbook.io/yq/commands/shell-completion)
## [Usage](https://mikefarah.gitbook.io/yq/)
Check out the [documentation](https://mikefarah.gitbook.io/yq/) for more detailed and advanced usage.
```
Usage:
yq [flags]
yq [command]
Available Commands:
compare yq x [--prettyPrint/-P] dataA.yaml dataB.yaml 'b.e(name==fr*).value'
delete yq d [--inplace/-i] [--doc/-d index] sample.yaml 'b.e(name==fred)'
help Help about any command
merge yq m [--inplace/-i] [--doc/-d index] [--overwrite/-x] [--append/-a] sample.yaml sample2.yaml
new yq n [--script/-s script_file] a.b.c newValue
prefix yq p [--inplace/-i] [--doc/-d index] sample.yaml a.b.c
read yq r [--printMode/-p pv] sample.yaml 'b.e(name==fr*).value'
shell-completion Generates shell completion scripts
validate yq v sample.yaml
write yq w [--inplace/-i] [--script/-s script_file] [--doc/-d index] sample.yaml 'b.e(name==fr*).value' newValue
Flags:
-C, --colors print with colors
-h, --help help for yq
-I, --indent int sets indent level for output (default 2)
-P, --prettyPrint pretty print
-j, --tojson output as json. By default it prints a json document in one line, use the prettyPrint flag to print a formatted doc.
-v, --verbose verbose mode
-V, --version Print version information and quit
Use "yq [command] --help" for more information about a command.
```

25
SUMMARY.md Normal file
View File

@ -0,0 +1,25 @@
# Table of contents
* [yq](README.md)
* [Upgrading from V2](upgrading-from-v2.md)
## Commands
* [Read](commands/read.md)
* [Validate](commands/validate.md)
* [Compare](commands/compare.md)
* [Write](commands/write-update.md)
* [Create](commands/create.md)
* [Delete](commands/delete.md)
* [Merge](commands/merge.md)
* [Prefix](commands/prefix.md)
* [Shell Completion](commands/shell-completion.md)
## Usage
* [Output format](usage/output-format.md)
* [Path Expressions](usage/path-expressions.md)
* [Value Parsing](usage/value-parsing.md)
* [Working with JSON](usage/convert.md)
* [Github Page](https://github.com/mikefarah/yq)

103
commands/compare.md Normal file
View File

@ -0,0 +1,103 @@
---
description: Deeply compare two yaml documents
---
# Compare
```bash
yq compare <yaml_file_1> <yaml_file_2> <path_expression>
```
Compares the matching yaml nodes at path expression in the two yaml documents. See [path expression](../usage/path-expressions.md) for more details. Difference calculated line by line, and is printed out line by line where the first character of each line is either:
* `` a space, indicating no change at this line
* `-` a minus ,indicating the line is not present in the second document \(it's removed\)
* `+` a plus, indicating that the line is not present in the first document \(it's added\)
If there are differences then `yq` will print out the differences and exit with code 1. If there are no differences, then nothing will be printed and the exit code will be 0.
## Example data
Given data1.yaml
```yaml
"apples": are nice
somethingElse: cool # this is nice
favouriteNumbers: [1,2,3]
noDifference: it's the same
```
and data2.yaml
```yaml
apples: are nice
somethingElse: cool # yeah i like it
favouriteNumbers:
- 1
- 3
- 4
noDifference: it's the same
```
## Basic
Basic will compare the yaml documents 'as-is'
```yaml
yq compare data1.yaml data2.yaml
```
yields
```text
-"apples": are nice
-somethingElse: cool # this is nice
-favouriteNumbers: [1, 2, 3]
+apples: are nice
+somethingElse: cool # yeah i like it
+favouriteNumbers:
+- 1
+- 3
+- 4
noDifference: it's the same
```
## Formatted
Most of the time, it will make sense to [format](../usage/output-format.md#pretty-print) the documents before comparing:
```text
yq compare --prettyPrint data1.yaml data2.yml
```
yields
```text
apples: are nice
-somethingElse: cool # this is nice
+somethingElse: cool # yeah i like it
favouriteNumbers:
- 1
-- 2
- 3
+- 4
noDifference: it's the same
```
## Using path expressions
Use [path expressions](../usage/path-expressions.md) to compare subsets of yaml documents
```text
yq compare -P data1.yaml data2.yml favouriteNumbers
```
yields
```text
- 1
-- 2
- 3
+- 4
```

58
commands/create.md Normal file
View File

@ -0,0 +1,58 @@
# Create
```text
yq n <path_expression> <new value>
```
This works in the same way as the write command, but you don't pass in an existing Yaml file. Currently this does not support creating multiple documents in a single yaml file.
See docs for [path expression](../usage/path-expressions.md) and [value parsing](../usage/value-parsing.md) for more details, including controlling quotes and tags.
## Creating a simple yaml file
```bash
yq n b.c cat
```
will output:
```yaml
b:
c: cat
```
## Creating using a create script
Create scripts follow the same format as the update scripts.
Given a script create\_instructions.yaml of:
```yaml
- command: update
path: b.c
value:
#great
things: frog # wow!
```
then
```bash
yq n -s create_instructions.yaml
```
will output:
```yaml
b:
c:
#great
things: frog # wow!
```
You can also pipe the instructions in:
```bash
cat create_instructions.yaml | yq n -s -
```

110
commands/delete.md Normal file
View File

@ -0,0 +1,110 @@
---
description: Deletes all the matching nodes for the path expression in the given yaml input
---
# Delete
```text
yq delete <yaml_file|-> <path_expression>
```
See docs for [path expression](../usage/path-expressions.md) for more details.
## Deleting from a simple document
Given a sample.yaml file of:
```yaml
b:
c: 2
apples: green
```
then
```bash
yq d sample.yaml b.c
```
will output
```yaml
b:
apples: green
```
## From STDIN
Use "-" \(without quotes\) in-place of a file name if you wish to pipe in input from STDIN.
```bash
cat sample.yaml | yq d - b.c
```
## Deleting in-place
```bash
yq d -i sample.yaml b.c
```
will update the sample.yaml file so that the 'c' node is deleted
## Multiple Documents
### Delete from single document
Given a sample.yaml file of:
```yaml
something: else
field: leaveMe
---
b:
c: 2
field: deleteMe
```
then
```bash
yq w -d1 sample.yaml field
```
will output:
```yaml
something: else
field: leaveMe
---
b:
c: 2
```
### Delete from all documents
Given a sample.yaml file of:
```yaml
something: else
field: deleteMe
---
b:
c: 2
field: deleteMeToo
```
then
```bash
yq w -d'*' sample.yaml field
```
will output:
```yaml
something: else
---
b:
c: 2
```

232
commands/merge.md Normal file
View File

@ -0,0 +1,232 @@
---
description: Merge multiple yaml files into a one
---
# Merge
Yaml files can be merged using the 'merge' command. Each additional file merged with the first file will set values for any key not existing already or where the key has no value.
```text
yq m <yaml_file> <yaml_file2> <yaml_file3>...
```
## Merge example
Given a data1.yaml file of:
```yaml
a: simple
b: [1, 2]
```
and data2.yaml file of:
```yaml
a: other
c:
test: 1
```
then
```bash
yq merge data1.yaml data2.yaml
```
will output:
```yaml
a: simple
b: [1, 2]
c:
test: 1
```
## Updating files in-place
```bash
yq m -i data1.yaml data2.yaml
```
will update the data1.yaml file with the merged result.
## Overwrite values
Given a data1.yaml file of:
```yaml
a: simple
b: [1, 2]
d: left alone
```
and data2.yaml file of:
```yaml
a: other
b: [3, 4]
c:
test: 1
```
then
```bash
yq m -x data1.yaml data2.yaml
```
will output:
```yaml
a: other
b: [3, 4]
c:
test: 1
d: left alone
```
Notice that 'b' does not result in the merging of the values within an array.
## Append values with arrays
Given a data1.yaml file of:
```yaml
a: simple
b: [1, 2]
d: hi
```
and data2.yaml file of:
```yaml
a: something
b: [3, 4]
c:
test: 2
other: true
```
then
```bash
yq m -a data1.yaml data2.yaml
```
will output:
```yaml
a: simple
b: [1, 2, 3, 4]
c:
test: 2
other: true
d: hi
```
Note that the 'b' array has concatenated the values from the second data file. Also note that other map keys are not overridden \(field a\).
## Auto-create
By default, `yq` will automatically create any missing entries in the target yaml file. This can be turned off so that only matching paths are merged in. When turned off - you will most likely want to use the [override flag](merge.md#overwrite-values).
Given a data1.yml file of:
```yaml
a: thing
b: something else
```
and data2.yml file of:
```yaml
b: new value
d: not in original
```
Then
```yaml
yq m --overwrite --autocreate=false data1.yml data2.yml
```
will yield
```yaml
a: thing
b: new value
```
## Multiple Documents
### Merge into single document
Currently yq only has multi-document support for the _first_ document being merged into. The remaining yaml files will have their first document selected.
Given a data1.yaml file of:
```yaml
something: else
---
a: simple
b: cat
```
and data3.yaml file of:
```yaml
b: dog
```
then
```bash
yq m -x -d1 data1.yaml data3.yaml
```
will output:
```yaml
something: else
---
a: simple
b: dog
```
### Merge into all documents
Currently yq only has multi-document support for the _first_ document being merged into. The remaining yaml files will have their first document selected.
Given a data1.yaml file of:
```yaml
something: else
---
a: simple
b: cat
```
and data3.yaml file of:
```yaml
b: dog
```
then
```bash
yq m -x -d'*' data1.yaml data3.yaml
```
will output:
```yaml
b: dog
something: else
---
a: simple
b: dog
```

103
commands/prefix.md Normal file
View File

@ -0,0 +1,103 @@
---
description: >-
Prefixes a yaml document with the given path expression. The complete yaml
content will be nested inside the new prefix path.
---
# Prefix
```text
yq p <yaml_file> <path>
```
See docs for [path expression](../usage/path-expressions.md) for more details.
## Prefix a document
Given a data1.yaml file of:
```yaml
a:
b: [1, 2]
```
then
```bash
yq p data1.yaml c.d
```
will output:
```yaml
c:
d:
a:
b: [1, 2]
```
## Updating files in-place
```bash
yq p -i data1.yaml c
```
will update the data1.yaml file so that the path 'c' prefixes the document.
## Multiple Documents
### Prefix a single document
Given a data1.yaml file of:
```yaml
something: else
---
a: simple
b: cat
```
then
```bash
yq p -d1 data1.yaml c
```
will output:
```yaml
something: else
---
c:
a: simple
b: cat
```
### Prefix all documents
Given a data1.yaml file of:
```yaml
something: else
---
a: simple
b: cat
```
then
```bash
yq p -d'*' data1.yaml c
```
will output:
```yaml
c:
something: else
---
c:
a: simple
b: cat
```

326
commands/read.md Normal file
View File

@ -0,0 +1,326 @@
---
description: Returns matching nodes/values of a path expression for a given yaml document
---
# Read
```text
yq r <yaml_file|json_file> <path_expression>
```
Returns the matching nodes of the path expression for the given yaml file \(or STDIN\).
See docs for [path expression](../usage/path-expressions.md) for more details.
## Basic
Given a sample.yaml file of:
```yaml
b:
c: 2
```
then
```bash
yq r sample.yaml b.c
```
will output the value of '2'.
## From Stdin
Given a sample.yaml file of:
```bash
cat sample.yaml | yq r - b.c
```
will output the value of '2'.
## Default values
Using the `--defaultValue/-D` flag you can specify a default value to be printed when no matching nodes are found for an expression
```text
yq r sample.yaml --defaultValue frog path.not.there
```
will yield \(assuming `path.not.there` does not match any nodes\):
```text
frog
```
## Printing matching paths
By default, yq will only print the value of the path expression for the yaml document. By specifying `--printMode` or `-p` you can print the matching paths.
```yaml
a:
thing_a:
animal: cat
other:
animal: frog
thing_b:
vehicle: car
```
### Path Only
```bash
yq r --printMode p "a.thing*.*"
```
will print
```text
a.thing_a.animal
a.thing_b.vehicle
```
### Path and Value
```bash
yq r --printMode pv "a.thing*.*"
```
will print
```text
a.thing_a.animal: cat
a.thing_b.vehicle: car
```
## Collect results into an array
By default, results are printed out line by line as independent matches. This is handy for both readability as well as piping into tools like `xargs`. However, if you would like to collect the matching results into an array then use the `--collect/-C` flag. This is particularly useful with the `length` flag described below.
Given:
```yaml
a:
thing_a:
animal: cat
other:
animal: frog
thing_b:
vehicle: car
```
```text
yq r sample.yaml --collect a.*.animal
```
will print
```text
- cat
- frog
```
## Printing length of the results
Use the `--length/-L` flag to print the length of results. For arrays this will be the number of items, objects the number of entries and scalars the length of the value.
Given
```text
animals:
- cats
- dog
- cheetah
```
```text
yq r sample.yml --length animals
```
will print
```text
3
```
### Length of filtered results
By default, filtered results are printed _independently_ so you will get the length of each result printed on a separate line:
```text
yq r sample.yaml --length --printMode pv 'animals.(.==c*)'
```
```text
animals.[0]: 4
animals.[2]: 7
```
However, you'll often want to know the count of the filtered results - use the `--collect` flag to collect the results in the array. The length will then return the size of the array.
```text
yq r sample.yaml --length --collect 'animals.(.==c*)'
```
```text
2
```
## Anchors and Aliases
The read command will print out the anchors of a document and can also traverse them.
Lets take a look at a simple example file:
```yaml
foo: &foo
a: 1
foobar: *foo
```
### Printing anchors
```bash
yq r sample.yml foo
```
will print out
```yaml
&foo
a: 1
```
Similarly,
```text
yq r sample.yml foobar
```
prints out
```yaml
*foo
```
### Traversing anchors
To traverse an anchor, we need to either explicitly reference merged in values:
```text
yq r sample.yml foobar.a
```
to get
```text
1
```
or we can use deep splat to get all the values
```bash
yq r sample.yml -p pv "foobar.**"
```
prints
```yaml
foobar.a: 1
```
The same methods work for the `<<: [*blah, *thing]`anchors.
### Exploding Anchors
By default anchors are not exploded \(or expanded/de-referenced\) for viewing, and the yaml is shown as-is. Use the `--explodeAnchors/-X` flag to show the anchor values.
Given sample.yml:
```yaml
foo: &foo
a: original
thing: coolasdf
thirsty: yep
bar: &bar
b: 2
thing: coconut
c: oldbar
foobarList:
<<: [*foo,*bar]
c: newbar
```
Then
```text
yq r -X sample.yml foobarList
```
yields
```text
c: newbar
b: 2
thing: coconut
a: original
thirsty: yep
```
Note that yq processes the merge anchor list in reverse order, to ensure that the last items in the list override the preceding.
## Multiple Documents
### Reading from a single document
Given a sample.yaml file of:
```yaml
something: else
---
b:
c: 2
```
then
```bash
yq r -d1 sample.yaml b.c
```
will output the value of '2'.
### Read from all documents
Reading all documents will return the result as an array. This can be converted to json using the '-j' flag if desired.
Given a sample.yaml file of:
```yaml
name: Fred
age: 22
---
name: Stella
age: 23
---
name: Android
age: 232
```
then
```bash
yq r -d'*' sample.yaml name
```
will output:
```text
Fred
Stella
Android
```

51
commands/shell-completion.md Normal file
View File

@ -0,0 +1,51 @@
---
description: >-
Generate a shell completion file for supported shells
(bash/fish/zsh/powershell)
---
# Shell Completion
```bash
yq shell-completion --variation=zsh
```
Prints to StdOut a shell completion script for zsh shell.
### Bash \(default\)
```bash
yq shell-completion
```
To configure your bash shell to load completions for each session add to your bashrc
```text
# ~/.bashrc or ~/.profile
. <(yq shell-completion)
```
### zsh
```bash
yq shell-completion --variation=zsh
```
The generated completion script should be put somewhere in your $fpath named \_yq
### fish
```bash
yq shell-completion --variation=fish
```
Save the output to a '.fish' file and add it to your completions directory.
### PowerShell
```bash
yq shell-completion --variation=powershell
```
Users need PowerShell version 5.0 or above, which comes with Windows 10 and can be downloaded separately for Windows 7 or 8.1. They can then write the completions to a file and source this file from their PowerShell profile, which is referenced by the $Profile environment variable.

54
commands/validate.md Normal file
View File

@ -0,0 +1,54 @@
---
description: Validate a given yaml file
---
# Validate
```text
yq v <yaml_file|->
```
Validates the given yaml file, does nothing if its valid, otherwise it will print errors to Stderr and exit with a non 0 exit code. This works like the [read command](read.md) - but does not take a path expression and does not print the yaml if it is valid.
## Basic - valid
```text
yq v valid.yaml
```
This will not print anything, and finish with a successful \(0\) exit code.
## Basic - invalid, from stdin
```text
echo '[1234' | yq v -
```
This will print the parsing error to stderr:
```bash
10:43:09 main [ERRO] yaml: line 1: did not find expected ',' or ']'
```
And return a error exit code \(1\)
## Multiple documents
Like other commands, by default the validate command will only run against the first document in the yaml file. Note that when running against other specific document indexes, _all previous documents will also be validated._
### Validating a single document
```bash
yq v -d1 multidoc.yml
```
This will validate both document 0 and document 1 \(but not document 2\)
### Validating all documents
```bash
yq v -d'*' multidoc.yml
```
This will validate all documents in the yaml file. Note that \* is quoted to avoid the CLI from processing the wildcard.

285
commands/write-update.md Normal file
View File

@ -0,0 +1,285 @@
---
description: >-
Updates all the matching nodes of path expression in a yaml file to the
supplied value.
---
# Write
```bash
yq w <yaml_file> <path_expression> <new value>
```
See docs for [path expression](../usage/path-expressions.md) and [value parsing](../usage/value-parsing.md) for more details, including controlling quotes and tags.
## Basic
Given a sample.yaml file of:
```yaml
b:
c: 2
```
then
```bash
yq w sample.yaml b.c cat
```
will output:
```yaml
b:
c: cat
```
### Updating files in-place
```bash
yq w -i sample.yaml b.c cat
```
will update the sample.yaml file so that the value of 'c' is cat.
## From STDIN
```bash
cat sample.yaml | yq w - b.c blah
```
## Adding new fields
Any missing fields in the path will be created on the fly.
Given a sample.yaml file of:
```yaml
b:
c: 2
```
then
```bash
yq w sample.yaml b.d[+] "new thing"
```
will output:
```yaml
b:
c: cat
d:
- new thing
```
## Appending value to an array field
Given a sample.yaml file of:
```yaml
b:
c: 2
d:
- new thing
- foo thing
```
then
```bash
yq w sample.yaml "b.d[+]" "bar thing"
```
will output:
```yaml
b:
c: cat
d:
- new thing
- foo thing
- bar thing
```
Note that the path is in quotes to avoid the square brackets being interpreted by your shell.
## Multiple Documents
### Update a single document
Given a sample.yaml file of:
```yaml
something: else
---
b:
c: 2
```
then
```bash
yq w -d1 sample.yaml b.c 5
```
will output:
```yaml
something: else
---
b:
c: 5
```
### Update all documents
Given a sample.yaml file of:
```yaml
something: else
---
b:
c: 2
```
then
```bash
yq w -d'*' sample.yaml b.c 5
```
will output:
```yaml
something: else
b:
c: 5
---
b:
c: 5
```
## Writing Anchors
The `---anchorName` flag can be used to set the anchor name of a node
Given a sample document of:
```yaml
commonStuff:
flavour: vanilla
```
Then:
```bash
yq write sample.yaml commonStuff --anchorName=commonBits
```
Will yield
```yaml
commonStuff: &commonBits
flavour: vanilla
```
## Writing Aliases
The `--makeAlias` flag can create \(or update\) a node to be an alias to an anchor.
Given a sample file of:
```yaml
commonStuff: &commonBits
flavour: vanilla
```
Then
```bash
yq write sample.yaml --makeAnchor foo commonBits
```
Will yield:
```yaml
commonStuff: &commonBits
flavour: vanilla
foo: *commonBits
```
## Updating only styles/tags without affecting values
You can use the write command to update the quoting style of nodes, or their tags, without re-specifying the values. This is done by omitting the value argument:
Given a sample document:
```yaml
a:
c: things
d: other things
```
Then
```bash
yq write sample.yaml --style=single a.*
```
Will yield:
```yaml
a:
c: 'things'
d: 'other things'
```
## Using a script file to update
Given a sample.yaml file of:
```yaml
b:
d: be gone
c: 2
e:
- name: Billy Bob # comment over here
```
and a script update\_instructions.yaml of:
```yaml
- command: update
path: b.c
value:
#great
things: frog # wow!
- command: delete
path: b.d
```
then
```bash
yq w -s update_instructions.yaml sample.yaml
```
will output:
```yaml
b:
c:
#great
things: frog # wow!
e:
- name: Billy Bob # comment over here
```
And, of course, you can pipe the instructions in using '-':
```bash
cat update_instructions.yaml | yq w -s - sample.yaml
```

107
upgrading-from-v2.md Normal file
View File

@ -0,0 +1,107 @@
---
description: New features and breaking changes
---
# Upgrading from V2
## New Features
* Keeps yaml comments and formatting, can specify yaml [tags](usage/value-parsing.md#using-the-tag-field-to-override) when updating.
* Handles anchors!
* Can print out matching paths and values when splatting, more info [here](commands/read.md#printing-matching-paths).
* JSON output works for all commands! Yaml files with multiple documents are printed out as one JSON document per line, more info [here](usage/convert.md)
* Deep splat \(`**`\) to match arbitrary paths and match nodes by their children, more info [here](usage/path-expressions.md)
## Breaking Changes
### Parsing values from the CLI
In V3 users are able to better control how values are treated when updating YAML by using a new `--tag` argument \(see more info [here](usage/value-parsing.md)\). A result of this however, is that quoting values, e.g. "true" will no longer have an effect on how the value is interpreted like it did in V2.
For instance, to get the _string_ "true" into a yaml file:
V2:
```text
yq n a.path '"true"'
```
V3
```text
yq n a.path --tag '!!str' true
```
### Reading paths that don't exist
In V2 this would return null, V3 does not return anything.
Similarly, reading null yaml values `null`, `~` and , V2 returns null whereas V3 returns the values as is.
This is a result of taking effort not to format values coming in and out of the original YAML.
### Update scripts file format has changed to be more powerful.
Comments can be added, and delete commands have been introduced.
V2
```text
b.e[+].name: Mike Farah
```
V3
```yaml
- command: update
path: b.e[+].thing
value:
#great
things: frog # wow!
- command: delete
path: b.d
```
### Reading and splatting, matching results are printed once per line.
e.g:
```yaml
parent:
childA:
no: matches here
childB:
there: matches
hi: no match
there2: also matches
```
```text
yq r sample.yaml 'parent.*.there*'
```
V2
```text
- null
- - matches
- also matches
```
V3
```text
matches
also matches
```
### Converting JSON to YAML
As JSON is a subset of YAML, and `yq` now preserves the formatting of the passed in document, you will most likely need to use the `--prettyPrint` flag to format the JSON document as idiomatic YAML. See [Working with JSON](usage/convert.md#json-to-yaml) for more info.

94
usage/convert.md Normal file
View File

@ -0,0 +1,94 @@
# Working with JSON
## Yaml to Json
To convert output to json, use the `--tojson` \(or `-j`\) flag. This is supported by all commands. You can change the json output format by using the [pretty print](output-format.md#pretty-print) or [indent](output-format.md#indent) flags. _Note that due to the implementation of the JSON marshaller in GO, object keys will be sorted on output \(_[_https://golang.org/pkg/encoding/json/\#Marshal_](https://golang.org/pkg/encoding/json/#Marshal)_\)._
Given a sample.yaml file of:
```yaml
b:
c: 2
```
then
```bash
yq r -j sample.yaml
```
will output
```javascript
{"b":{"c":2}}
```
To format the json:
```yaml
yq r --prettyPrint -j sample.yaml
```
will yield
```yaml
{
"b": {
"c": 2
}
}
```
### Multiple matches
Each matching yaml node will be converted to json and printed out on a separate line. The [prettyPrint](output-format.md#pretty-print) and [indent](output-format.md#indent) flags will still work too. ****
Given a sample.yaml file of:
```yaml
bob:
c: 2
bab:
c: 5
```
then
```bash
yq r -j sample.yaml b*
```
will output
```javascript
{"c":2}
{"c":5}
```
## Json to Yaml
To read in json, just pass in a json file instead of yaml, it will just work - as json is a subset of yaml. However, you will probably want to [pretty print the output](output-format.md#pretty-print) to look more like an idiomatic yaml document.
e.g given a json file
```javascript
{"a":"Easy! as one two three","b":{"c":2,"d":[3,4]}}
```
then
```bash
yq r --prettyPrint sample.json
```
will output
```yaml
a: Easy! as one two three
b:
c: 2
d:
- 3
- 4
```

161
usage/output-format.md Normal file
View File

@ -0,0 +1,161 @@
---
description: Flags to control yaml and json output format
---
# Output format
These flags are available for all `yq` commands.
## Colorize Output
Use the `--colors/-C`flag to print out yaml with colors. This does not work when outputing in JSON format.
## Pretty Print
Use the `--prettyPrint/-P` flag to enforce a formatting style for yaml documents. This is particularly useful when reading a json file \(which is a subset of yaml\) and wanting to format it in a more conventional yaml format.
Given:
```text
{
"apples": [
{
"are": "great"
}
]
}
```
Then:
```text
yq r --prettyPrint sample.json
```
Will print out:
```text
apples:
- are: great
```
This works in the same manner for yaml files:
```text
"apples": [are: great]
```
will format to:
```text
apples:
- are: great
```
## Indent
Use the indent flag `--indent/-I` to control the number of spaces used for indentation. This also works for JSON output. The default value is 2.
Note that lists are indented at the same level as the map key at indent level 2, but are more deeply indented at indent level 4 and greater. This is \(currently\) a quirk of the underlying [yaml parser](https://github.com/go-yaml/yaml/tree/v3).
Given:
```text
apples:
collection:
- name: Green
- name: Blue
favourite: Pink Lady
```
Then:
```text
yq r -I4 sample.yaml
```
Will print out:
```text
apples:
collection:
- name: Green
- name: Blue
favourite: Pink Lady
```
With json, you must also specify the `--prettyPrint/-P` flag
```text
yq r -j -P -I4 sample.yaml
```
yields
```text
{
"apples": {
"collection": [
{
"name": "Green"
},
{
"name": "Blue"
}
],
"favourite": "Pink Lady"
}
}
```
## Unwrap scalars
By default scalar values are 'unwrapped', that is only their value is printed \(except when outputting as JSON\). To print out the node as-is, with the original formatting an any comments pass in `--unwrapScalar=false`
Given data.yml:
```yaml
a: "Things" # cool stuff
```
Then:
`yq r --unwrapScalar=false data.yml a`
Will yield:
```yaml
"Things" # cool stuff
```
where as without setting the flag to false you would get:
```yaml
Things
```
## Strip comments
Use the `--stripComments` flag to print out the yaml file without any of the original comments.
Given data.yml of:
```yaml
a:
b: # there is where the good stuff is
c: hi
```
Then
```yaml
yq r data.yml a --stripComments
```
Will yield:
```yaml
b:
c: hi
```

251
usage/path-expressions.md Normal file
View File

@ -0,0 +1,251 @@
---
description: Path expressions are used to deeply navigate and match particular yaml nodes.
---
# Path Expressions
_As a general rule, you should wrap paths in quotes to prevent your CLI from processing `*`, `[]` and other special characters._
## Simple expressions
### Maps
`'a.b.c'`
```yaml
a:
b:
c: thing # MATCHES
```
### Arrays
`'a.b[1].c'`
```yaml
a:
b:
- c: thing0
- c: thing1 # MATCHES
- c: thing2
```
#### Appending to arrays
\(e.g. when using the write command\)
`'a.b[+].c'`
```yaml
a:
b:
- c: thing0
```
Will add a new entry:
```yaml
a:
b:
- c: thing0
- c: thing1 # NEW entry from [+] on B array.
```
#### Negative Array indexes
Negative array indexes can be used to traverse the array in reverse
`'a.b[-1].c'`
Will access the last element in the `b` array and yield:
```yaml
thing2
```
## Splat
### Maps
`'a.*.c'`
```yaml
a:
b1:
c: thing # MATCHES
d: whatever
b2:
c: thing # MATCHES
f: something irrelevant
```
#### Prefix splat
`'bob.item*.cats'`
```yaml
bob:
item:
cats: bananas # MATCHES
something:
cats: lemons
itemThing:
cats: more bananas # MATCHES
item2:
cats: apples # MATCHES
thing:
cats: oranges
```
### Arrays
`'a.b[*].c'`
```yaml
a:
b:
- c: thing0 # MATCHES
d: what..ever
- c: thing1 # MATCHES
d: blarh
- c: thing2 # MATCHES
f: thingamabob
```
## Deep Splat
`**` will match arbitrary nodes for both maps and arrays:
`'a.**.c'`
```yaml
a:
b1:
c: thing1 # MATCHES
d: cat cat
b2:
c: thing2 # MATCHES
d: dog dog
b3:
d:
- f:
c: thing3 # MATCHES
d: beep
- f:
g:
c: thing4 # MATCHES
d: boop
- d: mooo
```
## Search by children nodes
You can search children by nodes - note that this will return the _parent_ of the matching expression, in the example below the parent\(s\) will be the matching indices of the 'a' array. We can then navigate down to get 'b.c' of each matching indice.
`'a.(b.d==cat).b.c'`
```yaml
a:
- b:
c: thing0
d: leopard
ba: fast
- b:
c: thing1 # MATCHES
d: cat
ba: meowy
- b:
c: thing2
d: caterpillar
ba: icky
- b:
c: thing3 # MATCHES
d: cat
ba: also meowy
```
### With prefixes
`'a.(b.d==cat*).c'`
```yaml
a:
- b:
c: thing0
d: leopard
ba: fast
- b:
c: thing1 # MATCHES
d: cat
ba: meowy
- b:
c: thing2 # MATCHES
d: caterpillar
ba: icky
- b:
c: thing3 # MATCHES
d: cat
ba: also meowy
```
### Matching children values
`'animals(.==cat)'`
```yaml
animals:
- dog
- cat # MATCHES
- rat
```
this also works in maps, and with prefixes
`'animals(.==c*)'`
```yaml
animals:
friendliest: cow # MATCHES
favourite: cat # MATCHES
smallest: rat
```
## Special Characters
When your yaml field has special characters that overlap with `yq` path expression characters, you will need to escape them in order for the command to work.
### Keys with dots
When specifying a key that has a dot use key lookup indicator.
```yaml
b:
foo.bar: 7
```
```bash
yaml r sample.yaml 'b."foo.bar"'
```
```bash
yaml w sample.yaml 'b."foo.bar"' 9
```
Any valid yaml key can be specified as part of a key lookup.
Note that the path is in single quotes to avoid the double quotes being interpreted by your shell.
### Keys \(and values\) with leading dashes
The flag terminator needs to be used to stop the app from attempting to parse the subsequent arguments as flags, if they start if a dash.
```bash
yq n -j -- --key --value
```
Will result in
```text
--key: --value
```

288
usage/value-parsing.md Normal file
View File

@ -0,0 +1,288 @@
---
description: >-
How values are parsed from the CLI to commands that create/update yaml (e.g.
new/write).
---
# Value Parsing
`yq` attempts to parse values intelligently, e.g. when a number is passed it - it will assume it's a number as opposed to a string. `yq` will not alter the representation of what you give. So if you pass '03.0' in, it will assume it's a number and keep the value formatted as it was passed in, that is '03.0'.
The `--tag` flag can be used to override the tag type to force particular tags.
## Default behavior
### Integers
_Given_
```bash
yq new key 3
```
results in
```yaml
key: 3
```
_Given a formatted number_
```bash
yq new key 03
```
results in
```yaml
key: 03
```
`yq` keeps the number formatted as it was passed in.
### Float
_Given_
```bash
yq new key "3.1"
```
results in
```yaml
key: 3.1
```
Note that quoting the number does not make a difference.
_Given a formatted decimal number_
```bash
yq new key 03.0
```
results in
```yaml
key: 03.0
```
`yq` keeps the number formatted as it was passed in
### Booleans
```bash
yq new key true
```
results in
```yaml
key: true
```
### Nulls
```bash
yq new key null
```
results in
```yaml
key: null
```
```bash
yq new key '~'
```
results in
```yaml
key: ~
```
```bash
yq new key ''
```
results in
```yaml
key:
```
### Strings
```bash
yq new key whatever
```
results in
```yaml
key: whatever
```
```bash
yq new key ' whatever '
```
results in
```yaml
key: ' whatever '
```
## Using the tag flag to cast
Previous versions of yq required double quoting to force values to be strings, this no longer works - instead use the --tag flag.
### Casting booleans
```bash
yq new --tag '!!str' key true
```
results in
```yaml
key: "true"
```
### Casting nulls
```bash
yq new --tag '!!str' key null
```
results in
```yaml
key: "null"
```
### Custom types
```bash
yq new --tag '!!farah' key gold
```
results in
```yaml
key: !!farah gold
```
## The style flag
The `--style` flag can be used to specify the quote or block style of the node value. Valid values are
* single
* double
* folded
* flow
* literal
* tagged
For example, given:
```bash
MULTILINE=$(cat <<END
This is line one.
This is line two.
END
)
SINGLE="only one line"
```
### Single
```yaml
yq n --style single things "$MULTILINE"
```
```yaml
things: 'This is line one.
This is line two.'
```
### Double
```yaml
things: "This is line one.\nThis is line two."
```
### Folded:
```yaml
things: >-
This is line one.
This is line two.
```
#### Folded single line:
```yaml
things: >-
only one line
```
### Flow:
```yaml
things: |-
This is line one.
This is line two.
```
#### Flow single line:
```yaml
things: only one line
```
### Literal
```yaml
things: |-
This is line one.
This is line two.
```
#### Literal single line
```yaml
things: |-
only one line
```
### Tagged
Always show the tag, note - you must also pass in `--tag='!!str'`
```yaml
things: !!str |-
This is line one.
This is line two.
```
#### Tagged single line
```yaml
things: !!str only one line
```