mirror of
https://github.com/taigrr/yq
synced 2025-01-18 04:53:17 -08:00
Compare commits
58 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
fae2b2643c | ||
|
|
dd86b5e7f2 | ||
|
|
f1f75683c1 | ||
|
|
38b9856f50 | ||
|
|
48eeb2a9df | ||
|
|
af283315f2 | ||
|
|
d18a6963f6 | ||
|
|
77edbb9f5c | ||
|
|
179c44aacc | ||
|
|
bc70c1fb16 | ||
|
|
0b71a40797 | ||
|
|
3f51a44596 | ||
|
|
afebf0e621 | ||
|
|
dc464a5b10 | ||
|
|
5340ed0ad3 | ||
|
|
7b52c5fe0e | ||
|
|
f4392f8658 | ||
|
|
8e14b3b393 | ||
|
|
8627441705 | ||
|
|
aa95ecd012 | ||
|
|
a2bd463a91 | ||
|
|
6c3965dca3 | ||
|
|
bb3ffd40b5 | ||
|
|
cc08afc435 | ||
|
|
941a453163 | ||
|
|
77630ca179 | ||
|
|
ae4b606707 | ||
|
|
37f3e21970 | ||
|
|
25d0787011 | ||
|
|
b5b8da0a1d | ||
|
|
fa21510194 | ||
|
|
f541194250 | ||
|
|
38666f4db6 | ||
|
|
8b327d0414 | ||
|
|
c8630fe4f3 | ||
|
|
87df9b1ae6 | ||
|
|
2483c38eeb | ||
|
|
b2a538bdfc | ||
|
|
6c26344449 | ||
|
|
daf0bfe1b9 | ||
|
|
750a00ec35 | ||
|
|
25e0a824c5 | ||
|
|
095f921f62 | ||
|
|
12d3425b4a | ||
|
|
0249f00bd5 | ||
|
|
42b1bf9678 | ||
|
|
c632fd3641 | ||
|
|
0470e5cd69 | ||
|
|
35c6d07248 | ||
|
|
3f0be107d4 | ||
|
|
21a9e506cb | ||
|
|
3722367fbb | ||
|
|
f7b50e9853 | ||
|
|
4f3fe256aa | ||
|
|
2595a929c9 | ||
|
|
b2186d5404 | ||
|
|
e93c43f7a0 | ||
|
|
b1a5387cdd |
@ -1 +1 @@
|
||||
bin
|
||||
bin/*
|
||||
|
||||
@ -6,26 +6,5 @@ RUN set -e -x \
|
||||
&& /opt/devtools.sh
|
||||
ENV PATH=/go/bin:$PATH
|
||||
|
||||
# install mkdocs
|
||||
RUN set -ex \
|
||||
&& buildDeps=' \
|
||||
build-essential \
|
||||
python3-dev \
|
||||
' \
|
||||
&& apt-get update && apt-get install -y --no-install-recommends \
|
||||
$buildDeps \
|
||||
python3 \
|
||||
python3-setuptools \
|
||||
python3-wheel \
|
||||
python3-pip \
|
||||
&& pip3 install --upgrade \
|
||||
pip \
|
||||
'Markdown>=2.6.9' \
|
||||
'mkdocs>=0.16.3' \
|
||||
'mkdocs-material>=1.10.1' \
|
||||
'markdown-include>=0.5.1' \
|
||||
&& apt-get purge -y --auto-remove $buildDeps \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
ENV CGO_ENABLED 0
|
||||
ENV GOPATH /go:/yq
|
||||
|
||||
10
Makefile
10
Makefile
@ -17,6 +17,7 @@ help:
|
||||
@echo ' make vendor Install dependencies to vendor directory.'
|
||||
@echo ' make format Run code formatter.'
|
||||
@echo ' make check Run static code analysis (lint).'
|
||||
@echo ' make secure Run gosec.'
|
||||
@echo ' make test Run tests on project.'
|
||||
@echo ' make cover Run tests and capture code coverage metrics on project.'
|
||||
@echo ' make clean Clean the directory tree of produced artifacts.'
|
||||
@ -84,6 +85,10 @@ format: vendor
|
||||
check: format
|
||||
${DOCKRUN} bash ./scripts/check.sh
|
||||
|
||||
.PHONY: secure
|
||||
secure:
|
||||
${DOCKRUN} bash ./scripts/secure.sh
|
||||
|
||||
.PHONY: test
|
||||
test: check
|
||||
${DOCKRUN} bash ./scripts/test.sh
|
||||
@ -96,11 +101,6 @@ cover: check
|
||||
@find cover -type d -exec chmod 755 {} \; || :
|
||||
@find cover -type f -exec chmod 644 {} \; || :
|
||||
|
||||
.PHONY: build-docs
|
||||
build-docs: prepare mkdocs.yml mkdocs/*
|
||||
${DOCKRUN} mkdocs build
|
||||
@find docs -type d -exec chmod 755 {} \; || :
|
||||
@find docs -type f -exec chmod 644 {} \; || :
|
||||
|
||||
.PHONY: release
|
||||
release: xcompile
|
||||
|
||||
12
README.md
12
README.md
@ -119,6 +119,8 @@ See [webi](https://webinstall.dev/)
|
||||
Supported by @adithyasunil26 (https://github.com/webinstall/webi-installers/tree/master/yq)
|
||||
|
||||
### Windows:
|
||||
[](https://chocolatey.org/packages/yq)
|
||||
[](https://chocolatey.org/packages/yq)
|
||||
```
|
||||
choco install yq
|
||||
```
|
||||
@ -151,19 +153,21 @@ sudo apt install yq -y
|
||||
Supported by @rmescandon (https://launchpad.net/~rmescandon/+archive/ubuntu/yq)
|
||||
|
||||
## Features
|
||||
- [Detailed documentation with many examples](https://mikefarah.gitbook.io/yq/)
|
||||
- Written in portable go, so you can download a lovely dependency free binary
|
||||
- Uses similar syntax as `jq` but works with YAML and JSON files
|
||||
- Fully supports multi document yaml files
|
||||
- Colorized yaml output
|
||||
- [Deeply traverse yaml](https://mikefarah.gitbook.io/yq/v/v4.x/traverse)
|
||||
- [Sort yaml by keys](https://mikefarah.gitbook.io/yq/v/v4.x/sort-keys)
|
||||
- Manipulate yaml [comments](https://mikefarah.gitbook.io/yq/comment-operators), [styling](https://mikefarah.gitbook.io/yq/style), [tags](https://mikefarah.gitbook.io/yq/tag) and [anchors and aliases](https://mikefarah.gitbook.io/yq/anchor-and-alias-operators).
|
||||
- [Deeply traverse yaml](https://mikefarah.gitbook.io/yq/operators/traverse-read)
|
||||
- [Sort yaml by keys](https://mikefarah.gitbook.io/yq/operators/sort-keys)
|
||||
- Manipulate yaml [comments](https://mikefarah.gitbook.io/yq/operators/comment-operators), [styling](https://mikefarah.gitbook.io/yq/operators/style), [tags](https://mikefarah.gitbook.io/yq/operators/tag) and [anchors and aliases](https://mikefarah.gitbook.io/yq/operators/anchor-and-alias-operators).
|
||||
- [Update yaml inplace](https://mikefarah.gitbook.io/yq/v/v4.x/commands/evaluate#flags)
|
||||
- [Complex expressions to select and update](https://mikefarah.gitbook.io/yq/v/v4.x/select#select-and-update-matching-values-in-map)
|
||||
- [Complex expressions to select and update](https://mikefarah.gitbook.io/yq/operators/select#select-and-update-matching-values-in-map)
|
||||
- Keeps yaml formatting and comments when updating (though there are issues with whitespace)
|
||||
- [Convert to/from json to yaml](https://mikefarah.gitbook.io/yq/v/v4.x/usage/convert)
|
||||
- [Pipe data in by using '-'](https://mikefarah.gitbook.io/yq/v/v4.x/commands/evaluate)
|
||||
- [General shell completion scripts (bash/zsh/fish/powershell)](https://mikefarah.gitbook.io/yq/v/v4.x/commands/shell-completion)
|
||||
- [Reduce](https://mikefarah.gitbook.io/yq/operators/reduce) to merge multiple files or sum an array or other fancy things.
|
||||
|
||||
## [Usage](https://mikefarah.gitbook.io/yq/)
|
||||
|
||||
|
||||
@ -17,6 +17,9 @@ func createEvaluateAllCommand() *cobra.Command {
|
||||
Example: `
|
||||
# merges f2.yml into f1.yml (inplace)
|
||||
yq eval-all --inplace 'select(fileIndex == 0) * select(fileIndex == 1)' f1.yml f2.yml
|
||||
|
||||
# use '-' as a filename to read from STDIN
|
||||
cat file2.yml | yq ea '.a.b' file1.yml - file3.yml
|
||||
`,
|
||||
Long: "Evaluate All:\nUseful when you need to run an expression across several yaml documents or files. Consumes more memory than eval",
|
||||
RunE: evaluateAll,
|
||||
|
||||
@ -21,6 +21,9 @@ yq e '.a.b | length' f1.yml f2.yml
|
||||
# prints out the file
|
||||
yq e sample.yaml
|
||||
|
||||
# use '-' as a filename to read from STDIN
|
||||
cat file2.yml | yq e '.a.b' file1.yml - file3.yml
|
||||
|
||||
# prints a new yaml document
|
||||
yq e -n '.a.b.c = "cat"'
|
||||
|
||||
|
||||
@ -11,7 +11,7 @@ var (
|
||||
GitDescribe string
|
||||
|
||||
// Version is main version number that is being run at the moment.
|
||||
Version = "4.6.1"
|
||||
Version = "4.9.3"
|
||||
|
||||
// VersionPrerelease is a pre-release marker for the version. If this is "" (empty string)
|
||||
// then it means that it is a final release. Otherwise, this is a pre-release
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
FROM mikefarah/yq:4.6.1
|
||||
FROM mikefarah/yq:4.9.3
|
||||
|
||||
COPY entrypoint.sh /entrypoint.sh
|
||||
|
||||
|
||||
8
go.mod
8
go.mod
@ -3,12 +3,12 @@ module github.com/mikefarah/yq/v4
|
||||
require (
|
||||
github.com/elliotchance/orderedmap v1.4.0
|
||||
github.com/fatih/color v1.10.0
|
||||
github.com/goccy/go-yaml v1.8.8
|
||||
github.com/jinzhu/copier v0.2.3
|
||||
github.com/spf13/cobra v1.1.1
|
||||
github.com/goccy/go-yaml v1.8.9
|
||||
github.com/jinzhu/copier v0.2.8
|
||||
github.com/spf13/cobra v1.1.3
|
||||
github.com/timtadh/data-structures v0.5.3 // indirect
|
||||
github.com/timtadh/lexmachine v0.2.2
|
||||
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c // indirect
|
||||
golang.org/x/sys v0.0.0-20210317225723-c4fcb01b228e // indirect
|
||||
gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473
|
||||
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
|
||||
)
|
||||
|
||||
26
go.sum
26
go.sum
@ -52,8 +52,8 @@ github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTM
|
||||
github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
|
||||
github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4=
|
||||
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
|
||||
github.com/goccy/go-yaml v1.8.8 h1:MGfRB1GeSn/hWXYWS2Pt67iC2GJNnebdIro01ddyucA=
|
||||
github.com/goccy/go-yaml v1.8.8/go.mod h1:U/jl18uSupI5rdI2jmuCswEA2htH9eXfferR3KfscvA=
|
||||
github.com/goccy/go-yaml v1.8.9 h1:4AEXg2qx+/w29jXnXpMY6mTckmYu1TMoHteKuMf0HFg=
|
||||
github.com/goccy/go-yaml v1.8.9/go.mod h1:U/jl18uSupI5rdI2jmuCswEA2htH9eXfferR3KfscvA=
|
||||
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
|
||||
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
|
||||
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
|
||||
@ -101,8 +101,8 @@ github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2p
|
||||
github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=
|
||||
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
|
||||
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
|
||||
github.com/jinzhu/copier v0.2.3 h1:Oe09ju+9qft7TffZ7l/04AB2f8u1+V4ZMxmp/nnqeOs=
|
||||
github.com/jinzhu/copier v0.2.3/go.mod h1:24xnZezI2Yqac9J61UC6/dG/k76ttpq0DdJI3QmUvro=
|
||||
github.com/jinzhu/copier v0.2.8 h1:N8MbL5niMwE3P4dOwurJixz5rMkKfujmMRFmAanSzWE=
|
||||
github.com/jinzhu/copier v0.2.8/go.mod h1:24xnZezI2Yqac9J61UC6/dG/k76ttpq0DdJI3QmUvro=
|
||||
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
|
||||
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
|
||||
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
|
||||
@ -168,10 +168,9 @@ github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4k
|
||||
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
|
||||
github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
|
||||
github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
|
||||
github.com/spf13/cobra v1.1.1 h1:KfztREH0tPxJJ+geloSLaAkaPkr4ki2Er5quFV1TDo4=
|
||||
github.com/spf13/cobra v1.1.1/go.mod h1:WnodtKOvamDL/PwE2M4iKs8aMDBZ5Q5klgD3qfVJQMI=
|
||||
github.com/spf13/cobra v1.1.3 h1:xghbfqPkxzxP3C/f3n5DdpAbdKLj4ZE4BWQI362l53M=
|
||||
github.com/spf13/cobra v1.1.3/go.mod h1:pGADOWyqRD/YMrPZigI/zbliZ2wVD/23d+is3pSWzOo=
|
||||
github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
|
||||
github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
|
||||
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
|
||||
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
|
||||
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
||||
@ -180,7 +179,6 @@ github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+
|
||||
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
|
||||
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
|
||||
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
|
||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
@ -257,10 +255,9 @@ golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7w
|
||||
golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae h1:/WDfKMnPU+m5M4xB+6x4kaepxRw6jWvR5iDRdvjHgy8=
|
||||
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c h1:VwygUrnw9jn88c4u8GD3rZQbqrP/tgas88tPUbBxQrk=
|
||||
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210317225723-c4fcb01b228e h1:XNp2Flc/1eWQGk5BLzqTAN7fQIwIbfyVTuVxXxZh73M=
|
||||
golang.org/x/sys v0.0.0-20210317225723-c4fcb01b228e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
|
||||
@ -270,7 +267,6 @@ golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGm
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
|
||||
golang.org/x/tools v0.0.0-20190311212946-11955173bddd h1:/e+gpKk9r3dJobndpTytxS2gOy6m5uvpg+ISQoEcusQ=
|
||||
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
|
||||
golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
|
||||
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
|
||||
@ -283,7 +279,6 @@ golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgw
|
||||
golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc h1:NCy3Ohtk6Iny5V/reW2Ktypo4zIpWBdRJ1uFMjBxdg8=
|
||||
golang.org/x/tools v0.0.0-20191112195655-aa38f8e97acc/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
|
||||
@ -310,7 +305,6 @@ google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZi
|
||||
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
|
||||
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
|
||||
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
|
||||
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
@ -321,11 +315,9 @@ gopkg.in/op/go-logging.v1 v1.0.0-20160211212156-b2cb9fa56473/go.mod h1:N1eN2tsCx
|
||||
gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
|
||||
gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
|
||||
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
|
||||
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
|
||||
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
|
||||
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=
|
||||
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
|
||||
@ -9,6 +9,7 @@ import (
|
||||
|
||||
type CandidateNode struct {
|
||||
Node *yaml.Node // the actual node
|
||||
Parent *CandidateNode // parent node
|
||||
Path []interface{} /// the path we took to get to this node
|
||||
Document uint // the document index of this node
|
||||
Filename string
|
||||
@ -31,6 +32,7 @@ func (n *CandidateNode) CreateChild(path interface{}, node *yaml.Node) *Candidat
|
||||
return &CandidateNode{
|
||||
Node: node,
|
||||
Path: n.createChildPath(path),
|
||||
Parent: n,
|
||||
Document: n.Document,
|
||||
Filename: n.Filename,
|
||||
FileIndex: n.FileIndex,
|
||||
@ -86,7 +88,14 @@ func (n *CandidateNode) UpdateAttributesFrom(other *CandidateNode) {
|
||||
if n.Node.Style == 0 {
|
||||
n.Node.Style = other.Node.Style
|
||||
}
|
||||
n.Node.FootComment = n.Node.FootComment + other.Node.FootComment
|
||||
n.Node.HeadComment = n.Node.HeadComment + other.Node.HeadComment
|
||||
n.Node.LineComment = n.Node.LineComment + other.Node.LineComment
|
||||
|
||||
if other.Node.FootComment != "" {
|
||||
n.Node.FootComment = other.Node.FootComment
|
||||
}
|
||||
if other.Node.HeadComment != "" {
|
||||
n.Node.HeadComment = other.Node.HeadComment
|
||||
}
|
||||
if other.Node.LineComment != "" {
|
||||
n.Node.LineComment = other.Node.LineComment
|
||||
}
|
||||
}
|
||||
|
||||
@ -12,6 +12,14 @@ type Context struct {
|
||||
DontAutoCreate bool
|
||||
}
|
||||
|
||||
func (n *Context) SingleReadonlyChildContext(candidate *CandidateNode) Context {
|
||||
list := list.New()
|
||||
list.PushBack(candidate)
|
||||
newContext := n.ChildContext(list)
|
||||
newContext.DontAutoCreate = true
|
||||
return newContext
|
||||
}
|
||||
|
||||
func (n *Context) SingleChildContext(candidate *CandidateNode) Context {
|
||||
list := list.New()
|
||||
list.PushBack(candidate)
|
||||
@ -52,3 +60,9 @@ func (n *Context) Clone() Context {
|
||||
}
|
||||
return clone
|
||||
}
|
||||
|
||||
func (n *Context) ReadOnlyClone() Context {
|
||||
clone := n.Clone()
|
||||
clone.DontAutoCreate = true
|
||||
return clone
|
||||
}
|
||||
|
||||
@ -101,7 +101,7 @@ will output
|
||||
- hello
|
||||
```
|
||||
|
||||
## Update array (append)
|
||||
## Append to array
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
@ -127,6 +127,36 @@ b:
|
||||
- 4
|
||||
```
|
||||
|
||||
## Relative append
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
a1:
|
||||
b:
|
||||
- cat
|
||||
a2:
|
||||
b:
|
||||
- dog
|
||||
a3: {}
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a[].b += ["mouse"]' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
a1:
|
||||
b:
|
||||
- cat
|
||||
- mouse
|
||||
a2:
|
||||
b:
|
||||
- dog
|
||||
- mouse
|
||||
a3: {b: [mouse]}
|
||||
```
|
||||
|
||||
## String concatenation
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@ -143,22 +173,6 @@ a: catmeow
|
||||
b: meow
|
||||
```
|
||||
|
||||
## Relative string concatenation
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: cat
|
||||
b: meow
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a += .b' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: catmeow
|
||||
b: meow
|
||||
```
|
||||
|
||||
## Number addition - float
|
||||
If the lhs or rhs are floats then the expression will be calculated with floats.
|
||||
|
||||
@ -195,18 +209,20 @@ a: 7
|
||||
b: 4
|
||||
```
|
||||
|
||||
## Increment number
|
||||
## Increment numbers
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 3
|
||||
b: 5
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a += 1' sample.yml
|
||||
yq eval '.[] += 1' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: 4
|
||||
b: 6
|
||||
```
|
||||
|
||||
## Add to null
|
||||
|
||||
@ -1,8 +1,99 @@
|
||||
Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference aliases and remove anchor names).
|
||||
Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference (or expands) aliases and remove anchor names).
|
||||
|
||||
`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag.
|
||||
|
||||
|
||||
## Merge one map
|
||||
see https://yaml.org/type/merge.html
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- &CENTER
|
||||
x: 1
|
||||
y: 2
|
||||
- &LEFT
|
||||
x: 0
|
||||
y: 2
|
||||
- &BIG
|
||||
r: 10
|
||||
- &SMALL
|
||||
r: 1
|
||||
- !!merge <<: *CENTER
|
||||
r: 10
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.[4] | explode(.)' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
x: 1
|
||||
y: 2
|
||||
r: 10
|
||||
```
|
||||
|
||||
## Merge multiple maps
|
||||
see https://yaml.org/type/merge.html
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- &CENTER
|
||||
x: 1
|
||||
y: 2
|
||||
- &LEFT
|
||||
x: 0
|
||||
y: 2
|
||||
- &BIG
|
||||
r: 10
|
||||
- &SMALL
|
||||
r: 1
|
||||
- !!merge <<:
|
||||
- *CENTER
|
||||
- *BIG
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.[4] | explode(.)' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
r: 10
|
||||
x: 1
|
||||
y: 2
|
||||
```
|
||||
|
||||
## Override
|
||||
see https://yaml.org/type/merge.html
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- &CENTER
|
||||
x: 1
|
||||
y: 2
|
||||
- &LEFT
|
||||
x: 0
|
||||
y: 2
|
||||
- &BIG
|
||||
r: 10
|
||||
- &SMALL
|
||||
r: 1
|
||||
- !!merge <<:
|
||||
- *BIG
|
||||
- *LEFT
|
||||
- *SMALL
|
||||
x: 1
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.[4] | explode(.)' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
r: 10
|
||||
x: 1
|
||||
y: 2
|
||||
```
|
||||
|
||||
## Get anchor
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@ -78,6 +169,22 @@ b: &meow purr
|
||||
a: *meow
|
||||
```
|
||||
|
||||
## Set alias to blank does nothing
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
b: &meow purr
|
||||
a: cat
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a alias = ""' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
b: &meow purr
|
||||
a: cat
|
||||
```
|
||||
|
||||
## Set alias relatively using assign-update
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@ -183,9 +290,9 @@ bar:
|
||||
c: bar_c
|
||||
foobarList:
|
||||
b: bar_b
|
||||
a: foo_a
|
||||
thing: bar_thing
|
||||
thing: foo_thing
|
||||
c: foobarList_c
|
||||
a: foo_a
|
||||
foobar:
|
||||
c: foo_c
|
||||
a: foo_a
|
||||
|
||||
@ -1,10 +1,4 @@
|
||||
This operator is used to update node values. It can be used in either the:
|
||||
|
||||
### plain form: `=`
|
||||
Which will assign the LHS node values to the RHS node values. The RHS expression is run against the matching nodes in the pipeline.
|
||||
|
||||
### relative form: `|=`
|
||||
This will do a similar thing to the plain form, however, the RHS expression is run against _the LHS nodes_. This is useful for updating values based on old values, e.g. increment.
|
||||
## Create yaml file
|
||||
Running
|
||||
```bash
|
||||
@ -1,5 +1,14 @@
|
||||
The `or` and `and` operators take two parameters and return a boolean result. `not` flips a boolean from true to false, or vice versa. These are most commonly used with the `select` operator to filter particular nodes.
|
||||
## OR example
|
||||
The `or` and `and` operators take two parameters and return a boolean result.
|
||||
|
||||
`not` flips a boolean from true to false, or vice versa.
|
||||
|
||||
`any` will return `true` if there are any `true` values in a array sequence, and `all` will return true if _all_ elements in an array are true.
|
||||
|
||||
`any_c(condition)` and `all_c(condition)` are like `any` and `all` but they take a condition expression that is used against each element to determine if it's `true`. Note: in `jq` you can simply pass a condition to `any` or `all` and it simply works - `yq` isn't that clever..yet
|
||||
|
||||
These are most commonly used with the `select` operator to filter particular nodes.
|
||||
|
||||
## `or` example
|
||||
Running
|
||||
```bash
|
||||
yq eval --null-input 'true or false'
|
||||
@ -9,7 +18,7 @@ will output
|
||||
true
|
||||
```
|
||||
|
||||
## AND example
|
||||
## `and` example
|
||||
Running
|
||||
```bash
|
||||
yq eval --null-input 'true and false'
|
||||
@ -41,6 +50,104 @@ will output
|
||||
b: fly
|
||||
```
|
||||
|
||||
## `any` returns true if any boolean in a given array is true
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- false
|
||||
- true
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'any' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
true
|
||||
```
|
||||
|
||||
## `any` returns false for an empty array
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
[]
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'any' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
false
|
||||
```
|
||||
|
||||
## `any_c` returns true if any element in the array is true for the given condition.
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
- rad
|
||||
- awesome
|
||||
b:
|
||||
- meh
|
||||
- whatever
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.[] |= any_c(. == "awesome")' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: true
|
||||
b: false
|
||||
```
|
||||
|
||||
## `all` returns true if all booleans in a given array are true
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- true
|
||||
- true
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'all' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
true
|
||||
```
|
||||
|
||||
## `all` returns true for an empty array
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
[]
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'all' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
true
|
||||
```
|
||||
|
||||
## `all_c` returns true if all elements in the array are true for the given condition.
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
- rad
|
||||
- awesome
|
||||
b:
|
||||
- meh
|
||||
- 12
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.[] |= all_c(tag == "!!str")' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: true
|
||||
b: false
|
||||
```
|
||||
|
||||
## Not true is false
|
||||
Running
|
||||
```bash
|
||||
|
||||
@ -1,4 +1,15 @@
|
||||
Use these comment operators to set or retrieve comments.
|
||||
|
||||
Like the `=` and `|=` assign operators, the same syntax applies when updating comments:
|
||||
|
||||
|
||||
### plain form: `=`
|
||||
This will assign the LHS nodes comments to the expression on the RHS. The RHS is run against the matching nodes in the pipeline
|
||||
|
||||
### relative form: `|=`
|
||||
Similar to the plain form, however the RHS evaluates against each matching LHS node! This is useful if you want to set the comments as a relative expression of the node, for instance its value or path.
|
||||
|
||||
|
||||
## Set line comment
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@ -77,7 +88,7 @@ a: cat
|
||||
b: dog # leave this
|
||||
```
|
||||
|
||||
## Remove all comments
|
||||
## Remove (strip) all comments
|
||||
Note the use of `...` to ensure key nodes are included.
|
||||
|
||||
Given a sample.yml file of:
|
||||
|
||||
100
pkg/yqlib/doc/Entries.md
Normal file
100
pkg/yqlib/doc/Entries.md
Normal file
@ -0,0 +1,100 @@
|
||||
Similar to the same named functions in `jq` these functions convert to/from an object and an array of key-value pairs. This is most useful for performing operations on keys of maps.
|
||||
## to_entries Map
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 1
|
||||
b: 2
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'to_entries' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- key: a
|
||||
value: 1
|
||||
- key: b
|
||||
value: 2
|
||||
```
|
||||
|
||||
## to_entries Array
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- a
|
||||
- b
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'to_entries' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- key: 0
|
||||
value: a
|
||||
- key: 1
|
||||
value: b
|
||||
```
|
||||
|
||||
## to_entries null
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
null
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'to_entries' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
```
|
||||
|
||||
## from_entries map
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 1
|
||||
b: 2
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'to_entries | from_entries' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: 1
|
||||
b: 2
|
||||
```
|
||||
|
||||
## from_entries with numeric key indexes
|
||||
from_entries always creates a map, even for numeric keys
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- a
|
||||
- b
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'to_entries | from_entries' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
0: a
|
||||
1: b
|
||||
```
|
||||
|
||||
## Use with_entries to update keys
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 1
|
||||
b: 2
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'with_entries(.key |= "KEY_" + .)' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
KEY_a: 1
|
||||
KEY_b: 2
|
||||
```
|
||||
|
||||
@ -93,3 +93,31 @@ will output
|
||||
true
|
||||
```
|
||||
|
||||
## Non exisitant key doesn't equal a value
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: frog
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'select(.b != "thing")' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: frog
|
||||
```
|
||||
|
||||
## Two non existant keys are equal
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: frog
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'select(.b == .c)' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: frog
|
||||
```
|
||||
|
||||
|
||||
@ -19,6 +19,29 @@ true
|
||||
false
|
||||
```
|
||||
|
||||
## Select, checking for existence of deep paths
|
||||
Simply pipe in parent expressions into `has`
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- a:
|
||||
b:
|
||||
c: cat
|
||||
- a:
|
||||
b:
|
||||
d: dog
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.[] | select(.a.b | has("c"))' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
b:
|
||||
c: cat
|
||||
```
|
||||
|
||||
## Has array index
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@ -231,6 +231,46 @@ will output
|
||||
age: 32
|
||||
```
|
||||
|
||||
## Merge arrays of objects together, matching on a key
|
||||
It's a complex command, the trickyness comes from needing to have the right context in the expressions.
|
||||
First we save the second array into a variable '$two' which lets us reference it later.
|
||||
We then need to update the first array. We will use the relative update (|=) because we need to update relative to the current element of the array in the LHS in the RHS expression.
|
||||
We set the current element of the first array as $cur. Now we multiply (merge) $cur with the matching entry in $two, by passing $two through a select filter.
|
||||
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- a: apple
|
||||
b: appleB
|
||||
- a: kiwi
|
||||
b: kiwiB
|
||||
- a: banana
|
||||
b: bananaB
|
||||
```
|
||||
And another sample another.yml file of:
|
||||
```yaml
|
||||
- a: banana
|
||||
c: bananaC
|
||||
- a: apple
|
||||
b: appleB2
|
||||
- a: dingo
|
||||
c: dingoC
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval-all '(select(fi==1) | .[]) as $two | select(fi==0) | .[] |= (. as $cur | $cur * ($two | select(.a == $cur.a)))' sample.yml another.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- a: apple
|
||||
b: appleB2
|
||||
- a: kiwi
|
||||
b: kiwiB
|
||||
- a: banana
|
||||
b: bananaB
|
||||
c: bananaC
|
||||
```
|
||||
|
||||
## Merge to prefix an element
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
@ -18,6 +18,42 @@ will output
|
||||
cat; meow; 1; ; true
|
||||
```
|
||||
|
||||
## Substitute / Replace string
|
||||
This uses golang regex, described [here](https://github.com/google/re2/wiki/Syntax)
|
||||
Note the use of `|=` to run in context of the current string value.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: dogs are great
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a |= sub("dogs", "cats")' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: cats are great
|
||||
```
|
||||
|
||||
## Substitute / Replace string with regex
|
||||
This uses golang regex, described [here](https://github.com/google/re2/wiki/Syntax)
|
||||
Note the use of `|=` to run in context of the current string value.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: cat
|
||||
b: heat
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.[] |= sub("(a)", "${1}r")' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: cart
|
||||
b: heart
|
||||
```
|
||||
|
||||
## Split strings
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
@ -1,4 +1,42 @@
|
||||
The style operator can be used to get or set the style of nodes (e.g. string style, yaml style)
|
||||
## Update and set style of a particular node (simple)
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
b: thing
|
||||
c: something
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a.b = "new" | .a.b style="double"' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
b: "new"
|
||||
c: something
|
||||
```
|
||||
|
||||
## Update and set style of a particular node using path variables
|
||||
You can use a variable to re-use a path
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a:
|
||||
b: thing
|
||||
c: something
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a.b as $x | $x = "new" | $x style="double"' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a:
|
||||
b: "new"
|
||||
c: something
|
||||
```
|
||||
|
||||
## Set tagged style
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
71
pkg/yqlib/doc/Subtract.md
Normal file
71
pkg/yqlib/doc/Subtract.md
Normal file
@ -0,0 +1,71 @@
|
||||
|
||||
## Number subtraction - float
|
||||
If the lhs or rhs are floats then the expression will be calculated with floats.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 3
|
||||
b: 4.5
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a = .a - .b' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: -1.5
|
||||
b: 4.5
|
||||
```
|
||||
|
||||
## Number subtraction - float
|
||||
If the lhs or rhs are floats then the expression will be calculated with floats.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 3
|
||||
b: 4.5
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a = .a - .b' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: -1.5
|
||||
b: 4.5
|
||||
```
|
||||
|
||||
## Number subtraction - int
|
||||
If both the lhs and rhs are ints then the expression will be calculated with ints.
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 3
|
||||
b: 4
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a = .a - .b' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: -1
|
||||
b: 4
|
||||
```
|
||||
|
||||
## Decrement numbers
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
a: 3
|
||||
b: 5
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.[] -= 1' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
a: 2
|
||||
b: 4
|
||||
```
|
||||
|
||||
@ -32,6 +32,21 @@ b: apple
|
||||
c: banana
|
||||
```
|
||||
|
||||
## Optional Splat
|
||||
Just like splat, but won't error if you run it against scalars
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
cat
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.[]' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
```
|
||||
|
||||
## Special characters
|
||||
Use quotes with brackets around path elements with special characters
|
||||
|
||||
@ -98,6 +113,23 @@ will output
|
||||
null
|
||||
```
|
||||
|
||||
## Optional identifier
|
||||
Like jq, does not output an error when the yaml is not an array or object as expected
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- 1
|
||||
- 2
|
||||
- 3
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval '.a?' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
```
|
||||
|
||||
## Wildcard matching
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
|
||||
82
pkg/yqlib/doc/Unique.md
Normal file
82
pkg/yqlib/doc/Unique.md
Normal file
@ -0,0 +1,82 @@
|
||||
This is used to filter out duplicated items in an array.
|
||||
|
||||
## Unique array of scalars (string/numbers)
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- 1
|
||||
- 2
|
||||
- 3
|
||||
- 2
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'unique' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- 1
|
||||
- 2
|
||||
- 3
|
||||
```
|
||||
|
||||
## Unique nulls
|
||||
Unique works on the node value, so it considers different representations of nulls to be different
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- ~
|
||||
- null
|
||||
- ~
|
||||
- null
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'unique' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- ~
|
||||
- null
|
||||
```
|
||||
|
||||
## Unique all nulls
|
||||
Run against the node tag to unique all the nulls
|
||||
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- ~
|
||||
- null
|
||||
- ~
|
||||
- null
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'unique_by(tag)' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- ~
|
||||
```
|
||||
|
||||
## Unique array object fields
|
||||
Given a sample.yml file of:
|
||||
```yaml
|
||||
- name: harry
|
||||
pet: cat
|
||||
- name: billy
|
||||
pet: dog
|
||||
- name: harry
|
||||
pet: dog
|
||||
```
|
||||
then
|
||||
```bash
|
||||
yq eval 'unique_by(.name)' sample.yml
|
||||
```
|
||||
will output
|
||||
```yaml
|
||||
- name: harry
|
||||
pet: cat
|
||||
- name: billy
|
||||
pet: dog
|
||||
```
|
||||
|
||||
@ -1,12 +1,116 @@
|
||||
# Operators
|
||||
|
||||
In `yq` expressions are made up of operators. Operators have 0-2 arguments and run against the current 'matching' nodes in the expression tree.
|
||||
In `yq` expressions are made up of operators and pipes. A context of nodes is passed through the expression and each operation takes the context as input and returns a new context as output. That output is piped in as input for the next operation in the expression. To begin with, the context is set to the first yaml document of the first yaml file (if processing in sequence using eval).
|
||||
|
||||
Lets look at a couple of examples.
|
||||
|
||||
The `length` operator take no arguments, and will simply return the length of _each_ matching node. So if there were 2 nodes, one string and one array, length will update the 'matching' nodes context to be two new numeric scalar nodes representing the lengths of the orignal 'matching' nodes.
|
||||
## Example with a simple operator
|
||||
|
||||
The `=` operator takes two arguments, a `lhs` expression and `rhs` expression. It runs the 'matching' nodes context against the `lhs` expression to find the nodes to update, lets call it `lhsNodes`, and then runs the matching nodes against the `rhs` to find the new values, lets call that `rhsNodes`. It updates the `lhsNodes` values with the `rhsNodes` values and _returns the original matching nodes_. This is important, where length changed the matching nodes to be new nodes with the length values, `=` returns the original matching nodes, albeit with some of the nodes values updated. So `.a = 3` will still return the parent matching node, but with the matching child updated.
|
||||
Given a document like:
|
||||
|
||||
Please see the individual operator docs for more information and examples.
|
||||
```yaml
|
||||
- [a]
|
||||
- "cat"
|
||||
```
|
||||
|
||||
with an expression:
|
||||
|
||||
```
|
||||
.[] | length
|
||||
```
|
||||
|
||||
`yq` will initially set the context as single node of the entire yaml document, an array of two elements.
|
||||
|
||||
```yaml
|
||||
- [a]
|
||||
- "cat"
|
||||
```
|
||||
|
||||
This gets piped into the splat operator `.[]` which will split out the context into a collection of two nodes `[a]` and `"cat"`. Note that this is _not_ a yaml array.
|
||||
|
||||
The `length` operator take no arguments, and will simply return the length of _each_ matching node in the context. So for the context of `[a]` and `"cat"`, it will return a new context of `1` and `3`.
|
||||
|
||||
This being the last operation in the expression, the results will be printed out:
|
||||
|
||||
```
|
||||
1
|
||||
3
|
||||
```
|
||||
|
||||
# Example with an operator that takes arguments.
|
||||
|
||||
Given a document like:
|
||||
|
||||
```yaml
|
||||
a: cat
|
||||
b: dog
|
||||
```
|
||||
|
||||
with an expression:
|
||||
|
||||
```
|
||||
.a = .b
|
||||
```
|
||||
|
||||
The `=` operator takes two arguments, a `lhs` expression, which in this case is `.a` and `rhs` expression which is `.b`.
|
||||
|
||||
It pipes the current, lets call it 'root' context through the `lhs` expression of `.a` to return the node
|
||||
|
||||
```yaml
|
||||
cat
|
||||
```
|
||||
|
||||
Note that this node holds not only its value 'cat', but comments and metadata too, including path and parent information.
|
||||
|
||||
The `=` operator then pipes the 'root' context through the `rhs` expression of `.b` to return the node
|
||||
|
||||
```yaml
|
||||
dog
|
||||
```
|
||||
|
||||
Both sides have now been evaluated, so now the operator copies across the value from the RHS to the value on the LHS, and it returns the now updated context:
|
||||
|
||||
```yaml
|
||||
a: dog
|
||||
b: dog
|
||||
```
|
||||
|
||||
# Relative update (e.g. `|=`)
|
||||
There is another form of the `=` operator which we call the relative form. It's very similar to `=` but with one key difference when evaluating the RHS expression.
|
||||
|
||||
In the plain form, we pass in the 'root' level context to the RHS expression. In relative form, we pass in _each result of the LHS_ to the RHS expression. Let's go through an example.
|
||||
|
||||
Given a document like:
|
||||
|
||||
```yaml
|
||||
a: 1
|
||||
b: thing
|
||||
```
|
||||
|
||||
with an expression:
|
||||
|
||||
```
|
||||
.a |= . + 1
|
||||
```
|
||||
|
||||
Similar to the `=` operator, `|=` takes two operands, the LHS and RHS.
|
||||
|
||||
It pipes the current context (the whole document) through the LHS expression of `.a` to get the node value:
|
||||
|
||||
```
|
||||
1
|
||||
```
|
||||
|
||||
Now it pipes _that LHS context_ into the RHS expression `. + 1` (whereas in the `=` plain form it piped the original document context into the RHS) to yield:
|
||||
|
||||
|
||||
```
|
||||
2
|
||||
```
|
||||
|
||||
The assignment operator then copies across the value from the RHS to the value on the LHS, and it returns the now updated 'root' context:
|
||||
|
||||
```yaml
|
||||
a: 2
|
||||
b: thing
|
||||
```
|
||||
@ -1,4 +1,4 @@
|
||||
Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference aliases and remove anchor names).
|
||||
Use the `alias` and `anchor` operators to read and write yaml aliases and anchors. The `explode` operator normalises a yaml file (dereference (or expands) aliases and remove anchor names).
|
||||
|
||||
`yq` supports merge aliases (like `<<: *blah`) however this is no longer in the standard yaml spec (1.2) and so `yq` will automatically add the `!!merge` tag to these nodes as it is effectively a custom tag.
|
||||
|
||||
|
||||
@ -1 +1,9 @@
|
||||
The `or` and `and` operators take two parameters and return a boolean result. `not` flips a boolean from true to false, or vice versa. These are most commonly used with the `select` operator to filter particular nodes.
|
||||
The `or` and `and` operators take two parameters and return a boolean result.
|
||||
|
||||
`not` flips a boolean from true to false, or vice versa.
|
||||
|
||||
`any` will return `true` if there are any `true` values in a array sequence, and `all` will return true if _all_ elements in an array are true.
|
||||
|
||||
`any_c(condition)` and `all_c(condition)` are like `any` and `all` but they take a condition expression that is used against each element to determine if it's `true`. Note: in `jq` you can simply pass a condition to `any` or `all` and it simply works - `yq` isn't that clever..yet
|
||||
|
||||
These are most commonly used with the `select` operator to filter particular nodes.
|
||||
|
||||
@ -1 +1,11 @@
|
||||
Use these comment operators to set or retrieve comments.
|
||||
|
||||
Like the `=` and `|=` assign operators, the same syntax applies when updating comments:
|
||||
|
||||
|
||||
### plain form: `=`
|
||||
This will assign the LHS nodes comments to the expression on the RHS. The RHS is run against the matching nodes in the pipeline
|
||||
|
||||
### relative form: `|=`
|
||||
Similar to the plain form, however the RHS evaluates against each matching LHS node! This is useful if you want to set the comments as a relative expression of the node, for instance its value or path.
|
||||
|
||||
|
||||
1
pkg/yqlib/doc/headers/Entries.md
Normal file
1
pkg/yqlib/doc/headers/Entries.md
Normal file
@ -0,0 +1 @@
|
||||
Similar to the same named functions in `jq` these functions convert to/from an object and an array of key-value pairs. This is most useful for performing operations on keys of maps.
|
||||
1
pkg/yqlib/doc/headers/Unique.md
Normal file
1
pkg/yqlib/doc/headers/Unique.md
Normal file
@ -0,0 +1 @@
|
||||
This is used to filter out duplicated items in an array.
|
||||
@ -54,12 +54,26 @@ func (p *expressionPostFixerImpl) ConvertToPostfix(infixTokens []*token) ([]*Ope
|
||||
opStack = opStack[0 : len(opStack)-1]
|
||||
log.Debugf("deleteing open bracket from opstack")
|
||||
|
||||
//and append a collect to the opStack
|
||||
result = append(result, &Operation{OperationType: collectOperator})
|
||||
//and append a collect to the result
|
||||
// hack - see if there's the optional traverse flag
|
||||
// on the close op - move it to the collect op.
|
||||
// allows for .["cat"]?
|
||||
prefs := traversePreferences{}
|
||||
closeTokenMatch := string(currentToken.Match.Bytes)
|
||||
if closeTokenMatch[len(closeTokenMatch)-1:] == "?" {
|
||||
prefs.OptionalTraverse = true
|
||||
}
|
||||
result = append(result, &Operation{OperationType: collectOperator, Preferences: prefs})
|
||||
log.Debugf("put collect onto the result")
|
||||
result = append(result, &Operation{OperationType: shortPipeOpType})
|
||||
log.Debugf("put shortpipe onto the result")
|
||||
|
||||
//traverseArrayCollect is a sneaky op that needs to be included too
|
||||
//when closing a []
|
||||
if len(opStack) > 0 && opStack[len(opStack)-1].Operation != nil && opStack[len(opStack)-1].Operation.OperationType == traverseArrayOpType {
|
||||
opStack, result = popOpToResult(opStack, result)
|
||||
}
|
||||
|
||||
case closeBracket:
|
||||
for len(opStack) > 0 && opStack[len(opStack)-1].TokenType != openBracket {
|
||||
opStack, result = popOpToResult(opStack, result)
|
||||
|
||||
@ -12,6 +12,36 @@ var pathTests = []struct {
|
||||
expectedTokens []interface{}
|
||||
expectedPostFix []interface{}
|
||||
}{
|
||||
{
|
||||
`.[0]`,
|
||||
append(make([]interface{}, 0), "SELF", "TRAVERSE_ARRAY", "[", "0 (int64)", "]"),
|
||||
append(make([]interface{}, 0), "SELF", "0 (int64)", "COLLECT", "SHORT_PIPE", "TRAVERSE_ARRAY"),
|
||||
},
|
||||
{
|
||||
`.[0][1]`,
|
||||
append(make([]interface{}, 0), "SELF", "TRAVERSE_ARRAY", "[", "0 (int64)", "]", "TRAVERSE_ARRAY", "[", "1 (int64)", "]"),
|
||||
append(make([]interface{}, 0), "SELF", "0 (int64)", "COLLECT", "SHORT_PIPE", "TRAVERSE_ARRAY", "1 (int64)", "COLLECT", "SHORT_PIPE", "TRAVERSE_ARRAY"),
|
||||
},
|
||||
{
|
||||
`"\""`,
|
||||
append(make([]interface{}, 0), "\" (string)"),
|
||||
append(make([]interface{}, 0), "\" (string)"),
|
||||
},
|
||||
{
|
||||
`[]|join(".")`,
|
||||
append(make([]interface{}, 0), "[", "EMPTY", "]", "PIPE", "JOIN", "(", ". (string)", ")"),
|
||||
append(make([]interface{}, 0), "EMPTY", "COLLECT", "SHORT_PIPE", ". (string)", "JOIN", "PIPE"),
|
||||
},
|
||||
{
|
||||
`{"cool": .b or .c}`,
|
||||
append(make([]interface{}, 0), "{", "cool (string)", "CREATE_MAP", "b", "OR", "c", "}"),
|
||||
append(make([]interface{}, 0), "cool (string)", "b", "c", "OR", "CREATE_MAP", "COLLECT_OBJECT", "SHORT_PIPE"),
|
||||
},
|
||||
{
|
||||
`{"cool": []|join(".")}`,
|
||||
append(make([]interface{}, 0), "{", "cool (string)", "CREATE_MAP", "[", "EMPTY", "]", "PIPE", "JOIN", "(", ". (string)", ")", "}"),
|
||||
append(make([]interface{}, 0), "cool (string)", "EMPTY", "COLLECT", "SHORT_PIPE", ". (string)", "JOIN", "PIPE", "CREATE_MAP", "COLLECT_OBJECT", "SHORT_PIPE"),
|
||||
},
|
||||
{
|
||||
`.a as $item ireduce (0; . + $item)`, // note - add code to shuffle reduce to this position for postfix
|
||||
append(make([]interface{}, 0), "a", "ASSIGN_VARIABLE", "GET_VARIABLE", "REDUCE", "(", "0 (int64)", "BLOCK", "SELF", "ADD", "GET_VARIABLE", ")"),
|
||||
@ -47,6 +77,11 @@ var pathTests = []struct {
|
||||
append(make([]interface{}, 0), "b", "TRAVERSE_ARRAY", "[", "a", "]"),
|
||||
append(make([]interface{}, 0), "b", "a", "COLLECT", "SHORT_PIPE", "TRAVERSE_ARRAY"),
|
||||
},
|
||||
{
|
||||
`.b[.a]?`,
|
||||
append(make([]interface{}, 0), "b", "TRAVERSE_ARRAY", "[", "a", "]"),
|
||||
append(make([]interface{}, 0), "b", "a", "COLLECT", "SHORT_PIPE", "TRAVERSE_ARRAY"),
|
||||
},
|
||||
{
|
||||
`.[]`,
|
||||
append(make([]interface{}, 0), "SELF", "TRAVERSE_ARRAY", "[", "EMPTY", "]"),
|
||||
@ -57,6 +92,11 @@ var pathTests = []struct {
|
||||
append(make([]interface{}, 0), "a", "TRAVERSE_ARRAY", "[", "EMPTY", "]"),
|
||||
append(make([]interface{}, 0), "a", "EMPTY", "COLLECT", "SHORT_PIPE", "TRAVERSE_ARRAY"),
|
||||
},
|
||||
{
|
||||
`.a[]?`,
|
||||
append(make([]interface{}, 0), "a", "TRAVERSE_ARRAY", "[", "EMPTY", "]"),
|
||||
append(make([]interface{}, 0), "a", "EMPTY", "COLLECT", "SHORT_PIPE", "TRAVERSE_ARRAY"),
|
||||
},
|
||||
{
|
||||
`.a.[]`,
|
||||
append(make([]interface{}, 0), "a", "TRAVERSE_ARRAY", "[", "EMPTY", "]"),
|
||||
@ -67,6 +107,11 @@ var pathTests = []struct {
|
||||
append(make([]interface{}, 0), "a", "TRAVERSE_ARRAY", "[", "0 (int64)", "]"),
|
||||
append(make([]interface{}, 0), "a", "0 (int64)", "COLLECT", "SHORT_PIPE", "TRAVERSE_ARRAY"),
|
||||
},
|
||||
{
|
||||
`.a[0]?`,
|
||||
append(make([]interface{}, 0), "a", "TRAVERSE_ARRAY", "[", "0 (int64)", "]"),
|
||||
append(make([]interface{}, 0), "a", "0 (int64)", "COLLECT", "SHORT_PIPE", "TRAVERSE_ARRAY"),
|
||||
},
|
||||
{
|
||||
`.a.[0]`,
|
||||
append(make([]interface{}, 0), "a", "TRAVERSE_ARRAY", "[", "0 (int64)", "]"),
|
||||
|
||||
@ -31,6 +31,7 @@ type token struct {
|
||||
Operation *Operation
|
||||
AssignOperation *Operation // e.g. tag (GetTag) op becomes AssignTag if '=' follows it
|
||||
CheckForPostTraverse bool // e.g. [1]cat should really be [1].cat
|
||||
Match *machines.Match // match that created this token
|
||||
|
||||
}
|
||||
|
||||
@ -63,12 +64,19 @@ func (t *token) toString(detail bool) string {
|
||||
func pathToken(wrapped bool) lex.Action {
|
||||
return func(s *lex.Scanner, m *machines.Match) (interface{}, error) {
|
||||
value := string(m.Bytes)
|
||||
prefs := traversePreferences{}
|
||||
|
||||
if value[len(value)-1:] == "?" {
|
||||
prefs.OptionalTraverse = true
|
||||
value = value[:len(value)-1]
|
||||
}
|
||||
|
||||
value = value[1:]
|
||||
if wrapped {
|
||||
value = unwrap(value)
|
||||
}
|
||||
log.Debug("PathToken %v", value)
|
||||
op := &Operation{OperationType: traversePathOpType, Value: value, StringValue: value, Preferences: traversePreferences{}}
|
||||
op := &Operation{OperationType: traversePathOpType, Value: value, StringValue: value, Preferences: prefs}
|
||||
return &token{TokenType: operationToken, Operation: op, CheckForPostTraverse: true}, nil
|
||||
}
|
||||
}
|
||||
@ -138,7 +146,7 @@ func assignAllCommentsOp(updateAssign bool) lex.Action {
|
||||
|
||||
func literalToken(pType tokenType, checkForPost bool) lex.Action {
|
||||
return func(s *lex.Scanner, m *machines.Match) (interface{}, error) {
|
||||
return &token{TokenType: pType, CheckForPostTraverse: checkForPost}, nil
|
||||
return &token{TokenType: pType, CheckForPostTraverse: checkForPost, Match: m}, nil
|
||||
}
|
||||
}
|
||||
|
||||
@ -181,6 +189,7 @@ func stringValue(wrapped bool) lex.Action {
|
||||
if wrapped {
|
||||
value = unwrap(value)
|
||||
}
|
||||
value = strings.ReplaceAll(value, "\\\"", "\"")
|
||||
return &token{TokenType: operationToken, Operation: createValueOperation(value, value)}, nil
|
||||
}
|
||||
}
|
||||
@ -251,6 +260,8 @@ func initLexer() (*lex.Lexer, error) {
|
||||
lexer.Add([]byte(`sortKeys`), opToken(sortKeysOpType))
|
||||
lexer.Add([]byte(`select`), opToken(selectOpType))
|
||||
lexer.Add([]byte(`has`), opToken(hasOpType))
|
||||
lexer.Add([]byte(`unique`), opToken(uniqueOpType))
|
||||
lexer.Add([]byte(`unique_by`), opToken(uniqueByOpType))
|
||||
lexer.Add([]byte(`explode`), opToken(explodeOpType))
|
||||
lexer.Add([]byte(`or`), opToken(orOpType))
|
||||
lexer.Add([]byte(`and`), opToken(andOpType))
|
||||
@ -264,6 +275,13 @@ func initLexer() (*lex.Lexer, error) {
|
||||
lexer.Add([]byte(`splitDoc`), opToken(splitDocumentOpType))
|
||||
|
||||
lexer.Add([]byte(`join`), opToken(joinStringOpType))
|
||||
lexer.Add([]byte(`sub`), opToken(subStringOpType))
|
||||
|
||||
lexer.Add([]byte(`any`), opToken(anyOpType))
|
||||
lexer.Add([]byte(`any_c`), opToken(anyConditionOpType))
|
||||
lexer.Add([]byte(`all`), opToken(allOpType))
|
||||
lexer.Add([]byte(`all_c`), opToken(allConditionOpType))
|
||||
|
||||
lexer.Add([]byte(`split`), opToken(splitStringOpType))
|
||||
lexer.Add([]byte(`keys`), opToken(keysOpType))
|
||||
|
||||
@ -276,6 +294,9 @@ func initLexer() (*lex.Lexer, error) {
|
||||
lexer.Add([]byte(`fileIndex`), opToken(getFileIndexOpType))
|
||||
lexer.Add([]byte(`fi`), opToken(getFileIndexOpType))
|
||||
lexer.Add([]byte(`path`), opToken(getPathOpType))
|
||||
lexer.Add([]byte(`to_entries`), opToken(toEntriesOpType))
|
||||
lexer.Add([]byte(`from_entries`), opToken(fromEntriesOpType))
|
||||
lexer.Add([]byte(`with_entries`), opToken(withEntriesOpType))
|
||||
|
||||
lexer.Add([]byte(`lineComment`), opTokenWithPrefs(getCommentOpType, assignCommentOpType, commentOpPreferences{LineComment: true}))
|
||||
|
||||
@ -298,8 +319,8 @@ func initLexer() (*lex.Lexer, error) {
|
||||
|
||||
lexer.Add([]byte("( |\t|\n|\r)+"), skip)
|
||||
|
||||
lexer.Add([]byte(`\."[^ "]+"`), pathToken(true))
|
||||
lexer.Add([]byte(`\.[^ \}\{\:\[\],\|\.\[\(\)=]+`), pathToken(false))
|
||||
lexer.Add([]byte(`\."[^ "]+"\??`), pathToken(true))
|
||||
lexer.Add([]byte(`\.[^ \}\{\:\[\],\|\.\[\(\)=]+\??`), pathToken(false))
|
||||
lexer.Add([]byte(`\.`), selfToken())
|
||||
|
||||
lexer.Add([]byte(`\|`), opToken(pipeOpType))
|
||||
@ -314,17 +335,19 @@ func initLexer() (*lex.Lexer, error) {
|
||||
lexer.Add([]byte(`[Nn][Uu][Ll][Ll]`), nullValue())
|
||||
lexer.Add([]byte(`~`), nullValue())
|
||||
|
||||
lexer.Add([]byte(`"[^"]*"`), stringValue(true))
|
||||
lexer.Add([]byte(`"([^"\\]*(\\.[^"\\]*)*)"`), stringValue(true))
|
||||
lexer.Add([]byte(`strenv\([^\)]+\)`), envOp(true))
|
||||
lexer.Add([]byte(`env\([^\)]+\)`), envOp(false))
|
||||
|
||||
lexer.Add([]byte(`\[`), literalToken(openCollect, false))
|
||||
lexer.Add([]byte(`\]`), literalToken(closeCollect, true))
|
||||
lexer.Add([]byte(`\]\??`), literalToken(closeCollect, true))
|
||||
lexer.Add([]byte(`\{`), literalToken(openCollectObject, false))
|
||||
lexer.Add([]byte(`\}`), literalToken(closeCollectObject, true))
|
||||
lexer.Add([]byte(`\*[\+|\?d]*`), multiplyWithPrefs())
|
||||
lexer.Add([]byte(`\+`), opToken(addOpType))
|
||||
lexer.Add([]byte(`\+=`), opToken(addAssignOpType))
|
||||
lexer.Add([]byte(`\-`), opToken(subtractOpType))
|
||||
lexer.Add([]byte(`\-=`), opToken(subtractAssignOpType))
|
||||
lexer.Add([]byte(`\$[a-zA-Z_-0-9]+`), getVariableOpToken())
|
||||
lexer.Add([]byte(`as`), opToken(assignVariableOpType))
|
||||
|
||||
@ -388,16 +411,19 @@ func (p *expressionTokeniserImpl) handleToken(tokens []*token, index int, postPr
|
||||
skipNextToken = false
|
||||
currentToken := tokens[index]
|
||||
|
||||
log.Debug("processing %v", currentToken.toString(true))
|
||||
|
||||
if currentToken.TokenType == traverseArrayCollect {
|
||||
//need to put a traverse array then a collect currentToken
|
||||
// do this by adding traverse then converting currentToken to collect
|
||||
|
||||
if index == 0 || tokens[index-1].TokenType != operationToken ||
|
||||
tokens[index-1].Operation.OperationType != traversePathOpType {
|
||||
log.Debug(" adding self")
|
||||
op := &Operation{OperationType: selfReferenceOpType, StringValue: "SELF"}
|
||||
postProcessedTokens = append(postProcessedTokens, &token{TokenType: operationToken, Operation: op})
|
||||
}
|
||||
|
||||
log.Debug(" adding traverse array")
|
||||
op := &Operation{OperationType: traverseArrayOpType, StringValue: "TRAVERSE_ARRAY"}
|
||||
postProcessedTokens = append(postProcessedTokens, &token{TokenType: operationToken, Operation: op})
|
||||
|
||||
@ -408,17 +434,19 @@ func (p *expressionTokeniserImpl) handleToken(tokens []*token, index int, postPr
|
||||
if index != len(tokens)-1 && currentToken.AssignOperation != nil &&
|
||||
tokens[index+1].TokenType == operationToken &&
|
||||
tokens[index+1].Operation.OperationType == assignOpType {
|
||||
log.Debug(" its an update assign")
|
||||
currentToken.Operation = currentToken.AssignOperation
|
||||
currentToken.Operation.UpdateAssign = tokens[index+1].Operation.UpdateAssign
|
||||
skipNextToken = true
|
||||
}
|
||||
|
||||
log.Debug(" adding token to the fixed list")
|
||||
postProcessedTokens = append(postProcessedTokens, currentToken)
|
||||
|
||||
if index != len(tokens)-1 &&
|
||||
((currentToken.TokenType == openCollect && tokens[index+1].TokenType == closeCollect) ||
|
||||
(currentToken.TokenType == openCollectObject && tokens[index+1].TokenType == closeCollectObject)) {
|
||||
|
||||
log.Debug(" adding empty")
|
||||
op := &Operation{OperationType: emptyOpType, StringValue: "EMPTY"}
|
||||
postProcessedTokens = append(postProcessedTokens, &token{TokenType: operationToken, Operation: op})
|
||||
}
|
||||
@ -426,12 +454,19 @@ func (p *expressionTokeniserImpl) handleToken(tokens []*token, index int, postPr
|
||||
if index != len(tokens)-1 && currentToken.CheckForPostTraverse &&
|
||||
tokens[index+1].TokenType == operationToken &&
|
||||
tokens[index+1].Operation.OperationType == traversePathOpType {
|
||||
log.Debug(" adding pipe because the next thing is traverse")
|
||||
op := &Operation{OperationType: shortPipeOpType, Value: "PIPE"}
|
||||
postProcessedTokens = append(postProcessedTokens, &token{TokenType: operationToken, Operation: op})
|
||||
}
|
||||
if index != len(tokens)-1 && currentToken.CheckForPostTraverse &&
|
||||
tokens[index+1].TokenType == openCollect {
|
||||
|
||||
// if tokens[index].TokenType == closeCollect {
|
||||
// log.Debug(" adding pipe because next is opencollect")
|
||||
// op := &Operation{OperationType: shortPipeOpType, Value: "PIPE"}
|
||||
// postProcessedTokens = append(postProcessedTokens, &token{TokenType: operationToken, Operation: op})
|
||||
// }
|
||||
log.Debug(" adding traverArray because next is opencollect")
|
||||
op := &Operation{OperationType: traverseArrayOpType}
|
||||
postProcessedTokens = append(postProcessedTokens, &token{TokenType: operationToken, Operation: op})
|
||||
}
|
||||
|
||||
@ -20,9 +20,6 @@ type operationType struct {
|
||||
Handler operatorHandler
|
||||
}
|
||||
|
||||
// operators TODO:
|
||||
// - mergeEmpty (sets only if the document is empty, do I do that now?)
|
||||
|
||||
var orOpType = &operationType{Type: "OR", NumArgs: 2, Precedence: 20, Handler: orOperator}
|
||||
var andOpType = &operationType{Type: "AND", NumArgs: 2, Precedence: 20, Handler: andOperator}
|
||||
var reduceOpType = &operationType{Type: "REDUCE", NumArgs: 2, Precedence: 35, Handler: reduceOperator}
|
||||
@ -35,6 +32,7 @@ var pipeOpType = &operationType{Type: "PIPE", NumArgs: 2, Precedence: 30, Handle
|
||||
|
||||
var assignOpType = &operationType{Type: "ASSIGN", NumArgs: 2, Precedence: 40, Handler: assignUpdateOperator}
|
||||
var addAssignOpType = &operationType{Type: "ADD_ASSIGN", NumArgs: 2, Precedence: 40, Handler: addAssignOperator}
|
||||
var subtractAssignOpType = &operationType{Type: "SUBTRACT_ASSIGN", NumArgs: 2, Precedence: 40, Handler: subtractAssignOperator}
|
||||
|
||||
var assignAttributesOpType = &operationType{Type: "ASSIGN_ATTRIBUTES", NumArgs: 2, Precedence: 40, Handler: assignAttributesOperator}
|
||||
var assignStyleOpType = &operationType{Type: "ASSIGN_STYLE", NumArgs: 2, Precedence: 40, Handler: assignStyleOperator}
|
||||
@ -46,16 +44,29 @@ var assignAliasOpType = &operationType{Type: "ASSIGN_ALIAS", NumArgs: 2, Precede
|
||||
|
||||
var multiplyOpType = &operationType{Type: "MULTIPLY", NumArgs: 2, Precedence: 42, Handler: multiplyOperator}
|
||||
var addOpType = &operationType{Type: "ADD", NumArgs: 2, Precedence: 42, Handler: addOperator}
|
||||
var subtractOpType = &operationType{Type: "SUBTRACT", NumArgs: 2, Precedence: 42, Handler: subtractOperator}
|
||||
var alternativeOpType = &operationType{Type: "ALTERNATIVE", NumArgs: 2, Precedence: 42, Handler: alternativeOperator}
|
||||
|
||||
var equalsOpType = &operationType{Type: "EQUALS", NumArgs: 2, Precedence: 40, Handler: equalsOperator}
|
||||
var notEqualsOpType = &operationType{Type: "EQUALS", NumArgs: 2, Precedence: 40, Handler: notEqualsOperator}
|
||||
var createMapOpType = &operationType{Type: "CREATE_MAP", NumArgs: 2, Precedence: 40, Handler: createMapOperator}
|
||||
|
||||
//createmap needs to be above union, as we use union to build the components of the objects
|
||||
var createMapOpType = &operationType{Type: "CREATE_MAP", NumArgs: 2, Precedence: 15, Handler: createMapOperator}
|
||||
|
||||
var shortPipeOpType = &operationType{Type: "SHORT_PIPE", NumArgs: 2, Precedence: 45, Handler: pipeOperator}
|
||||
|
||||
var lengthOpType = &operationType{Type: "LENGTH", NumArgs: 0, Precedence: 50, Handler: lengthOperator}
|
||||
var collectOpType = &operationType{Type: "COLLECT", NumArgs: 0, Precedence: 50, Handler: collectOperator}
|
||||
|
||||
var anyOpType = &operationType{Type: "ANY", NumArgs: 0, Precedence: 50, Handler: anyOperator}
|
||||
var allOpType = &operationType{Type: "ALL", NumArgs: 0, Precedence: 50, Handler: allOperator}
|
||||
var anyConditionOpType = &operationType{Type: "ANY_CONDITION", NumArgs: 1, Precedence: 50, Handler: anyOperator}
|
||||
var allConditionOpType = &operationType{Type: "ALL_CONDITION", NumArgs: 1, Precedence: 50, Handler: allOperator}
|
||||
|
||||
var toEntriesOpType = &operationType{Type: "TO_ENTRIES", NumArgs: 0, Precedence: 50, Handler: toEntriesOperator}
|
||||
var fromEntriesOpType = &operationType{Type: "FROM_ENTRIES", NumArgs: 0, Precedence: 50, Handler: fromEntriesOperator}
|
||||
var withEntriesOpType = &operationType{Type: "WITH_ENTRIES", NumArgs: 1, Precedence: 50, Handler: withEntriesOperator}
|
||||
|
||||
var splitDocumentOpType = &operationType{Type: "SPLIT_DOC", NumArgs: 0, Precedence: 50, Handler: splitDocumentOperator}
|
||||
var getVariableOpType = &operationType{Type: "GET_VARIABLE", NumArgs: 0, Precedence: 55, Handler: getVariableOperator}
|
||||
var getStyleOpType = &operationType{Type: "GET_STYLE", NumArgs: 0, Precedence: 50, Handler: getStyleOperator}
|
||||
@ -71,6 +82,7 @@ var getPathOpType = &operationType{Type: "GET_PATH", NumArgs: 0, Precedence: 50,
|
||||
var explodeOpType = &operationType{Type: "EXPLODE", NumArgs: 1, Precedence: 50, Handler: explodeOperator}
|
||||
var sortKeysOpType = &operationType{Type: "SORT_KEYS", NumArgs: 1, Precedence: 50, Handler: sortKeysOperator}
|
||||
var joinStringOpType = &operationType{Type: "JOIN", NumArgs: 1, Precedence: 50, Handler: joinStringOperator}
|
||||
var subStringOpType = &operationType{Type: "SUBSTR", NumArgs: 1, Precedence: 50, Handler: substituteStringOperator}
|
||||
var splitStringOpType = &operationType{Type: "SPLIT", NumArgs: 1, Precedence: 50, Handler: splitStringOperator}
|
||||
|
||||
var keysOpType = &operationType{Type: "KEYS", NumArgs: 0, Precedence: 50, Handler: keysOperator}
|
||||
@ -89,8 +101,9 @@ var recursiveDescentOpType = &operationType{Type: "RECURSIVE_DESCENT", NumArgs:
|
||||
|
||||
var selectOpType = &operationType{Type: "SELECT", NumArgs: 1, Precedence: 50, Handler: selectOperator}
|
||||
var hasOpType = &operationType{Type: "HAS", NumArgs: 1, Precedence: 50, Handler: hasOperator}
|
||||
var uniqueOpType = &operationType{Type: "UNIQUE", NumArgs: 0, Precedence: 50, Handler: unique}
|
||||
var uniqueByOpType = &operationType{Type: "UNIQUE_BY", NumArgs: 1, Precedence: 50, Handler: uniqueBy}
|
||||
var deleteChildOpType = &operationType{Type: "DELETE", NumArgs: 1, Precedence: 40, Handler: deleteChildOperator}
|
||||
var deleteImmediateChildOpType = &operationType{Type: "DELETE_IMMEDIATE_CHILD", NumArgs: 1, Precedence: 40, Handler: deleteImmediateChildOperator}
|
||||
|
||||
type Operation struct {
|
||||
OperationType *operationType
|
||||
|
||||
@ -16,9 +16,9 @@ func createAddOp(lhs *ExpressionNode, rhs *ExpressionNode) *ExpressionNode {
|
||||
|
||||
func addAssignOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
assignmentOp := &Operation{OperationType: assignOpType}
|
||||
assignmentOp.UpdateAssign = false
|
||||
|
||||
assignmentOpNode := &ExpressionNode{Operation: assignmentOp, Lhs: expressionNode.Lhs, Rhs: createAddOp(expressionNode.Lhs, expressionNode.Rhs)}
|
||||
assignmentOp.UpdateAssign = true
|
||||
selfExpression := &ExpressionNode{Operation: &Operation{OperationType: selfReferenceOpType}}
|
||||
assignmentOpNode := &ExpressionNode{Operation: assignmentOp, Lhs: expressionNode.Lhs, Rhs: createAddOp(selfExpression, expressionNode.Rhs)}
|
||||
return d.GetMatchingNodes(context, assignmentOpNode)
|
||||
}
|
||||
|
||||
@ -39,7 +39,7 @@ func toNodes(candidate *CandidateNode) []*yaml.Node {
|
||||
func addOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("Add operator")
|
||||
|
||||
return crossFunction(d, context, expressionNode, add)
|
||||
return crossFunction(d, context.ReadOnlyClone(), expressionNode, add, false)
|
||||
}
|
||||
|
||||
func add(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
|
||||
|
||||
@ -14,6 +14,22 @@ var addOperatorScenarios = []expressionScenario{
|
||||
"D0, P[1 a], (!!int)::3\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{}`,
|
||||
expression: "(.a + .b) as $x",
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `a: 0`,
|
||||
expression: ".a += .b.c",
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: 0\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Concatenate and assign arrays",
|
||||
document: `{a: {val: thing, b: [cat,dog]}}`,
|
||||
@ -63,7 +79,7 @@ var addOperatorScenarios = []expressionScenario{
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Update array (append)",
|
||||
description: "Append to array",
|
||||
document: `{a: [1,2], b: [3,4]}`,
|
||||
expression: `.a = .a + .b`,
|
||||
expected: []string{
|
||||
@ -71,17 +87,17 @@ var addOperatorScenarios = []expressionScenario{
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "String concatenation",
|
||||
document: `{a: cat, b: meow}`,
|
||||
expression: `.a = .a + .b`,
|
||||
description: "Relative append",
|
||||
document: `a: { a1: {b: [cat]}, a2: {b: [dog]}, a3: {} }`,
|
||||
expression: `.a[].b += ["mouse"]`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: catmeow, b: meow}\n",
|
||||
"D0, P[], (doc)::a: {a1: {b: [cat, mouse]}, a2: {b: [dog, mouse]}, a3: {b: [mouse]}}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Relative string concatenation",
|
||||
description: "String concatenation",
|
||||
document: `{a: cat, b: meow}`,
|
||||
expression: `.a += .b`,
|
||||
expression: `.a = .a + .b`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: catmeow, b: meow}\n",
|
||||
},
|
||||
@ -105,11 +121,11 @@ var addOperatorScenarios = []expressionScenario{
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Increment number",
|
||||
document: `{a: 3}`,
|
||||
expression: `.a += 1`,
|
||||
description: "Increment numbers",
|
||||
document: `{a: 3, b: 5}`,
|
||||
expression: `.[] += 1`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: 4}\n",
|
||||
"D0, P[], (doc)::{a: 4, b: 6}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
|
||||
@ -1,14 +1,14 @@
|
||||
package yqlib
|
||||
|
||||
// corssFunction no matches
|
||||
// can boolean use crossfunction
|
||||
|
||||
func alternativeOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("-- alternative")
|
||||
return crossFunction(d, context, expressionNode, alternativeFunc)
|
||||
return crossFunction(d, context.ReadOnlyClone(), expressionNode, alternativeFunc, true)
|
||||
}
|
||||
|
||||
func alternativeFunc(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
|
||||
if lhs == nil {
|
||||
return rhs, nil
|
||||
}
|
||||
lhs.Node = unwrapDoc(lhs.Node)
|
||||
rhs.Node = unwrapDoc(rhs.Node)
|
||||
log.Debugf("Alternative LHS: %v", lhs.Node.Tag)
|
||||
|
||||
@ -5,6 +5,20 @@ import (
|
||||
)
|
||||
|
||||
var alternativeOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
skipDoc: true,
|
||||
expression: `.b // .c`,
|
||||
document: `a: bridge`,
|
||||
expected: []string{},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
expression: `(.b // "hello") as $x`,
|
||||
document: `a: bridge`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: bridge\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "LHS is defined",
|
||||
expression: `.a // "hello"`,
|
||||
@ -13,6 +27,14 @@ var alternativeOperatorScenarios = []expressionScenario{
|
||||
"D0, P[a], (!!str)::bridge\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
expression: `select(tag == "seq") // "cat"`,
|
||||
skipDoc: true,
|
||||
document: `a: frog`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!str)::cat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "LHS is not defined",
|
||||
expression: `.a // "hello"`,
|
||||
|
||||
@ -12,7 +12,7 @@ func assignAliasOperator(d *dataTreeNavigator, context Context, expressionNode *
|
||||
|
||||
aliasName := ""
|
||||
if !expressionNode.Operation.UpdateAssign {
|
||||
rhs, err := d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
@ -32,7 +32,7 @@ func assignAliasOperator(d *dataTreeNavigator, context Context, expressionNode *
|
||||
log.Debugf("Setting aliasName : %v", candidate.GetKey())
|
||||
|
||||
if expressionNode.Operation.UpdateAssign {
|
||||
rhs, err := d.GetMatchingNodes(context.SingleChildContext(candidate), expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.SingleReadonlyChildContext(candidate), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
@ -41,9 +41,11 @@ func assignAliasOperator(d *dataTreeNavigator, context Context, expressionNode *
|
||||
}
|
||||
}
|
||||
|
||||
if aliasName != "" {
|
||||
candidate.Node.Kind = yaml.AliasNode
|
||||
candidate.Node.Value = aliasName
|
||||
}
|
||||
}
|
||||
return context, nil
|
||||
}
|
||||
|
||||
@ -66,7 +68,7 @@ func assignAnchorOperator(d *dataTreeNavigator, context Context, expressionNode
|
||||
|
||||
anchorName := ""
|
||||
if !expressionNode.Operation.UpdateAssign {
|
||||
rhs, err := d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
@ -87,7 +89,7 @@ func assignAnchorOperator(d *dataTreeNavigator, context Context, expressionNode
|
||||
log.Debugf("Setting anchorName of : %v", candidate.GetKey())
|
||||
|
||||
if expressionNode.Operation.UpdateAssign {
|
||||
rhs, err := d.GetMatchingNodes(context.SingleChildContext(candidate), expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.SingleReadonlyChildContext(candidate), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
@ -176,7 +178,7 @@ func explodeNode(node *yaml.Node, context Context) error {
|
||||
} else {
|
||||
if valueNode.Kind == yaml.SequenceNode {
|
||||
log.Debugf("an alias merge list!")
|
||||
for index := 0; index < len(valueNode.Content); index = index + 1 {
|
||||
for index := len(valueNode.Content) - 1; index >= 0; index = index - 1 {
|
||||
aliasNode := valueNode.Content[index]
|
||||
err := applyAlias(node, aliasNode.Alias, index, context.ChildContext(newContent))
|
||||
if err != nil {
|
||||
|
||||
@ -4,7 +4,36 @@ import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
var specDocument = `- &CENTER { x: 1, y: 2 }
|
||||
- &LEFT { x: 0, y: 2 }
|
||||
- &BIG { r: 10 }
|
||||
- &SMALL { r: 1 }
|
||||
`
|
||||
|
||||
var expectedSpecResult = "D0, P[4], (!!map)::x: 1\ny: 2\nr: 10\n"
|
||||
|
||||
var anchorOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "Merge one map",
|
||||
subdescription: "see https://yaml.org/type/merge.html",
|
||||
document: specDocument + "- << : *CENTER\n r: 10\n",
|
||||
expression: ".[4] | explode(.)",
|
||||
expected: []string{expectedSpecResult},
|
||||
},
|
||||
{
|
||||
description: "Merge multiple maps",
|
||||
subdescription: "see https://yaml.org/type/merge.html",
|
||||
document: specDocument + "- << : [ *CENTER, *BIG ]\n",
|
||||
expression: ".[4] | explode(.)",
|
||||
expected: []string{"D0, P[4], (!!map)::r: 10\nx: 1\ny: 2\n"},
|
||||
},
|
||||
{
|
||||
description: "Override",
|
||||
subdescription: "see https://yaml.org/type/merge.html",
|
||||
document: specDocument + "- << : [ *BIG, *LEFT, *SMALL ]\n x: 1\n",
|
||||
expression: ".[4] | explode(.)",
|
||||
expected: []string{"D0, P[4], (!!map)::r: 10\nx: 1\ny: 2\n"},
|
||||
},
|
||||
{
|
||||
description: "Get anchor",
|
||||
document: `a: &billyBob cat`,
|
||||
@ -29,6 +58,22 @@ var anchorOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (doc)::a: &cat {b: cat}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `a: {c: cat}`,
|
||||
expression: `.a anchor |= .b`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: {c: cat}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `a: {c: cat}`,
|
||||
expression: `.a anchor = .b`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: {c: cat}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Get alias",
|
||||
document: `{b: &billyBob meow, a: *billyBob}`,
|
||||
@ -45,6 +90,30 @@ var anchorOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (doc)::{b: &meow purr, a: *meow}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Set alias to blank does nothing",
|
||||
document: `{b: &meow purr, a: cat}`,
|
||||
expression: `.a alias = ""`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{b: &meow purr, a: cat}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{b: &meow purr, a: cat}`,
|
||||
expression: `.a alias = .c`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{b: &meow purr, a: cat}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{b: &meow purr, a: cat}`,
|
||||
expression: `.a alias |= .c`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{b: &meow purr, a: cat}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Set alias relatively using assign-update",
|
||||
document: `{b: &meow purr, a: {f: meow}}`,
|
||||
@ -91,9 +160,9 @@ bar:
|
||||
c: bar_c
|
||||
foobarList:
|
||||
b: bar_b
|
||||
a: foo_a
|
||||
thing: bar_thing
|
||||
thing: foo_thing
|
||||
c: foobarList_c
|
||||
a: foo_a
|
||||
foobar:
|
||||
c: foo_c
|
||||
a: foo_a
|
||||
@ -106,7 +175,7 @@ foobar:
|
||||
expression: `.foo* | explode(.) | (. style="flow")`,
|
||||
expected: []string{
|
||||
"D0, P[foo], (!!map)::{a: foo_a, thing: foo_thing, c: foo_c}\n",
|
||||
"D0, P[foobarList], (!!map)::{b: bar_b, a: foo_a, thing: bar_thing, c: foobarList_c}\n",
|
||||
"D0, P[foobarList], (!!map)::{b: bar_b, thing: foo_thing, c: foobarList_c, a: foo_a}\n",
|
||||
"D0, P[foobar], (!!map)::{c: foo_c, a: foo_a, thing: foobar_thing}\n",
|
||||
},
|
||||
},
|
||||
@ -116,7 +185,7 @@ foobar:
|
||||
expression: `.foo* | explode(explode(.)) | (. style="flow")`,
|
||||
expected: []string{
|
||||
"D0, P[foo], (!!map)::{a: foo_a, thing: foo_thing, c: foo_c}\n",
|
||||
"D0, P[foobarList], (!!map)::{b: bar_b, a: foo_a, thing: bar_thing, c: foobarList_c}\n",
|
||||
"D0, P[foobarList], (!!map)::{b: bar_b, thing: foo_thing, c: foobarList_c, a: foo_a}\n",
|
||||
"D0, P[foobar], (!!map)::{c: foo_c, a: foo_a, thing: foobar_thing}\n",
|
||||
},
|
||||
},
|
||||
@ -130,7 +199,7 @@ foobar:
|
||||
},
|
||||
}
|
||||
|
||||
func TestAnchorAliaseOperatorScenarios(t *testing.T) {
|
||||
func TestAnchorAliasOperatorScenarios(t *testing.T) {
|
||||
for _, tt := range anchorOperatorScenarios {
|
||||
testScenario(t, &tt)
|
||||
}
|
||||
|
||||
@ -7,7 +7,7 @@ func assignUpdateOperator(d *dataTreeNavigator, context Context, expressionNode
|
||||
}
|
||||
var rhs Context
|
||||
if !expressionNode.Operation.UpdateAssign {
|
||||
rhs, err = d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
rhs, err = d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
}
|
||||
|
||||
for el := lhs.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
@ -44,7 +44,7 @@ func assignAttributesOperator(d *dataTreeNavigator, context Context, expressionN
|
||||
for el := lhs.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
|
||||
rhs, err := d.GetMatchingNodes(context.SingleChildContext(candidate), expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.SingleReadonlyChildContext(candidate), expressionNode.Rhs)
|
||||
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
|
||||
@ -12,6 +12,22 @@ var assignOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], ()::a:\n b: cat\nx: frog\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "{}",
|
||||
expression: `.a |= .b`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: null}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "{}",
|
||||
expression: `.a = .b`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: null}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Update node to be the child value",
|
||||
document: `{a: {b: {g: foof}}}`,
|
||||
@ -143,5 +159,5 @@ func TestAssignOperatorScenarios(t *testing.T) {
|
||||
for _, tt := range assignOperatorScenarios {
|
||||
testScenario(t, &tt)
|
||||
}
|
||||
documentScenarios(t, "Assign", assignOperatorScenarios)
|
||||
documentScenarios(t, "Assign (Update)", assignOperatorScenarios)
|
||||
}
|
||||
|
||||
@ -2,14 +2,13 @@ package yqlib
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"fmt"
|
||||
|
||||
yaml "gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
func isTruthy(c *CandidateNode) (bool, error) {
|
||||
node := unwrapDoc(c.Node)
|
||||
func isTruthyNode(node *yaml.Node) (bool, error) {
|
||||
value := true
|
||||
|
||||
if node.Tag == "!!null" {
|
||||
return false, nil
|
||||
}
|
||||
@ -23,41 +22,131 @@ func isTruthy(c *CandidateNode) (bool, error) {
|
||||
return value, nil
|
||||
}
|
||||
|
||||
func isTruthy(c *CandidateNode) (bool, error) {
|
||||
node := unwrapDoc(c.Node)
|
||||
return isTruthyNode(node)
|
||||
}
|
||||
|
||||
type boolOp func(bool, bool) bool
|
||||
|
||||
func performBoolOp(op boolOp) func(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
|
||||
return func(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
|
||||
owner := lhs
|
||||
|
||||
if lhs == nil && rhs == nil {
|
||||
owner = &CandidateNode{}
|
||||
} else if lhs == nil {
|
||||
owner = rhs
|
||||
}
|
||||
|
||||
var errDecoding error
|
||||
lhsTrue := false
|
||||
if lhs != nil {
|
||||
lhs.Node = unwrapDoc(lhs.Node)
|
||||
lhsTrue, errDecoding = isTruthy(lhs)
|
||||
|
||||
if errDecoding != nil {
|
||||
return nil, errDecoding
|
||||
}
|
||||
}
|
||||
log.Debugf("-- lhsTrue", lhsTrue)
|
||||
|
||||
rhsTrue := false
|
||||
if rhs != nil {
|
||||
rhs.Node = unwrapDoc(rhs.Node)
|
||||
|
||||
lhsTrue, errDecoding := isTruthy(lhs)
|
||||
rhsTrue, errDecoding = isTruthy(rhs)
|
||||
if errDecoding != nil {
|
||||
return nil, errDecoding
|
||||
}
|
||||
}
|
||||
log.Debugf("-- rhsTrue", rhsTrue)
|
||||
|
||||
rhsTrue, errDecoding := isTruthy(rhs)
|
||||
if errDecoding != nil {
|
||||
return nil, errDecoding
|
||||
return createBooleanCandidate(owner, op(lhsTrue, rhsTrue)), nil
|
||||
}
|
||||
}
|
||||
|
||||
return createBooleanCandidate(lhs, op(lhsTrue, rhsTrue)), nil
|
||||
func findBoolean(wantBool bool, d *dataTreeNavigator, context Context, expressionNode *ExpressionNode, sequenceNode *yaml.Node) (bool, error) {
|
||||
for _, node := range sequenceNode.Content {
|
||||
|
||||
if expressionNode != nil {
|
||||
//need to evaluate the expression against the node
|
||||
candidate := &CandidateNode{Node: node}
|
||||
rhs, err := d.GetMatchingNodes(context.SingleReadonlyChildContext(candidate), expressionNode)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if rhs.MatchingNodes.Len() > 0 {
|
||||
node = rhs.MatchingNodes.Front().Value.(*CandidateNode).Node
|
||||
} else {
|
||||
// no results found, ignore this entry
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
truthy, err := isTruthyNode(node)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if truthy == wantBool {
|
||||
return true, nil
|
||||
}
|
||||
}
|
||||
return false, nil
|
||||
}
|
||||
|
||||
func allOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
var results = list.New()
|
||||
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
candidateNode := unwrapDoc(candidate.Node)
|
||||
if candidateNode.Kind != yaml.SequenceNode {
|
||||
return Context{}, fmt.Errorf("any only supports arrays, was %v", candidateNode.Tag)
|
||||
}
|
||||
booleanResult, err := findBoolean(false, d, context, expressionNode.Rhs, candidateNode)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
result := createBooleanCandidate(candidate, !booleanResult)
|
||||
results.PushBack(result)
|
||||
}
|
||||
return context.ChildContext(results), nil
|
||||
}
|
||||
|
||||
func anyOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
var results = list.New()
|
||||
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
candidateNode := unwrapDoc(candidate.Node)
|
||||
if candidateNode.Kind != yaml.SequenceNode {
|
||||
return Context{}, fmt.Errorf("any only supports arrays, was %v", candidateNode.Tag)
|
||||
}
|
||||
booleanResult, err := findBoolean(true, d, context, expressionNode.Rhs, candidateNode)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
result := createBooleanCandidate(candidate, booleanResult)
|
||||
results.PushBack(result)
|
||||
}
|
||||
return context.ChildContext(results), nil
|
||||
}
|
||||
|
||||
func orOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("-- orOp")
|
||||
return crossFunction(d, context, expressionNode, performBoolOp(
|
||||
return crossFunction(d, context.ReadOnlyClone(), expressionNode, performBoolOp(
|
||||
func(b1 bool, b2 bool) bool {
|
||||
log.Debugf("-- peformingOrOp with %v and %v", b1, b2)
|
||||
return b1 || b2
|
||||
}))
|
||||
}), true)
|
||||
}
|
||||
|
||||
func andOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("-- AndOp")
|
||||
return crossFunction(d, context, expressionNode, performBoolOp(
|
||||
return crossFunction(d, context.ReadOnlyClone(), expressionNode, performBoolOp(
|
||||
func(b1 bool, b2 bool) bool {
|
||||
return b1 && b2
|
||||
}))
|
||||
}), true)
|
||||
}
|
||||
|
||||
func notOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
|
||||
@ -6,14 +6,38 @@ import (
|
||||
|
||||
var booleanOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "OR example",
|
||||
description: "`or` example",
|
||||
expression: `true or false`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::true\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "AND example",
|
||||
skipDoc: true,
|
||||
document: "b: hi",
|
||||
expression: `.a or .c`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::false\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "b: hi",
|
||||
expression: `select(.a or .b)`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::b: hi\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "b: hi",
|
||||
expression: `select((.a and .b) | not)`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::b: hi\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "`and` example",
|
||||
expression: `true and false`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::false\n",
|
||||
@ -27,6 +51,86 @@ var booleanOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!seq)::- {a: bird, b: dog}\n- {a: cat, b: fly}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "`any` returns true if any boolean in a given array is true",
|
||||
document: `[false, true]`,
|
||||
expression: "any",
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::true\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "`any` returns false for an empty array",
|
||||
document: `[]`,
|
||||
expression: "any",
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::false\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "`any_c` returns true if any element in the array is true for the given condition.",
|
||||
document: "a: [rad, awesome]\nb: [meh, whatever]",
|
||||
expression: `.[] |= any_c(. == "awesome")`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: true\nb: false\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[{pet: cat}]`,
|
||||
expression: `any_c(.name == "harry") as $c`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::[{pet: cat}]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[{pet: cat}]`,
|
||||
expression: `all_c(.name == "harry") as $c`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::[{pet: cat}]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[false, false]`,
|
||||
expression: "any",
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::false\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "`all` returns true if all booleans in a given array are true",
|
||||
document: `[true, true]`,
|
||||
expression: "all",
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::true\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[false, true]`,
|
||||
expression: "all",
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::false\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "`all` returns true for an empty array",
|
||||
document: `[]`,
|
||||
expression: "all",
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::true\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "`all_c` returns true if all elements in the array are true for the given condition.",
|
||||
document: "a: [rad, awesome]\nb: [meh, 12]",
|
||||
expression: `.[] |= all_c(tag == "!!str")`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: true\nb: false\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
expression: `false or false`,
|
||||
@ -45,6 +149,22 @@ var booleanOperatorScenarios = []expressionScenario{
|
||||
"D0, P[b], (!!bool)::true\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{}`,
|
||||
expression: `(.a.b or .c) as $x`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{}`,
|
||||
expression: `(.a.b and .c) as $x`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Not true is false",
|
||||
expression: `true | not`,
|
||||
|
||||
@ -17,9 +17,12 @@ import (
|
||||
...
|
||||
*/
|
||||
|
||||
func collectObjectOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
func collectObjectOperator(d *dataTreeNavigator, originalContext Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("-- collectObjectOperation")
|
||||
|
||||
context := originalContext.Clone()
|
||||
context.DontAutoCreate = false
|
||||
|
||||
if context.MatchingNodes.Len() == 0 {
|
||||
node := &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map", Value: "{}"}
|
||||
candidate := &CandidateNode{Node: node}
|
||||
|
||||
@ -5,6 +5,30 @@ import (
|
||||
)
|
||||
|
||||
var collectObjectOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "a: []",
|
||||
expression: `.a += [{"key": "att2", "value": "val2"}]`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: [{key: att2, value: val2}]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "",
|
||||
expression: `.a += {"key": "att2", "value": "val2"}`,
|
||||
expected: []string{
|
||||
"D0, P[], ()::a:\n key: att2\n value: val2\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "",
|
||||
expression: `.a += [0]`,
|
||||
expected: []string{
|
||||
"D0, P[], ()::a:\n - 0\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: `Collect empty object`,
|
||||
document: ``,
|
||||
|
||||
@ -5,6 +5,14 @@ import (
|
||||
)
|
||||
|
||||
var collectOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
skipDoc: true,
|
||||
document: ``,
|
||||
expression: `.a += [0]`,
|
||||
expected: []string{
|
||||
"D0, P[], ()::a:\n - 0\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Collect empty",
|
||||
document: ``,
|
||||
|
||||
@ -27,7 +27,7 @@ func assignCommentsOperator(d *dataTreeNavigator, context Context, expressionNod
|
||||
|
||||
comment := ""
|
||||
if !expressionNode.Operation.UpdateAssign {
|
||||
rhs, err := d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
@ -41,7 +41,7 @@ func assignCommentsOperator(d *dataTreeNavigator, context Context, expressionNod
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
|
||||
if expressionNode.Operation.UpdateAssign {
|
||||
rhs, err := d.GetMatchingNodes(context.SingleChildContext(candidate), expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.SingleReadonlyChildContext(candidate), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
@ -62,6 +62,22 @@ var commentOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (doc)::a: cat\n\n# cat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `a: cat`,
|
||||
expression: `. footComment=.b.d`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: cat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `a: cat`,
|
||||
expression: `. footComment|=.b.d`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: cat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Remove comment",
|
||||
document: "a: cat # comment\nb: dog # leave this",
|
||||
@ -71,7 +87,7 @@ var commentOperatorScenarios = []expressionScenario{
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Remove all comments",
|
||||
description: "Remove (strip) all comments",
|
||||
subdescription: "Note the use of `...` to ensure key nodes are included.",
|
||||
document: "# hi\n\na: cat # comment\n\n# great\n\nb: # key comment",
|
||||
expression: `... comments=""`,
|
||||
|
||||
@ -62,7 +62,7 @@ func sequenceFor(d *dataTreeNavigator, context Context, matchingNode *CandidateN
|
||||
}
|
||||
|
||||
return &CandidateNode{Node: &node, Document: document, Path: path}, nil
|
||||
})
|
||||
}, false)
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
@ -7,9 +7,7 @@ import (
|
||||
)
|
||||
|
||||
func deleteChildOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
contextToUse := context.Clone()
|
||||
contextToUse.DontAutoCreate = true
|
||||
nodesToDelete, err := d.GetMatchingNodes(contextToUse, expressionNode.Rhs)
|
||||
nodesToDelete, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
@ -18,48 +16,23 @@ func deleteChildOperator(d *dataTreeNavigator, context Context, expressionNode *
|
||||
for el := nodesToDelete.MatchingNodes.Back(); el != nil; el = el.Prev() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
|
||||
if len(candidate.Path) > 0 {
|
||||
deleteImmediateChildOp := &Operation{
|
||||
OperationType: deleteImmediateChildOpType,
|
||||
Value: candidate.Path[len(candidate.Path)-1],
|
||||
}
|
||||
|
||||
deleteImmediateChildOpNode := &ExpressionNode{
|
||||
Operation: deleteImmediateChildOp,
|
||||
Rhs: createTraversalTree(candidate.Path[0:len(candidate.Path)-1], traversePreferences{}, false),
|
||||
}
|
||||
|
||||
_, err := d.GetMatchingNodes(contextToUse, deleteImmediateChildOpNode)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
}
|
||||
}
|
||||
//problem: context may already be '.a' and then I pass in '.a.a2'.
|
||||
// should pass in .a2.
|
||||
if candidate.Parent == nil {
|
||||
log.Info("Could not find parent of %v", candidate.GetKey())
|
||||
return context, nil
|
||||
}
|
||||
|
||||
func deleteImmediateChildOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
parents, err := d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
parentNode := candidate.Parent.Node
|
||||
childPath := candidate.Path[len(candidate.Path)-1]
|
||||
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
childPath := expressionNode.Operation.Value
|
||||
|
||||
log.Debug("childPath to remove %v", childPath)
|
||||
|
||||
for el := parents.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
parent := el.Value.(*CandidateNode)
|
||||
parentNode := unwrapDoc(parent.Node)
|
||||
if parentNode.Kind == yaml.MappingNode {
|
||||
deleteFromMap(parent, childPath)
|
||||
deleteFromMap(candidate.Parent, childPath)
|
||||
} else if parentNode.Kind == yaml.SequenceNode {
|
||||
deleteFromArray(parent, childPath)
|
||||
deleteFromArray(candidate.Parent, childPath)
|
||||
} else {
|
||||
return Context{}, fmt.Errorf("Cannot delete nodes from parent of tag %v", parentNode.Tag)
|
||||
}
|
||||
|
||||
}
|
||||
return context, nil
|
||||
}
|
||||
|
||||
@ -21,6 +21,70 @@ var deleteOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (doc)::{a: {a2: frood}}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{a: {a1: fred, a2: frood}}`,
|
||||
expression: `.a | del(.a1)`,
|
||||
expected: []string{
|
||||
"D0, P[a], (!!map)::{a2: frood}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `a: [1,2,3]`,
|
||||
expression: `.a | del(.[1])`,
|
||||
expected: []string{
|
||||
"D0, P[a], (!!seq)::[1, 3]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[0, {a: cat, b: dog}]`,
|
||||
expression: `.[1] | del(.a)`,
|
||||
expected: []string{
|
||||
"D0, P[1], (!!map)::{b: dog}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[{a: cat, b: dog}]`,
|
||||
expression: `.[0] | del(.a)`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!map)::{b: dog}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[{a: {b: thing, c: frog}}]`,
|
||||
expression: `.[0].a | del(.b)`,
|
||||
expected: []string{
|
||||
"D0, P[0 a], (!!map)::{c: frog}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[{a: {b: thing, c: frog}}]`,
|
||||
expression: `.[0] | del(.a.b)`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!map)::{a: {c: frog}}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{a: [0, {b: thing, c: frog}]}`,
|
||||
expression: `.a[1] | del(.b)`,
|
||||
expected: []string{
|
||||
"D0, P[a 1], (!!map)::{c: frog}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{a: [0, {b: thing, c: frog}]}`,
|
||||
expression: `.a | del(.[1].b)`,
|
||||
expected: []string{
|
||||
"D0, P[a], (!!seq)::[0, {c: frog}]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{a: {a1: fred, a2: frood}}`,
|
||||
|
||||
162
pkg/yqlib/operator_entries.go
Normal file
162
pkg/yqlib/operator_entries.go
Normal file
@ -0,0 +1,162 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"fmt"
|
||||
|
||||
yaml "gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
func entrySeqFor(key *yaml.Node, value *yaml.Node) *yaml.Node {
|
||||
var keyKey = &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: "key"}
|
||||
var valueKey = &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!str", Value: "value"}
|
||||
|
||||
return &yaml.Node{
|
||||
Kind: yaml.MappingNode,
|
||||
Tag: "!!map",
|
||||
Content: []*yaml.Node{keyKey, key, valueKey, value},
|
||||
}
|
||||
}
|
||||
|
||||
func toEntriesFromMap(candidateNode *CandidateNode) *CandidateNode {
|
||||
var sequence = &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"}
|
||||
var entriesNode = candidateNode.CreateChild(nil, sequence)
|
||||
|
||||
var contents = unwrapDoc(candidateNode.Node).Content
|
||||
for index := 0; index < len(contents); index = index + 2 {
|
||||
key := contents[index]
|
||||
value := contents[index+1]
|
||||
|
||||
sequence.Content = append(sequence.Content, entrySeqFor(key, value))
|
||||
}
|
||||
return entriesNode
|
||||
}
|
||||
|
||||
func toEntriesfromSeq(candidateNode *CandidateNode) *CandidateNode {
|
||||
var sequence = &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"}
|
||||
var entriesNode = candidateNode.CreateChild(nil, sequence)
|
||||
|
||||
var contents = unwrapDoc(candidateNode.Node).Content
|
||||
for index := 0; index < len(contents); index = index + 1 {
|
||||
key := &yaml.Node{Kind: yaml.ScalarNode, Tag: "!!int", Value: fmt.Sprintf("%v", index)}
|
||||
value := contents[index]
|
||||
|
||||
sequence.Content = append(sequence.Content, entrySeqFor(key, value))
|
||||
}
|
||||
return entriesNode
|
||||
}
|
||||
|
||||
func toEntriesOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
var results = list.New()
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
candidateNode := unwrapDoc(candidate.Node)
|
||||
|
||||
switch candidateNode.Kind {
|
||||
case yaml.MappingNode:
|
||||
results.PushBack(toEntriesFromMap(candidate))
|
||||
|
||||
case yaml.SequenceNode:
|
||||
results.PushBack(toEntriesfromSeq(candidate))
|
||||
default:
|
||||
if candidateNode.Tag != "!!null" {
|
||||
return Context{}, fmt.Errorf("%v has no keys", candidate.Node.Tag)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return context.ChildContext(results), nil
|
||||
}
|
||||
|
||||
func parseEntry(d *dataTreeNavigator, entry *yaml.Node, position int) (*yaml.Node, *yaml.Node, error) {
|
||||
prefs := traversePreferences{DontAutoCreate: true}
|
||||
candidateNode := &CandidateNode{Node: entry}
|
||||
|
||||
keyResults, err := traverseMap(Context{}, candidateNode, "key", prefs, false)
|
||||
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
} else if keyResults.Len() != 1 {
|
||||
return nil, nil, fmt.Errorf("Expected to find one 'key' entry but found %v in position %v", keyResults.Len(), position)
|
||||
}
|
||||
|
||||
valueResults, err := traverseMap(Context{}, candidateNode, "value", prefs, false)
|
||||
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
} else if valueResults.Len() != 1 {
|
||||
return nil, nil, fmt.Errorf("Expected to find one 'value' entry but found %v in position %v", valueResults.Len(), position)
|
||||
}
|
||||
|
||||
return keyResults.Front().Value.(*CandidateNode).Node, valueResults.Front().Value.(*CandidateNode).Node, nil
|
||||
|
||||
}
|
||||
|
||||
func fromEntries(d *dataTreeNavigator, candidateNode *CandidateNode) (*CandidateNode, error) {
|
||||
var node = &yaml.Node{Kind: yaml.MappingNode, Tag: "!!map"}
|
||||
var mapCandidateNode = candidateNode.CreateChild(nil, node)
|
||||
|
||||
var contents = unwrapDoc(candidateNode.Node).Content
|
||||
|
||||
for index := 0; index < len(contents); index = index + 1 {
|
||||
key, value, err := parseEntry(d, contents[index], index)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
node.Content = append(node.Content, key, value)
|
||||
}
|
||||
return mapCandidateNode, nil
|
||||
}
|
||||
|
||||
func fromEntriesOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
var results = list.New()
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
candidateNode := unwrapDoc(candidate.Node)
|
||||
|
||||
switch candidateNode.Kind {
|
||||
case yaml.SequenceNode:
|
||||
mapResult, err := fromEntries(d, candidate)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
results.PushBack(mapResult)
|
||||
default:
|
||||
return Context{}, fmt.Errorf("from entries only runs against arrays")
|
||||
}
|
||||
}
|
||||
|
||||
return context.ChildContext(results), nil
|
||||
}
|
||||
|
||||
func withEntriesOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
|
||||
//to_entries on the context
|
||||
toEntries, err := toEntriesOperator(d, context, expressionNode)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
//run expression against entries
|
||||
// splat toEntries and pipe it into Rhs
|
||||
splatted, err := splat(d, toEntries, traversePreferences{})
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
result, err := d.GetMatchingNodes(splatted, expressionNode.Rhs)
|
||||
log.Debug("expressionNode.Rhs %v", expressionNode.Rhs.Operation.OperationType)
|
||||
log.Debug("result %v", result)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
collected, err := collectOperator(d, result, expressionNode)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
//from_entries on the result
|
||||
return fromEntriesOperator(d, collected, expressionNode)
|
||||
}
|
||||
62
pkg/yqlib/operator_entries_test.go
Normal file
62
pkg/yqlib/operator_entries_test.go
Normal file
@ -0,0 +1,62 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
var entriesOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "to_entries Map",
|
||||
document: `{a: 1, b: 2}`,
|
||||
expression: `to_entries`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::- key: a\n value: 1\n- key: b\n value: 2\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "to_entries Array",
|
||||
document: `[a, b]`,
|
||||
expression: `to_entries`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::- key: 0\n value: a\n- key: 1\n value: b\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "to_entries null",
|
||||
document: `null`,
|
||||
expression: `to_entries`,
|
||||
expected: []string{},
|
||||
},
|
||||
{
|
||||
description: "from_entries map",
|
||||
document: `{a: 1, b: 2}`,
|
||||
expression: `to_entries | from_entries`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!map)::a: 1\nb: 2\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "from_entries with numeric key indexes",
|
||||
subdescription: "from_entries always creates a map, even for numeric keys",
|
||||
document: `[a,b]`,
|
||||
expression: `to_entries | from_entries`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!map)::0: a\n1: b\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Use with_entries to update keys",
|
||||
document: `{a: 1, b: 2}`,
|
||||
expression: `with_entries(.key |= "KEY_" + .)`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!map)::KEY_a: 1\nKEY_b: 2\n",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
func TestEntriesOperatorScenarios(t *testing.T) {
|
||||
for _, tt := range entriesOperatorScenarios {
|
||||
testScenario(t, &tt)
|
||||
}
|
||||
documentScenarios(t, "Entries", entriesOperatorScenarios)
|
||||
}
|
||||
@ -4,12 +4,33 @@ import "gopkg.in/yaml.v3"
|
||||
|
||||
func equalsOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("-- equalsOperation")
|
||||
return crossFunction(d, context, expressionNode, isEquals(false))
|
||||
return crossFunction(d, context.ReadOnlyClone(), expressionNode, isEquals(false), true)
|
||||
}
|
||||
|
||||
func isEquals(flip bool) func(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
|
||||
return func(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
|
||||
value := false
|
||||
log.Debugf("-- isEquals cross function")
|
||||
if lhs == nil && rhs == nil {
|
||||
owner := &CandidateNode{}
|
||||
return createBooleanCandidate(owner, !flip), nil
|
||||
} else if lhs == nil {
|
||||
log.Debugf("lhs nil, but rhs is not")
|
||||
rhsNode := unwrapDoc(rhs.Node)
|
||||
value := rhsNode.Tag == "!!null"
|
||||
if flip {
|
||||
value = !value
|
||||
}
|
||||
return createBooleanCandidate(rhs, value), nil
|
||||
} else if rhs == nil {
|
||||
log.Debugf("lhs not nil, but rhs is")
|
||||
lhsNode := unwrapDoc(lhs.Node)
|
||||
value := lhsNode.Tag == "!!null"
|
||||
if flip {
|
||||
value = !value
|
||||
}
|
||||
return createBooleanCandidate(lhs, value), nil
|
||||
}
|
||||
|
||||
lhsNode := unwrapDoc(lhs.Node)
|
||||
rhsNode := unwrapDoc(rhs.Node)
|
||||
@ -28,6 +49,6 @@ func isEquals(flip bool) func(d *dataTreeNavigator, context Context, lhs *Candid
|
||||
}
|
||||
|
||||
func notEqualsOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("-- equalsOperation")
|
||||
return crossFunction(d, context, expressionNode, isEquals(true))
|
||||
log.Debugf("-- notEqualsOperator")
|
||||
return crossFunction(d, context.ReadOnlyClone(), expressionNode, isEquals(true), true)
|
||||
}
|
||||
|
||||
@ -5,6 +5,13 @@ import (
|
||||
)
|
||||
|
||||
var equalsOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
skipDoc: true,
|
||||
expression: ".a == .b",
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::true\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "cat",
|
||||
@ -14,6 +21,60 @@ var equalsOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!bool)::false\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "{}",
|
||||
expression: "(.a == .b) as $x",
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "{}",
|
||||
expression: ".a == .b",
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::true\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "{}",
|
||||
expression: "(.a != .b) as $x",
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "{}",
|
||||
expression: ".a != .b",
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::false\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "{a: {b: 10}}",
|
||||
expression: "select(.c != null)",
|
||||
expected: []string{},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "{a: {b: 10}}",
|
||||
expression: "select(.d == .c)",
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: {b: 10}}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "{a: {b: 10}}",
|
||||
expression: "select(null == .c)",
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: {b: 10}}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "{a: { b: {things: \"\"}, f: [1], g: [] }}",
|
||||
@ -87,6 +148,22 @@ var equalsOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!bool)::true\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Non exisitant key doesn't equal a value",
|
||||
document: "a: frog",
|
||||
expression: `select(.b != "thing")`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: frog\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Two non existant keys are equal",
|
||||
document: "a: frog",
|
||||
expression: `select(.b == .c)`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: frog\n",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
func TestEqualOperatorScenarios(t *testing.T) {
|
||||
|
||||
@ -12,14 +12,19 @@ func hasOperator(d *dataTreeNavigator, context Context, expressionNode *Expressi
|
||||
log.Debugf("-- hasOperation")
|
||||
var results = list.New()
|
||||
|
||||
rhs, err := d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
wanted := rhs.MatchingNodes.Front().Value.(*CandidateNode).Node
|
||||
wantedKey := wanted.Value
|
||||
rhs, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
wantedKey := "null"
|
||||
wanted := &yaml.Node{Tag: "!!null"}
|
||||
if rhs.MatchingNodes.Len() != 0 {
|
||||
wanted = rhs.MatchingNodes.Front().Value.(*CandidateNode).Node
|
||||
wantedKey = wanted.Value
|
||||
}
|
||||
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
|
||||
|
||||
@ -13,6 +13,22 @@ var hasOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!bool)::true\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `a: hello`,
|
||||
expression: `has(.b) as $c`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: hello\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `a: hello`,
|
||||
expression: `has(.b)`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!bool)::false\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Has map key",
|
||||
document: `- a: "yes"
|
||||
@ -28,6 +44,15 @@ var hasOperatorScenarios = []expressionScenario{
|
||||
"D0, P[3], (!!bool)::false\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Select, checking for existence of deep paths",
|
||||
subdescription: "Simply pipe in parent expressions into `has`",
|
||||
document: "- {a: {b: {c: cat}}}\n- {a: {b: {d: dog}}}",
|
||||
expression: `.[] | select(.a.b | has("c"))`,
|
||||
expected: []string{
|
||||
"D0, P[0], (!!map)::{a: {b: {c: cat}}}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
dontFormatInputForDoc: true,
|
||||
description: "Has array index",
|
||||
|
||||
@ -17,7 +17,7 @@ type multiplyPreferences struct {
|
||||
|
||||
func multiplyOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("-- MultiplyOperator")
|
||||
return crossFunction(d, context, expressionNode, multiply(expressionNode.Operation.Preferences.(multiplyPreferences)))
|
||||
return crossFunction(d, context, expressionNode, multiply(expressionNode.Operation.Preferences.(multiplyPreferences)), false)
|
||||
}
|
||||
|
||||
func multiply(preferences multiplyPreferences) func(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
|
||||
@ -31,6 +31,7 @@ func multiply(preferences multiplyPreferences) func(d *dataTreeNavigator, contex
|
||||
(lhs.Node.Kind == yaml.SequenceNode && rhs.Node.Kind == yaml.SequenceNode) {
|
||||
|
||||
var newBlank = lhs.CreateChild(nil, &yaml.Node{})
|
||||
|
||||
var newThing, err = mergeObjects(d, context, newBlank, lhs, multiplyPreferences{})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
@ -4,6 +4,32 @@ import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
var doc1 = `list:
|
||||
# Hi this is a comment.
|
||||
# Hello this is another comment.
|
||||
- "abc"`
|
||||
|
||||
var doc2 = `list2:
|
||||
# This is yet another comment.
|
||||
# Indeed this is yet another comment.
|
||||
- "123"`
|
||||
|
||||
var docExpected = `D0, P[], (!!map)::list:
|
||||
# Hi this is a comment.
|
||||
# Hello this is another comment.
|
||||
- "abc"
|
||||
list2:
|
||||
# This is yet another comment.
|
||||
# Indeed this is yet another comment.
|
||||
- "123"
|
||||
`
|
||||
|
||||
var mergeArraysObjectKeysText = `It's a complex command, the trickyness comes from needing to have the right context in the expressions.
|
||||
First we save the second array into a variable '$two' which lets us reference it later.
|
||||
We then need to update the first array. We will use the relative update (|=) because we need to update relative to the current element of the array in the LHS in the RHS expression.
|
||||
We set the current element of the first array as $cur. Now we multiply (merge) $cur with the matching entry in $two, by passing $two through a select filter.
|
||||
`
|
||||
|
||||
var multiplyOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "Multiply integers",
|
||||
@ -12,6 +38,15 @@ var multiplyOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!int)::12\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: doc1,
|
||||
document2: doc2,
|
||||
expression: `select(fi == 0) * select(fi == 1)`,
|
||||
expected: []string{
|
||||
docExpected,
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
expression: `3 * 4.5`,
|
||||
@ -216,6 +251,16 @@ var multiplyOperatorScenarios = []expressionScenario{
|
||||
"D0, P[a], (!!seq)::[{name: fred, age: 34}, {name: bob, age: 32}]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Merge arrays of objects together, matching on a key",
|
||||
subdescription: mergeArraysObjectKeysText,
|
||||
document: `[{a: apple, b: appleB}, {a: kiwi, b: kiwiB}, {a: banana, b: bananaB}]`,
|
||||
document2: `[{a: banana, c: bananaC}, {a: apple, b: appleB2}, {a: dingo, c: dingoC}]`,
|
||||
expression: `(select(fi==1) | .[]) as $two | select(fi==0) | .[] |= (. as $cur | $cur * ($two | select(.a == $cur.a)))`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::[{a: apple, b: appleB2}, {a: kiwi, b: kiwiB}, {a: banana, b: bananaB, c: bananaC}]\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Merge to prefix an element",
|
||||
document: `{a: cat, b: dog}`,
|
||||
@ -254,5 +299,5 @@ func TestMultiplyOperatorScenarios(t *testing.T) {
|
||||
for _, tt := range multiplyOperatorScenarios {
|
||||
testScenario(t, &tt)
|
||||
}
|
||||
documentScenarios(t, "Multiply", multiplyOperatorScenarios)
|
||||
documentScenarios(t, "Multiply (Merge)", multiplyOperatorScenarios)
|
||||
}
|
||||
|
||||
@ -11,9 +11,7 @@ func selectOperator(d *dataTreeNavigator, context Context, expressionNode *Expre
|
||||
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
childContext := context.SingleChildContext(candidate)
|
||||
childContext.DontAutoCreate = true
|
||||
rhs, err := d.GetMatchingNodes(childContext, expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.SingleReadonlyChildContext(candidate), expressionNode.Rhs)
|
||||
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
@ -24,7 +22,9 @@ func selectOperator(d *dataTreeNavigator, context Context, expressionNode *Expre
|
||||
|
||||
if first != nil {
|
||||
result := first.Value.(*CandidateNode)
|
||||
log.Debugf("result %v", NodeToString(result))
|
||||
includeResult, errDecoding := isTruthy(result)
|
||||
log.Debugf("isTruthy %v", includeResult)
|
||||
if errDecoding != nil {
|
||||
return Context{}, errDecoding
|
||||
}
|
||||
|
||||
@ -14,6 +14,16 @@ var selectOperatorScenarios = []expressionScenario{
|
||||
"D0, P[1], (!!str)::goat\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: "a: hello",
|
||||
document2: "b: world",
|
||||
expression: `select(.a == "hello" or .b == "world")`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: hello\n",
|
||||
"D0, P[], (doc)::b: world\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[{animal: cat, legs: {cool: true}}, {animal: fish}]`,
|
||||
|
||||
@ -10,7 +10,7 @@ func sortKeysOperator(d *dataTreeNavigator, context Context, expressionNode *Exp
|
||||
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
rhs, err := d.GetMatchingNodes(context.SingleChildContext(candidate), expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.SingleReadonlyChildContext(candidate), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
@ -13,6 +13,14 @@ var sortKeysOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (doc)::{a: blah, b: bing, c: frog}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{c: frog}`,
|
||||
expression: `sortKeys(.d)`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{c: frog}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Sort keys recursively",
|
||||
subdescription: "Note the array elements are left unsorted, but maps inside arrays are sorted",
|
||||
|
||||
@ -3,16 +3,82 @@ package yqlib
|
||||
import (
|
||||
"container/list"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
func getSubstituteParameters(d *dataTreeNavigator, block *ExpressionNode, context Context) (string, string, error) {
|
||||
regEx := ""
|
||||
replacementText := ""
|
||||
|
||||
regExNodes, err := d.GetMatchingNodes(context.ReadOnlyClone(), block.Lhs)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
if regExNodes.MatchingNodes.Front() != nil {
|
||||
regEx = regExNodes.MatchingNodes.Front().Value.(*CandidateNode).Node.Value
|
||||
}
|
||||
|
||||
log.Debug("regEx %v", regEx)
|
||||
|
||||
replacementNodes, err := d.GetMatchingNodes(context, block.Rhs)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
if replacementNodes.MatchingNodes.Front() != nil {
|
||||
replacementText = replacementNodes.MatchingNodes.Front().Value.(*CandidateNode).Node.Value
|
||||
}
|
||||
|
||||
return regEx, replacementText, nil
|
||||
}
|
||||
|
||||
func substitute(original string, regex *regexp.Regexp, replacement string) *yaml.Node {
|
||||
replacedString := regex.ReplaceAllString(original, replacement)
|
||||
return &yaml.Node{Kind: yaml.ScalarNode, Value: replacedString, Tag: "!!str"}
|
||||
}
|
||||
|
||||
func substituteStringOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
//rhs block operator
|
||||
//lhs of block = regex
|
||||
//rhs of block = replacement expression
|
||||
block := expressionNode.Rhs
|
||||
|
||||
regExStr, replacementText, err := getSubstituteParameters(d, block, context)
|
||||
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
regEx, err := regexp.Compile(regExStr)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
var results = list.New()
|
||||
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
node := unwrapDoc(candidate.Node)
|
||||
if node.Tag != "!!str" {
|
||||
return Context{}, fmt.Errorf("cannot substitute with %v, can only substitute strings. Hint: Most often you'll want to use '|=' over '=' for this operation.", node.Tag)
|
||||
}
|
||||
|
||||
targetNode := substitute(node.Value, regEx, replacementText)
|
||||
result := candidate.CreateChild(nil, targetNode)
|
||||
results.PushBack(result)
|
||||
}
|
||||
|
||||
return context.ChildContext(results), nil
|
||||
|
||||
}
|
||||
|
||||
func joinStringOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("-- joinStringOperator")
|
||||
joinStr := ""
|
||||
|
||||
rhs, err := d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
@ -26,7 +92,7 @@ func joinStringOperator(d *dataTreeNavigator, context Context, expressionNode *E
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
node := unwrapDoc(candidate.Node)
|
||||
if node.Kind != yaml.SequenceNode {
|
||||
return Context{}, fmt.Errorf("Cannot join with %v, can only join arrays of scalars", node.Tag)
|
||||
return Context{}, fmt.Errorf("cannot join with %v, can only join arrays of scalars", node.Tag)
|
||||
}
|
||||
targetNode := join(node.Content, joinStr)
|
||||
result := candidate.CreateChild(nil, targetNode)
|
||||
@ -53,7 +119,7 @@ func splitStringOperator(d *dataTreeNavigator, context Context, expressionNode *
|
||||
log.Debugf("-- splitStringOperator")
|
||||
splitStr := ""
|
||||
|
||||
rhs, err := d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
@ -13,6 +13,24 @@ var stringsOperatorScenarios = []expressionScenario{
|
||||
"D0, P[], (!!str)::cat; meow; 1; ; true\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Substitute / Replace string",
|
||||
subdescription: "This uses golang regex, described [here](https://github.com/google/re2/wiki/Syntax)\nNote the use of `|=` to run in context of the current string value.",
|
||||
document: `a: dogs are great`,
|
||||
expression: `.a |= sub("dogs", "cats")`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: cats are great\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Substitute / Replace string with regex",
|
||||
subdescription: "This uses golang regex, described [here](https://github.com/google/re2/wiki/Syntax)\nNote the use of `|=` to run in context of the current string value.",
|
||||
document: "a: cat\nb: heat",
|
||||
expression: `.[] |= sub("(a)", "${1}r")`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: cart\nb: heart\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Split strings",
|
||||
document: `"cat; meow; 1; ; true"`,
|
||||
|
||||
@ -31,7 +31,7 @@ func assignStyleOperator(d *dataTreeNavigator, context Context, expressionNode *
|
||||
log.Debugf("AssignStyleOperator: %v")
|
||||
var style yaml.Style
|
||||
if !expressionNode.Operation.UpdateAssign {
|
||||
rhs, err := d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
@ -54,7 +54,7 @@ func assignStyleOperator(d *dataTreeNavigator, context Context, expressionNode *
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
log.Debugf("Setting style of : %v", candidate.GetKey())
|
||||
if expressionNode.Operation.UpdateAssign {
|
||||
rhs, err := d.GetMatchingNodes(context.SingleChildContext(candidate), expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.SingleReadonlyChildContext(candidate), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
@ -5,6 +5,23 @@ import (
|
||||
)
|
||||
|
||||
var styleOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "Update and set style of a particular node (simple)",
|
||||
document: `a: {b: thing, c: something}`,
|
||||
expression: `.a.b = "new" | .a.b style="double"`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: {b: \"new\", c: something}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Update and set style of a particular node using path variables",
|
||||
subdescription: "You can use a variable to re-use a path",
|
||||
document: `a: {b: thing, c: something}`,
|
||||
expression: `.a.b as $x | $x = "new" | $x style="double"`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::a: {b: \"new\", c: something}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Set tagged style",
|
||||
document: `{a: cat, b: 5, c: 3.2, e: true}`,
|
||||
|
||||
97
pkg/yqlib/operator_subtract.go
Normal file
97
pkg/yqlib/operator_subtract.go
Normal file
@ -0,0 +1,97 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"strconv"
|
||||
|
||||
yaml "gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
func createSubtractOp(lhs *ExpressionNode, rhs *ExpressionNode) *ExpressionNode {
|
||||
return &ExpressionNode{Operation: &Operation{OperationType: subtractOpType},
|
||||
Lhs: lhs,
|
||||
Rhs: rhs}
|
||||
}
|
||||
|
||||
func subtractAssignOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
assignmentOp := &Operation{OperationType: assignOpType}
|
||||
assignmentOp.UpdateAssign = true
|
||||
selfExpression := &ExpressionNode{Operation: &Operation{OperationType: selfReferenceOpType}}
|
||||
assignmentOpNode := &ExpressionNode{Operation: assignmentOp, Lhs: expressionNode.Lhs, Rhs: createSubtractOp(selfExpression, expressionNode.Rhs)}
|
||||
return d.GetMatchingNodes(context, assignmentOpNode)
|
||||
}
|
||||
|
||||
func subtractOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
log.Debugf("Subtract operator")
|
||||
|
||||
return crossFunction(d, context.ReadOnlyClone(), expressionNode, subtract, false)
|
||||
}
|
||||
|
||||
func subtract(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error) {
|
||||
lhs.Node = unwrapDoc(lhs.Node)
|
||||
rhs.Node = unwrapDoc(rhs.Node)
|
||||
|
||||
lhsNode := lhs.Node
|
||||
|
||||
if lhsNode.Tag == "!!null" {
|
||||
return lhs.CreateChild(nil, rhs.Node), nil
|
||||
}
|
||||
|
||||
target := lhs.CreateChild(nil, &yaml.Node{})
|
||||
|
||||
switch lhsNode.Kind {
|
||||
case yaml.MappingNode:
|
||||
return nil, fmt.Errorf("Maps not yet supported for subtraction")
|
||||
case yaml.SequenceNode:
|
||||
return nil, fmt.Errorf("Sequences not yet supported for subtraction")
|
||||
// target.Node.Kind = yaml.SequenceNode
|
||||
// target.Node.Style = lhsNode.Style
|
||||
// target.Node.Tag = "!!seq"
|
||||
// target.Node.Content = append(lhsNode.Content, toNodes(rhs)...)
|
||||
case yaml.ScalarNode:
|
||||
if rhs.Node.Kind != yaml.ScalarNode {
|
||||
return nil, fmt.Errorf("%v (%v) cannot be added to a %v", rhs.Node.Tag, rhs.Path, lhsNode.Tag)
|
||||
}
|
||||
target.Node.Kind = yaml.ScalarNode
|
||||
target.Node.Style = lhsNode.Style
|
||||
return subtractScalars(target, lhsNode, rhs.Node)
|
||||
}
|
||||
|
||||
return target, nil
|
||||
}
|
||||
|
||||
func subtractScalars(target *CandidateNode, lhs *yaml.Node, rhs *yaml.Node) (*CandidateNode, error) {
|
||||
|
||||
if lhs.Tag == "!!str" {
|
||||
return nil, fmt.Errorf("strings cannot be subtracted")
|
||||
} else if lhs.Tag == "!!int" && rhs.Tag == "!!int" {
|
||||
lhsNum, err := strconv.Atoi(lhs.Value)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rhsNum, err := strconv.Atoi(rhs.Value)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result := lhsNum - rhsNum
|
||||
target.Node.Tag = "!!int"
|
||||
target.Node.Value = fmt.Sprintf("%v", result)
|
||||
} else if (lhs.Tag == "!!int" || lhs.Tag == "!!float") && (rhs.Tag == "!!int" || rhs.Tag == "!!float") {
|
||||
lhsNum, err := strconv.ParseFloat(lhs.Value, 64)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rhsNum, err := strconv.ParseFloat(rhs.Value, 64)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result := lhsNum - rhsNum
|
||||
target.Node.Tag = "!!float"
|
||||
target.Node.Value = fmt.Sprintf("%v", result)
|
||||
} else {
|
||||
return nil, fmt.Errorf("%v cannot be added to %v", lhs.Tag, rhs.Tag)
|
||||
}
|
||||
|
||||
return target, nil
|
||||
}
|
||||
58
pkg/yqlib/operator_subtract_test.go
Normal file
58
pkg/yqlib/operator_subtract_test.go
Normal file
@ -0,0 +1,58 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
var subtractOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{}`,
|
||||
expression: "(.a - .b) as $x",
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Number subtraction - float",
|
||||
subdescription: "If the lhs or rhs are floats then the expression will be calculated with floats.",
|
||||
document: `{a: 3, b: 4.5}`,
|
||||
expression: `.a = .a - .b`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: -1.5, b: 4.5}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Number subtraction - float",
|
||||
subdescription: "If the lhs or rhs are floats then the expression will be calculated with floats.",
|
||||
document: `{a: 3, b: 4.5}`,
|
||||
expression: `.a = .a - .b`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: -1.5, b: 4.5}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Number subtraction - int",
|
||||
subdescription: "If both the lhs and rhs are ints then the expression will be calculated with ints.",
|
||||
document: `{a: 3, b: 4}`,
|
||||
expression: `.a = .a - .b`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: -1, b: 4}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Decrement numbers",
|
||||
document: `{a: 3, b: 5}`,
|
||||
expression: `.[] -= 1`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{a: 2, b: 4}\n",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
func TestSubtractOperatorScenarios(t *testing.T) {
|
||||
for _, tt := range subtractOperatorScenarios {
|
||||
testScenario(t, &tt)
|
||||
}
|
||||
documentScenarios(t, "Subtract", subtractOperatorScenarios)
|
||||
}
|
||||
@ -12,7 +12,7 @@ func assignTagOperator(d *dataTreeNavigator, context Context, expressionNode *Ex
|
||||
tag := ""
|
||||
|
||||
if !expressionNode.Operation.UpdateAssign {
|
||||
rhs, err := d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
@ -32,7 +32,7 @@ func assignTagOperator(d *dataTreeNavigator, context Context, expressionNode *Ex
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
log.Debugf("Setting tag of : %v", candidate.GetKey())
|
||||
if expressionNode.Operation.UpdateAssign {
|
||||
rhs, err := d.GetMatchingNodes(context.SingleChildContext(candidate), expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.SingleReadonlyChildContext(candidate), expressionNode.Rhs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
@ -14,6 +14,7 @@ type traversePreferences struct {
|
||||
IncludeMapKeys bool
|
||||
DontAutoCreate bool // by default, we automatically create entries on the fly.
|
||||
DontIncludeMapValues bool
|
||||
OptionalTraverse bool // e.g. .adf?
|
||||
}
|
||||
|
||||
func splat(d *dataTreeNavigator, context Context, prefs traversePreferences) (Context, error) {
|
||||
@ -60,7 +61,7 @@ func traverse(d *dataTreeNavigator, context Context, matchingNode *CandidateNode
|
||||
|
||||
case yaml.SequenceNode:
|
||||
log.Debug("its a sequence of %v things!", len(value.Content))
|
||||
return traverseArray(matchingNode, operation)
|
||||
return traverseArray(matchingNode, operation, operation.Preferences.(traversePreferences))
|
||||
|
||||
case yaml.AliasNode:
|
||||
log.Debug("its an alias!")
|
||||
@ -88,15 +89,20 @@ func traverseArrayOperator(d *dataTreeNavigator, context Context, expressionNode
|
||||
|
||||
// rhs is a collect expression that will yield indexes to retreive of the arrays
|
||||
|
||||
rhs, err := d.GetMatchingNodes(context, expressionNode.Rhs)
|
||||
rhs, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Rhs)
|
||||
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
prefs := traversePreferences{}
|
||||
|
||||
if expressionNode.Rhs.Rhs != nil && expressionNode.Rhs.Rhs.Operation.Preferences != nil {
|
||||
prefs = expressionNode.Rhs.Rhs.Operation.Preferences.(traversePreferences)
|
||||
}
|
||||
var indicesToTraverse = rhs.MatchingNodes.Front().Value.(*CandidateNode).Node.Content
|
||||
|
||||
//now we traverse the result of the lhs against the indices we found
|
||||
result, err := traverseNodesWithArrayIndices(lhs, indicesToTraverse, traversePreferences{})
|
||||
result, err := traverseNodesWithArrayIndices(lhs, indicesToTraverse, prefs)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
@ -130,7 +136,7 @@ func traverseArrayIndices(context Context, matchingNode *CandidateNode, indicesT
|
||||
matchingNode.Node = node.Alias
|
||||
return traverseArrayIndices(context, matchingNode, indicesToTraverse, prefs)
|
||||
} else if node.Kind == yaml.SequenceNode {
|
||||
return traverseArrayWithIndices(matchingNode, indicesToTraverse)
|
||||
return traverseArrayWithIndices(matchingNode, indicesToTraverse, prefs)
|
||||
} else if node.Kind == yaml.MappingNode {
|
||||
return traverseMapWithIndices(context, matchingNode, indicesToTraverse, prefs)
|
||||
} else if node.Kind == yaml.DocumentNode {
|
||||
@ -159,7 +165,7 @@ func traverseMapWithIndices(context Context, candidate *CandidateNode, indices [
|
||||
return matchingNodeMap, nil
|
||||
}
|
||||
|
||||
func traverseArrayWithIndices(candidate *CandidateNode, indices []*yaml.Node) (*list.List, error) {
|
||||
func traverseArrayWithIndices(candidate *CandidateNode, indices []*yaml.Node, prefs traversePreferences) (*list.List, error) {
|
||||
log.Debug("traverseArrayWithIndices")
|
||||
var newMatches = list.New()
|
||||
node := unwrapDoc(candidate.Node)
|
||||
@ -177,6 +183,9 @@ func traverseArrayWithIndices(candidate *CandidateNode, indices []*yaml.Node) (*
|
||||
for _, indexNode := range indices {
|
||||
log.Debug("traverseArrayWithIndices: '%v'", indexNode.Value)
|
||||
index, err := strconv.ParseInt(indexNode.Value, 10, 64)
|
||||
if err != nil && prefs.OptionalTraverse {
|
||||
continue
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Cannot index array with '%v' (%v)", indexNode.Value, err)
|
||||
}
|
||||
@ -297,8 +306,8 @@ func traverseMergeAnchor(newMatches *orderedmap.OrderedMap, originalCandidate *C
|
||||
return nil
|
||||
}
|
||||
|
||||
func traverseArray(candidate *CandidateNode, operation *Operation) (*list.List, error) {
|
||||
func traverseArray(candidate *CandidateNode, operation *Operation, prefs traversePreferences) (*list.List, error) {
|
||||
log.Debug("operation Value %v", operation.Value)
|
||||
indices := []*yaml.Node{&yaml.Node{Value: operation.StringValue}}
|
||||
return traverseArrayWithIndices(candidate, indices)
|
||||
indices := []*yaml.Node{{Value: operation.StringValue}}
|
||||
return traverseArrayWithIndices(candidate, indices, prefs)
|
||||
}
|
||||
|
||||
@ -27,6 +27,22 @@ foobar:
|
||||
`
|
||||
|
||||
var traversePathOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[[1]]`,
|
||||
expression: `.[0][0]`,
|
||||
expected: []string{
|
||||
"D0, P[0 0], (!!int)::1\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[[[1]]]`,
|
||||
expression: `.[0][0][0]`,
|
||||
expected: []string{
|
||||
"D0, P[0 0 0], (!!int)::1\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Simple map navigation",
|
||||
document: `{a: {b: apple}}`,
|
||||
@ -45,6 +61,13 @@ var traversePathOperatorScenarios = []expressionScenario{
|
||||
"D0, P[1], (!!map)::{c: banana}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Optional Splat",
|
||||
subdescription: "Just like splat, but won't error if you run it against scalars",
|
||||
document: `"cat"`,
|
||||
expression: `.[]`,
|
||||
expected: []string{},
|
||||
},
|
||||
{
|
||||
description: "Special characters",
|
||||
subdescription: "Use quotes with brackets around path elements with special characters",
|
||||
@ -71,6 +94,14 @@ var traversePathOperatorScenarios = []expressionScenario{
|
||||
"D0, P[flying fox], (!!str)::frog\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `c: dog`,
|
||||
expression: `.[.a.b] as $x`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::c: dog\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Dynamic keys",
|
||||
subdescription: `Expressions within [] can be used to dynamically lookup / calculate keys`,
|
||||
@ -97,6 +128,19 @@ var traversePathOperatorScenarios = []expressionScenario{
|
||||
"D0, P[a b], (!!null)::null\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Optional identifier",
|
||||
subdescription: "Like jq, does not output an error when the yaml is not an array or object as expected",
|
||||
document: `[1,2,3]`,
|
||||
expression: `.a?`,
|
||||
expected: []string{},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[[1,2,3], {a: frog}]`,
|
||||
expression: `.[] | .["a"]?`,
|
||||
expected: []string{"D0, P[1 a], (!!str)::frog\n"},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: ``,
|
||||
|
||||
@ -5,6 +5,20 @@ import (
|
||||
)
|
||||
|
||||
var unionOperatorScenarios = []expressionScenario{
|
||||
// {
|
||||
// skipDoc: true,
|
||||
// document: "{}",
|
||||
// expression: `(.a, .b.c) as $x`,
|
||||
// expected: []string{
|
||||
// "D0, P[], (doc)::{}\n",
|
||||
// },
|
||||
// },
|
||||
// {
|
||||
// skipDoc: true,
|
||||
// document: "{}",
|
||||
// expression: `(.a, .b.c)`,
|
||||
// expected: []string{},
|
||||
// },
|
||||
{
|
||||
description: "Combine scalars",
|
||||
expression: `1, true, "cat"`,
|
||||
|
||||
64
pkg/yqlib/operator_unique.go
Normal file
64
pkg/yqlib/operator_unique.go
Normal file
@ -0,0 +1,64 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"fmt"
|
||||
|
||||
"github.com/elliotchance/orderedmap"
|
||||
yaml "gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
func unique(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
selfExpression := &ExpressionNode{Operation: &Operation{OperationType: selfReferenceOpType}}
|
||||
uniqueByExpression := &ExpressionNode{Operation: &Operation{OperationType: uniqueByOpType}, Rhs: selfExpression}
|
||||
return uniqueBy(d, context, uniqueByExpression)
|
||||
|
||||
}
|
||||
|
||||
func uniqueBy(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
|
||||
log.Debugf("-- uniqueBy Operator")
|
||||
var results = list.New()
|
||||
|
||||
for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
candidate := el.Value.(*CandidateNode)
|
||||
candidateNode := unwrapDoc(candidate.Node)
|
||||
|
||||
if candidateNode.Kind != yaml.SequenceNode {
|
||||
return Context{}, fmt.Errorf("Only arrays are supported for unique")
|
||||
}
|
||||
|
||||
var newMatches = orderedmap.NewOrderedMap()
|
||||
for _, node := range candidateNode.Content {
|
||||
child := &CandidateNode{Node: node}
|
||||
rhs, err := d.GetMatchingNodes(context.SingleReadonlyChildContext(child), expressionNode.Rhs)
|
||||
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
keyValue := "null"
|
||||
|
||||
if rhs.MatchingNodes.Len() > 0 {
|
||||
first := rhs.MatchingNodes.Front()
|
||||
keyCandidate := first.Value.(*CandidateNode)
|
||||
keyValue = keyCandidate.Node.Value
|
||||
}
|
||||
|
||||
_, exists := newMatches.Get(keyValue)
|
||||
|
||||
if !exists {
|
||||
newMatches.Set(keyValue, child.Node)
|
||||
}
|
||||
}
|
||||
resultNode := &yaml.Node{Kind: yaml.SequenceNode, Tag: "!!seq"}
|
||||
for el := newMatches.Front(); el != nil; el = el.Next() {
|
||||
resultNode.Content = append(resultNode.Content, el.Value.(*yaml.Node))
|
||||
}
|
||||
|
||||
results.PushBack(candidate.CreateChild(nil, resultNode))
|
||||
}
|
||||
|
||||
return context.ChildContext(results), nil
|
||||
|
||||
}
|
||||
65
pkg/yqlib/operator_unique_test.go
Normal file
65
pkg/yqlib/operator_unique_test.go
Normal file
@ -0,0 +1,65 @@
|
||||
package yqlib
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
var uniqueOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
description: "Unique array of scalars (string/numbers)",
|
||||
document: `[1,2,3,2]`,
|
||||
expression: `unique`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::- 1\n- 2\n- 3\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Unique nulls",
|
||||
subdescription: "Unique works on the node value, so it considers different representations of nulls to be different",
|
||||
document: `[~,null, ~, null]`,
|
||||
expression: `unique`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::- ~\n- null\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Unique all nulls",
|
||||
subdescription: "Run against the node tag to unique all the nulls",
|
||||
document: `[~,null, ~, null]`,
|
||||
expression: `unique_by(tag)`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::- ~\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Unique array object fields",
|
||||
document: `[{name: harry, pet: cat}, {name: billy, pet: dog}, {name: harry, pet: dog}]`,
|
||||
expression: `unique_by(.name)`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::- {name: harry, pet: cat}\n- {name: billy, pet: dog}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[{name: harry, pet: cat}, {pet: fish}, {name: harry, pet: dog}]`,
|
||||
expression: `unique_by(.name)`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::- {name: harry, pet: cat}\n- {pet: fish}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `[{name: harry, pet: cat}, {pet: fish}, {name: harry, pet: dog}]`,
|
||||
expression: `unique_by(.cat.dog)`,
|
||||
expected: []string{
|
||||
"D0, P[], (!!seq)::- {name: harry, pet: cat}\n",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
func TestUniqueOperatorScenarios(t *testing.T) {
|
||||
for _, tt := range uniqueOperatorScenarios {
|
||||
testScenario(t, &tt)
|
||||
}
|
||||
documentScenarios(t, "Unique", uniqueOperatorScenarios)
|
||||
}
|
||||
@ -16,7 +16,7 @@ func getVariableOperator(d *dataTreeNavigator, context Context, expressionNode *
|
||||
}
|
||||
|
||||
func assignVariableOperator(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode) (Context, error) {
|
||||
lhs, err := d.GetMatchingNodes(context, expressionNode.Lhs)
|
||||
lhs, err := d.GetMatchingNodes(context.ReadOnlyClone(), expressionNode.Lhs)
|
||||
if err != nil {
|
||||
return Context{}, nil
|
||||
}
|
||||
|
||||
@ -5,6 +5,14 @@ import (
|
||||
)
|
||||
|
||||
var variableOperatorScenarios = []expressionScenario{
|
||||
{
|
||||
skipDoc: true,
|
||||
document: `{}`,
|
||||
expression: `.a.b as $foo`,
|
||||
expected: []string{
|
||||
"D0, P[], (doc)::{}\n",
|
||||
},
|
||||
},
|
||||
{
|
||||
description: "Single value variable",
|
||||
document: `a: cat`,
|
||||
|
||||
@ -24,7 +24,34 @@ func emptyOperator(d *dataTreeNavigator, context Context, expressionNode *Expres
|
||||
|
||||
type crossFunctionCalculation func(d *dataTreeNavigator, context Context, lhs *CandidateNode, rhs *CandidateNode) (*CandidateNode, error)
|
||||
|
||||
func doCrossFunc(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode, calculation crossFunctionCalculation) (Context, error) {
|
||||
func resultsForRhs(d *dataTreeNavigator, context Context, lhsCandidate *CandidateNode, rhs Context, calculation crossFunctionCalculation, results *list.List, calcWhenEmpty bool) error {
|
||||
|
||||
if calcWhenEmpty && rhs.MatchingNodes.Len() == 0 {
|
||||
resultCandidate, err := calculation(d, context, lhsCandidate, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resultCandidate != nil {
|
||||
results.PushBack(resultCandidate)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
for rightEl := rhs.MatchingNodes.Front(); rightEl != nil; rightEl = rightEl.Next() {
|
||||
log.Debugf("Applying calc")
|
||||
rhsCandidate := rightEl.Value.(*CandidateNode)
|
||||
resultCandidate, err := calculation(d, context, lhsCandidate, rhsCandidate)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resultCandidate != nil {
|
||||
results.PushBack(resultCandidate)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func doCrossFunc(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode, calculation crossFunctionCalculation, calcWhenEmpty bool) (Context, error) {
|
||||
var results = list.New()
|
||||
lhs, err := d.GetMatchingNodes(context, expressionNode.Lhs)
|
||||
if err != nil {
|
||||
@ -38,24 +65,26 @@ func doCrossFunc(d *dataTreeNavigator, context Context, expressionNode *Expressi
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
for el := lhs.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
lhsCandidate := el.Value.(*CandidateNode)
|
||||
|
||||
for rightEl := rhs.MatchingNodes.Front(); rightEl != nil; rightEl = rightEl.Next() {
|
||||
log.Debugf("Applying calc")
|
||||
rhsCandidate := rightEl.Value.(*CandidateNode)
|
||||
resultCandidate, err := calculation(d, context, lhsCandidate, rhsCandidate)
|
||||
if calcWhenEmpty && lhs.MatchingNodes.Len() == 0 {
|
||||
err := resultsForRhs(d, context, nil, rhs, calculation, results, calcWhenEmpty)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
results.PushBack(resultCandidate)
|
||||
}
|
||||
|
||||
for el := lhs.MatchingNodes.Front(); el != nil; el = el.Next() {
|
||||
lhsCandidate := el.Value.(*CandidateNode)
|
||||
|
||||
err := resultsForRhs(d, context, lhsCandidate, rhs, calculation, results, calcWhenEmpty)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
}
|
||||
return context.ChildContext(results), nil
|
||||
}
|
||||
|
||||
func crossFunction(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode, calculation crossFunctionCalculation) (Context, error) {
|
||||
func crossFunction(d *dataTreeNavigator, context Context, expressionNode *ExpressionNode, calculation crossFunctionCalculation, calcWhenEmpty bool) (Context, error) {
|
||||
var results = list.New()
|
||||
|
||||
var evaluateAllTogether = true
|
||||
@ -66,11 +95,11 @@ func crossFunction(d *dataTreeNavigator, context Context, expressionNode *Expres
|
||||
}
|
||||
}
|
||||
if evaluateAllTogether {
|
||||
return doCrossFunc(d, context, expressionNode, calculation)
|
||||
return doCrossFunc(d, context, expressionNode, calculation, calcWhenEmpty)
|
||||
}
|
||||
|
||||
for matchEl := context.MatchingNodes.Front(); matchEl != nil; matchEl = matchEl.Next() {
|
||||
innerResults, err := doCrossFunc(d, context.SingleChildContext(matchEl.Value.(*CandidateNode)), expressionNode, calculation)
|
||||
innerResults, err := doCrossFunc(d, context.SingleChildContext(matchEl.Value.(*CandidateNode)), expressionNode, calculation, calcWhenEmpty)
|
||||
if err != nil {
|
||||
return Context{}, err
|
||||
}
|
||||
|
||||
@ -71,7 +71,7 @@ func testScenario(t *testing.T, s *expressionScenario) {
|
||||
t.Error(fmt.Errorf("%v: %v", err, s.expression))
|
||||
return
|
||||
}
|
||||
test.AssertResultComplexWithContext(t, s.expected, resultsToString(context.MatchingNodes), fmt.Sprintf("exp: %v\ndoc: %v", s.expression, s.document))
|
||||
test.AssertResultComplexWithContext(t, s.expected, resultsToString(context.MatchingNodes), fmt.Sprintf("desc: %v\nexp: %v\ndoc: %v", s.description, s.expression, s.document))
|
||||
}
|
||||
|
||||
func resultsToString(results *list.List) []string {
|
||||
|
||||
@ -34,7 +34,7 @@ func NewPrinter(writer io.Writer, outputToJSON bool, unwrapScalar bool, colorsEn
|
||||
unwrapScalar: unwrapScalar,
|
||||
colorsEnabled: colorsEnabled,
|
||||
indent: indent,
|
||||
printDocSeparators: printDocSeparators,
|
||||
printDocSeparators: !outputToJSON && printDocSeparators,
|
||||
firstTimePrinting: true,
|
||||
treeNavigator: NewDataTreeNavigator(),
|
||||
}
|
||||
|
||||
@ -130,7 +130,9 @@ func TestPrinterMultipleDocsInSinglePrint(t *testing.T) {
|
||||
func TestPrinterMultipleDocsJson(t *testing.T) {
|
||||
var output bytes.Buffer
|
||||
var writer = bufio.NewWriter(&output)
|
||||
printer := NewPrinter(writer, true, true, false, 0, false)
|
||||
// note printDocSeparators is true, it should still not print document separators
|
||||
// when outputing JSON.
|
||||
printer := NewPrinter(writer, true, true, false, 0, true)
|
||||
|
||||
inputs, err := readDocuments(strings.NewReader(multiDocSample), "sample.yml", 0)
|
||||
if err != nil {
|
||||
|
||||
@ -2,6 +2,7 @@
|
||||
- increment version in snapcraft.yaml
|
||||
- increment version in github-action/Dockerfile
|
||||
- make sure local build passes
|
||||
- run ./scripts/secure.sh (manual because docker platforms)
|
||||
- commit version update changes
|
||||
- tag git with same version number
|
||||
- commit vX tag - this will trigger github actions
|
||||
|
||||
@ -10,12 +10,3 @@ else
|
||||
./bin/golangci-lint run --timeout=5m
|
||||
fi
|
||||
|
||||
# ./bin/golangci-lint \
|
||||
# --tests \
|
||||
# --vendor \
|
||||
# --disable=aligncheck \
|
||||
# --disable=gotype \
|
||||
# --disable=goconst \
|
||||
# --disable=gocyclo \
|
||||
# --deadline=300s \
|
||||
# ./...
|
||||
|
||||
@ -1,4 +1,5 @@
|
||||
#!/bin/sh
|
||||
set -ex
|
||||
go get golang.org/x/tools/cmd/goimports
|
||||
wget -O- -nv https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s v1.24.0
|
||||
wget -O- -nv https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s v1.37.1
|
||||
# wget -O- -nv https://raw.githubusercontent.com/securego/gosec/master/install.sh | sh -s v2.6.1
|
||||
|
||||
@ -1,5 +1,6 @@
|
||||
#!/bin/bash
|
||||
|
||||
find . \( -path ./vendor \) -prune -o -name "*.go" -exec goimports -w {} \;
|
||||
gofmt -w -s .
|
||||
go mod tidy
|
||||
go mod vendor
|
||||
11
scripts/secure.sh
Executable file
11
scripts/secure.sh
Executable file
@ -0,0 +1,11 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -o errexit
|
||||
set -o pipefail
|
||||
|
||||
if command -v gosec &> /dev/null
|
||||
then
|
||||
gosec ${PWD}
|
||||
else
|
||||
./bin/gosec ${PWD}
|
||||
fi
|
||||
@ -1,5 +1,5 @@
|
||||
name: yq
|
||||
version: '4.6.1'
|
||||
version: '4.9.3'
|
||||
summary: A lightweight and portable command-line YAML processor
|
||||
description: |
|
||||
The aim of the project is to be the jq or sed of yaml files.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user