Compare commits

..

131 Commits

Author SHA1 Message Date
4e979d76a3 move scanip from cmdline to config
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2024-09-11 13:28:09 +02:00
6561a77d6d updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2024-08-19 03:25:49 +02:00
b8521f050a updated ipbl
All checks were successful
continuous-integration/drone/push Build is passing
* added version in user agent
* lint code
* removed compat with config v2
2024-07-17 23:13:21 +02:00
1f56bd64ca Merge pull request 'dependencies + new index' (#6) from develop into master
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
Reviewed-on: #6
2024-05-30 09:12:54 +02:00
eb9bdf35cd update indices on event table
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2024-05-30 09:10:36 +02:00
938ef9fb3f updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2024-05-30 09:09:13 +02:00
f8ecc86006 updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2024-05-11 18:03:51 +02:00
63855a9e21 updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2024-03-21 07:42:39 +01:00
45f86d5598 Merge pull request 'updated dependencies - fosdem 2024 commit' (#5) from develop into master
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone Build is passing
Reviewed-on: #5
2024-02-03 15:24:43 +01:00
c782383171 updated dependencies - fosdem 2024 commit
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2024-02-03 13:23:32 +01:00
08f9378de8 added indices to event table
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2023-12-31 01:14:40 +01:00
d7fcd209a4 updated gcConnOnError func
All checks were successful
continuous-integration/drone/push Build is passing
2023-12-26 16:26:17 +01:00
5858b1398f added conn close on ping call
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2023-12-26 13:15:11 +01:00
583742eef6 added ping method
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2023-12-24 13:57:51 +01:00
f1f6b2dbc9 replaced classic map by sync.Map
All checks were successful
continuous-integration/drone/push Build is passing
2023-12-24 10:22:26 +01:00
2a52ef3157 updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2023-12-24 09:36:41 +01:00
137cbd8d01 updated functions names 2023-12-24 09:36:33 +01:00
d03dc8b6cd Merge pull request 'update to 1.6.0' (#4) from develop into master
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
Reviewed-on: #4
2023-11-10 23:34:38 +01:00
8deaf69baf added switch to manually migrate data
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-11-10 23:24:23 +01:00
246406f638 updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2023-11-10 23:16:44 +01:00
8d45b04e99 code cleanup
All checks were successful
continuous-integration/drone/push Build is passing
2023-11-03 11:02:06 +01:00
410d4dd810 feat: implement new config handling
All checks were successful
continuous-integration/drone/push Build is passing
* new config endpoint with v=2 param
* fix messages in websockets handlers
2023-11-03 10:06:17 +01:00
580ded68bb removed zmq from drone
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2023-11-01 18:03:40 +01:00
b1d681f13f feat: added ip type on event
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2023-10-28 16:08:50 +02:00
491718ff80 change on ip model
All checks were successful
continuous-integration/drone/push Build is passing
2023-10-21 02:39:28 +02:00
9e5ffdd6b6 updated dependencies 2023-10-21 02:38:57 +02:00
74c266d853 updates on multithreading (#3)
All checks were successful
continuous-integration/drone/tag Build is passing
continuous-integration/drone/push Build is passing
Reviewed-on: #3
2023-09-16 09:57:43 +02:00
995a10ca17 Merge pull request 'updates on dependencies + multithreading fixes' (#2) from develop into master
All checks were successful
continuous-integration/drone/push Build is passing
Reviewed-on: #2
2023-09-14 15:42:49 +02:00
ee28c124a0 fix on database package
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-09-14 15:39:42 +02:00
9bea58a890 fix: fixes on debug mode + multithreading
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/pr Build is failing
2023-09-14 15:33:09 +02:00
945b53f28c updated dependencies (#1)
All checks were successful
continuous-integration/drone/push Build is passing
Reviewed-on: #1
2023-09-10 20:41:55 +02:00
46b3c1abc3 updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is passing
2023-09-10 20:40:56 +02:00
fbf9382ca4 updated ipbl
All checks were successful
continuous-integration/drone/push Build is passing
2023-09-10 20:37:45 +02:00
e068b4740c updated ipbl
All checks were successful
continuous-integration/drone/push Build is passing
2023-09-10 20:36:57 +02:00
8862ac15b0 Merge branch 'develop-websocket'
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2023-06-27 12:00:30 +02:00
16b150503f updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2023-06-09 13:44:44 +02:00
62ed0e030c added error handling
All checks were successful
continuous-integration/drone/push Build is passing
2023-06-05 22:00:52 +02:00
670f8a6f9f updated ipbl
All checks were successful
continuous-integration/drone/push Build is passing
2023-05-27 18:36:15 +02:00
e5658ff99f updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2023-05-11 07:36:48 +02:00
2f60ee84e1 added timeout on ip scan when error 2023-05-01 08:20:03 +02:00
a735f54024 updated WelcomeAgents func
All checks were successful
continuous-integration/drone/push Build is passing
2023-04-29 11:14:35 +02:00
efc26fb27a updated README.md
All checks were successful
continuous-integration/drone/push Build is passing
2023-04-28 18:18:35 +02:00
6b569681fe updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2023-04-28 17:51:43 +02:00
2ed127c238 updated dependencies 2023-04-10 18:08:34 +02:00
5da8a5c952 fix: re-added gcConnOnError, addcheck on send 2023-04-10 17:10:04 +02:00
b99badcb87 fix: tmp remove gcConnOnError 2023-04-10 16:59:48 +02:00
1802abe553 fix: gcConnOnError 2023-04-10 16:53:53 +02:00
77919b208f fix: gcConnOnError 2023-04-10 15:28:36 +02:00
8b617a8f9f fix: gcConn 2023-04-10 12:04:40 +02:00
447f756922 updated ipbl websocket feat 2023-04-10 11:57:25 +02:00
abc02caf92 updated ipbl websocket feat 2023-04-09 17:38:19 +02:00
14d72693dd more stable ipbl websocket feat 2023-04-09 15:05:31 +02:00
f5ed3ee455 updated ipbl websocket feat 2023-04-09 01:41:52 +02:00
e84ca893d0 updated dependencies
Some checks failed
continuous-integration/drone/push Build is failing
2023-03-26 11:18:08 +02:00
0354fe3255 updated ws sockets
All checks were successful
continuous-integration/drone/push Build is passing
2023-03-12 14:25:40 +01:00
697df68640 updated ws feature
All checks were successful
continuous-integration/drone/push Build is passing
2023-03-05 23:05:05 +01:00
bb76849f33 remove zmq from project
All checks were successful
continuous-integration/drone/push Build is passing
2023-03-05 22:37:17 +01:00
98f07aaff8 updated dependencies
Some checks failed
continuous-integration/drone/push Build is failing
2023-03-05 22:32:40 +01:00
118588e5bb added initial websocket feature / remove zeromq
Some checks failed
continuous-integration/drone/push Build is failing
2023-03-05 22:30:42 +01:00
05a275c9d3 updated dependencies
Some checks failed
continuous-integration/drone/push Build is failing
2023-02-18 22:47:12 +01:00
90b7f68004 Merge branch 'develop-xorm'
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone/tag Build is failing
2023-01-21 09:59:32 +01:00
8439926037 added arch arm64 in ci
Some checks failed
continuous-integration/drone/push Build is failing
2023-01-21 09:39:07 +01:00
e1d5d7bdfe added arch arm64 in ci
Some checks failed
continuous-integration/drone/push Build is failing
2023-01-21 09:37:27 +01:00
c81d89d417 updated dependencies
All checks were successful
continuous-integration/drone/tag Build is passing
2023-01-19 08:51:51 +01:00
53f78abaf3 stabilized zmq loop 2023-01-19 08:51:03 +01:00
1cfe9e83bb updated dependencies 2023-01-19 08:50:42 +01:00
5b95e817de updated ipbl
All checks were successful
continuous-integration/drone/push Build is passing
* removed unused columns in tables
* added multithreaded support for ip scans
* added health endpoint
2023-01-17 18:37:31 +01:00
698510b19a updated messaging system, removed cache
All checks were successful
continuous-integration/drone/push Build is passing
2023-01-16 01:01:14 +01:00
4bada1c166 updated messaging system
All checks were successful
continuous-integration/drone/push Build is passing
2023-01-15 16:04:49 +01:00
9244e05fc3 updated ScanIP function
All checks were successful
continuous-integration/drone/push Build is passing
2023-01-14 20:58:32 +01:00
1868bb2ea4 misc update for develop-xorm
All checks were successful
continuous-integration/drone/push Build is passing
2023-01-14 17:30:42 +01:00
46095f778c updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2023-01-14 17:13:25 +01:00
589c6a5861 updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2023-01-14 17:11:15 +01:00
467ce67fa6 updated ipbl
All checks were successful
continuous-integration/drone/push Build is passing
2022-10-23 18:16:43 +02:00
2a2199e3eb updated models
All checks were successful
continuous-integration/drone/push Build is passing
2022-10-16 23:27:57 +02:00
7e8437c2cd updated dependencies 2022-10-16 23:27:39 +02:00
a3d1f84271 updated ipbl 2022-09-18 10:24:55 +02:00
b81adb4f95 updated ipbl 2022-09-18 10:24:44 +02:00
77e57b4f7d updated ipbl 2022-09-18 10:23:24 +02:00
e92d332269 updated ipbl to handle init of ipblc
All checks were successful
continuous-integration/drone/push Build is passing
2022-09-11 23:34:51 +02:00
3f000cbf8f updated ipbl, bug fixes, change models
All checks were successful
continuous-integration/drone Build is passing
continuous-integration/drone/tag Build is passing
2022-08-28 16:23:21 +02:00
a1329fc544 updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2022-07-16 11:42:03 +02:00
be9446abab updated ipbl
All checks were successful
continuous-integration/drone/push Build is passing
2022-07-01 12:47:01 +02:00
0518d4445c added hostinfo collect on ipbl
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-07-01 10:25:48 +02:00
3fd0689366 updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
2022-06-05 23:07:32 +02:00
459c74bbf0 misc fixes
Some checks failed
continuous-integration/drone/push Build is failing
2022-06-04 17:45:37 +02:00
f854a3b219 added per set blocktime
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-06-04 11:32:56 +02:00
9d70341f41 added transactions, cleaned code
All checks were successful
continuous-integration/drone/push Build is passing
2022-05-10 13:00:22 +02:00
632c4ba101 updated ipbl limit on ScanIP
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-04-10 19:11:26 +02:00
c362158dfb updated ipbl
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-04-10 12:46:39 +02:00
2c4af80b3d Merge branch 'master' of ssh://git.paulbsd.com:2222/paulbsd/ipbl
All checks were successful
continuous-integration/drone/push Build is passing
2022-04-10 12:43:32 +02:00
60943adc66 updated ipbl api endpoints 2022-04-10 12:43:28 +02:00
bdaf6c49a3 fix bug when inserting~ ips
All checks were successful
continuous-integration/drone/push Build is passing
2022-03-29 10:14:40 +02:00
9803adfbb9 updated ipbl
All checks were successful
continuous-integration/drone/push Build is passing
2022-03-27 18:32:58 +02:00
9677593085 Merge branch 'master' of https://git.paulbsd.com/paulbsd/ipbl
All checks were successful
continuous-integration/drone/push Build is passing
2022-03-26 19:46:13 +01:00
d8ab04aedf updated ipbl with some functions reliability 2022-03-26 19:46:04 +01:00
a6d2de6f69 Update 'README.md'
All checks were successful
continuous-integration/drone/push Build is passing
2022-03-25 11:56:28 +01:00
551b3311a6 updated dependencies
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-03-19 16:21:31 +01:00
cbce240dc2 updated dependencies 2022-03-19 16:21:19 +01:00
1ef4b1789a fix bug in InsertIPBulk 2022-03-19 16:21:08 +01:00
b1b7302c7a updated .drone.yml
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-03-19 15:32:49 +01:00
83995c26a5 changed tables schemas
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-03-19 14:48:50 +01:00
881969f81e updated ipbl 2022-03-14 18:49:28 +01:00
32784ff442 misc updates 2022-03-13 12:08:44 +01:00
c41747e146 updated .drone.yml
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-03-11 23:35:24 +01:00
90136fe906 added zeromq handling
Some checks failed
continuous-integration/drone/push Build is failing
2022-03-11 23:34:09 +01:00
e828164392 merge conflict
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-03-10 19:57:32 +01:00
b93cd8306e added zmq configs 2022-03-10 19:56:03 +01:00
1df597e1d6 Merge branch 'master' of https://git.paulbsd.com/paulbsd/ipbl
Some checks reported errors
continuous-integration/drone/push Build encountered an error
continuous-integration/drone/tag Build is passing
2022-02-27 01:17:35 +01:00
8cb9c51bde misc changes 2022-02-27 01:17:21 +01:00
2bf11fdb75 updated misc ipbl functions
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-25 13:39:19 +01:00
52e767b941 added folders config update
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-25 12:12:57 +01:00
de8717dc99 changed archive filename
Some checks reported errors
continuous-integration/drone/push Build encountered an error
continuous-integration/drone/tag Build is passing
2022-02-24 19:53:12 +01:00
14f8cbebb0 added go generate
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-02-24 19:50:05 +01:00
561b5be477 changed .drone.yml
Some checks failed
continuous-integration/drone/push Build is failing
2022-02-24 19:48:23 +01:00
ee33ee3fd6 updated .drone.yml
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2022-02-24 18:46:12 +01:00
29a5c5927b added folders config in database
Some checks reported errors
continuous-integration/drone/push Build encountered an error
continuous-integration/drone/tag Build encountered an error
2022-02-24 17:28:29 +01:00
ef228e3750 updates on rdns resolve messages
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2022-02-23 11:17:21 +01:00
8512402488 replace abusive keywords
Some checks reported errors
continuous-integration/drone/push Build encountered an error
continuous-integration/drone/tag Build is passing
2022-02-22 19:46:01 +01:00
878239d5d5 updated msg in funcs.go after IP POST
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2022-02-22 14:44:52 +01:00
eb63a8dd07 updated msg in funcs.go after IP POST
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2022-02-22 14:44:28 +01:00
40185c4abb updated dependencies
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2022-02-22 14:43:07 +01:00
ea09905151 added groupby to http://host/ip/last endpoint
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-12 11:43:45 +01:00
c759df10d0 updated functions
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-12 00:33:58 +01:00
Paul Lecuq
ac71103475 updated ci
All checks were successful
continuous-integration/drone/push Build is passing
continuous-integration/drone/tag Build is passing
2022-02-11 09:47:36 +01:00
Paul Lecuq
7d36b8e6fe updated "last" endpoint
Some checks failed
continuous-integration/drone/push Build is failing
continuous-integration/drone Build is passing
continuous-integration/drone/tag Build is passing
2022-02-09 10:18:28 +01:00
Paul Lecuq
4f66f6fddd skipping duplicates whitelist inserts
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-08 15:32:10 +01:00
9dbe473cb7 updated README.md
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-07 23:09:22 +01:00
aa42f182b1 added endpoints for cfg update
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-07 23:08:39 +01:00
ee06baeb17 added endpoints for cfg update 2022-02-07 23:08:22 +01:00
cf2785121b updated README.md
All checks were successful
continuous-integration/drone/push Build is passing
2022-02-07 19:49:18 +01:00
657 changed files with 80509 additions and 22856 deletions

View File

@ -1,89 +1,52 @@
---
kind: pipeline
type: docker
name: cleanup-before
name: build-linux-amd64
steps:
- name: clean
image: alpine
commands:
- rm -rf /build/*
volumes:
- name: build
path: /build
when:
event: tag
volumes:
- name: build
host:
path: /tmp/ipbl/build
---
kind: pipeline
type: docker
name: default-linux-amd64
steps:
- name: build
image: golang
commands:
- ./ci-build.sh build
environment:
GOOS: linux
GOARCH: amd64
volumes:
- name: build
path: /build
GOOPTIONS: -mod=vendor
SRCFILES: cmd/ipbl/*.go
PROJECTNAME: ipbl
volumes:
steps:
- name: build
host:
path: /tmp/ipbl/build
depends_on:
- cleanup-before
image: golang
environment:
VERSION: dev
commands:
- go generate $SRCFILES
- go build -o $PROJECTNAME $GOOPTIONS $SRCFILES
when:
event:
exclude:
- tag
---
kind: pipeline
type: docker
name: default-linux-arm64
name: gitea-release-linux-amd64
environment:
GOOS: linux
GOARCH: amd64
GOOPTIONS: -mod=vendor
SRCFILES: cmd/ipbl/*.go
PROJECTNAME: ipbl
steps:
- name: build
image: golang
commands:
- ./ci-build.sh build
environment:
GOOS: linux
GOARCH: arm64
volumes:
- name: build
path: /build
volumes:
- name: build
host:
path: /tmp/ipbl/build
depends_on:
- cleanup-before
---
kind: pipeline
type: docker
name: gitea-release
steps:
- name: move
image: alpine
commands:
- mv build/* ./
volumes:
- name: build
path: /drone/src/build
- export VERSION=$DRONE_TAG
- go generate $SRCFILES
- go build -o $PROJECTNAME $GOOPTIONS $SRCFILES
- tar -czvf $PROJECTNAME-$DRONE_TAG-$GOOS-$GOARCH.tar.gz $PROJECTNAME
- echo $PROJECTNAME $DRONE_TAG > VERSION
when:
event: tag
event:
- tag
- name: release
image: plugins/gitea-release
settings:
@ -95,50 +58,70 @@ steps:
- sha256
- sha512
title: VERSION
volumes:
- name: build
path: /drone/src/build
when:
event: tag
- name: ls
image: alpine
commands:
- find .
volumes:
- name: build
path: /drone/src/build
when:
event: tag
volumes:
- name: build
host:
path: /tmp/ipbl/build
depends_on:
- default-linux-amd64
- default-linux-arm64
event:
- tag
---
kind: pipeline
type: docker
name: cleanup-after
name: build-linux-arm64
environment:
GOOS: linux
GOARCH: arm64
GOOPTIONS: -mod=vendor
SRCFILES: cmd/ipbl/*.go
PROJECTNAME: ipbl
steps:
- name: clean
image: alpine
- name: build
image: golang
environment:
VERSION: dev
commands:
- rm -rf /build/*
volumes:
- name: build
path: /build
- go generate $SRCFILES
- go build -o $PROJECTNAME $GOOPTIONS $SRCFILES
when:
event: tag
event:
exclude:
- tag
volumes:
---
kind: pipeline
type: docker
name: gitea-release-linux-arm64
environment:
GOOS: linux
GOARCH: arm64
GOOPTIONS: -mod=vendor
SRCFILES: cmd/ipbl/*.go
PROJECTNAME: ipbl
steps:
- name: build
host:
path: /tmp/ipbl/build
depends_on:
- gitea-release
image: golang
commands:
- export VERSION=$DRONE_TAG
- go generate $SRCFILES
- go build -o $PROJECTNAME $GOOPTIONS $SRCFILES
- tar -czvf $PROJECTNAME-$DRONE_TAG-$GOOS-$GOARCH.tar.gz $PROJECTNAME
- echo $PROJECTNAME $DRONE_TAG > VERSION
when:
event:
- tag
- name: release
image: plugins/gitea-release
settings:
base_url: https://git.paulbsd.com
api_key:
from_secret: gitea_token
files: "*.tar.gz"
checksum:
- sha256
- sha512
title: VERSION
when:
event:
- tag

1
.gitignore vendored
View File

@ -2,4 +2,5 @@
*.ini
*.swp
/test
/old
version.go

View File

@ -1,18 +0,0 @@
# ipbl Makefile
GOCMD=go
GOBUILDCMD=${GOCMD} build
GOOPTIONS=-mod=vendor -ldflags="-s -w"
RMCMD=rm
BINNAME=ipbl
SRCFILES=cmd/ipbl/*.go
all: build
build:
${GOBUILDCMD} ${GOOPTIONS} ${SRCFILES}
clean:
${RMCMD} -f ${BINNAME}

View File

@ -1,16 +1,18 @@
# ipbl
[![Build Status](https://drone.paulbsd.com/api/badges/paulbsd/ipbl/status.svg)](https://drone.paulbsd.com/paulbsd/ipbl)
## Summary
ipbl is ...
ipbl is a webservice collecting / storing / distributing abuse IP addresses
## Howto
### Build
```bash
make
go generate cmd/ipbl/*.go
go build cmd/ipbl/*.go
```
### Sample config in ipbl.ini
@ -21,19 +23,19 @@ db_hostname="hostname"
db_name="database"
db_username="username"
db_password="password"
db_table="ipbl_test"
db_table="ipbl"
```
### Run
```bash
./ipbl -configfile ipbl.ini -port 8080
./ipbl -configfile ipbl.ini -port 8099
```
## License
```text
Copyright (c) 2020, 2021 PaulBSD
Copyright (c) 2021, 2022, 2023 PaulBSD
All rights reserved.
Redistribution and use in source and binary forms, with or without

View File

@ -1,63 +0,0 @@
#!/bin/bash
set -e
PROJECTNAME=ipbl
RELEASENAME=${PROJECTNAME}
VERSION="0"
GOOPTIONS="-mod=vendor"
SRCFILES=cmd/${PROJECTNAME}/*.go
build() {
echo "Begin of build"
if [[ ! -z ${DRONE_TAG} ]]
then
echo "Drone tag set, let's do a release"
VERSION=${DRONE_TAG}
echo "${PROJECTNAME} ${VERSION}" > /build/VERSION
elif [[ ! -z ${DRONE_COMMIT} ]]
then
echo "Drone not set, let's only do a build"
VERSION=${DRONE_COMMIT}
fi
if [[ ! -z ${VERSION} && ! -z ${GOOS} && ! -z ${GOARCH} ]]
then
echo "Let's set a release name"
RELEASENAME=${PROJECTNAME}-${VERSION}-${GOOS}-${GOARCH}
fi
echo "Building project"
go generate ${SRCFILES}
go build -o ${PROJECTNAME} ${GOOPTIONS} ${SRCFILES}
if [[ ! -z ${DRONE_TAG} ]]
then
echo "Let's make archives"
mkdir -p /build
tar -czvf /build/${RELEASENAME}.tar.gz ${PROJECTNAME}
fi
echo "Removing binary file"
rm ${PROJECTNAME}
echo "End of build"
}
clean() {
rm -rf ${RELEASEDIR}
}
case $1 in
"build")
build
;;
"clean")
clean
;;
*)
echo "No options choosen"
exit 1
;;
esac

View File

@ -2,12 +2,19 @@ package main
import (
"context"
"fmt"
"log"
"os"
"net/http"
_ "net/http/pprof"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"git.paulbsd.com/paulbsd/ipbl/src/database"
"git.paulbsd.com/paulbsd/ipbl/src/models"
"git.paulbsd.com/paulbsd/ipbl/src/routers"
"git.paulbsd.com/paulbsd/ipbl/src/ws"
"git.paulbsd.com/paulbsd/ipbl/utils"
_ "github.com/lib/pq"
)
@ -18,27 +25,31 @@ func main() {
var cfg config.Config
cfg.GetConfig()
cfg.Options.Version = version
if cfg.Switchs.Version {
fmt.Printf("ipbl version %s\n", cfg.Options.Version)
os.Exit(0)
}
// Initialize database app context
err := database.Initialize(&ctx, &cfg)
if err != nil {
log.Fatalln(err)
}
defer cfg.Db.Close()
// Handles IP with no reverse DNS
if cfg.Options.ScanIP {
go models.ScanIP(&cfg)
}
// Add cron task to handle them
//cr := cron.New()
//cr.AddFunc("0 * * * * *", func() {
// models.ScanIP(&cfg)
//})
//cr.Start()
if cfg.Options.Debug {
go func() {
log.Println(http.ListenAndServe("localhost:6060", nil))
}()
}
// Run the ipbl web service
err = routers.RunServer(&ctx, &cfg)
ws.Init(&cfg)
go func() { err = routers.RunServer(&ctx, &cfg) }()
if err != nil {
log.Fatalln(err)
}
utils.Server()
}

36
go.mod
View File

@ -1,32 +1,32 @@
module git.paulbsd.com/paulbsd/ipbl
go 1.17
go 1.23
require (
github.com/labstack/echo/v4 v4.6.1
github.com/lib/pq v1.10.4
github.com/robfig/cron v1.2.0
gopkg.in/ini.v1 v1.66.2
xorm.io/xorm v1.2.5
github.com/labstack/echo/v4 v4.12.0
github.com/lib/pq v1.10.9
golang.org/x/net v0.28.0
gopkg.in/ini.v1 v1.67.0
xorm.io/xorm v1.3.9
)
require (
github.com/goccy/go-json v0.8.1 // indirect
github.com/goccy/go-json v0.10.3 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/go-cmp v0.5.9 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/labstack/gommon v0.3.1 // indirect
github.com/mattn/go-colorable v0.1.12 // indirect
github.com/mattn/go-isatty v0.0.14 // indirect
github.com/labstack/gommon v0.4.2 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/onsi/ginkgo v1.16.4 // indirect
github.com/onsi/gomega v1.14.0 // indirect
github.com/onsi/ginkgo v1.16.5 // indirect
github.com/onsi/gomega v1.22.1 // indirect
github.com/syndtr/goleveldb v1.0.0 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasttemplate v1.2.1 // indirect
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b // indirect
golang.org/x/net v0.0.0-20211209124913-491a49abca63 // indirect
golang.org/x/sys v0.0.0-20211210111614-af8b64212486 // indirect
golang.org/x/text v0.3.7 // indirect
xorm.io/builder v0.3.9 // indirect
github.com/valyala/fasttemplate v1.2.2 // indirect
golang.org/x/crypto v0.26.0 // indirect
golang.org/x/sys v0.24.0 // indirect
golang.org/x/text v0.17.0 // indirect
xorm.io/builder v0.3.13 // indirect
)

605
go.sum
View File

@ -1,623 +1,180 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
gitea.com/xorm/sqlfiddle v0.0.0-20180821085327-62ce714f951a h1:lSA0F4e9A2NcQSqGqTOXqu2aRi/XEQxDCBwM8yJtE6s=
gitea.com/xorm/sqlfiddle v0.0.0-20180821085327-62ce714f951a/go.mod h1:EXuID2Zs0pAQhH8yz+DNjUbjppKQzKFAn28TMYPB6IU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible/go.mod h1:r7JcOSlj0wfOMncg0iLm8Leh48TZaKVeNIfJntJ2wa0=
github.com/Masterminds/semver/v3 v3.1.1/go.mod h1:VPu/7SZ7ePZ3QOrcuXROw5FAcLl4a0cBrbBpGY/8hQs=
github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWXgklEdEo=
github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI=
github.com/VividCortex/gohistogram v1.0.0/go.mod h1:Pf5mBqqDxYaXu3hDrrU+w6nw50o/4+TcAqDqk/vUH7g=
github.com/afex/hystrix-go v0.0.0-20180502004556-fa1af6a1f4f5/go.mod h1:SkGFH1ia65gfNATL8TAiHDNxPzPdmEL5uirI2Uyuz6c=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/apache/thrift v0.13.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=
github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=
github.com/aryann/difflib v0.0.0-20170710044230-e206f873d14a/go.mod h1:DAHtR1m6lCRdSC2Tm3DSWRPvIPr6xNKyeHdqDQSQT+A=
github.com/aws/aws-lambda-go v1.13.3/go.mod h1:4UKl9IzQMoD+QF79YdCuzCwp8VbmG4VAQwij/eHl5CU=
github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-sdk-go-v2 v0.18.0/go.mod h1:JWVYvqSMppoMJC0x5wdwiImzgXTI9FuZwxzkQq9wy+g=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=
github.com/casbin/casbin/v2 v2.1.2/go.mod h1:YcPU1XXisHhLzuxH9coDNf2FbKpjGlbCg3n9yuLkIJQ=
github.com/cenkalti/backoff v2.2.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/clbanning/x2j v0.0.0-20191024224557-825249438eec/go.mod h1:jMjuTZXRI4dUb/I5gc9Hdhagfvm9+RyrPryS/auMzxE=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cockroachdb/apd v1.1.0/go.mod h1:8Sl8LxpKi29FqWXR16WEFZRNSz3SoPzUzeMeY4+DwBQ=
github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8=
github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd/go.mod h1:sE/e/2PUdi/liOCUjSTXgM1o87ZssimdTWN964YiIeI=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/go-systemd v0.0.0-20190719114852-fd7a80b32e1f/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/denisenkom/go-mssqldb v0.10.0/go.mod h1:xbL0rPBG9cCiLr28tMa8zpbdarY27NDyej4t/EjAShU=
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo=
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/eapache/go-resiliency v1.1.0/go.mod h1:kFI+JgMyC7bLPUVY133qvEBtVayf5mFgVsvEsIPBvNs=
github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21/go.mod h1:+020luEh2TKB4/GOp8oxxtq0Daoen/Cii55CzbTV6DU=
github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=
github.com/edsrzf/mmap-go v1.0.0/go.mod h1:YO35OhQPt3KJa3ryjFM5Bs14WD66h8eGKpfaBNrHW5M=
github.com/envoyproxy/go-control-plane v0.6.9/go.mod h1:SBwIajubJHhxtWwsL9s8ss4safvEdbitLhGGK48rN6g=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/franela/goblin v0.0.0-20200105215937-c9ffbefa60db/go.mod h1:7dvUGVsVBjqR7JHJk0brhHOZYGmfBYOrK0ZhYMEtBr4=
github.com/franela/goreq v0.0.0-20171204163338-bcd34c9993f8/go.mod h1:ZhphrRTfi2rbfLwlschooIH4+wKKDR4Pdxhh+TRoA20=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.10.0/go.mod h1:xUsJbQ/Fp4kEt7AFgCuvyX4a71u8h9jB8tj/ORgOZ7o=
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-sql-driver/mysql v1.6.0 h1:BCTh4TKNUYmOmMUcQ3IipzF5prigylS7XXjEkfCHuOE=
github.com/go-sql-driver/mysql v1.6.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/go-sql-driver/mysql v1.7.0 h1:ueSltNNllEqE3qcWBTD0iQd3IpL/6U+mJxLkazJ7YPc=
github.com/go-sql-driver/mysql v1.7.0/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI=
github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0/go.mod h1:fyg7847qk6SyHyPtNmDHnmrv/HOrqktSC+C9fM+CJOE=
github.com/goccy/go-json v0.7.4/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/goccy/go-json v0.8.1 h1:4/Wjm0JIJaTDm8K1KcGrLHJoa8EsJ13YWeX+6Kfq6uI=
github.com/goccy/go-json v0.8.1/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/gofrs/uuid v3.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/gofrs/uuid v4.0.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/gogo/googleapis v1.1.0/go.mod h1:gf4bu3Q80BeJ6H1S1vYPm8/ELATdvryBaNFGgqEef3s=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/golang-jwt/jwt v3.2.2+incompatible/go.mod h1:8pz2t5EyA70fFQQSrl6XZXzqecmYZeUEB8OUGHkxJ+I=
github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe/go.mod h1:8vg3r2VgvsThLBIFL93Qb5yWzgyZWhEmBwUJWevAkK0=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/goccy/go-json v0.10.3 h1:KZ5WoDbxAIgm2HNbYckL0se1fHD6rz5j4ywS6ebzDqA=
github.com/goccy/go-json v0.10.3/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.4 h1:yAGX7huGHXlcLOEtBnF4w7FQwA26wojNCwOYAEhLjQM=
github.com/golang/snappy v0.0.4/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/hashicorp/consul/api v1.3.0/go.mod h1:MmDNSzIMUjNpY/mQ398R4bk2FnqQLoPndWW5VkKPlCE=
github.com/hashicorp/consul/sdk v0.3.0/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=
github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=
github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=
github.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU=
github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU=
github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4=
github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-version v1.2.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=
github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=
github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/hudl/fargo v1.3.0/go.mod h1:y3CKSmjA+wD2gak7sUSXTAoopbhU08POFhmITJgmKTg=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/influxdata/influxdb1-client v0.0.0-20191209144304-8bf82d3c094d/go.mod h1:qj24IKcXYK6Iy9ceXlo3Tc+vtHo9lIhSX5JddghvEPo=
github.com/jackc/chunkreader v1.0.0/go.mod h1:RT6O25fNZIuasFJRyZ4R/Y2BbhasbmZXF9QQ7T3kePo=
github.com/jackc/chunkreader/v2 v2.0.0/go.mod h1:odVSm741yZoC3dpHEUXIqA9tQRhFrgOHwnPIn9lDKlk=
github.com/jackc/chunkreader/v2 v2.0.1/go.mod h1:odVSm741yZoC3dpHEUXIqA9tQRhFrgOHwnPIn9lDKlk=
github.com/jackc/pgconn v0.0.0-20190420214824-7e0022ef6ba3/go.mod h1:jkELnwuX+w9qN5YIfX0fl88Ehu4XC3keFuOJJk9pcnA=
github.com/jackc/pgconn v0.0.0-20190824142844-760dd75542eb/go.mod h1:lLjNuW/+OfW9/pnVKPazfWOgNfH2aPem8YQ7ilXGvJE=
github.com/jackc/pgconn v0.0.0-20190831204454-2fabfa3c18b7/go.mod h1:ZJKsE/KZfsUgOEh9hBm+xYTstcNHg7UPMVJqRfQxq4s=
github.com/jackc/pgconn v1.4.0/go.mod h1:Y2O3ZDF0q4mMacyWV3AstPJpeHXWGEetiFttmq5lahk=
github.com/jackc/pgconn v1.5.0/go.mod h1:QeD3lBfpTFe8WUnPZWN5KY/mB8FGMIYRdd8P8Jr0fAI=
github.com/jackc/pgconn v1.5.1-0.20200601181101-fa742c524853/go.mod h1:QeD3lBfpTFe8WUnPZWN5KY/mB8FGMIYRdd8P8Jr0fAI=
github.com/jackc/pgconn v1.8.0/go.mod h1:1C2Pb36bGIP9QHGBYCjnyhqu7Rv3sGshaQUvmfGIB/o=
github.com/jackc/pgconn v1.8.1/go.mod h1:JV6m6b6jhjdmzchES0drzCcYcAHS1OPD5xu3OZ/lE2g=
github.com/jackc/pgconn v1.9.0/go.mod h1:YctiPyvzfU11JFxoXokUOOKQXQmDMoJL9vJzHH8/2JY=
github.com/jackc/pgio v1.0.0/go.mod h1:oP+2QK2wFfUWgr+gxjoBH9KGBb31Eio69xUb0w5bYf8=
github.com/jackc/pgmock v0.0.0-20190831213851-13a1b77aafa2/go.mod h1:fGZlG77KXmcq05nJLRkk0+p82V8B8Dw8KN2/V9c/OAE=
github.com/jackc/pgmock v0.0.0-20201204152224-4fe30f7445fd/go.mod h1:hrBW0Enj2AZTNpt/7Y5rr2xe/9Mn757Wtb2xeBzPv2c=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgproto3 v1.1.0/go.mod h1:eR5FA3leWg7p9aeAqi37XOTgTIbkABlvcPB3E5rlc78=
github.com/jackc/pgproto3/v2 v2.0.0-alpha1.0.20190420180111-c116219b62db/go.mod h1:bhq50y+xrl9n5mRYyCBFKkpRVTLYJVWeCc+mEAI3yXA=
github.com/jackc/pgproto3/v2 v2.0.0-alpha1.0.20190609003834-432c2951c711/go.mod h1:uH0AWtUmuShn0bcesswc4aBTWGvw0cAxIJp+6OB//Wg=
github.com/jackc/pgproto3/v2 v2.0.0-rc3/go.mod h1:ryONWYqW6dqSg1Lw6vXNMXoBJhpzvWKnT95C46ckYeM=
github.com/jackc/pgproto3/v2 v2.0.0-rc3.0.20190831210041-4c03ce451f29/go.mod h1:ryONWYqW6dqSg1Lw6vXNMXoBJhpzvWKnT95C46ckYeM=
github.com/jackc/pgproto3/v2 v2.0.1/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA=
github.com/jackc/pgproto3/v2 v2.0.6/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA=
github.com/jackc/pgproto3/v2 v2.1.1/go.mod h1:WfJCnwN3HIg9Ish/j3sgWXnAfK8A9Y0bwXYU5xKaEdA=
github.com/jackc/pgservicefile v0.0.0-20200307190119-3430c5407db8/go.mod h1:vsD4gTJCa9TptPL8sPkXrLZ+hDuNrZCnj29CQpr4X1E=
github.com/jackc/pgservicefile v0.0.0-20200714003250-2b9c44734f2b/go.mod h1:vsD4gTJCa9TptPL8sPkXrLZ+hDuNrZCnj29CQpr4X1E=
github.com/jackc/pgtype v0.0.0-20190421001408-4ed0de4755e0/go.mod h1:hdSHsc1V01CGwFsrv11mJRHWJ6aifDLfdV3aVjFF0zg=
github.com/jackc/pgtype v0.0.0-20190824184912-ab885b375b90/go.mod h1:KcahbBH1nCMSo2DXpzsoWOAfFkdEtEJpPbVLq8eE+mc=
github.com/jackc/pgtype v0.0.0-20190828014616-a8802b16cc59/go.mod h1:MWlu30kVJrUS8lot6TQqcg7mtthZ9T0EoIBFiJcmcyw=
github.com/jackc/pgtype v1.2.0/go.mod h1:5m2OfMh1wTK7x+Fk952IDmI4nw3nPrvtQdM0ZT4WpC0=
github.com/jackc/pgtype v1.3.1-0.20200510190516-8cd94a14c75a/go.mod h1:vaogEUkALtxZMCH411K+tKzNpwzCKU+AnPzBKZ+I+Po=
github.com/jackc/pgtype v1.3.1-0.20200606141011-f6355165a91c/go.mod h1:cvk9Bgu/VzJ9/lxTO5R5sf80p0DiucVtN7ZxvaC4GmQ=
github.com/jackc/pgtype v1.7.0/go.mod h1:ZnHF+rMePVqDKaOfJVI4Q8IVvAQMryDlDkZnKOI75BE=
github.com/jackc/pgtype v1.8.0/go.mod h1:PqDKcEBtllAtk/2p6z6SHdXW5UB+MhE75tUol2OKexE=
github.com/jackc/pgx/v4 v4.0.0-20190420224344-cc3461e65d96/go.mod h1:mdxmSJJuR08CZQyj1PVQBHy9XOp5p8/SHH6a0psbY9Y=
github.com/jackc/pgx/v4 v4.0.0-20190421002000-1b8f0016e912/go.mod h1:no/Y67Jkk/9WuGR0JG/JseM9irFbnEPbuWV2EELPNuM=
github.com/jackc/pgx/v4 v4.0.0-pre1.0.20190824185557-6972a5742186/go.mod h1:X+GQnOEnf1dqHGpw7JmHqHc1NxDoalibchSk9/RWuDc=
github.com/jackc/pgx/v4 v4.5.0/go.mod h1:EpAKPLdnTorwmPUUsqrPxy5fphV18j9q3wrfRXgo+kA=
github.com/jackc/pgx/v4 v4.6.1-0.20200510190926-94ba730bb1e9/go.mod h1:t3/cdRQl6fOLDxqtlyhe9UWgfIi9R8+8v8GKV5TRA/o=
github.com/jackc/pgx/v4 v4.6.1-0.20200606145419-4e5062306904/go.mod h1:ZDaNWkt9sW1JMiNn0kdYBaLelIhw7Pg4qd+Vk6tw7Hg=
github.com/jackc/pgx/v4 v4.11.0/go.mod h1:i62xJgdrtVDsnL3U8ekyrQXEwGNTRoG7/8r+CIdYfcc=
github.com/jackc/pgx/v4 v4.12.0/go.mod h1:fE547h6VulLPA3kySjfnSG/e2D861g/50JlVUa/ub60=
github.com/jackc/puddle v0.0.0-20190413234325-e4ced69a3a2b/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk=
github.com/jackc/puddle v0.0.0-20190608224051-11cab39313c9/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk=
github.com/jackc/puddle v1.1.0/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk=
github.com/jackc/puddle v1.1.1/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk=
github.com/jackc/puddle v1.1.3/go.mod h1:m4B5Dj62Y0fbyuIc15OsIqK0+JU8nkqQjsgx7dvjSWk=
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.8/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.8/go.mod h1:O1sed60cT9XZ5uDucP5qwvh+TE3NnUj51EiZO/lmSfw=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/labstack/echo/v4 v4.6.1 h1:OMVsrnNFzYlGSdaiYGHbgWQnr+JM7NG+B9suCPie14M=
github.com/labstack/echo/v4 v4.6.1/go.mod h1:RnjgMWNDB9g/HucVWhQYNQP9PvbYf6adqftqryo7s9k=
github.com/labstack/gommon v0.3.0/go.mod h1:MULnywXg0yavhxWKc+lOruYdAhDwPK9wf0OL7NoOu+k=
github.com/labstack/gommon v0.3.1 h1:OomWaJXm7xR6L1HmEtGyQf26TEn7V6X88mktX9kee9o=
github.com/labstack/gommon v0.3.1/go.mod h1:uW6kP17uPlLJsD3ijUYn3/M5bAxtlZhMI6m3MFxTMTM=
github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/lib/pq v1.1.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/lib/pq v1.2.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/lib/pq v1.3.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/lib/pq v1.10.2/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/lib/pq v1.10.4 h1:SO9z7FRPzA03QhHKJrH5BXA6HU1rS4V2nIVrrNC1iYk=
github.com/lib/pq v1.10.4/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-bc2310a04743/go.mod h1:qklhhLq1aX+mtWk9cPHPzaBjWImj5ULL6C7HFJtXQMM=
github.com/lightstep/lightstep-tracer-go v0.18.1/go.mod h1:jlF1pusYV4pidLvZ+XD0UBX0ZE6WURAspgAczcDHrL4=
github.com/lyft/protoc-gen-validate v0.0.13/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0QoUACkjt2znoq26NVQ=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-colorable v0.1.1/go.mod h1:FuOcm+DKB9mbwrcAfNl7/TZVBZ6rcnceauSikq3lYCQ=
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.11/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
github.com/mattn/go-colorable v0.1.12 h1:jF+Du6AlPIjs2BiUiQlKOX0rt3SujHxPnksPKZbaA40=
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.5/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.7/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.9/go.mod h1:YNRxwqDuOph6SZLI9vUUz6OYw3QyUt7WiY2yME+cCiQ=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-isatty v0.0.14 h1:yVuAays6BHfxijgZPzw+3Zlu5yQgKGP2/hcQbHb7S9Y=
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-sqlite3 v1.14.6/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
github.com/mattn/go-sqlite3 v1.14.8 h1:gDp86IdQsN/xWjIEmr9MF6o9mpksUgh0fu+9ByFxzIU=
github.com/mattn/go-sqlite3 v1.14.8/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=
github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=
github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=
github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=
github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/labstack/echo/v4 v4.12.0 h1:IKpw49IMryVB2p1a4dzwlhP1O2Tf2E0Ir/450lH+kI0=
github.com/labstack/echo/v4 v4.12.0/go.mod h1:UP9Cr2DJXbOK3Kr9ONYzNowSh7HP0aG0ShAyycHSJvM=
github.com/labstack/gommon v0.4.2 h1:F8qTUNXgG1+6WQmqoUWnz8WiEU60mXVVw0P4ht1WRA0=
github.com/labstack/gommon v0.4.2/go.mod h1:QlUFxVM+SNXhDL/Z7YhocGIBYOiwB0mXm1+1bAPHPyU=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-sqlite3 v1.14.16 h1:yOQRA0RpS5PFz/oikGwBEqvAWhWg5ufRz4ETLjwpU1Y=
github.com/mattn/go-sqlite3 v1.14.16/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/nats-io/jwt v0.3.0/go.mod h1:fRYCDE99xlTsqUzISS1Bi75UBJ6ljOJQOAAu5VglpSg=
github.com/nats-io/jwt v0.3.2/go.mod h1:/euKqTS1ZD+zzjYrY7pseZrTtWQSjujC7xjPc8wL6eU=
github.com/nats-io/nats-server/v2 v2.1.2/go.mod h1:Afk+wRZqkMQs/p45uXdrVLuab3gwv3Z8C4HTBu8GD/k=
github.com/nats-io/nats.go v1.9.1/go.mod h1:ZjDU1L/7fJ09jvUSRVBR2e7+RnLiiIQyqyzEE/Zbp4w=
github.com/nats-io/nkeys v0.1.0/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w=
github.com/nats-io/nkeys v0.1.3/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE=
github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=
github.com/oklog/oklog v0.3.2/go.mod h1:FCV+B7mhrz4o+ueLpx+KqkyXRGMWOYEvfiXtdGtbWGs=
github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA=
github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
github.com/onsi/ginkgo v1.16.4 h1:29JGrr5oVBm5ulCWet69zQkzWipVXIol6ygQUe/EzNc=
github.com/onsi/ginkgo v1.16.4/go.mod h1:dX+/inL/fNMqNlz0e9LfyB9TswhZpCVdJM/Z6Vvnwo0=
github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=
github.com/onsi/ginkgo v1.16.5/go.mod h1:+E8gABHa3K6zRBolWtd+ROzc/U5bkGt0FwiG042wbpU=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
github.com/onsi/gomega v1.14.0 h1:ep6kpPVwmr/nTbklSx2nrLNSIO62DoYAhnPNIMhK8gI=
github.com/onsi/gomega v1.14.0/go.mod h1:cIuvLEne0aoVhAgh/O6ac0Op8WWw9H6eYCriF+tEHG0=
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/opentracing-contrib/go-observer v0.0.0-20170622124052-a52f23424492/go.mod h1:Ngi6UdF0k5OKD5t5wlmGhe/EDKPoUM3BXZSSfIuJbis=
github.com/opentracing/basictracer-go v1.0.0/go.mod h1:QfBfYuafItcjQuMwinw9GhYKwFXS9KnPs5lxoYwgW74=
github.com/opentracing/opentracing-go v1.0.2/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
github.com/openzipkin-contrib/zipkin-go-opentracing v0.4.5/go.mod h1:/wsWhb9smxSfWAKL3wpBW7V8scJMt8N8gnaMCS9E/cA=
github.com/openzipkin/zipkin-go v0.1.6/go.mod h1:QgAqvLzwWbR/WpD4A3cGpPtJrZXNIiJc5AZX7/PBEpw=
github.com/openzipkin/zipkin-go v0.2.1/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnhQw8ySjnjRyN4=
github.com/openzipkin/zipkin-go v0.2.2/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnhQw8ySjnjRyN4=
github.com/pact-foundation/pact-go v1.0.4/go.mod h1:uExwJY4kCzNPcHRj+hCR/HBbOOIwwtUjcrb0b5/5kLM=
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k=
github.com/performancecopilot/speed v3.0.0+incompatible/go.mod h1:/CLtqpZ5gBg1M9iaPbIdPPGyKcA8hKdoy6hAWba7Yac=
github.com/pierrec/lz4 v1.0.2-0.20190131084431-473cd7ce01a1/go.mod h1:3/3N9NVKO0jef7pBehbT1qWhCMrIgbYNnFAZCqQ5LRc=
github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/profile v1.2.1/go.mod h1:hJw3o1OdXxsrSjjVksARp5W95eeEaEfptyVZyv6JUPA=
github.com/onsi/gomega v1.22.1 h1:pY8O4lBfsHKZHM/6nrxkhVPUznOlIu3quZcKP/M20KI=
github.com/onsi/gomega v1.22.1/go.mod h1:x6n7VNe4hw0vkyYUM4mjIXx3JbLiPaBPNgB7PRQ1tuM=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs=
github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
github.com/prometheus/client_golang v1.3.0/go.mod h1:hJaj2vgQTGQmVCsAACORcieXFeDPbaTKGT+JTgUa3og=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.1.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/common v0.2.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.7.0/go.mod h1:DjGbpBbp5NYNiECxcL/VnbXCCaQpKd3tt26CguLLsqA=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0 h1:OdAsTTz6OkFY5QxjkYwrChwuRruF69c169dPK26NUlk=
github.com/remyoudompheng/bigfft v0.0.0-20200410134404-eec4a21b6bb0/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/robfig/cron v1.2.0 h1:ZjScXvvxeQ63Dbyxy76Fj3AT3Ut0aKsyd2/tl3DTMuQ=
github.com/robfig/cron v1.2.0/go.mod h1:JGuDeoQd7Z6yL4zQhZ3OPEVHB7fL6Ka6skscFHfmt2k=
github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rs/xid v1.2.1/go.mod h1:+uKXf+4Djp6Md1KODXJxgGQPKngRmWyn10oCKFzNHOQ=
github.com/rs/zerolog v1.13.0/go.mod h1:YbFCdg8HfsridGWAh22vktObvhZbQsZXe4/zB0OKkWU=
github.com/rs/zerolog v1.15.0/go.mod h1:xYTKnLHcpfU2225ny5qZjxnj9NvkumZYjJHlAThCjNc=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da/go.mod h1:gi+0XIa01GRL2eRQVjQkKGqKF3SF9vZR/HnPullcV2E=
github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24/go.mod h1:M+9NzErvs504Cn4c5DxATwIqPbtswREoFCre64PpcG4=
github.com/shopspring/decimal v0.0.0-20200227202807-02e2044944cc/go.mod h1:DKyhrW/HYNuLGql+MJL6WCR6knT2jwCFRcu2hWCYk4o=
github.com/shopspring/decimal v1.2.0/go.mod h1:DKyhrW/HYNuLGql+MJL6WCR6knT2jwCFRcu2hWCYk4o=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
github.com/sony/gobreaker v0.4.1/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY=
github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/streadway/amqp v0.0.0-20190404075320-75d898a42a94/go.mod h1:AZpEONHx3DKn8O/DFsRAY58/XVQiIPMTMB1SddzLXVw=
github.com/streadway/amqp v0.0.0-20190827072141-edfb9018d271/go.mod h1:AZpEONHx3DKn8O/DFsRAY58/XVQiIPMTMB1SddzLXVw=
github.com/streadway/handy v0.0.0-20190108123426-d5acb3125c2a/go.mod h1:qNTQ5P5JnDBl6z3cMAg/SywNDC5ABu5ApDIw6lUbRmI=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/syndtr/goleveldb v1.0.0 h1:fBdIW9lB4Iz0n9khmH8w27SJ3QEJ7+IgjPEwGSZiFdE=
github.com/syndtr/goleveldb v1.0.0/go.mod h1:ZVVdQEZoIme9iO1Ch2Jdy24qqXrMMOU6lpPAyBWyWuQ=
github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasttemplate v1.0.1/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPUpymEIMZ47gx8=
github.com/valyala/fasttemplate v1.2.1 h1:TVEnxayobAdVkhQfrfes2IzOB6o+z4roRkPF52WA1u4=
github.com/valyala/fasttemplate v1.2.1/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ=
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/valyala/fasttemplate v1.2.2 h1:lxLXG0uE3Qnshl9QyaK6XJxMXlQZELvChBOCmQD0Loo=
github.com/valyala/fasttemplate v1.2.2/go.mod h1:KHLXt3tVN2HBp8eijSv/kGJopbvo7S+qRAEEKiv+SiQ=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/zenazn/goji v0.9.0/go.mod h1:7S9M489iMyHBNxwZnk9/EHS098H4/F6TATF2mIxtB1Q=
github.com/ziutek/mymysql v1.5.4/go.mod h1:LMSpPZ6DbqWFxNCHW77HeMg9I646SAhApZ/wKdgO/C0=
go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738/go.mod h1:dnLIgRNXwCJa5e+c6mIZCrds/GIG4ncV9HhK5PX7jPg=
go.opencensus.io v0.20.1/go.mod h1:6WKK9ahsWS3RSO+PY9ZHZUfv2irvY6gN279GOPZjmmk=
go.opencensus.io v0.20.2/go.mod h1:6WKK9ahsWS3RSO+PY9ZHZUfv2irvY6gN279GOPZjmmk=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/atomic v1.5.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ=
go.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ=
go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
go.uber.org/multierr v1.3.0/go.mod h1:VgVr7evmIr6uPjLBxg28wmKNXyqE9akIJ5XnfpiKl+4=
go.uber.org/multierr v1.5.0/go.mod h1:FeouvMocqHpRaaGuG9EjoKcStLC43Zu/fmqdUMPcKYU=
go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA=
go.uber.org/zap v1.9.1/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
go.uber.org/zap v1.13.0/go.mod h1:zwrFLgMcdUuIBviXEYEH1YKNaOBnKXsx2IPda5bBwHM=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190325154230-a5d413f7728c/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190411191339-88737f569e3a/go.mod h1:WFFai1msRO1wXaEeE5yQxYXgSfI8pQAWXbQop6sCtWE=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190911031432-227b76d455e7/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200323165209-0ec3e9974c59/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201203163018-be400aefbc4c/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210616213533-5ff15b29337e/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b h1:QAqMVf3pSa6eeTsuklijukjXBlj7Es2QQplab+/RbQ4=
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.3.0 h1:RM4zey1++hCTbCVQfnWeKs9/IEsaBLA8vTkd0WVtmH4=
golang.org/x/crypto v0.23.0 h1:dIJU/v2J8Mdglj/8rJ6UUOM3Zc9zLZxVZwwxMooUSAI=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.26.0 h1:RrRspgV4mU+YwB4FYnuBoKsUapNIL5cohGAmSH3azsw=
golang.org/x/crypto v0.26.0/go.mod h1:GY7jblb9wI+FOo5y8/S2oY4zWP07AkOJ4+jxCqdqn54=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/mod v0.8.0 h1:LUYupSeNrTNCGzR/hVBk2NHZO4hXcVaW1k4Qx7rjPx8=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190125091013-d26f9f9a57f3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
golang.org/x/net v0.0.0-20210913180222-943fd674d43e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20211209124913-491a49abca63 h1:iocB37TsdFuN6IBRZ+ry36wrkoV51/tl5vOWqkcPGvY=
golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.28.0 h1:a9JDOJc5GMUJ0+UDqmLT86WiEy7iWyIhz8gz8E4e5hE=
golang.org/x/net v0.28.0/go.mod h1:yqtgsTWOOnlGLG9GFRrK3++bGOUEkNBoHZc8MEDWPNg=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181122145206-62eef0e2fa9b/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190403152447-81d4e9dc473e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190813064441-fde4db37ae7a/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190904154756-749cb33beabd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191120155948-bd437916bb0e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191220142924-d4481acd189f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201126233918-771906719818/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210112080510-489259a85091/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210910150752-751e447fb3d0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211103235746-7861aae1554b/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211210111614-af8b64212486 h1:5hpz5aRr+W1erYCL5JRhSUBJRph7l9XkNveoExlrKYk=
golang.org/x/sys v0.0.0-20211210111614-af8b64212486/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.24.0 h1:Twjiwq9dn6R1fQcyiK+wQyHWfaz/BJB+YIpzU/Cv3Xg=
golang.org/x/sys v0.24.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20201208040808-7e3f01d25324/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/text v0.15.0 h1:h1V/4gjBv8v9cjcR6+AR5+/cIYK5N/WAgiv4xlsEtAk=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.17.0 h1:XtiM5bkSOt+ewxlOE/aE/AKEHibwj/6gvWMl9Rsh0Qc=
golang.org/x/text v0.17.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190425163242-31fd60d6bfdc/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20190823170909-c4a336ef6a2f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200103221440-774c71fcf114/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20201124115921-2c860bdd6e78/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e h1:4nW4NLDYnU28ojHaHO8OVxFHk/aQ33U01a9cjED+pzE=
golang.org/x/tools v0.0.0-20201224043029-2b0845dc783e/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/xerrors v0.0.0-20190410155217-1f06c39b4373/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20190513163551-3ee3066db522/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/tools v0.6.0 h1:BOw41kyTf3PuCW1pVQf8+Cyg8pMlkYB1oo9iJ6D/lKM=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.3.1/go.mod h1:6wY9I6uQWHQ8EM57III9mq/AjF+i8G65rmVagqKMtkk=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190530194941-fb225487d101/go.mod h1:z3L6/3dTEVtUr6QSP8miRzeRqwQOioJ9I66odjN4I7s=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.0/go.mod h1:chYK+tFQF0nDUGJgXMSgLCQk3phJEuONr2DCgLDdAQM=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.22.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.23.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/gcfg.v1 v1.2.3/go.mod h1:yesOnuUOFQAhST5vPY4nbZsb/huCgGGXlipJsBn0b3o=
gopkg.in/inconshreveable/log15.v2 v2.0.0-20180818164646-67afb5ed74ec/go.mod h1:aPpfJ7XW+gOuirDoZ8gHhLh3kZ1B08FtV2bbmy7Jv3s=
gopkg.in/ini.v1 v1.66.2 h1:XfR1dOYubytKy4Shzc2LHrrGhU0lDCfDGG1yLPmpgsI=
gopkg.in/ini.v1 v1.66.2/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo=
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.0.0-20180728063816-88497007e858/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
lukechampine.com/uint128 v1.1.1 h1:pnxCASz787iMf+02ssImqk6OLt+Z5QHMoZyUXR4z6JU=
lukechampine.com/uint128 v1.1.1/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk=
modernc.org/cc/v3 v3.33.6 h1:r63dgSzVzRxUpAJFPQWHy1QeZeY1ydNENUDaBx1GqYc=
modernc.org/cc/v3 v3.33.6/go.mod h1:iPJg1pkwXqAV16SNgFBVYmggfMg6xhs+2oiO0vclK3g=
modernc.org/ccgo/v3 v3.9.5 h1:dEuUSf8WN51rDkprFuAqjfchKEzN0WttP/Py3enBwjk=
modernc.org/ccgo/v3 v3.9.5/go.mod h1:umuo2EP2oDSBnD3ckjaVUXMrmeAw8C8OSICVa0iFf60=
modernc.org/httpfs v1.0.6/go.mod h1:7dosgurJGp0sPaRanU53W4xZYKh14wfzX420oZADeHM=
modernc.org/libc v1.7.13-0.20210308123627-12f642a52bb8/go.mod h1:U1eq8YWr/Kc1RWCMFUWEdkTg8OTcfLw2kY8EDwl039w=
modernc.org/libc v1.9.8/go.mod h1:U1eq8YWr/Kc1RWCMFUWEdkTg8OTcfLw2kY8EDwl039w=
modernc.org/libc v1.9.11 h1:QUxZMs48Ahg2F7SN41aERvMfGLY2HU/ADnB9DC4Yts8=
modernc.org/libc v1.9.11/go.mod h1:NyF3tsA5ArIjJ83XB0JlqhjTabTCHm9aX4XMPHyQn0Q=
modernc.org/mathutil v1.1.1/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
modernc.org/mathutil v1.2.2/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
modernc.org/mathutil v1.4.0 h1:GCjoRaBew8ECCKINQA2nYjzvufFW9YiEuuB+rQ9bn2E=
modernc.org/mathutil v1.4.0/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
modernc.org/memory v1.0.4 h1:utMBrFcpnQDdNsmM6asmyH/FM9TqLPS7XF7otpJmrwM=
modernc.org/memory v1.0.4/go.mod h1:nV2OApxradM3/OVbs2/0OsP6nPfakXpi50C7dcoHXlc=
modernc.org/opt v0.1.1 h1:/0RX92k9vwVeDXj+Xn23DKp2VJubL7k8qNffND6qn3A=
modernc.org/opt v0.1.1/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sqlite v1.11.2 h1:ShWQpeD3ag/bmx6TqidBlIWonWmQaSQKls3aenCbt+w=
modernc.org/sqlite v1.11.2/go.mod h1:+mhs/P1ONd+6G7hcAs6irwDi/bjTQ7nLW6LHRBsEa3A=
modernc.org/strutil v1.1.1 h1:xv+J1BXY3Opl2ALrBwyfEikFAj8pmqcpnfmuwUwcozs=
modernc.org/strutil v1.1.1/go.mod h1:DE+MQQ/hjKBZS2zNInV5hhcipt5rLPWkmpbGeW5mmdw=
modernc.org/tcl v1.5.5/go.mod h1:ADkaTUuwukkrlhqwERyq0SM8OvyXo7+TjFz7yAF56EI=
modernc.org/token v1.0.0 h1:a0jaWiNMDhDUtqOj09wvjWWAqd3q7WpBulmL9H2egsk=
modernc.org/token v1.0.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
modernc.org/z v1.0.1/go.mod h1:8/SRk5C/HgiQWCgXdfpb+1RvhORdkz5sw72d3jjtyqA=
sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o=
sourcegraph.com/sourcegraph/appdash v0.0.0-20190731080439-ebfcffb1b5c0/go.mod h1:hI742Nqp5OhwiqlzhgfbWU4mW4yO10fP+LoT9WOswdU=
xorm.io/builder v0.3.9 h1:Sd65/LdWyO7LR8+Cbd+e7mm3sK/7U9k0jS3999IDHMc=
xorm.io/builder v0.3.9/go.mod h1:aUW0S9eb9VCaPohFCH3j7czOx1PMW3i1HrSzbLYGBSE=
xorm.io/xorm v1.2.5 h1:tqN7OhN8P9xi52qBb76I8m5maAJMz/SSbgK2RGPCPbo=
xorm.io/xorm v1.2.5/go.mod h1:fTG8tSjk6O1BYxwuohZUK+S1glnRycsCF05L1qQyEU0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
lukechampine.com/uint128 v1.2.0 h1:mBi/5l91vocEN8otkC5bDLhi2KdCticRiwbdB0O+rjI=
lukechampine.com/uint128 v1.2.0/go.mod h1:c4eWIwlEGaxC/+H1VguhU4PHXNWDCDMUlWdIWl2j1gk=
modernc.org/cc/v3 v3.40.0 h1:P3g79IUS/93SYhtoeaHW+kRCIrYaxJ27MFPv+7kaTOw=
modernc.org/cc/v3 v3.40.0/go.mod h1:/bTg4dnWkSXowUO6ssQKnOV0yMVxDYNIsIrzqTFDGH0=
modernc.org/ccgo/v3 v3.16.13 h1:Mkgdzl46i5F/CNR/Kj80Ri59hC8TKAhZrYSaqvkwzUw=
modernc.org/ccgo/v3 v3.16.13/go.mod h1:2Quk+5YgpImhPjv2Qsob1DnZ/4som1lJTodubIcoUkY=
modernc.org/libc v1.22.2 h1:4U7v51GyhlWqQmwCHj28Rdq2Yzwk55ovjFrdPjs8Hb0=
modernc.org/libc v1.22.2/go.mod h1:uvQavJ1pZ0hIoC/jfqNoMLURIMhKzINIWypNM17puug=
modernc.org/mathutil v1.5.0 h1:rV0Ko/6SfM+8G+yKiyI830l3Wuz1zRutdslNoQ0kfiQ=
modernc.org/mathutil v1.5.0/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
modernc.org/memory v1.4.0 h1:crykUfNSnMAXaOJnnxcSzbUGMqkLWjklJKkBK2nwZwk=
modernc.org/memory v1.4.0/go.mod h1:PkUhL0Mugw21sHPeskwZW4D6VscE/GQJOnIpCnW6pSU=
modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sqlite v1.20.4 h1:J8+m2trkN+KKoE7jglyHYYYiaq5xmz2HoHJIiBlRzbE=
modernc.org/sqlite v1.20.4/go.mod h1:zKcGyrICaxNTMEHSr1HQ2GUraP0j+845GYw37+EyT6A=
modernc.org/strutil v1.1.3 h1:fNMm+oJklMGYfU9Ylcywl0CO5O6nTfaowNsh2wpPjzY=
modernc.org/strutil v1.1.3/go.mod h1:MEHNA7PdEnEwLvspRMtWTNnp2nnyvMfkimT1NKNAGbw=
modernc.org/token v1.0.1 h1:A3qvTqOwexpfZZeyI0FeGPDlSWX5pjZu9hF4lU+EKWg=
modernc.org/token v1.0.1/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
xorm.io/builder v0.3.13 h1:a3jmiVVL19psGeXx8GIurTp7p0IIgqeDmwhcR6BAOAo=
xorm.io/builder v0.3.13/go.mod h1:aUW0S9eb9VCaPohFCH3j7czOx1PMW3i1HrSzbLYGBSE=
xorm.io/xorm v1.3.9 h1:TUovzS0ko+IQ1XnNLfs5dqK1cJl1H5uHpWbWqAQ04nU=
xorm.io/xorm v1.3.9/go.mod h1:LsCCffeeYp63ssk0pKumP6l96WZcHix7ChpurcLNuMw=

View File

@ -1,6 +1,5 @@
#!/bin/bash
VERSION=$(git tag | tail -n 1)
echo 'package main
var version = "'${VERSION}'"' > version.go

View File

@ -1,9 +0,0 @@
package api
// IP describe IP objects via API calls
type IP struct {
ID int `json:"id"`
IP string `json:"ip"`
Rdns string `json:"rdns"`
Src string `json:"src"`
}

View File

@ -8,28 +8,30 @@ import (
"xorm.io/xorm"
)
// GetConfig fetch configuration
func (cfg *Config) GetConfig() error {
var configfile string
var debug bool
var drop bool
var init bool
var port int
var version bool
var migrate bool
flag.Usage = utils.Usage
flag.StringVar(&configfile, "configfile", "ipbl.ini", "Configuration file to use with ipbl section")
flag.IntVar(&port, "port", 8099, "Web service port to use")
flag.BoolVar(&debug, "debug", false, "If debug logging must be enabled")
flag.BoolVar(&drop, "drop", false, "If dropping tables must occur")
flag.BoolVar(&init, "init", false, "If init of database must be done")
flag.BoolVar(&version, "version", false, "Show version")
flag.BoolVar(&migrate, "migrate", false, "Do migrations on version changes")
flag.Parse()
cfg.Switchs.Debug = debug
cfg.Switchs.Drop = drop
cfg.Switchs.Init = init
cfg.Switchs.Port = port
cfg.Switchs.Version = version
cfg.Switchs.Migrate = migrate
var inicfg, err = ini.Load(configfile)
if err != nil {
@ -42,29 +44,36 @@ func (cfg *Config) GetConfig() error {
cfg.DbParams.DbUsername = ipblsection.Key("username").MustString("username")
cfg.DbParams.DbPassword = ipblsection.Key("password").MustString("password")
cfg.DbParams.DbDatabase = ipblsection.Key("database").MustString("database")
cfg.Options.SocketChannel = "ipbl"
cfg.Options.HideBanner = ipblsection.Key("hidebanner").MustBool(false)
cfg.Options.Debug = ipblsection.Key("debug").MustBool(false)
cfg.Options.ScanIP = ipblsection.Key("scanip").MustBool(false)
return nil
}
// Config is the global config
type Config struct {
Db *xorm.Engine `json:"-"`
Db *xorm.Engine
DbParams struct {
DbHostname string `json:"dbhostname"`
DbUsername string `json:"dbusername"`
DbPassword string `json:"dbpassword"`
DbDatabase string `json:"dbdatabase"`
} `json:"dbparams"`
Options struct {
Version string `json:"version"`
HideBanner bool `json:"hidebanner"`
} `json:"-"`
Switchs struct {
Port int `json:"port"`
NoFeed bool `json:"nofeed"`
Debug bool `json:"debug"`
Drop bool `json:"drop"`
Init bool `json:"init"`
} `json:"-"`
DbHostname string
DbUsername string
DbPassword string
DbDatabase string
}
Options struct {
Version string
HideBanner bool
SocketChannel string
Debug bool
ScanIP bool
}
Switchs struct {
Port int
NoFeed bool
Drop bool
Init bool
Version bool
Migrate bool
}
}

View File

@ -5,18 +5,19 @@ import (
"fmt"
"log"
"os"
"strings"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"git.paulbsd.com/paulbsd/ipbl/src/models"
_ "github.com/lib/pq"
"xorm.io/xorm"
"xorm.io/xorm/dialects"
"xorm.io/xorm/names"
)
// Init creates connection to database and exec Schema
func Initialize(ctx *context.Context, cfg *config.Config) (err error) {
var databaseEngine = "postgres"
var tables = []interface{}{models.IP{}, models.Cfg{}, models.Src{}}
//cacher := caches.NewLRUCacher(caches.NewMemoryStore(), 10)
cfg.Db, err = xorm.NewEngine(databaseEngine,
fmt.Sprintf("%s://%s:%s@%s/%s",
@ -29,18 +30,48 @@ func Initialize(ctx *context.Context, cfg *config.Config) (err error) {
log.Fatalln(err)
}
cfg.Db.SetMapper(names.GonicMapper{})
cfg.Db.ShowSQL(cfg.Switchs.Debug)
cfg.Db.SetQuotePolicy(dialects.QuotePolicyReserved)
cfg.Db.ShowSQL(cfg.Options.Debug)
//cfg.Db.SetDefaultCacher(cacher)
if cfg.Switchs.Drop {
log.Println("Dropping tables")
cfg.Db.DropTables(tables)
models.DropTables(ctx, cfg)
os.Exit(0)
}
log.Println("Syncing tables")
for _, table := range tables {
cfg.Db.CreateTables(table)
err = cfg.Db.Sync2(table)
err = models.NewEngine(ctx, cfg)
if cfg.Switchs.Migrate {
migrate(cfg)
}
return
}
func migrate(cfg *config.Config) (err error) {
var cfgsets []models.CfgSet
num, err := cfg.Db.FindAndCount(&cfgsets)
fmt.Printf("%d sets found\n", num)
if err != nil {
fmt.Println(err)
}
for _, cfgset := range cfgsets {
var reg = cfgset.Regex
reg = strings.Trim(reg, "()")
rs := strings.Split(reg, "|")
for _, v := range rs {
if v != "" {
n := models.CfgExpr{
Expr: v,
Set: &cfgset,
Enabled: true,
}
cfg.Db.Insert(&n)
}
}
}
return
}

62
src/models/as.go Normal file
View File

@ -0,0 +1,62 @@
package models
import (
"time"
"xorm.io/xorm"
)
func (as *AutonomousSystem) GetOrCreate(session *xorm.Session) (apias *APIAutonomousSystem, err error) {
has, err := session.Get(as)
if err != nil {
return
}
if !has {
session.Insert(as)
} else {
session.ID(as.ID).Update(as)
}
as.Get(session)
apias = as.APIFormat()
return
}
func (as *AutonomousSystem) Get(session *xorm.Session) (apias *APIAutonomousSystem, err error) {
has, err := session.Get(as)
if !has || err != nil {
return
}
apias = as.APIFormat()
return
}
func (as *AutonomousSystem) APIFormat() *APIAutonomousSystem {
if as == nil {
return &APIAutonomousSystem{}
}
return &APIAutonomousSystem{
ASID: as.ASID,
ASName: as.ASName,
}
}
func (as *AutonomousSystem) APIParse(apias APIAutonomousSystem) (err error) {
*as = AutonomousSystem{
ASID: apias.ASID,
ASName: apias.ASName}
return
}
type AutonomousSystem struct {
ID int `xorm:"pk autoincr"`
ASID int `xorm:"integer unique(asindex) as_id"`
ASName string `xorm:"text unique(asindex) as_name"`
Created time.Time `xorm:"created notnull"`
Updated time.Time `xorm:"updated notnull"`
}
type APIAutonomousSystem struct {
ID int `json:"-"`
ASID int `json:"as_id"`
ASName string `json:"as_name"`
}

View File

@ -1,22 +1,218 @@
package models
import (
"encoding/json"
"fmt"
"regexp"
"strings"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"github.com/labstack/echo/v4"
)
// GetWhitelists ...
func GetWhitelists(cfg config.Config) (res []string) {
var w = Cfg{Key: "whitelist"}
cfg.Db.Get(&w)
res = strings.Split(w.Value, ",")
const ipv4_cidr_regex = `^(((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.|)){4}\/([1-3])?([0-9])?$)`
func GetTrustlists(cfg *config.Config) (res []string, err error) {
var ctl []CfgTrustlist
err = cfg.Db.Find(&ctl)
if len(ctl) > 0 {
for _, a := range ctl {
res = append(res, a.IP)
}
}
return
}
// Cfg is ipbl config
func (tl CfgTrustlist) InsertOrUpdate(cfg *config.Config) (err error) {
var w = Cfg{Key: "trustlist"}
exists, _ := cfg.Db.Get(&w)
if exists {
existing, _ := GetTrustlists(cfg)
for _, j := range existing {
if j == tl.IP {
return fmt.Errorf("ip %s already in config", j)
}
}
existing = append(existing, tl.IP)
w.Value = strings.Join(existing, ",")
cfg.Db.ID(w.ID).Update(&w)
}
return fmt.Errorf("no trustlist updated")
}
func (tl CfgTrustlist) Delete(cfg *config.Config, ip string) (affected int64, err error) {
var w = CfgTrustlist{IP: ip}
exists, _ := cfg.Db.Get(&w)
if exists {
affected, err = cfg.Db.ID(w.ID).Update(&w)
return
}
return
}
func (tl CfgTrustlist) Verify() bool {
reg := regexp.MustCompile(ipv4_cidr_regex)
return reg.MatchString(tl.IP)
}
func (cfgset *CfgSet) BuildRegex(cfg *config.Config) {
var rr []CfgExpr
err := cfg.Db.Where("cfgset_id = $1 AND enabled", cfgset.ID).OrderBy("expr").Find(&rr)
if err != nil {
fmt.Println(err)
}
regs := []string{}
for _, v := range rr {
regs = append(regs, v.Expr)
}
cfgset.Regex = fmt.Sprintf("(%s)", strings.Join(regs, "|"))
}
func GetSets(cfg *config.Config) (res []CfgSet, err error) {
var w = []CfgSet{}
if err := cfg.Db.Where("enabled").Find(&w); err == nil {
return w, err
}
return
}
func GetGlobalConfig(cfg *config.Config) (res []Cfg, err error) {
var w = []Cfg{}
if err := cfg.Db.Find(&w); err == nil {
return w, err
}
return
}
func GetAllConfigv2(cfg *config.Config) (res CfgAllv2, err error) {
var c = make(map[string]string)
var tmpcfg = []Cfg{}
if err := cfg.Db.Find(&tmpcfg); err == nil {
for _, v := range tmpcfg {
c[v.Key] = v.Value
}
res.Cfg = c
}
var sets = []CfgSet{}
if err := cfg.Db.Find(&sets); err == nil {
for i, _ := range sets {
sets[i].BuildRegex(cfg)
}
res.Sets = sets
}
var tl = []CfgTrustlist{}
if err := cfg.Db.Find(&tl); err == nil {
for _, v := range tl {
res.Trustlists = append(res.Trustlists, v.IP)
}
}
var ws = []CfgWS{}
if err := cfg.Db.Find(&ws); err == nil {
res.WS = ws
}
return
}
func InsertOrUpdateSets(cfg *config.Config, folders []CfgSet) (res string, err error) {
var w = Cfg{Key: "folders"}
if exists, _ := cfg.Db.Get(&w); exists {
resbytes, err := json.Marshal(folders)
if err != nil {
return "", err
}
w.Value = string(resbytes)
_, err = cfg.Db.ID(w.ID).Update(&w)
if err != nil {
return "", err
}
}
return
}
func GetWS(cfg *config.Config) (res []CfgWS, err error) {
var w = []CfgWS{}
if err = cfg.Db.Find(&w); err == nil {
return w, err
}
return
}
func DiscoverURLS(cfg *config.Config, routes []*echo.Route) (Discovery, error) {
var disc Discovery
var urls = make(map[string]Url)
for _, j := range routes {
if strings.Contains(j.Path, ":") || j.Path == "/" || j.Path == "/discovery" {
continue
}
names := strings.Split(j.Path, "/")
name := names[len(names)-1]
urls[name] = Url{Key: name, Path: j.Path}
}
disc = Discovery{Version: "1.0", URLs: urls}
return disc, nil
}
type CfgAllv2 struct {
Cfg map[string]string `json:"cfg"`
Sets []CfgSet `json:"sets"`
Trustlists []string `json:"trustlists"`
WS []CfgWS `json:"ws"`
}
type Cfg struct {
ID int `xorm:"pk autoincr" json:"-"`
Key string `xorm:"text notnull unique" json:"key"`
Value string `xorm:"text default" json:"value"`
}
type CfgSet struct {
ID int `xorm:"pk autoincr" json:"-"`
Path string `xorm:"text notnull default" json:"path"`
Src string `xorm:"text notnull" json:"src"`
Filename string `xorm:"text notnull" json:"filename"`
Regex string `xorm:"-" json:"regex"`
Blocktime int64 `xorm:"notnull default 60" json:"blocktime"`
TryFail int64 `xorm:"notnull default 5" json:"tryfail"`
Enabled bool `xorm:"notnull default true" json:"-"`
}
type CfgExpr struct {
ID int `xorm:"pk autoincr" json:"-"`
Expr string `xorm:"text notnull unique(exprindex) index default ''"`
Enabled bool `xorm:"notnull default true" json:"-"`
Set *CfgSet `xorm:"cfgset_id int unique(exprindex) default null"`
}
type CfgTrustlist struct {
ID int `xorm:"pk autoincr" json:"-"`
IP string `xorm:"text notnull" json:"ip"`
}
type CfgWS struct {
ID int `xorm:"pk autoincr" json:"-"`
Type string `xorm:"text notnull" json:"type"`
Endpoint string `xorm:"text notnull" json:"endpoint"`
Subscription string `json:"subscription"`
}
type Discovery struct {
Version string `json:"version"`
URLs map[string]Url `json:"urls"`
}
type Url struct {
Key string `json:"key"`
Path string `json:"path"`
}

58
src/models/city.go Normal file
View File

@ -0,0 +1,58 @@
package models
import (
"time"
"xorm.io/xorm"
)
func (city *City) GetOrCreate(session *xorm.Session) (apicity *APICity, err error) {
has, err := session.Get(city)
if err != nil {
return
}
if !has {
session.Insert(city)
} else {
session.ID(city.ID).Update(city)
}
city.Get(session)
apicity = city.APIFormat()
return
}
func (city *City) Get(session *xorm.Session) (apicity *APICity, err error) {
has, err := session.Get(city)
if !has || err != nil {
return
}
apicity = city.APIFormat()
return
}
func (city *City) APIFormat() *APICity {
if city == nil {
return &APICity{}
}
return &APICity{
CityName: city.CityName,
}
}
func (city *City) APIParse(apicity APICity) (err error) {
*city = City{
CityName: apicity.CityName}
return
}
type City struct {
ID int `xorm:"pk autoincr"`
CityName string `xorm:"text unique(cityindex) city_name" json:"city_name"`
Created time.Time `xorm:"created notnull"`
Updated time.Time `xorm:"updated notnull"`
}
type APICity struct {
ID int `json:"-"`
CityName string `json:"city_name"`
}

58
src/models/country.go Normal file
View File

@ -0,0 +1,58 @@
package models
import (
"time"
"xorm.io/xorm"
)
func (country *Country) GetOrCreate(session *xorm.Session) (apicountry *APICountry, err error) {
has, err := session.Get(country)
if err != nil {
return
}
if !has {
session.Insert(country)
} else {
session.ID(country.ID).Update(country)
}
apicountry = country.APIFormat()
country.Get(session)
return
}
func (country *Country) Get(session *xorm.Session) (apicountry *APICountry, err error) {
has, err := session.Get(country)
if !has || err != nil {
return
}
apicountry = country.APIFormat()
return
}
func (country *Country) APIFormat() *APICountry {
if country == nil {
return &APICountry{}
}
return &APICountry{
CountryName: country.CountryName,
}
}
func (country *Country) APIParse(apicountry APICountry) (err error) {
*country = Country{
CountryName: apicountry.CountryName}
return
}
type Country struct {
ID int `xorm:"pk autoincr"`
CountryName string `xorm:"text unique(countryindex) country_name" json:"country_name"`
Created time.Time `xorm:"created notnull"`
Updated time.Time `xorm:"updated notnull"`
}
type APICountry struct {
ID int `json:"-"`
CountryName string `json:"country_name"`
}

50
src/models/event.go Normal file
View File

@ -0,0 +1,50 @@
package models
import (
"time"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"xorm.io/xorm"
)
func (event *Event) Insert(cfg *config.Config) (err error) {
session := cfg.Db.NewSession()
defer session.Close()
_, err = session.Insert(event)
err = session.Commit()
return
}
func (event *Event) APIParse(session *xorm.Session, apievent APIEvent) (err error) {
*event = Event{
IP: &IP{IP: apievent.IPData.IP},
Host: &Host{Host: apievent.IPData.Hostname},
Src: &Src{Src: apievent.IPData.Src},
}
event.IP.GetOrCreate(session)
event.Host.GetOrCreate(session)
event.Src.GetOrCreate(session)
return
}
type Event struct {
ID int `xorm:"pk autoincr"`
Src *Src `xorm:"int src_id index"`
Host *Host `xorm:"int host_id index"`
IP *IP `xorm:"int ip_id index"`
Created time.Time `xorm:"created notnull index"`
}
type APIEvent struct {
MsgType string `json:"msgtype"`
Mode string `json:"mode"`
Hostname string `json:"hostname"`
IPData struct {
IP string `json:"ip"`
Type int `json:"t"`
Src string `json:"src"`
Hostname string `json:"hostname"`
Date string `json:"date"`
Created string `json:"created"`
} `json:"ipdata"`
}

58
src/models/host.go Normal file
View File

@ -0,0 +1,58 @@
package models
import (
"time"
"xorm.io/xorm"
)
func (host *Host) GetOrCreate(session *xorm.Session) (apihost *APIHost, err error) {
has, err := session.Get(host)
if err != nil {
return
}
if !has {
session.Insert(host)
} else {
session.ID(host.ID).Update(host)
}
host.Get(session)
apihost = host.APIFormat()
return
}
func (host *Host) Get(session *xorm.Session) (apihost *APIHost, err error) {
has, err := session.Get(host)
if !has || err != nil {
return
}
apihost = host.APIFormat()
return
}
func (host *Host) APIFormat() *APIHost {
if host == nil {
return &APIHost{}
}
return &APIHost{
Host: host.Host,
}
}
func (host *Host) APIParse(apihost APIHost) (err error) {
*host = Host{
Host: apihost.Host}
return
}
type Host struct {
ID int `xorm:"pk autoincr"`
Host string `xorm:"text unique host" json:"host"`
Created time.Time `xorm:"created notnull"`
Updated time.Time `xorm:"updated notnull"`
}
type APIHost struct {
ID int `json:"-"`
Host string `json:"host"`
}

55
src/models/hostinfo.go Normal file
View File

@ -0,0 +1,55 @@
package models
import (
"context"
"database/sql"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"xorm.io/xorm"
)
func GetIPsWithoutHostInfo(ctx *context.Context, config *config.Config) (apiips []string, err error) {
res, err := config.Db.Query(`
SELECT ip
FROM ip
WHERE ip NOT IN (
SELECT ip FROM host_info)
ORDER BY RANDOM()
LIMIT 10;`)
for _, r := range res {
apiips = append(apiips, string(r["ip"]))
}
return
}
func ProcessHostInfo(cfg *config.Config, hostinfos []HostInfo) (err error) {
session := cfg.Db.NewSession()
defer session.Close()
for _, hi := range hostinfos {
var i IP = HostInfoToIP(session, &hi)
i.GetOrCreate(session)
}
session.Commit()
return
}
func HostInfoToIP(session *xorm.Session, hi *HostInfo) (ip IP) {
ip = IP{
IP: hi.IP,
Rdns: sql.NullString{
String: hi.Rdns,
Valid: true},
City: &City{CityName: hi.City},
Country: &Country{CountryName: hi.Country},
AutonomousSystem: &AutonomousSystem{ID: 0},
}
return
}
type HostInfo struct {
IP string `json:"ip"`
Whois string `json:"whois"`
Rdns string `json:"rdns"`
City string `json:"city"`
Country string `json:"country"`
}

View File

@ -3,167 +3,291 @@ package models
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"io"
"log"
"net"
"reflect"
"net/http"
"runtime"
"sync"
"time"
"git.paulbsd.com/paulbsd/ipbl/src/api"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"xorm.io/xorm"
)
const SCANLIMIT = 100
const IPINFO_WS = "https://ip.paulbsd.com"
var lastday = time.Now().Add(-(time.Hour * 24))
// GetIPs ...
func GetIPs(ctx *context.Context, config *config.Config, limit int) (apimailboxes []*api.IP, err error) {
func GetIPs(ctx *context.Context, config *config.Config, limit int) (apiips []*APIIP, err error) {
var ips []IP
err = config.Db.Limit(limit).Desc("created").Find(&ips)
for _, ml := range ips {
apimailboxes = append(apimailboxes, ml.APIFormat())
apiips = append(apiips, ml.APIFormat())
}
return
}
// GetIPs ...
func GetIPsLastDay(ctx *context.Context, config *config.Config, interval string) (apimailboxes []*api.IP, err error) {
func GetIPsLast(ctx *context.Context, config *config.Config, interval string) (apiips []string, err error) {
var ips []IP
err = config.Db.Where(fmt.Sprintf("updated >= (now()-'%s'::interval)", interval)).Asc("id").Find(&ips)
err = config.Db.Where("updated >= (now()-?::interval)", interval).GroupBy("ip").Find(&ips)
for _, ml := range ips {
apimailboxes = append(apimailboxes, ml.APIFormat())
apiips = append(apiips, ml.IP)
}
return
}
// GetIP ...
func GetIP(ctx *context.Context, config *config.Config, ipquery interface{}) (apiip *api.IP, err error) {
var ip IP
has, err := config.Db.Where("ip = ?", ipquery).Get(&ip)
if !has {
err = fmt.Errorf("Not Found")
return nil, err
func (ip *IP) GetOrCreate(session *xorm.Session) (apiip *APIIP, err error) {
ip.City.GetOrCreate(session)
ip.Country.GetOrCreate(session)
ip.AutonomousSystem.GetOrCreate(session)
session.Commit()
var tmpip *IP
if ip.ID != 0 {
tmpip = &IP{ID: ip.ID, IP: ip.IP}
} else {
tmpip = &IP{IP: ip.IP}
}
has, err := session.Get(tmpip)
if err != nil {
log.Println(err)
}
if !has {
session.Insert(ip)
} else {
ip.ID = tmpip.ID
session.ID(ip.ID).AllCols().Update(ip)
session.ID(ip.ID).Cols("city_id", "country_id", "as_id").Update(ip)
}
session.Commit()
ip.Get(session)
apiip = ip.APIFormat()
return
}
func (ip *IP) Get(session *xorm.Session) (apiip *APIIP, err error) {
has, err := session.Get(ip)
if !has || err != nil {
err = fmt.Errorf("not found")
return nil, err
}
apiip = ip.APIFormat()
return
}
// UpdateRDNS ...
func (i *IP) UpdateRDNS() (result string, err error) {
res, err := net.LookupAddr(i.IP)
if err != nil {
result = ""
} else {
result = res[0]
func (ip *IP) InsertOrUpdate(session *xorm.Session) (numinsert int64, numupdate int64, err error) {
has, err := session.Get(ip)
if has {
session.ID(ip.ID).Update(&IP{})
}
session.Commit()
return
}
// InsertIP ...
func (i *IP) InsertIP(cfg *config.Config) (num int64, err error) {
num, err = cfg.Db.Insert(i)
return
}
// InsertIPBulk ...
func InsertIPBulk(cfg *config.Config, ips *[]IP) (numinserts int64, numupdates int64, numfail int64, err error) {
var iplist []string
func InsertIPBulk(session *xorm.Session, ips *[]IP) (numinsert int64, numupdate int64, err error) {
for _, ip := range *ips {
iplist = append(iplist, ip.IP)
numinsert, numupdate, err = ip.InsertOrUpdate(session)
}
var searchips []IP
cfg.Db.In("ip", iplist).Find(&searchips)
var toupdateips []string
for _, ip := range searchips {
toupdateips = append(toupdateips, ip.IP)
}
cfg.Db.In("ip", toupdateips).Cols("updated").Update(&IP{})
var toinsertip, _ = differ(*ips, searchips)
numinserts, err = cfg.Db.Insert(toinsertip)
Cleanup(session)
return
}
// ScanIP ...
func ScanIP(cfg *config.Config) (err error) {
var numthreads int = runtime.NumCPU() / 2
for {
var orphans = []IP{}
cfg.Db.Where("rdns IS NULL").Asc("ip").Find(&orphans)
if len(orphans) > 0 {
for _, i := range orphans {
reverse, _ := i.UpdateRDNS()
if reverse == "" {
log.Printf("Set \"none\" rdns to IP %s\n", i.IP)
i.Rdns.String = "none"
} else {
log.Printf("%s %s\n", i.IP, reverse)
i.Rdns.String = reverse
session := cfg.Db.NewSession()
orphans := []IP{}
err = session.Where(`
(
(
as_id IS NULL
OR country_id IS NULL
OR city_id IS NULL
OR rdns = ''
OR rdns IS NULL
)
AND updated < now()-'1d'::interval
)
OR
(
as_id IS NULL
AND country_id IS NULL
AND city_id IS NULL
AND rdns IS NULL
)
`).Desc("updated").Limit(SCANLIMIT).Find(&orphans)
session.Close()
if err == nil && len(orphans) > 0 {
orphanchan := make(chan IP)
var wg sync.WaitGroup
for thr := range make([]int, numthreads) {
go ScanOrphan(&wg, orphanchan, thr, cfg)
}
i.Rdns.Valid = true
_, err = cfg.Db.ID(i.ID).Cols("rdns").Update(&i)
for _, orphan := range orphans {
orphanchan <- orphan
}
close(orphanchan)
wg.Wait()
} else {
time.Sleep(2 * time.Second)
}
}
}
func ScanOrphan(wg *sync.WaitGroup, orphans chan IP, thr int, cfg *config.Config) (err error) {
wg.Add(1)
session := cfg.Db.NewSession()
defer session.Close()
queryclient := http.Client{}
for {
orphan, more := <-orphans
if more {
if err != nil {
log.Println(err)
}
}
} else {
time.Sleep(60 * time.Second)
}
}
var query QueryIP
query, err := QueryInfo(cfg, &queryclient, orphan.IP)
if err != nil {
log.Println(err)
time.Sleep(1 * time.Minute)
continue
}
// APIFormat returns a JSON formatted object of IP
func (ip *IP) APIFormat() *api.IP {
if ip == nil {
var as = AutonomousSystem{ASID: query.APIAS.Number, ASName: query.APIAS.Org}
orphan.AutonomousSystem = &as
if query.APICity != "" {
var city = City{CityName: query.APICity}
orphan.City = &city
}
if query.APICountry != "" {
var country = Country{CountryName: query.APICountry}
orphan.Country = &country
}
orphan.Rdns = sql.NullString{String: query.Rdns, Valid: true}
if cfg.Options.Debug {
log.Printf("t%d: %s -> \"%s\"\n", thr, orphan.IP, query.Rdns)
}
_, err = orphan.GetOrCreate(session)
if err != nil {
continue
}
} else {
log.Printf("All orphan migrated on thread num %d\n", thr)
wg.Done()
break
}
}
err = session.Commit()
if err != nil {
log.Println(err)
}
return nil
}
return &api.IP{
ID: ip.ID,
IP: ip.IP,
Rdns: ip.Rdns.String,
Src: ip.Src,
}
}
func differ(sl1 []IP, sl2 []IP) (toinsert []IP, err error) {
var m = make(map[string]IPDiffer)
longslice := append(sl1, sl2...)
for _, v2 := range longslice {
if _, v := m[v2.IP]; !v {
m[v2.IP] = IPDiffer{IP: v2, Num: 1}
} else {
if this, ok := m[v2.IP]; ok {
this.Num += 1
m[v2.IP] = this
func QueryInfo(cfg *config.Config, client *http.Client, ip string) (query QueryIP, err error) {
var url = fmt.Sprintf("%s/%s", IPINFO_WS, ip)
req, err := http.NewRequest("GET", url, nil)
if err != nil {
log.Println(err)
}
req.Header.Add("Accept", "*/*")
req.Header.Add("User-Agent", fmt.Sprintf("ipbl %s", cfg.Options.Version))
res, err := client.Do(req)
if err != nil {
log.Println(err)
return
}
data, err := io.ReadAll(res.Body)
if err != nil {
log.Println(err)
}
for _, j := range reflect.ValueOf(m).MapKeys() {
if m[j.String()].Num == 1 {
toinsert = append(toinsert, m[j.String()].IP)
}
err = json.Unmarshal(data, &query)
if err != nil {
log.Println(err)
}
return
}
// IP describe IP objects
type IP struct {
ID int `xorm:"pk autoincr" json:"-"`
IP string `xorm:"text notnull unique(ipsrc)" json:"ip"`
Rdns sql.NullString `xorm:"text default" json:"rdns"`
Src string `xorm:"text notnull unique(ipsrc)" json:"src"`
Created time.Time `xorm:"created notnull" json:"-"`
Updated time.Time `xorm:"updated notnull" json:"-"`
func Cleanup(session *xorm.Session) (err error) {
return
}
type IPDiffer struct {
IP IP
Num int
func (ip *IP) APIFormat() *APIIP {
if ip == nil {
return &APIIP{}
}
return &APIIP{
IP: ip.IP,
Rdns: ip.Rdns.String,
APIAS: *ip.AutonomousSystem.APIFormat(),
APICity: ip.City.APIFormat().CityName,
APICountry: ip.Country.APIFormat().CountryName,
}
}
func (ip *IP) APIParse(apiip APIIP) (err error) {
*ip = IP{
IP: apiip.IP,
Rdns: sql.NullString{
String: apiip.Rdns,
Valid: true},
AutonomousSystem: &AutonomousSystem{
ASID: apiip.APIAS.ASID,
ASName: apiip.APIAS.ASName,
},
}
return
}
func (ip *APIIP) BeforeInsert() (err error) {
return
}
type IP struct {
ID int `xorm:"pk autoincr"`
IP string `xorm:"text notnull unique"`
Rdns sql.NullString `xorm:"text index default ''"`
AutonomousSystem *AutonomousSystem `xorm:"as_id int index default null"`
City *City `xorm:"city_id int index default null"`
Country *Country `xorm:"country_id int index default null"`
Created time.Time `xorm:"created notnull"`
Updated time.Time `xorm:"updated index notnull"`
}
type APIIP struct {
IP string `json:"ip"`
Rdns string `json:"rdns"`
APIAS APIAutonomousSystem `json:"as"`
APICity string `json:"city"`
APICountry string `json:"country"`
APIWhois string `json:"whois"`
}
type QueryIP struct {
IP string `json:"ip"`
Rdns string `json:"hostname"`
APIAS QueryAutonomousSystem `json:"as"`
APICity string `json:"city"`
APICountry string `json:"country"`
}
type QueryAutonomousSystem struct {
Number int `json:"number"`
Org string `json:"org"`
}

64
src/models/models.go Normal file
View File

@ -0,0 +1,64 @@
package models
import (
"context"
"fmt"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"xorm.io/xorm/names"
)
var (
tables []interface{}
HasEngine bool
)
func init() {
tables = append(tables,
new(AutonomousSystem),
new(Cfg),
new(CfgSet),
new(CfgExpr),
new(CfgTrustlist),
new(CfgWS),
new(City),
new(Country),
new(Event),
new(IP),
new(Src),
new(Host),
)
for _, name := range []string{"SSL", "UID"} {
names.LintGonicMapper[name] = true
}
}
// NewEngine initializes a new xorm.Engine
func NewEngine(ctx *context.Context, config *config.Config) (err error) {
var x = config.Db
if err = x.Ping(); err != nil {
return err
}
if err = x.Sync2(tables...); err != nil {
return fmt.Errorf("sync database struct error: %v", err)
}
return nil
}
// DropTables initializes a new xorm.Engine
func DropTables(ctx *context.Context, config *config.Config) (err error) {
var x = config.Db
if err = x.Ping(); err != nil {
return err
}
if err = x.Sync2(tables...); err != nil {
return fmt.Errorf("sync database struct error: %v", err)
}
return nil
}

65
src/models/scanresult.go Normal file
View File

@ -0,0 +1,65 @@
package models
import (
"time"
"git.paulbsd.com/paulbsd/ipbl/src/config"
)
func (sr *ScanResult) GetOrCreate(cfg *config.Config) (apisr *APIScanResult, err error) {
has, err := cfg.Db.Get(sr)
if !has {
cfg.Db.Insert(sr)
} else {
cfg.Db.ID(sr.ID).Update(sr)
}
sr.Get(cfg)
apisr = sr.APIFormat()
return
}
func (sr *ScanResult) Get(cfg *config.Config) (apisr *APIScanResult, err error) {
has, err := cfg.Db.Get(sr)
if !has || err != nil {
return
}
apisr = sr.APIFormat()
return
}
func (sr *ScanResult) APIFormat() *APIScanResult {
if sr == nil {
return &APIScanResult{}
}
return &APIScanResult{
Protocol: sr.Protocol,
PortID: sr.PortID,
State: sr.State,
ServiceName: sr.ServiceName,
}
}
func (sr *ScanResult) APIParse(apisr APIScanResult) (err error) {
*sr = ScanResult{
Protocol: apisr.Protocol,
PortID: apisr.PortID}
return
}
type ScanResult struct {
ID int `xorm:"pk autoincr"`
Protocol string `xorm:"text default ''"`
PortID int `xorm:"default ''"`
State string `xorm:"text default ''"`
ServiceName string `xorm:"text default ''"`
Created time.Time `xorm:"created notnull"`
Updated time.Time `xorm:"updated notnull"`
}
type APIScanResult struct {
ID int `json:"id"`
Protocol string `json:"proto"`
PortID int `json:"port_id"`
State string `json:"state"`
ServiceName string `json:"service"`
}

View File

@ -1,7 +1,58 @@
package models
// Src is src types
type Src struct {
ID int `xorm:"pk autoincr" json:"-"`
Src string `xorm:"text notnull unique" json:"src"`
import (
"time"
"xorm.io/xorm"
)
func (src *Src) GetOrCreate(session *xorm.Session) (apisrc *APISrc, err error) {
has, err := session.Get(src)
if err != nil {
return
}
if !has {
session.Insert(src)
} else {
session.ID(src.ID).Update(src)
}
src.Get(session)
apisrc = src.APIFormat()
return
}
func (src *Src) Get(session *xorm.Session) (apisrc *APISrc, err error) {
has, err := session.Get(src)
if !has || err != nil {
return
}
apisrc = src.APIFormat()
return
}
func (src *Src) APIFormat() *APISrc {
if src == nil {
return &APISrc{}
}
return &APISrc{
Src: src.Src,
}
}
func (src *Src) APIParse(apisrc APISrc) (err error) {
*src = Src{
Src: apisrc.Src}
return
}
type Src struct {
ID int `xorm:"pk autoincr"`
Src string `xorm:"text unique(srcindex) src" json:"src"`
Created time.Time `xorm:"created notnull"`
Updated time.Time `xorm:"updated notnull"`
}
type APISrc struct {
ID int `json:"-"`
Src string `json:"src"`
}

View File

@ -1,3 +1,31 @@
package models
const keyname string = "id"
import "reflect"
func Differ(sl1 []IP, sl2 []IP) (toinsert []IP, err error) {
var m = make(map[string]IPDiffer)
longslice := append(sl1, sl2...)
for _, v2 := range longslice {
if _, v := m[v2.IP]; !v {
m[v2.IP] = IPDiffer{IP: v2, Num: 1}
} else {
if this, ok := m[v2.IP]; ok {
this.Num++
m[v2.IP] = this
}
}
}
for _, j := range reflect.ValueOf(m).MapKeys() {
if m[j.String()].Num == 1 {
toinsert = append(toinsert, m[j.String()].IP)
}
}
return
}
type IPDiffer struct {
IP IP
Num int
}

View File

@ -3,46 +3,54 @@ package routers
import (
"context"
"fmt"
"log"
"net/http"
"strconv"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"git.paulbsd.com/paulbsd/ipbl/src/models"
"git.paulbsd.com/paulbsd/ipbl/src/ws"
"github.com/labstack/echo/v4"
)
// RegisterRoutes runs the main echo HTTP server
func RegisterRoutes(e *echo.Echo, ctx *context.Context, cfg *config.Config) {
e.GET("/", func(c echo.Context) error {
return c.HTML(http.StatusOK, `<html>
<body style="background-color: black">
<p style="color:white">Welcome to ipbl software (https://git.paulbsd.com/paulbsd/ipbl)</p>
return c.HTML(http.StatusOK, `
<html>
<head></head>
<body>
<p>Welcome to ipbl software (<a href="https://git.paulbsd.com/paulbsd/ipbl">https://git.paulbsd.com/paulbsd/ipbl</a>)</p>
</body>
</html>`)
</html>
`)
})
e.GET("/health", func(c echo.Context) error {
return c.HTML(http.StatusOK, `OK`)
})
// IPs
e.GET("/ip/:ip", func(c echo.Context) (err error) {
ret, err := models.GetIP(ctx, cfg, c.Param("ip"))
return JSONResult(c, err, ret)
session := cfg.Db.NewSession()
defer session.Close()
var ip = models.IP{IP: c.Param("ip")}
ret, err := ip.Get(session)
return Result(c, err, ret)
})
e.POST("/ip", func(c echo.Context) (err error) {
session := cfg.Db.NewSession()
var apiip = new(models.APIIP)
var ip = new(models.IP)
var msg string
err = c.Bind(ip)
var msg = "No IP inserted"
err = c.Bind(apiip)
if err != nil {
return JSONResult(c, fmt.Errorf("error when parsing body"), "")
return Result(c, fmt.Errorf("error when parsing body"), "")
}
num, err := ip.InsertIP(cfg)
ip.APIParse(*apiip)
_, err = ip.GetOrCreate(session)
if err != nil {
return JSONResult(c, fmt.Errorf("error inserting data"), "")
return Result(c, err, "")
}
if num > 0 {
msg = fmt.Sprintf("Inserted %d IP", num)
}
return c.JSON(http.StatusOK, msg)
return Result(c, err, msg)
})
e.GET("/ips", func(c echo.Context) (err error) {
limit := 50
@ -50,42 +58,106 @@ func RegisterRoutes(e *echo.Echo, ctx *context.Context, cfg *config.Config) {
limit, _ = strconv.Atoi(c.QueryParam("limit"))
}
ret, err := models.GetIPs(ctx, cfg, limit)
return JSONResult(c, err, ret)
return Result(c, err, ret)
})
e.GET("/ips/lastday", func(c echo.Context) (err error) {
interval := "1 day"
ret, err := models.GetIPsLastDay(ctx, cfg, interval)
return JSONResult(c, err, ret)
e.GET("/ips/last", func(c echo.Context) (err error) {
interval := "30m"
if c.QueryParam("interval") != "" {
interval = c.QueryParam("interval")
}
ret, err := models.GetIPsLast(ctx, cfg, interval)
return Result(c, err, ret)
})
e.GET("/ips/withouthostinfo", func(c echo.Context) (err error) {
ret, err := models.GetIPsWithoutHostInfo(ctx, cfg)
return Result(c, err, ret)
})
e.POST("/ips", func(c echo.Context) (err error) {
session := cfg.Db.NewSession()
defer session.Close()
var apiips = []models.APIIP{}
var ips = []models.IP{}
var msg string
err = c.Bind(&ips)
err = c.Bind(&apiips)
if err != nil {
return c.JSON(http.StatusInternalServerError, "Error when parsing body")
return Result(c, err, apiips)
}
numinsert, numupdate, _, _ := models.InsertIPBulk(cfg, &ips)
if numinsert > 0 {
msg = fmt.Sprintf("Inserted %d IP", numinsert)
log.Println(msg)
for _, apiip := range apiips {
var ip = new(models.IP)
ip.APIParse(apiip)
ips = append(ips, *ip)
}
if numupdate > 0 {
msg = fmt.Sprintf("Updated %d IP", numupdate)
log.Println(msg)
}
return c.JSON(http.StatusOK, msg)
numinsert, numupdate, _ := models.InsertIPBulk(session, &ips)
msg = fmt.Sprintf("Inserted %d IP, Updated %d IP", numinsert, numupdate)
return Result(c, err, msg)
})
e.GET("/ips/whitelist", func(c echo.Context) (err error) {
var whitelists = models.GetWhitelists(*cfg)
if len(whitelists) > 0 {
return c.JSON(http.StatusOK, whitelists)
e.POST("/event", func(c echo.Context) (err error) {
session := cfg.Db.NewSession()
defer session.Close()
var apievent = new(models.APIEvent)
var event = new(models.Event)
err = c.Bind(apievent)
if err != nil {
return Result(c, fmt.Errorf("error when parsing body"), "")
}
return c.JSON(http.StatusInternalServerError, "")
event.APIParse(session, *apievent)
err = event.Insert(cfg)
if err != nil {
return Result(c, err, "")
}
return Result(c, err, "OK")
})
e.POST("/hostinfo", func(c echo.Context) (err error) {
var hostinfos = []models.HostInfo{}
err = c.Bind(&hostinfos)
if err != nil {
return Result(c, err, hostinfos)
}
models.ProcessHostInfo(cfg, hostinfos)
return Result(c, err, "")
})
e.GET("/config", func(c echo.Context) (err error) {
globalconfig, err := models.GetAllConfigv2(cfg)
return Result(c, err, globalconfig)
})
e.GET("/config/trustlist", func(c echo.Context) (err error) {
trustlists, err := models.GetTrustlists(cfg)
return Result(c, err, trustlists)
})
e.POST("/config/trustlist", func(c echo.Context) (err error) {
var cidr models.CfgTrustlist
err = c.Bind(&cidr)
if err == nil && cidr.Verify() {
err = cidr.InsertOrUpdate(cfg)
return Result(c, err, cidr)
}
return Result(c, err, nil)
})
e.DELETE("/config/trustlist/:ip", func(c echo.Context) (err error) {
var ip = c.Param("ip")
var cidr models.CfgTrustlist
_, err = cidr.Delete(cfg, ip)
return
})
e.GET("/config/sets", func(c echo.Context) (err error) {
sets, err := models.GetSets(cfg)
return Result(c, err, sets)
})
e.GET("/config/ws", func(c echo.Context) (err error) {
folders, err := models.GetWS(cfg)
return Result(c, err, folders)
})
e.GET("/discovery", func(c echo.Context) (err error) {
disc, err := models.DiscoverURLS(cfg, e.Routes())
return Result(c, err, disc)
})
e.File("/test.html", "/home/paul/test.html")
e.GET("/wsps", func(c echo.Context) (err error) {
ws.HandleWSPS(&c, cfg)
return
})
e.GET("/wsrr", func(c echo.Context) (err error) {
ws.HandleWSRR(&c, cfg)
return
})
e.Logger.Fatal(
e.Start(
fmt.Sprintf(":%d",
cfg.Switchs.Port)))
}

View File

@ -4,37 +4,46 @@ import (
"context"
"fmt"
"net/http"
"strings"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"github.com/labstack/echo/v4"
)
// RunServer runs the main echo server
func RunServer(ctx *context.Context, cfg *config.Config) (err error) {
e := echo.New()
e.HideBanner = true
RegisterRoutes(e, ctx, cfg)
err = e.Start(
fmt.Sprintf(":%d",
cfg.Switchs.Port))
e.Logger.Fatal(e.Start(fmt.Sprintf(":%d", cfg.Switchs.Port)))
return
}
// JSONResult handles returns and error management on backend api
func JSONResult(c echo.Context, inputerr error, data interface{}) (err error) {
func Result(c echo.Context, inputerr error, data interface{}) (err error) {
if inputerr != nil {
if inputerr.Error() == "not found" {
return c.String(http.StatusNotFound, "Content not found")
}
if inputerr.Error() == "Not Found" {
return c.JSON(http.StatusNotFound, inputerr.Error())
return c.String(http.StatusNotFound, "Content not found")
}
if strings.Contains(inputerr.Error(), "duplicate key value violates unique constraint") {
return c.String(http.StatusConflict, "Content already exists")
}
if strings.Contains(inputerr.Error(), "already exists") {
return c.String(http.StatusConflict, "Content already exists")
}
if inputerr.Error() == "Error when parsing body" {
return c.JSON(http.StatusBadRequest, inputerr.Error())
return c.String(http.StatusBadRequest, "Content not conform")
}
return c.JSON(http.StatusInternalServerError, inputerr.Error())
return c.String(http.StatusInternalServerError, inputerr.Error())
}
return c.JSON(http.StatusOK, data)
}
// ConfigAccess make ip authorization to configuration
func ConfigAccess(cfg config.Config, ip string) (ret bool) {
switch ip {
case "127.0.0.1":

66
src/ws/init.go Normal file
View File

@ -0,0 +1,66 @@
package ws
import (
"sync"
"time"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"golang.org/x/net/websocket"
)
var listeners sync.Map
func Init(cfg *config.Config) {
}
func welcomeAgents(ws *websocket.Conn, welcome wsWelcome, t string) {
connectinfoVal, ok := listeners.Load(welcome.Hostname)
if !ok {
switch t {
case "ps":
connectinfo := connectionInfo{
ConnectionPS: ws,
InitDate: time.Now(),
}
listeners.Store(welcome.Hostname, &connectinfo)
case "rr":
connectinfo := connectionInfo{
ConnectionRR: ws,
InitDate: time.Now(),
}
listeners.Store(welcome.Hostname, &connectinfo)
}
} else {
connectinfo := connectinfoVal.(*connectionInfo)
switch t {
case "ps":
connectinfo.ConnectionPS = ws
case "rr":
connectinfo.ConnectionRR = ws
}
}
}
func gcConnOnError(ws *websocket.Conn) (err error) {
listeners.Range(func(index, value interface{}) bool {
if value.(*connectionInfo).ConnectionPS == ws || value.(*connectionInfo).ConnectionRR == ws {
value.(*connectionInfo).ConnectionPS.Close()
value.(*connectionInfo).ConnectionRR.Close()
listeners.Delete(index)
}
return true
})
return err
}
type connectionInfo struct {
ConnectionPS *websocket.Conn
ConnectionRR *websocket.Conn
InitDate time.Time
}
// WSWelcome
type wsWelcome struct {
Hostname string
}

44
src/ws/pubsub.go Normal file
View File

@ -0,0 +1,44 @@
package ws
import (
"encoding/json"
"log"
"time"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"github.com/labstack/echo/v4"
"golang.org/x/net/websocket"
)
// HandleWSPS handle pub sub flows
func HandleWSPS(c *echo.Context, cfg *config.Config) (err error) {
websocket.Handler(func(ws *websocket.Conn) {
defer ws.Close()
var welcome = wsWelcome{}
var msg []byte
err := websocket.Message.Receive(ws, &msg)
if err != nil {
log.Println(err)
}
err = json.Unmarshal(msg, &welcome)
if err != nil {
log.Println(err)
} else {
welcomeAgents(ws, welcome, "ps")
}
err = websocket.Message.Receive(ws, "OK")
if err != nil {
log.Printf("disconnect: %s (from pubsub channel)\n", welcome.Hostname)
ws.Close()
}
for {
time.Sleep(50 * time.Millisecond)
}
}).ServeHTTP((*c).Response(), (*c).Request())
return nil
}

123
src/ws/reqrep.go Normal file
View File

@ -0,0 +1,123 @@
package ws
import (
"encoding/json"
"log"
"git.paulbsd.com/paulbsd/ipbl/src/config"
"git.paulbsd.com/paulbsd/ipbl/src/models"
"github.com/labstack/echo/v4"
"golang.org/x/net/websocket"
)
// HandleWSRR handle req rep flows
func HandleWSRR(c *echo.Context, cfg *config.Config) error {
websocket.Handler(func(ws *websocket.Conn) {
defer ws.Close()
var welcome = wsWelcome{}
var msg []byte
err := websocket.Message.Receive(ws, &msg)
if err != nil {
log.Println(err)
}
err = json.Unmarshal(msg, &welcome)
if err == nil {
welcomeAgents(ws, welcome, "rr")
}
var lastip string
for {
var apievent = models.APIEvent{}
var event = models.Event{}
var msg []byte
err := websocket.Message.Receive(ws, &msg)
if err != nil {
log.Printf("disconnect: %s (from reqrep channel)", welcome.Hostname)
return
}
err = json.Unmarshal(msg, &apievent)
if err != nil {
log.Println(err)
}
if lastip != "" && apievent.IPData.IP == lastip {
continue
}
switch apievent.MsgType {
case "bootstrap":
log.Printf("bootstrap: %s\n", apievent.Hostname)
listeners.Range(func(index, value interface{}) bool {
if index != apievent.Hostname && value.(*connectionInfo).ConnectionPS != nil {
err = websocket.Message.Send(value.(*connectionInfo).ConnectionPS, msg)
if err != nil {
log.Println(err)
ws.Close()
gcConnOnError(ws)
}
}
return true
})
case "add":
session := cfg.Db.NewSession()
event.APIParse(session, apievent)
session.Close()
err := event.Insert(cfg)
if err != nil {
log.Println(err)
}
listeners.Range(func(index, value interface{}) bool {
if value.(*connectionInfo).ConnectionPS != nil {
err = websocket.Message.Send(value.(*connectionInfo).ConnectionPS, msg)
if err != nil {
log.Println(err)
ws.Close()
gcConnOnError(ws)
}
}
return true
})
log.Printf("ws: Inserted event")
case "init":
listeners.Range(func(index, value interface{}) bool {
if value.(*connectionInfo).ConnectionPS != nil {
err = websocket.Message.Send(value.(*connectionInfo).ConnectionPS, msg)
if err != nil {
log.Println(err)
ws.Close()
gcConnOnError(ws)
}
}
return true
})
case "ping":
listeners.Range(func(index, value interface{}) bool {
if index == apievent.Hostname && value.(*connectionInfo).ConnectionPS != nil {
err = websocket.Message.Send(value.(*connectionInfo).ConnectionPS, msg)
if err != nil {
log.Println(err)
ws.Close()
gcConnOnError(ws)
}
}
return true
})
default:
}
err = websocket.Message.Send(ws, "OK")
if err != nil {
log.Println(err)
gcConnOnError(ws)
}
}
}).ServeHTTP((*c).Response(), (*c).Request())
return nil
}

View File

@ -3,12 +3,14 @@ package utils
import (
"flag"
"log"
"os"
"time"
)
// Usage displays possible arguments
func Usage() {
flag.PrintDefaults()
log.Fatal()
os.Exit(0)
}
// Advice displays possible arguments with warning advices
@ -17,5 +19,11 @@ func Advice(advice string) {
if advice != "" {
log.Fatalln(advice)
}
log.Fatal()
os.Exit(0)
}
func Server() {
for {
time.Sleep(time.Second)
}
}

View File

@ -48,6 +48,17 @@ linters:
- nlreturn
- testpackage
- wsl
- varnamelen
- nilnil
- ireturn
- govet
- forcetypeassert
- cyclop
- containedctx
- revive
- nosnakecase
- exhaustruct
- depguard
issues:
exclude-rules:

View File

@ -1,3 +1,151 @@
# v0.10.2 - 2023/03/20
### New features
* Support DebugDOT option for debugging encoder ( #440 )
### Fix bugs
* Fix combination of embedding structure and omitempty option ( #442 )
# v0.10.1 - 2023/03/13
### Fix bugs
* Fix checkptr error for array decoder ( #415 )
* Fix added buffer size check when decoding key ( #430 )
* Fix handling of anonymous fields other than struct ( #431 )
* Fix to not optimize when lower conversion can't handle byte-by-byte ( #432 )
* Fix a problem that MarshalIndent does not work when UnorderedMap is specified ( #435 )
* Fix mapDecoder.DecodeStream() for empty objects containing whitespace ( #425 )
* Fix an issue that could not set the correct NextField for fields in the embedded structure ( #438 )
# v0.10.0 - 2022/11/29
### New features
* Support JSON Path ( #250 )
### Fix bugs
* Fix marshaler for map's key ( #409 )
# v0.9.11 - 2022/08/18
### Fix bugs
* Fix unexpected behavior when buffer ends with backslash ( #383 )
* Fix stream decoding of escaped character ( #387 )
# v0.9.10 - 2022/07/15
### Fix bugs
* Fix boundary exception of type caching ( #382 )
# v0.9.9 - 2022/07/15
### Fix bugs
* Fix encoding of directed interface with typed nil ( #377 )
* Fix embedded primitive type encoding using alias ( #378 )
* Fix slice/array type encoding with types implementing MarshalJSON ( #379 )
* Fix unicode decoding when the expected buffer state is not met after reading ( #380 )
# v0.9.8 - 2022/06/30
### Fix bugs
* Fix decoding of surrogate-pair ( #365 )
* Fix handling of embedded primitive type ( #366 )
* Add validation of escape sequence for decoder ( #367 )
* Fix stream tokenizing respecting UseNumber ( #369 )
* Fix encoding when struct pointer type that implements Marshal JSON is embedded ( #375 )
### Improve performance
* Improve performance of linkRecursiveCode ( #368 )
# v0.9.7 - 2022/04/22
### Fix bugs
#### Encoder
* Add filtering process for encoding on slow path ( #355 )
* Fix encoding of interface{} with pointer type ( #363 )
#### Decoder
* Fix map key decoder that implements UnmarshalJSON ( #353 )
* Fix decoding of []uint8 type ( #361 )
### New features
* Add DebugWith option for encoder ( #356 )
# v0.9.6 - 2022/03/22
### Fix bugs
* Correct the handling of the minimum value of int type for decoder ( #344 )
* Fix bugs of stream decoder's bufferSize ( #349 )
* Add a guard to use typeptr more safely ( #351 )
### Improve decoder performance
* Improve escapeString's performance ( #345 )
### Others
* Update go version for CI ( #347 )
# v0.9.5 - 2022/03/04
### Fix bugs
* Fix panic when decoding time.Time with context ( #328 )
* Fix reading the next character in buffer to nul consideration ( #338 )
* Fix incorrect handling on skipValue ( #341 )
### Improve decoder performance
* Improve performance when a payload contains escape sequence ( #334 )
# v0.9.4 - 2022/01/21
* Fix IsNilForMarshaler for string type with omitempty ( #323 )
* Fix the case where the embedded field is at the end ( #326 )
# v0.9.3 - 2022/01/14
* Fix logic of removing struct field for decoder ( #322 )
# v0.9.2 - 2022/01/14
* Add invalid decoder to delay type error judgment at decode ( #321 )
# v0.9.1 - 2022/01/11
* Fix encoding of MarshalText/MarshalJSON operation with head offset ( #319 )
# v0.9.0 - 2022/01/05
### New feature
* Supports dynamic filtering of struct fields ( #314 )
### Improve encoding performance
* Improve map encoding performance ( #310 )
* Optimize encoding path for escaped string ( #311 )
* Add encoding option for performance ( #312 )
### Fix bugs
* Fix panic at encoding map value on 1.18 ( #310 )
* Fix MarshalIndent for interface type ( #317 )
# v0.8.1 - 2021/12/05
* Fix operation conversion from PtrHead to Head in Recursive type ( #305 )

View File

@ -22,7 +22,7 @@ cover-html: cover
.PHONY: lint
lint: golangci-lint
golangci-lint run
$(BIN_DIR)/golangci-lint run
golangci-lint: | $(BIN_DIR)
@{ \
@ -30,7 +30,7 @@ golangci-lint: | $(BIN_DIR)
GOLANGCI_LINT_TMP_DIR=$$(mktemp -d); \
cd $$GOLANGCI_LINT_TMP_DIR; \
go mod init tmp; \
GOBIN=$(BIN_DIR) go get github.com/golangci/golangci-lint/cmd/golangci-lint@v1.36.0; \
GOBIN=$(BIN_DIR) go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.54.2; \
rm -rf $$GOLANGCI_LINT_TMP_DIR; \
}

View File

@ -13,7 +13,7 @@ Fast JSON encoder/decoder compatible with encoding/json for Go
```
* version ( expected release date )
* v0.8.0
* v0.9.0
|
| while maintaining compatibility with encoding/json, we will add convenient APIs
|
@ -21,9 +21,8 @@ Fast JSON encoder/decoder compatible with encoding/json for Go
* v1.0.0
```
We are accepting requests for features that will be implemented between v0.8.0 and v.1.0.0.
We are accepting requests for features that will be implemented between v0.9.0 and v.1.0.0.
If you have the API you need, please submit your issue [here](https://github.com/goccy/go-json/issues).
For example, I'm thinking of supporting `context.Context` of `json.Marshaler` and decoding using JSON Path.
# Features
@ -32,6 +31,7 @@ For example, I'm thinking of supporting `context.Context` of `json.Marshaler` an
- Flexible customization with options
- Coloring the encoded string
- Can propagate context.Context to `MarshalJSON` or `UnmarshalJSON`
- Can dynamically filter the fields of the structure type-safely
# Installation
@ -184,7 +184,7 @@ func Marshal(v interface{}) ([]byte, error) {
`json.Marshal` and `json.Unmarshal` receive `interface{}` value and they perform type determination dynamically to process.
In normal case, you need to use the `reflect` library to determine the type dynamically, but since `reflect.Type` is defined as `interface`, when you call the method of `reflect.Type`, The reflect's argument is escaped.
Therefore, the arguments for `Marshal` and `Unmarshal` are always escape to the heap.
Therefore, the arguments for `Marshal` and `Unmarshal` are always escaped to the heap.
However, `go-json` can use the feature of `reflect.Type` while avoiding escaping.
`reflect.Type` is defined as `interface`, but in reality `reflect.Type` is implemented only by the structure `rtype` defined in the `reflect` package.

View File

@ -83,6 +83,37 @@ func unmarshalContext(ctx context.Context, data []byte, v interface{}, optFuncs
return validateEndBuf(src, cursor)
}
var (
pathDecoder = decoder.NewPathDecoder()
)
func extractFromPath(path *Path, data []byte, optFuncs ...DecodeOptionFunc) ([][]byte, error) {
if path.path.RootSelectorOnly {
return [][]byte{data}, nil
}
src := make([]byte, len(data)+1) // append nul byte to the end
copy(src, data)
ctx := decoder.TakeRuntimeContext()
ctx.Buf = src
ctx.Option.Flags = 0
ctx.Option.Flags |= decoder.PathOption
ctx.Option.Path = path.path
for _, optFunc := range optFuncs {
optFunc(ctx.Option)
}
paths, cursor, err := pathDecoder.DecodePath(ctx, 0, 0)
if err != nil {
decoder.ReleaseRuntimeContext(ctx)
return nil, err
}
decoder.ReleaseRuntimeContext(ctx)
if err := validateEndBuf(src, cursor); err != nil {
return nil, err
}
return paths, nil
}
func unmarshalNoEscape(data []byte, v interface{}, optFuncs ...DecodeOptionFunc) error {
src := make([]byte, len(data)+1) // append nul byte to the end
copy(src, data)

View File

@ -1,7 +1,7 @@
version: '2'
services:
go-json:
image: golang:1.17
image: golang:1.18
volumes:
- '.:/go/src/go-json'
deploy:

View File

@ -3,6 +3,7 @@ package json
import (
"context"
"io"
"os"
"unsafe"
"github.com/goccy/go-json/internal/encoder"
@ -51,7 +52,7 @@ func (e *Encoder) EncodeContext(ctx context.Context, v interface{}, optFuncs ...
rctx.Option.Flag |= encoder.ContextOption
rctx.Option.Context = ctx
err := e.encodeWithOption(rctx, v, optFuncs...)
err := e.encodeWithOption(rctx, v, optFuncs...) //nolint: contextcheck
encoder.ReleaseRuntimeContext(rctx)
return err
@ -61,6 +62,8 @@ func (e *Encoder) encodeWithOption(ctx *encoder.RuntimeContext, v interface{}, o
if e.enabledHTMLEscape {
ctx.Option.Flag |= encoder.HTMLEscapeOption
}
ctx.Option.Flag |= encoder.NormalizeUTF8Option
ctx.Option.DebugOut = os.Stdout
for _, optFunc := range optFuncs {
optFunc(ctx.Option)
}
@ -111,13 +114,13 @@ func (e *Encoder) SetIndent(prefix, indent string) {
func marshalContext(ctx context.Context, v interface{}, optFuncs ...EncodeOptionFunc) ([]byte, error) {
rctx := encoder.TakeRuntimeContext()
rctx.Option.Flag = 0
rctx.Option.Flag = encoder.HTMLEscapeOption | encoder.ContextOption
rctx.Option.Flag = encoder.HTMLEscapeOption | encoder.NormalizeUTF8Option | encoder.ContextOption
rctx.Option.Context = ctx
for _, optFunc := range optFuncs {
optFunc(rctx.Option)
}
buf, err := encode(rctx, v)
buf, err := encode(rctx, v) //nolint: contextcheck
if err != nil {
encoder.ReleaseRuntimeContext(rctx)
return nil, err
@ -139,7 +142,7 @@ func marshal(v interface{}, optFuncs ...EncodeOptionFunc) ([]byte, error) {
ctx := encoder.TakeRuntimeContext()
ctx.Option.Flag = 0
ctx.Option.Flag |= encoder.HTMLEscapeOption
ctx.Option.Flag |= (encoder.HTMLEscapeOption | encoder.NormalizeUTF8Option)
for _, optFunc := range optFuncs {
optFunc(ctx.Option)
}
@ -166,7 +169,7 @@ func marshalNoEscape(v interface{}) ([]byte, error) {
ctx := encoder.TakeRuntimeContext()
ctx.Option.Flag = 0
ctx.Option.Flag |= encoder.HTMLEscapeOption
ctx.Option.Flag |= (encoder.HTMLEscapeOption | encoder.NormalizeUTF8Option)
buf, err := encodeNoEscape(ctx, v)
if err != nil {
@ -190,7 +193,7 @@ func marshalIndent(v interface{}, prefix, indent string, optFuncs ...EncodeOptio
ctx := encoder.TakeRuntimeContext()
ctx.Option.Flag = 0
ctx.Option.Flag |= (encoder.HTMLEscapeOption | encoder.IndentOption)
ctx.Option.Flag |= (encoder.HTMLEscapeOption | encoder.NormalizeUTF8Option | encoder.IndentOption)
for _, optFunc := range optFuncs {
optFunc(ctx.Option)
}
@ -220,7 +223,7 @@ func encode(ctx *encoder.RuntimeContext, v interface{}) ([]byte, error) {
typ := header.typ
typeptr := uintptr(unsafe.Pointer(typ))
codeSet, err := encoder.CompileToGetCodeSet(typeptr)
codeSet, err := encoder.CompileToGetCodeSet(ctx, typeptr)
if err != nil {
return nil, err
}
@ -248,7 +251,7 @@ func encodeNoEscape(ctx *encoder.RuntimeContext, v interface{}) ([]byte, error)
typ := header.typ
typeptr := uintptr(unsafe.Pointer(typ))
codeSet, err := encoder.CompileToGetCodeSet(typeptr)
codeSet, err := encoder.CompileToGetCodeSet(ctx, typeptr)
if err != nil {
return nil, err
}
@ -275,7 +278,7 @@ func encodeIndent(ctx *encoder.RuntimeContext, v interface{}, prefix, indent str
typ := header.typ
typeptr := uintptr(unsafe.Pointer(typ))
codeSet, err := encoder.CompileToGetCodeSet(typeptr)
codeSet, err := encoder.CompileToGetCodeSet(ctx, typeptr)
if err != nil {
return nil, err
}

View File

@ -37,3 +37,5 @@ type UnmarshalTypeError = errors.UnmarshalTypeError
type UnsupportedTypeError = errors.UnsupportedTypeError
type UnsupportedValueError = errors.UnsupportedValueError
type PathError = errors.PathError

View File

@ -35,3 +35,7 @@ func (d *anonymousFieldDecoder) Decode(ctx *RuntimeContext, cursor, depth int64,
p = *(*unsafe.Pointer)(p)
return d.dec.Decode(ctx, cursor, depth, unsafe.Pointer(uintptr(p)+d.offset))
}
func (d *anonymousFieldDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return d.dec.DecodePath(ctx, cursor, depth)
}

View File

@ -1,6 +1,7 @@
package decoder
import (
"fmt"
"unsafe"
"github.com/goccy/go-json/internal/errors"
@ -18,7 +19,9 @@ type arrayDecoder struct {
}
func newArrayDecoder(dec Decoder, elemType *runtime.Type, alen int, structName, fieldName string) *arrayDecoder {
zeroValue := *(*unsafe.Pointer)(unsafe_New(elemType))
// workaround to avoid checkptr errors. cannot use `*(*unsafe.Pointer)(unsafe_New(elemType))` directly.
zeroValuePtr := unsafe_New(elemType)
zeroValue := **(**unsafe.Pointer)(unsafe.Pointer(&zeroValuePtr))
return &arrayDecoder{
valueDecoder: dec,
elemType: elemType,
@ -167,3 +170,7 @@ func (d *arrayDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe
}
}
}
func (d *arrayDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: array decoder does not support decode path")
}

View File

@ -0,0 +1,438 @@
package decoder
import (
"fmt"
"reflect"
"strconv"
)
var (
nilValue = reflect.ValueOf(nil)
)
func AssignValue(src, dst reflect.Value) error {
if dst.Type().Kind() != reflect.Ptr {
return fmt.Errorf("invalid dst type. required pointer type: %T", dst.Type())
}
casted, err := castValue(dst.Elem().Type(), src)
if err != nil {
return err
}
dst.Elem().Set(casted)
return nil
}
func castValue(t reflect.Type, v reflect.Value) (reflect.Value, error) {
switch t.Kind() {
case reflect.Int:
vv, err := castInt(v)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(int(vv.Int())), nil
case reflect.Int8:
vv, err := castInt(v)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(int8(vv.Int())), nil
case reflect.Int16:
vv, err := castInt(v)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(int16(vv.Int())), nil
case reflect.Int32:
vv, err := castInt(v)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(int32(vv.Int())), nil
case reflect.Int64:
return castInt(v)
case reflect.Uint:
vv, err := castUint(v)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(uint(vv.Uint())), nil
case reflect.Uint8:
vv, err := castUint(v)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(uint8(vv.Uint())), nil
case reflect.Uint16:
vv, err := castUint(v)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(uint16(vv.Uint())), nil
case reflect.Uint32:
vv, err := castUint(v)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(uint32(vv.Uint())), nil
case reflect.Uint64:
return castUint(v)
case reflect.Uintptr:
vv, err := castUint(v)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(uintptr(vv.Uint())), nil
case reflect.String:
return castString(v)
case reflect.Bool:
return castBool(v)
case reflect.Float32:
vv, err := castFloat(v)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(float32(vv.Float())), nil
case reflect.Float64:
return castFloat(v)
case reflect.Array:
return castArray(t, v)
case reflect.Slice:
return castSlice(t, v)
case reflect.Map:
return castMap(t, v)
case reflect.Struct:
return castStruct(t, v)
}
return v, nil
}
func castInt(v reflect.Value) (reflect.Value, error) {
switch v.Type().Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return v, nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return reflect.ValueOf(int64(v.Uint())), nil
case reflect.String:
i64, err := strconv.ParseInt(v.String(), 10, 64)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(i64), nil
case reflect.Bool:
if v.Bool() {
return reflect.ValueOf(int64(1)), nil
}
return reflect.ValueOf(int64(0)), nil
case reflect.Float32, reflect.Float64:
return reflect.ValueOf(int64(v.Float())), nil
case reflect.Array:
if v.Len() > 0 {
return castInt(v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to int64 from empty array")
case reflect.Slice:
if v.Len() > 0 {
return castInt(v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to int64 from empty slice")
case reflect.Interface:
return castInt(reflect.ValueOf(v.Interface()))
case reflect.Map:
return nilValue, fmt.Errorf("failed to cast to int64 from map")
case reflect.Struct:
return nilValue, fmt.Errorf("failed to cast to int64 from struct")
case reflect.Ptr:
return castInt(v.Elem())
}
return nilValue, fmt.Errorf("failed to cast to int64 from %s", v.Type().Kind())
}
func castUint(v reflect.Value) (reflect.Value, error) {
switch v.Type().Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return reflect.ValueOf(uint64(v.Int())), nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return v, nil
case reflect.String:
u64, err := strconv.ParseUint(v.String(), 10, 64)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(u64), nil
case reflect.Bool:
if v.Bool() {
return reflect.ValueOf(uint64(1)), nil
}
return reflect.ValueOf(uint64(0)), nil
case reflect.Float32, reflect.Float64:
return reflect.ValueOf(uint64(v.Float())), nil
case reflect.Array:
if v.Len() > 0 {
return castUint(v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to uint64 from empty array")
case reflect.Slice:
if v.Len() > 0 {
return castUint(v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to uint64 from empty slice")
case reflect.Interface:
return castUint(reflect.ValueOf(v.Interface()))
case reflect.Map:
return nilValue, fmt.Errorf("failed to cast to uint64 from map")
case reflect.Struct:
return nilValue, fmt.Errorf("failed to cast to uint64 from struct")
case reflect.Ptr:
return castUint(v.Elem())
}
return nilValue, fmt.Errorf("failed to cast to uint64 from %s", v.Type().Kind())
}
func castString(v reflect.Value) (reflect.Value, error) {
switch v.Type().Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return reflect.ValueOf(fmt.Sprint(v.Int())), nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return reflect.ValueOf(fmt.Sprint(v.Uint())), nil
case reflect.String:
return v, nil
case reflect.Bool:
if v.Bool() {
return reflect.ValueOf("true"), nil
}
return reflect.ValueOf("false"), nil
case reflect.Float32, reflect.Float64:
return reflect.ValueOf(fmt.Sprint(v.Float())), nil
case reflect.Array:
if v.Len() > 0 {
return castString(v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to string from empty array")
case reflect.Slice:
if v.Len() > 0 {
return castString(v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to string from empty slice")
case reflect.Interface:
return castString(reflect.ValueOf(v.Interface()))
case reflect.Map:
return nilValue, fmt.Errorf("failed to cast to string from map")
case reflect.Struct:
return nilValue, fmt.Errorf("failed to cast to string from struct")
case reflect.Ptr:
return castString(v.Elem())
}
return nilValue, fmt.Errorf("failed to cast to string from %s", v.Type().Kind())
}
func castBool(v reflect.Value) (reflect.Value, error) {
switch v.Type().Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
switch v.Int() {
case 0:
return reflect.ValueOf(false), nil
case 1:
return reflect.ValueOf(true), nil
}
return nilValue, fmt.Errorf("failed to cast to bool from %d", v.Int())
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
switch v.Uint() {
case 0:
return reflect.ValueOf(false), nil
case 1:
return reflect.ValueOf(true), nil
}
return nilValue, fmt.Errorf("failed to cast to bool from %d", v.Uint())
case reflect.String:
b, err := strconv.ParseBool(v.String())
if err != nil {
return nilValue, err
}
return reflect.ValueOf(b), nil
case reflect.Bool:
return v, nil
case reflect.Float32, reflect.Float64:
switch v.Float() {
case 0:
return reflect.ValueOf(false), nil
case 1:
return reflect.ValueOf(true), nil
}
return nilValue, fmt.Errorf("failed to cast to bool from %f", v.Float())
case reflect.Array:
if v.Len() > 0 {
return castBool(v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to string from empty array")
case reflect.Slice:
if v.Len() > 0 {
return castBool(v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to string from empty slice")
case reflect.Interface:
return castBool(reflect.ValueOf(v.Interface()))
case reflect.Map:
return nilValue, fmt.Errorf("failed to cast to string from map")
case reflect.Struct:
return nilValue, fmt.Errorf("failed to cast to string from struct")
case reflect.Ptr:
return castBool(v.Elem())
}
return nilValue, fmt.Errorf("failed to cast to bool from %s", v.Type().Kind())
}
func castFloat(v reflect.Value) (reflect.Value, error) {
switch v.Type().Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return reflect.ValueOf(float64(v.Int())), nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return reflect.ValueOf(float64(v.Uint())), nil
case reflect.String:
f64, err := strconv.ParseFloat(v.String(), 64)
if err != nil {
return nilValue, err
}
return reflect.ValueOf(f64), nil
case reflect.Bool:
if v.Bool() {
return reflect.ValueOf(float64(1)), nil
}
return reflect.ValueOf(float64(0)), nil
case reflect.Float32, reflect.Float64:
return v, nil
case reflect.Array:
if v.Len() > 0 {
return castFloat(v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to float64 from empty array")
case reflect.Slice:
if v.Len() > 0 {
return castFloat(v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to float64 from empty slice")
case reflect.Interface:
return castFloat(reflect.ValueOf(v.Interface()))
case reflect.Map:
return nilValue, fmt.Errorf("failed to cast to float64 from map")
case reflect.Struct:
return nilValue, fmt.Errorf("failed to cast to float64 from struct")
case reflect.Ptr:
return castFloat(v.Elem())
}
return nilValue, fmt.Errorf("failed to cast to float64 from %s", v.Type().Kind())
}
func castArray(t reflect.Type, v reflect.Value) (reflect.Value, error) {
kind := v.Type().Kind()
if kind == reflect.Interface {
return castArray(t, reflect.ValueOf(v.Interface()))
}
if kind != reflect.Slice && kind != reflect.Array {
return nilValue, fmt.Errorf("failed to cast to array from %s", kind)
}
if t.Elem() == v.Type().Elem() {
return v, nil
}
if t.Len() != v.Len() {
return nilValue, fmt.Errorf("failed to cast [%d]array from slice of %d length", t.Len(), v.Len())
}
ret := reflect.New(t).Elem()
for i := 0; i < v.Len(); i++ {
vv, err := castValue(t.Elem(), v.Index(i))
if err != nil {
return nilValue, err
}
ret.Index(i).Set(vv)
}
return ret, nil
}
func castSlice(t reflect.Type, v reflect.Value) (reflect.Value, error) {
kind := v.Type().Kind()
if kind == reflect.Interface {
return castSlice(t, reflect.ValueOf(v.Interface()))
}
if kind != reflect.Slice && kind != reflect.Array {
return nilValue, fmt.Errorf("failed to cast to slice from %s", kind)
}
if t.Elem() == v.Type().Elem() {
return v, nil
}
ret := reflect.MakeSlice(t, v.Len(), v.Len())
for i := 0; i < v.Len(); i++ {
vv, err := castValue(t.Elem(), v.Index(i))
if err != nil {
return nilValue, err
}
ret.Index(i).Set(vv)
}
return ret, nil
}
func castMap(t reflect.Type, v reflect.Value) (reflect.Value, error) {
ret := reflect.MakeMap(t)
switch v.Type().Kind() {
case reflect.Map:
iter := v.MapRange()
for iter.Next() {
key, err := castValue(t.Key(), iter.Key())
if err != nil {
return nilValue, err
}
value, err := castValue(t.Elem(), iter.Value())
if err != nil {
return nilValue, err
}
ret.SetMapIndex(key, value)
}
return ret, nil
case reflect.Interface:
return castMap(t, reflect.ValueOf(v.Interface()))
case reflect.Slice:
if v.Len() > 0 {
return castMap(t, v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to map from empty slice")
}
return nilValue, fmt.Errorf("failed to cast to map from %s", v.Type().Kind())
}
func castStruct(t reflect.Type, v reflect.Value) (reflect.Value, error) {
ret := reflect.New(t).Elem()
switch v.Type().Kind() {
case reflect.Map:
iter := v.MapRange()
for iter.Next() {
key := iter.Key()
k, err := castString(key)
if err != nil {
return nilValue, err
}
fieldName := k.String()
field, ok := t.FieldByName(fieldName)
if ok {
value, err := castValue(field.Type, iter.Value())
if err != nil {
return nilValue, err
}
ret.FieldByName(fieldName).Set(value)
}
}
return ret, nil
case reflect.Struct:
for i := 0; i < v.Type().NumField(); i++ {
name := v.Type().Field(i).Name
ret.FieldByName(name).Set(v.FieldByName(name))
}
return ret, nil
case reflect.Interface:
return castStruct(t, reflect.ValueOf(v.Interface()))
case reflect.Slice:
if v.Len() > 0 {
return castStruct(t, v.Index(0))
}
return nilValue, fmt.Errorf("failed to cast to struct from empty slice")
default:
return nilValue, fmt.Errorf("failed to cast to struct from %s", v.Type().Kind())
}
}

View File

@ -1,6 +1,7 @@
package decoder
import (
"fmt"
"unsafe"
"github.com/goccy/go-json/internal/errors"
@ -76,3 +77,7 @@ func (d *boolDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe.
}
return 0, errors.ErrUnexpectedEndOfJSON("bool", cursor)
}
func (d *boolDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: bool decoder does not support decode path")
}

View File

@ -2,6 +2,7 @@ package decoder
import (
"encoding/base64"
"fmt"
"unsafe"
"github.com/goccy/go-json/internal/errors"
@ -23,9 +24,8 @@ func byteUnmarshalerSliceDecoder(typ *runtime.Type, structName string, fieldName
unmarshalDecoder = newUnmarshalJSONDecoder(runtime.PtrTo(typ), structName, fieldName)
case runtime.PtrTo(typ).Implements(unmarshalTextType):
unmarshalDecoder = newUnmarshalTextDecoder(runtime.PtrTo(typ), structName, fieldName)
}
if unmarshalDecoder == nil {
return nil
default:
unmarshalDecoder, _ = compileUint8(typ, structName, fieldName)
}
return newSliceDecoder(unmarshalDecoder, typ, 1, structName, fieldName)
}
@ -79,6 +79,10 @@ func (d *bytesDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe
return cursor, nil
}
func (d *bytesDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: []byte decoder does not support decode path")
}
func (d *bytesDecoder) decodeStreamBinary(s *Stream, depth int64, p unsafe.Pointer) ([]byte, error) {
c := s.skipWhiteSpace()
if c == '[' {

View File

@ -9,7 +9,6 @@ import (
"unicode"
"unsafe"
"github.com/goccy/go-json/internal/errors"
"github.com/goccy/go-json/internal/runtime"
)
@ -25,7 +24,7 @@ func init() {
if typeAddr == nil {
typeAddr = &runtime.TypeAddr{}
}
cachedDecoder = make([]Decoder, typeAddr.AddrRange>>typeAddr.AddrShift)
cachedDecoder = make([]Decoder, typeAddr.AddrRange>>typeAddr.AddrShift+1)
}
func loadDecoderMap() map[uintptr]Decoder {
@ -126,13 +125,7 @@ func compile(typ *runtime.Type, structName, fieldName string, structTypeToDecode
case reflect.Func:
return compileFunc(typ, structName, fieldName)
}
return nil, &errors.UnmarshalTypeError{
Value: "object",
Type: runtime.RType2Type(typ),
Offset: 0,
Struct: structName,
Field: fieldName,
}
return newInvalidDecoder(typ, structName, fieldName), nil
}
func isStringTagSupportedType(typ *runtime.Type) bool {
@ -161,6 +154,9 @@ func compileMapKey(typ *runtime.Type, structName, fieldName string, structTypeTo
if runtime.PtrTo(typ).Implements(unmarshalTextType) {
return newUnmarshalTextDecoder(runtime.PtrTo(typ), structName, fieldName), nil
}
if typ.Kind() == reflect.String {
return newStringDecoder(structName, fieldName), nil
}
dec, err := compile(typ, structName, fieldName, structTypeToDecoder)
if err != nil {
return nil, err
@ -174,17 +170,9 @@ func compileMapKey(typ *runtime.Type, structName, fieldName string, structTypeTo
case *ptrDecoder:
dec = t.dec
default:
goto ERROR
return newInvalidDecoder(typ, structName, fieldName), nil
}
}
ERROR:
return nil, &errors.UnmarshalTypeError{
Value: "object",
Type: runtime.RType2Type(typ),
Offset: 0,
Struct: structName,
Field: fieldName,
}
}
func compilePtr(typ *runtime.Type, structName, fieldName string, structTypeToDecoder map[uintptr]Decoder) (Decoder, error) {
@ -322,64 +310,21 @@ func compileFunc(typ *runtime.Type, strutName, fieldName string) (Decoder, error
return newFuncDecoder(typ, strutName, fieldName), nil
}
func removeConflictFields(fieldMap map[string]*structFieldSet, conflictedMap map[string]struct{}, dec *structDecoder, field reflect.StructField) {
for k, v := range dec.fieldMap {
if _, exists := conflictedMap[k]; exists {
// already conflicted key
func typeToStructTags(typ *runtime.Type) runtime.StructTags {
tags := runtime.StructTags{}
fieldNum := typ.NumField()
for i := 0; i < fieldNum; i++ {
field := typ.Field(i)
if runtime.IsIgnoredStructField(field) {
continue
}
set, exists := fieldMap[k]
if !exists {
fieldSet := &structFieldSet{
dec: v.dec,
offset: field.Offset + v.offset,
isTaggedKey: v.isTaggedKey,
key: k,
keyLen: int64(len(k)),
}
fieldMap[k] = fieldSet
lower := strings.ToLower(k)
if _, exists := fieldMap[lower]; !exists {
fieldMap[lower] = fieldSet
}
continue
}
if set.isTaggedKey {
if v.isTaggedKey {
// conflict tag key
delete(fieldMap, k)
delete(fieldMap, strings.ToLower(k))
conflictedMap[k] = struct{}{}
conflictedMap[strings.ToLower(k)] = struct{}{}
}
} else {
if v.isTaggedKey {
fieldSet := &structFieldSet{
dec: v.dec,
offset: field.Offset + v.offset,
isTaggedKey: v.isTaggedKey,
key: k,
keyLen: int64(len(k)),
}
fieldMap[k] = fieldSet
lower := strings.ToLower(k)
if _, exists := fieldMap[lower]; !exists {
fieldMap[lower] = fieldSet
}
} else {
// conflict tag key
delete(fieldMap, k)
delete(fieldMap, strings.ToLower(k))
conflictedMap[k] = struct{}{}
conflictedMap[strings.ToLower(k)] = struct{}{}
}
}
tags = append(tags, runtime.StructTagFromField(field))
}
return tags
}
func compileStruct(typ *runtime.Type, structName, fieldName string, structTypeToDecoder map[uintptr]Decoder) (Decoder, error) {
fieldNum := typ.NumField()
conflictedMap := map[string]struct{}{}
fieldMap := map[string]*structFieldSet{}
typeptr := uintptr(unsafe.Pointer(typ))
if dec, exists := structTypeToDecoder[typeptr]; exists {
@ -388,6 +333,8 @@ func compileStruct(typ *runtime.Type, structName, fieldName string, structTypeTo
structDec := newStructDecoder(structName, fieldName, fieldMap)
structTypeToDecoder[typeptr] = structDec
structName = typ.Name()
tags := typeToStructTags(typ)
allFields := []*structFieldSet{}
for i := 0; i < fieldNum; i++ {
field := typ.Field(i)
if runtime.IsIgnoredStructField(field) {
@ -405,7 +352,19 @@ func compileStruct(typ *runtime.Type, structName, fieldName string, structTypeTo
// recursive definition
continue
}
removeConflictFields(fieldMap, conflictedMap, stDec, field)
for k, v := range stDec.fieldMap {
if tags.ExistsKey(k) {
continue
}
fieldSet := &structFieldSet{
dec: v.dec,
offset: field.Offset + v.offset,
isTaggedKey: v.isTaggedKey,
key: k,
keyLen: int64(len(k)),
}
allFields = append(allFields, fieldSet)
}
} else if pdec, ok := dec.(*ptrDecoder); ok {
contentDec := pdec.contentDecoder()
if pdec.typ == typ {
@ -421,12 +380,9 @@ func compileStruct(typ *runtime.Type, structName, fieldName string, structTypeTo
}
if dec, ok := contentDec.(*structDecoder); ok {
for k, v := range dec.fieldMap {
if _, exists := conflictedMap[k]; exists {
// already conflicted key
if tags.ExistsKey(k) {
continue
}
set, exists := fieldMap[k]
if !exists {
fieldSet := &structFieldSet{
dec: newAnonymousFieldDecoder(pdec.typ, v.offset, v.dec),
offset: field.Offset,
@ -435,46 +391,27 @@ func compileStruct(typ *runtime.Type, structName, fieldName string, structTypeTo
keyLen: int64(len(k)),
err: fieldSetErr,
}
fieldMap[k] = fieldSet
lower := strings.ToLower(k)
if _, exists := fieldMap[lower]; !exists {
fieldMap[lower] = fieldSet
}
continue
}
if set.isTaggedKey {
if v.isTaggedKey {
// conflict tag key
delete(fieldMap, k)
delete(fieldMap, strings.ToLower(k))
conflictedMap[k] = struct{}{}
conflictedMap[strings.ToLower(k)] = struct{}{}
allFields = append(allFields, fieldSet)
}
} else {
if v.isTaggedKey {
fieldSet := &structFieldSet{
dec: newAnonymousFieldDecoder(pdec.typ, v.offset, v.dec),
dec: pdec,
offset: field.Offset,
isTaggedKey: v.isTaggedKey,
key: k,
keyLen: int64(len(k)),
err: fieldSetErr,
isTaggedKey: tag.IsTaggedKey,
key: field.Name,
keyLen: int64(len(field.Name)),
}
fieldMap[k] = fieldSet
lower := strings.ToLower(k)
if _, exists := fieldMap[lower]; !exists {
fieldMap[lower] = fieldSet
allFields = append(allFields, fieldSet)
}
} else {
// conflict tag key
delete(fieldMap, k)
delete(fieldMap, strings.ToLower(k))
conflictedMap[k] = struct{}{}
conflictedMap[strings.ToLower(k)] = struct{}{}
}
}
}
fieldSet := &structFieldSet{
dec: dec,
offset: field.Offset,
isTaggedKey: tag.IsTaggedKey,
key: field.Name,
keyLen: int64(len(field.Name)),
}
allFields = append(allFields, fieldSet)
}
} else {
if tag.IsString && isStringTagSupportedType(runtime.Type2RType(field.Type)) {
@ -493,18 +430,58 @@ func compileStruct(typ *runtime.Type, structName, fieldName string, structTypeTo
key: key,
keyLen: int64(len(key)),
}
fieldMap[key] = fieldSet
lower := strings.ToLower(key)
if _, exists := fieldMap[lower]; !exists {
fieldMap[lower] = fieldSet
allFields = append(allFields, fieldSet)
}
}
for _, set := range filterDuplicatedFields(allFields) {
fieldMap[set.key] = set
lower := strings.ToLower(set.key)
if _, exists := fieldMap[lower]; !exists {
// first win
fieldMap[lower] = set
}
}
delete(structTypeToDecoder, typeptr)
structDec.tryOptimize()
return structDec, nil
}
func filterDuplicatedFields(allFields []*structFieldSet) []*structFieldSet {
fieldMap := map[string][]*structFieldSet{}
for _, field := range allFields {
fieldMap[field.key] = append(fieldMap[field.key], field)
}
duplicatedFieldMap := map[string]struct{}{}
for k, sets := range fieldMap {
sets = filterFieldSets(sets)
if len(sets) != 1 {
duplicatedFieldMap[k] = struct{}{}
}
}
filtered := make([]*structFieldSet, 0, len(allFields))
for _, field := range allFields {
if _, exists := duplicatedFieldMap[field.key]; exists {
continue
}
filtered = append(filtered, field)
}
return filtered
}
func filterFieldSets(sets []*structFieldSet) []*structFieldSet {
if len(sets) == 1 {
return sets
}
filtered := make([]*structFieldSet, 0, len(sets))
for _, set := range sets {
if set.isTaggedKey {
filtered = append(filtered, set)
}
}
return filtered
}
func implementsUnmarshalJSONType(typ *runtime.Type) bool {
return typ.Implements(unmarshalJSONType) || typ.Implements(unmarshalJSONContextType)
}

View File

@ -1,3 +1,4 @@
//go:build !race
// +build !race
package decoder

View File

@ -1,3 +1,4 @@
//go:build race
// +build race
package decoder

View File

@ -156,3 +156,15 @@ func (d *floatDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe
d.op(p, f64)
return cursor, nil
}
func (d *floatDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
buf := ctx.Buf
bytes, c, err := d.decodeByte(buf, cursor)
if err != nil {
return nil, 0, err
}
if bytes == nil {
return [][]byte{nullbytes}, c, nil
}
return [][]byte{bytes}, c, nil
}

View File

@ -2,6 +2,7 @@ package decoder
import (
"bytes"
"fmt"
"unsafe"
"github.com/goccy/go-json/internal/errors"
@ -139,3 +140,7 @@ func (d *funcDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe.
}
return cursor, errors.ErrInvalidBeginningOfValue(buf[cursor], cursor)
}
func (d *funcDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: func decoder does not support decode path")
}

View File

@ -192,15 +192,15 @@ func (d *intDecoder) DecodeStream(s *Stream, depth int64, p unsafe.Pointer) erro
}
switch d.kind {
case reflect.Int8:
if i64 <= -1*(1<<7) || (1<<7) <= i64 {
if i64 < -1*(1<<7) || (1<<7) <= i64 {
return d.typeError(bytes, s.totalOffset())
}
case reflect.Int16:
if i64 <= -1*(1<<15) || (1<<15) <= i64 {
if i64 < -1*(1<<15) || (1<<15) <= i64 {
return d.typeError(bytes, s.totalOffset())
}
case reflect.Int32:
if i64 <= -1*(1<<31) || (1<<31) <= i64 {
if i64 < -1*(1<<31) || (1<<31) <= i64 {
return d.typeError(bytes, s.totalOffset())
}
}
@ -225,18 +225,22 @@ func (d *intDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe.P
}
switch d.kind {
case reflect.Int8:
if i64 <= -1*(1<<7) || (1<<7) <= i64 {
if i64 < -1*(1<<7) || (1<<7) <= i64 {
return 0, d.typeError(bytes, cursor)
}
case reflect.Int16:
if i64 <= -1*(1<<15) || (1<<15) <= i64 {
if i64 < -1*(1<<15) || (1<<15) <= i64 {
return 0, d.typeError(bytes, cursor)
}
case reflect.Int32:
if i64 <= -1*(1<<31) || (1<<31) <= i64 {
if i64 < -1*(1<<31) || (1<<31) <= i64 {
return 0, d.typeError(bytes, cursor)
}
}
d.op(p, i64)
return cursor, nil
}
func (d *intDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: int decoder does not support decode path")
}

View File

@ -94,6 +94,7 @@ func (d *interfaceDecoder) numDecoder(s *Stream) Decoder {
var (
emptyInterfaceType = runtime.Type2RType(reflect.TypeOf((*interface{})(nil)).Elem())
EmptyInterfaceType = emptyInterfaceType
interfaceMapType = runtime.Type2RType(
reflect.TypeOf((*map[string]interface{})(nil)).Elem(),
)
@ -456,3 +457,72 @@ func (d *interfaceDecoder) decodeEmptyInterface(ctx *RuntimeContext, cursor, dep
}
return cursor, errors.ErrInvalidBeginningOfValue(buf[cursor], cursor)
}
func NewPathDecoder() Decoder {
ifaceDecoder := &interfaceDecoder{
typ: emptyInterfaceType,
structName: "",
fieldName: "",
floatDecoder: newFloatDecoder("", "", func(p unsafe.Pointer, v float64) {
*(*interface{})(p) = v
}),
numberDecoder: newNumberDecoder("", "", func(p unsafe.Pointer, v json.Number) {
*(*interface{})(p) = v
}),
stringDecoder: newStringDecoder("", ""),
}
ifaceDecoder.sliceDecoder = newSliceDecoder(
ifaceDecoder,
emptyInterfaceType,
emptyInterfaceType.Size(),
"", "",
)
ifaceDecoder.mapDecoder = newMapDecoder(
interfaceMapType,
stringType,
ifaceDecoder.stringDecoder,
interfaceMapType.Elem(),
ifaceDecoder,
"", "",
)
return ifaceDecoder
}
var (
truebytes = []byte("true")
falsebytes = []byte("false")
)
func (d *interfaceDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
buf := ctx.Buf
cursor = skipWhiteSpace(buf, cursor)
switch buf[cursor] {
case '{':
return d.mapDecoder.DecodePath(ctx, cursor, depth)
case '[':
return d.sliceDecoder.DecodePath(ctx, cursor, depth)
case '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
return d.floatDecoder.DecodePath(ctx, cursor, depth)
case '"':
return d.stringDecoder.DecodePath(ctx, cursor, depth)
case 't':
if err := validateTrue(buf, cursor); err != nil {
return nil, 0, err
}
cursor += 4
return [][]byte{truebytes}, cursor, nil
case 'f':
if err := validateFalse(buf, cursor); err != nil {
return nil, 0, err
}
cursor += 5
return [][]byte{falsebytes}, cursor, nil
case 'n':
if err := validateNull(buf, cursor); err != nil {
return nil, 0, err
}
cursor += 4
return [][]byte{nullbytes}, cursor, nil
}
return nil, cursor, errors.ErrInvalidBeginningOfValue(buf[cursor], cursor)
}

View File

@ -0,0 +1,55 @@
package decoder
import (
"reflect"
"unsafe"
"github.com/goccy/go-json/internal/errors"
"github.com/goccy/go-json/internal/runtime"
)
type invalidDecoder struct {
typ *runtime.Type
kind reflect.Kind
structName string
fieldName string
}
func newInvalidDecoder(typ *runtime.Type, structName, fieldName string) *invalidDecoder {
return &invalidDecoder{
typ: typ,
kind: typ.Kind(),
structName: structName,
fieldName: fieldName,
}
}
func (d *invalidDecoder) DecodeStream(s *Stream, depth int64, p unsafe.Pointer) error {
return &errors.UnmarshalTypeError{
Value: "object",
Type: runtime.RType2Type(d.typ),
Offset: s.totalOffset(),
Struct: d.structName,
Field: d.fieldName,
}
}
func (d *invalidDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe.Pointer) (int64, error) {
return 0, &errors.UnmarshalTypeError{
Value: "object",
Type: runtime.RType2Type(d.typ),
Offset: cursor,
Struct: d.structName,
Field: d.fieldName,
}
}
func (d *invalidDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, &errors.UnmarshalTypeError{
Value: "object",
Type: runtime.RType2Type(d.typ),
Offset: cursor,
Struct: d.structName,
Field: d.fieldName,
}
}

View File

@ -87,13 +87,13 @@ func (d *mapDecoder) DecodeStream(s *Stream, depth int64, p unsafe.Pointer) erro
if mapValue == nil {
mapValue = makemap(d.mapType, 0)
}
if s.buf[s.cursor+1] == '}' {
s.cursor++
if s.skipWhiteSpace() == '}' {
*(*unsafe.Pointer)(p) = mapValue
s.cursor += 2
s.cursor++
return nil
}
for {
s.cursor++
k := unsafe_New(d.keyType)
if err := d.keyDecoder.DecodeStream(s, depth, k); err != nil {
return err
@ -117,6 +117,7 @@ func (d *mapDecoder) DecodeStream(s *Stream, depth int64, p unsafe.Pointer) erro
if !s.equalChar(',') {
return errors.ErrExpected("comma after object value", s.totalOffset())
}
s.cursor++
}
}
@ -184,3 +185,96 @@ func (d *mapDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe.P
cursor++
}
}
func (d *mapDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
buf := ctx.Buf
depth++
if depth > maxDecodeNestingDepth {
return nil, 0, errors.ErrExceededMaxDepth(buf[cursor], cursor)
}
cursor = skipWhiteSpace(buf, cursor)
buflen := int64(len(buf))
if buflen < 2 {
return nil, 0, errors.ErrExpected("{} for map", cursor)
}
switch buf[cursor] {
case 'n':
if err := validateNull(buf, cursor); err != nil {
return nil, 0, err
}
cursor += 4
return [][]byte{nullbytes}, cursor, nil
case '{':
default:
return nil, 0, errors.ErrExpected("{ character for map value", cursor)
}
cursor++
cursor = skipWhiteSpace(buf, cursor)
if buf[cursor] == '}' {
cursor++
return nil, cursor, nil
}
keyDecoder, ok := d.keyDecoder.(*stringDecoder)
if !ok {
return nil, 0, &errors.UnmarshalTypeError{
Value: "string",
Type: reflect.TypeOf(""),
Offset: cursor,
Struct: d.structName,
Field: d.fieldName,
}
}
ret := [][]byte{}
for {
key, keyCursor, err := keyDecoder.decodeByte(buf, cursor)
if err != nil {
return nil, 0, err
}
cursor = skipWhiteSpace(buf, keyCursor)
if buf[cursor] != ':' {
return nil, 0, errors.ErrExpected("colon after object key", cursor)
}
cursor++
child, found, err := ctx.Option.Path.Field(string(key))
if err != nil {
return nil, 0, err
}
if found {
if child != nil {
oldPath := ctx.Option.Path.node
ctx.Option.Path.node = child
paths, c, err := d.valueDecoder.DecodePath(ctx, cursor, depth)
if err != nil {
return nil, 0, err
}
ctx.Option.Path.node = oldPath
ret = append(ret, paths...)
cursor = c
} else {
start := cursor
end, err := skipValue(buf, cursor, depth)
if err != nil {
return nil, 0, err
}
ret = append(ret, buf[start:end])
cursor = end
}
} else {
c, err := skipValue(buf, cursor, depth)
if err != nil {
return nil, 0, err
}
cursor = c
}
cursor = skipWhiteSpace(buf, cursor)
if buf[cursor] == '}' {
cursor++
return ret, cursor, nil
}
if buf[cursor] != ',' {
return nil, 0, errors.ErrExpected("comma after object value", cursor)
}
cursor++
}
}

View File

@ -51,6 +51,17 @@ func (d *numberDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsaf
return cursor, nil
}
func (d *numberDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
bytes, c, err := d.decodeByte(ctx.Buf, cursor)
if err != nil {
return nil, 0, err
}
if bytes == nil {
return [][]byte{nullbytes}, c, nil
}
return [][]byte{bytes}, c, nil
}
func (d *numberDecoder) decodeStreamByte(s *Stream) ([]byte, error) {
start := s.cursor
for {

View File

@ -7,9 +7,11 @@ type OptionFlags uint8
const (
FirstWinOption OptionFlags = 1 << iota
ContextOption
PathOption
)
type Option struct {
Flags OptionFlags
Context context.Context
Path *Path
}

View File

@ -0,0 +1,670 @@
package decoder
import (
"fmt"
"reflect"
"strconv"
"github.com/goccy/go-json/internal/errors"
"github.com/goccy/go-json/internal/runtime"
)
type PathString string
func (s PathString) Build() (*Path, error) {
builder := new(PathBuilder)
return builder.Build([]rune(s))
}
type PathBuilder struct {
root PathNode
node PathNode
singleQuotePathSelector bool
doubleQuotePathSelector bool
}
func (b *PathBuilder) Build(buf []rune) (*Path, error) {
node, err := b.build(buf)
if err != nil {
return nil, err
}
return &Path{
node: node,
RootSelectorOnly: node == nil,
SingleQuotePathSelector: b.singleQuotePathSelector,
DoubleQuotePathSelector: b.doubleQuotePathSelector,
}, nil
}
func (b *PathBuilder) build(buf []rune) (PathNode, error) {
if len(buf) == 0 {
return nil, errors.ErrEmptyPath()
}
if buf[0] != '$' {
return nil, errors.ErrInvalidPath("JSON Path must start with a $ character")
}
if len(buf) == 1 {
return nil, nil
}
buf = buf[1:]
offset, err := b.buildNext(buf)
if err != nil {
return nil, err
}
if len(buf) > offset {
return nil, errors.ErrInvalidPath("remain invalid path %q", buf[offset:])
}
return b.root, nil
}
func (b *PathBuilder) buildNextCharIfExists(buf []rune, cursor int) (int, error) {
if len(buf) > cursor {
offset, err := b.buildNext(buf[cursor:])
if err != nil {
return 0, err
}
return cursor + 1 + offset, nil
}
return cursor, nil
}
func (b *PathBuilder) buildNext(buf []rune) (int, error) {
switch buf[0] {
case '.':
if len(buf) == 1 {
return 0, errors.ErrInvalidPath("JSON Path ends with dot character")
}
offset, err := b.buildSelector(buf[1:])
if err != nil {
return 0, err
}
return offset + 1, nil
case '[':
if len(buf) == 1 {
return 0, errors.ErrInvalidPath("JSON Path ends with left bracket character")
}
offset, err := b.buildIndex(buf[1:])
if err != nil {
return 0, err
}
return offset + 1, nil
default:
return 0, errors.ErrInvalidPath("expect dot or left bracket character. but found %c character", buf[0])
}
}
func (b *PathBuilder) buildSelector(buf []rune) (int, error) {
switch buf[0] {
case '.':
if len(buf) == 1 {
return 0, errors.ErrInvalidPath("JSON Path ends with double dot character")
}
offset, err := b.buildPathRecursive(buf[1:])
if err != nil {
return 0, err
}
return 1 + offset, nil
case '[', ']', '$', '*':
return 0, errors.ErrInvalidPath("found invalid path character %c after dot", buf[0])
}
for cursor := 0; cursor < len(buf); cursor++ {
switch buf[cursor] {
case '$', '*', ']':
return 0, errors.ErrInvalidPath("found %c character in field selector context", buf[cursor])
case '.':
if cursor+1 >= len(buf) {
return 0, errors.ErrInvalidPath("JSON Path ends with dot character")
}
selector := buf[:cursor]
b.addSelectorNode(string(selector))
offset, err := b.buildSelector(buf[cursor+1:])
if err != nil {
return 0, err
}
return cursor + 1 + offset, nil
case '[':
if cursor+1 >= len(buf) {
return 0, errors.ErrInvalidPath("JSON Path ends with left bracket character")
}
selector := buf[:cursor]
b.addSelectorNode(string(selector))
offset, err := b.buildIndex(buf[cursor+1:])
if err != nil {
return 0, err
}
return cursor + 1 + offset, nil
case '"':
if cursor+1 >= len(buf) {
return 0, errors.ErrInvalidPath("JSON Path ends with double quote character")
}
offset, err := b.buildQuoteSelector(buf[cursor+1:], DoubleQuotePathSelector)
if err != nil {
return 0, err
}
return cursor + 1 + offset, nil
}
}
b.addSelectorNode(string(buf))
return len(buf), nil
}
func (b *PathBuilder) buildQuoteSelector(buf []rune, sel QuotePathSelector) (int, error) {
switch buf[0] {
case '[', ']', '$', '.', '*', '\'', '"':
return 0, errors.ErrInvalidPath("found invalid path character %c after quote", buf[0])
}
for cursor := 0; cursor < len(buf); cursor++ {
switch buf[cursor] {
case '\'':
if sel != SingleQuotePathSelector {
return 0, errors.ErrInvalidPath("found double quote character in field selector with single quote context")
}
if len(buf) <= cursor+1 {
return 0, errors.ErrInvalidPath("JSON Path ends with single quote character in field selector context")
}
if buf[cursor+1] != ']' {
return 0, errors.ErrInvalidPath("expect right bracket for field selector with single quote but found %c", buf[cursor+1])
}
selector := buf[:cursor]
b.addSelectorNode(string(selector))
b.singleQuotePathSelector = true
return b.buildNextCharIfExists(buf, cursor+2)
case '"':
if sel != DoubleQuotePathSelector {
return 0, errors.ErrInvalidPath("found single quote character in field selector with double quote context")
}
selector := buf[:cursor]
b.addSelectorNode(string(selector))
b.doubleQuotePathSelector = true
return b.buildNextCharIfExists(buf, cursor+1)
}
}
return 0, errors.ErrInvalidPath("couldn't find quote character in selector quote path context")
}
func (b *PathBuilder) buildPathRecursive(buf []rune) (int, error) {
switch buf[0] {
case '.', '[', ']', '$', '*':
return 0, errors.ErrInvalidPath("found invalid path character %c after double dot", buf[0])
}
for cursor := 0; cursor < len(buf); cursor++ {
switch buf[cursor] {
case '$', '*', ']':
return 0, errors.ErrInvalidPath("found %c character in field selector context", buf[cursor])
case '.':
if cursor+1 >= len(buf) {
return 0, errors.ErrInvalidPath("JSON Path ends with dot character")
}
selector := buf[:cursor]
b.addRecursiveNode(string(selector))
offset, err := b.buildSelector(buf[cursor+1:])
if err != nil {
return 0, err
}
return cursor + 1 + offset, nil
case '[':
if cursor+1 >= len(buf) {
return 0, errors.ErrInvalidPath("JSON Path ends with left bracket character")
}
selector := buf[:cursor]
b.addRecursiveNode(string(selector))
offset, err := b.buildIndex(buf[cursor+1:])
if err != nil {
return 0, err
}
return cursor + 1 + offset, nil
}
}
b.addRecursiveNode(string(buf))
return len(buf), nil
}
func (b *PathBuilder) buildIndex(buf []rune) (int, error) {
switch buf[0] {
case '.', '[', ']', '$':
return 0, errors.ErrInvalidPath("found invalid path character %c after left bracket", buf[0])
case '\'':
if len(buf) == 1 {
return 0, errors.ErrInvalidPath("JSON Path ends with single quote character")
}
offset, err := b.buildQuoteSelector(buf[1:], SingleQuotePathSelector)
if err != nil {
return 0, err
}
return 1 + offset, nil
case '*':
if len(buf) == 1 {
return 0, errors.ErrInvalidPath("JSON Path ends with star character")
}
if buf[1] != ']' {
return 0, errors.ErrInvalidPath("expect right bracket character for index all path but found %c character", buf[1])
}
b.addIndexAllNode()
offset := len("*]")
if len(buf) > 2 {
buildOffset, err := b.buildNext(buf[2:])
if err != nil {
return 0, err
}
return offset + buildOffset, nil
}
return offset, nil
}
for cursor := 0; cursor < len(buf); cursor++ {
switch buf[cursor] {
case ']':
index, err := strconv.ParseInt(string(buf[:cursor]), 10, 64)
if err != nil {
return 0, errors.ErrInvalidPath("%q is unexpected index path", buf[:cursor])
}
b.addIndexNode(int(index))
return b.buildNextCharIfExists(buf, cursor+1)
}
}
return 0, errors.ErrInvalidPath("couldn't find right bracket character in index path context")
}
func (b *PathBuilder) addIndexAllNode() {
node := newPathIndexAllNode()
if b.root == nil {
b.root = node
b.node = node
} else {
b.node = b.node.chain(node)
}
}
func (b *PathBuilder) addRecursiveNode(selector string) {
node := newPathRecursiveNode(selector)
if b.root == nil {
b.root = node
b.node = node
} else {
b.node = b.node.chain(node)
}
}
func (b *PathBuilder) addSelectorNode(name string) {
node := newPathSelectorNode(name)
if b.root == nil {
b.root = node
b.node = node
} else {
b.node = b.node.chain(node)
}
}
func (b *PathBuilder) addIndexNode(idx int) {
node := newPathIndexNode(idx)
if b.root == nil {
b.root = node
b.node = node
} else {
b.node = b.node.chain(node)
}
}
type QuotePathSelector int
const (
SingleQuotePathSelector QuotePathSelector = 1
DoubleQuotePathSelector QuotePathSelector = 2
)
type Path struct {
node PathNode
RootSelectorOnly bool
SingleQuotePathSelector bool
DoubleQuotePathSelector bool
}
func (p *Path) Field(sel string) (PathNode, bool, error) {
if p.node == nil {
return nil, false, nil
}
return p.node.Field(sel)
}
func (p *Path) Get(src, dst reflect.Value) error {
if p.node == nil {
return nil
}
return p.node.Get(src, dst)
}
func (p *Path) String() string {
if p.node == nil {
return "$"
}
return p.node.String()
}
type PathNode interface {
fmt.Stringer
Index(idx int) (PathNode, bool, error)
Field(fieldName string) (PathNode, bool, error)
Get(src, dst reflect.Value) error
chain(PathNode) PathNode
target() bool
single() bool
}
type BasePathNode struct {
child PathNode
}
func (n *BasePathNode) chain(node PathNode) PathNode {
n.child = node
return node
}
func (n *BasePathNode) target() bool {
return n.child == nil
}
func (n *BasePathNode) single() bool {
return true
}
type PathSelectorNode struct {
*BasePathNode
selector string
}
func newPathSelectorNode(selector string) *PathSelectorNode {
return &PathSelectorNode{
BasePathNode: &BasePathNode{},
selector: selector,
}
}
func (n *PathSelectorNode) Index(idx int) (PathNode, bool, error) {
return nil, false, &errors.PathError{}
}
func (n *PathSelectorNode) Field(fieldName string) (PathNode, bool, error) {
if n.selector == fieldName {
return n.child, true, nil
}
return nil, false, nil
}
func (n *PathSelectorNode) Get(src, dst reflect.Value) error {
switch src.Type().Kind() {
case reflect.Map:
iter := src.MapRange()
for iter.Next() {
key, ok := iter.Key().Interface().(string)
if !ok {
return fmt.Errorf("invalid map key type %T", src.Type().Key())
}
child, found, err := n.Field(key)
if err != nil {
return err
}
if found {
if child != nil {
return child.Get(iter.Value(), dst)
}
return AssignValue(iter.Value(), dst)
}
}
case reflect.Struct:
typ := src.Type()
for i := 0; i < typ.Len(); i++ {
tag := runtime.StructTagFromField(typ.Field(i))
child, found, err := n.Field(tag.Key)
if err != nil {
return err
}
if found {
if child != nil {
return child.Get(src.Field(i), dst)
}
return AssignValue(src.Field(i), dst)
}
}
case reflect.Ptr:
return n.Get(src.Elem(), dst)
case reflect.Interface:
return n.Get(reflect.ValueOf(src.Interface()), dst)
case reflect.Float64, reflect.String, reflect.Bool:
return AssignValue(src, dst)
}
return fmt.Errorf("failed to get %s value from %s", n.selector, src.Type())
}
func (n *PathSelectorNode) String() string {
s := fmt.Sprintf(".%s", n.selector)
if n.child != nil {
s += n.child.String()
}
return s
}
type PathIndexNode struct {
*BasePathNode
selector int
}
func newPathIndexNode(selector int) *PathIndexNode {
return &PathIndexNode{
BasePathNode: &BasePathNode{},
selector: selector,
}
}
func (n *PathIndexNode) Index(idx int) (PathNode, bool, error) {
if n.selector == idx {
return n.child, true, nil
}
return nil, false, nil
}
func (n *PathIndexNode) Field(fieldName string) (PathNode, bool, error) {
return nil, false, &errors.PathError{}
}
func (n *PathIndexNode) Get(src, dst reflect.Value) error {
switch src.Type().Kind() {
case reflect.Array, reflect.Slice:
if src.Len() > n.selector {
if n.child != nil {
return n.child.Get(src.Index(n.selector), dst)
}
return AssignValue(src.Index(n.selector), dst)
}
case reflect.Ptr:
return n.Get(src.Elem(), dst)
case reflect.Interface:
return n.Get(reflect.ValueOf(src.Interface()), dst)
}
return fmt.Errorf("failed to get [%d] value from %s", n.selector, src.Type())
}
func (n *PathIndexNode) String() string {
s := fmt.Sprintf("[%d]", n.selector)
if n.child != nil {
s += n.child.String()
}
return s
}
type PathIndexAllNode struct {
*BasePathNode
}
func newPathIndexAllNode() *PathIndexAllNode {
return &PathIndexAllNode{
BasePathNode: &BasePathNode{},
}
}
func (n *PathIndexAllNode) Index(idx int) (PathNode, bool, error) {
return n.child, true, nil
}
func (n *PathIndexAllNode) Field(fieldName string) (PathNode, bool, error) {
return nil, false, &errors.PathError{}
}
func (n *PathIndexAllNode) Get(src, dst reflect.Value) error {
switch src.Type().Kind() {
case reflect.Array, reflect.Slice:
var arr []interface{}
for i := 0; i < src.Len(); i++ {
var v interface{}
rv := reflect.ValueOf(&v)
if n.child != nil {
if err := n.child.Get(src.Index(i), rv); err != nil {
return err
}
} else {
if err := AssignValue(src.Index(i), rv); err != nil {
return err
}
}
arr = append(arr, v)
}
if err := AssignValue(reflect.ValueOf(arr), dst); err != nil {
return err
}
return nil
case reflect.Ptr:
return n.Get(src.Elem(), dst)
case reflect.Interface:
return n.Get(reflect.ValueOf(src.Interface()), dst)
}
return fmt.Errorf("failed to get all value from %s", src.Type())
}
func (n *PathIndexAllNode) String() string {
s := "[*]"
if n.child != nil {
s += n.child.String()
}
return s
}
type PathRecursiveNode struct {
*BasePathNode
selector string
}
func newPathRecursiveNode(selector string) *PathRecursiveNode {
node := newPathSelectorNode(selector)
return &PathRecursiveNode{
BasePathNode: &BasePathNode{
child: node,
},
selector: selector,
}
}
func (n *PathRecursiveNode) Field(fieldName string) (PathNode, bool, error) {
if n.selector == fieldName {
return n.child, true, nil
}
return nil, false, nil
}
func (n *PathRecursiveNode) Index(_ int) (PathNode, bool, error) {
return n, true, nil
}
func valueToSliceValue(v interface{}) []interface{} {
rv := reflect.ValueOf(v)
ret := []interface{}{}
if rv.Type().Kind() == reflect.Slice || rv.Type().Kind() == reflect.Array {
for i := 0; i < rv.Len(); i++ {
ret = append(ret, rv.Index(i).Interface())
}
return ret
}
return []interface{}{v}
}
func (n *PathRecursiveNode) Get(src, dst reflect.Value) error {
if n.child == nil {
return fmt.Errorf("failed to get by recursive path ..%s", n.selector)
}
var arr []interface{}
switch src.Type().Kind() {
case reflect.Map:
iter := src.MapRange()
for iter.Next() {
key, ok := iter.Key().Interface().(string)
if !ok {
return fmt.Errorf("invalid map key type %T", src.Type().Key())
}
child, found, err := n.Field(key)
if err != nil {
return err
}
if found {
var v interface{}
rv := reflect.ValueOf(&v)
_ = child.Get(iter.Value(), rv)
arr = append(arr, valueToSliceValue(v)...)
} else {
var v interface{}
rv := reflect.ValueOf(&v)
_ = n.Get(iter.Value(), rv)
if v != nil {
arr = append(arr, valueToSliceValue(v)...)
}
}
}
_ = AssignValue(reflect.ValueOf(arr), dst)
return nil
case reflect.Struct:
typ := src.Type()
for i := 0; i < typ.Len(); i++ {
tag := runtime.StructTagFromField(typ.Field(i))
child, found, err := n.Field(tag.Key)
if err != nil {
return err
}
if found {
var v interface{}
rv := reflect.ValueOf(&v)
_ = child.Get(src.Field(i), rv)
arr = append(arr, valueToSliceValue(v)...)
} else {
var v interface{}
rv := reflect.ValueOf(&v)
_ = n.Get(src.Field(i), rv)
if v != nil {
arr = append(arr, valueToSliceValue(v)...)
}
}
}
_ = AssignValue(reflect.ValueOf(arr), dst)
return nil
case reflect.Array, reflect.Slice:
for i := 0; i < src.Len(); i++ {
var v interface{}
rv := reflect.ValueOf(&v)
_ = n.Get(src.Index(i), rv)
if v != nil {
arr = append(arr, valueToSliceValue(v)...)
}
}
_ = AssignValue(reflect.ValueOf(arr), dst)
return nil
case reflect.Ptr:
return n.Get(src.Elem(), dst)
case reflect.Interface:
return n.Get(reflect.ValueOf(src.Interface()), dst)
}
return fmt.Errorf("failed to get %s value from %s", n.selector, src.Type())
}
func (n *PathRecursiveNode) String() string {
s := fmt.Sprintf("..%s", n.selector)
if n.child != nil {
s += n.child.String()
}
return s
}

View File

@ -1,6 +1,7 @@
package decoder
import (
"fmt"
"unsafe"
"github.com/goccy/go-json/internal/runtime"
@ -34,6 +35,10 @@ func (d *ptrDecoder) contentDecoder() Decoder {
//go:linkname unsafe_New reflect.unsafe_New
func unsafe_New(*runtime.Type) unsafe.Pointer
func UnsafeNew(t *runtime.Type) unsafe.Pointer {
return unsafe_New(t)
}
func (d *ptrDecoder) DecodeStream(s *Stream, depth int64, p unsafe.Pointer) error {
if s.skipWhiteSpace() == nul {
s.read()
@ -80,8 +85,13 @@ func (d *ptrDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe.P
}
c, err := d.dec.Decode(ctx, cursor, depth, newptr)
if err != nil {
*(*unsafe.Pointer)(p) = nil
return 0, err
}
cursor = c
return cursor, nil
}
func (d *ptrDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: ptr decoder does not support decode path")
}

View File

@ -299,3 +299,82 @@ func (d *sliceDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe
}
}
}
func (d *sliceDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
buf := ctx.Buf
depth++
if depth > maxDecodeNestingDepth {
return nil, 0, errors.ErrExceededMaxDepth(buf[cursor], cursor)
}
ret := [][]byte{}
for {
switch buf[cursor] {
case ' ', '\n', '\t', '\r':
cursor++
continue
case 'n':
if err := validateNull(buf, cursor); err != nil {
return nil, 0, err
}
cursor += 4
return [][]byte{nullbytes}, cursor, nil
case '[':
cursor++
cursor = skipWhiteSpace(buf, cursor)
if buf[cursor] == ']' {
cursor++
return ret, cursor, nil
}
idx := 0
for {
child, found, err := ctx.Option.Path.node.Index(idx)
if err != nil {
return nil, 0, err
}
if found {
if child != nil {
oldPath := ctx.Option.Path.node
ctx.Option.Path.node = child
paths, c, err := d.valueDecoder.DecodePath(ctx, cursor, depth)
if err != nil {
return nil, 0, err
}
ctx.Option.Path.node = oldPath
ret = append(ret, paths...)
cursor = c
} else {
start := cursor
end, err := skipValue(buf, cursor, depth)
if err != nil {
return nil, 0, err
}
ret = append(ret, buf[start:end])
cursor = end
}
} else {
c, err := skipValue(buf, cursor, depth)
if err != nil {
return nil, 0, err
}
cursor = c
}
cursor = skipWhiteSpace(buf, cursor)
switch buf[cursor] {
case ']':
cursor++
return ret, cursor, nil
case ',':
idx++
default:
return nil, 0, errors.ErrInvalidCharacter(buf[cursor], "slice", cursor)
}
cursor++
}
case '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
return nil, 0, d.errNumber(cursor)
default:
return nil, 0, errors.ErrUnexpectedEndOfJSON("slice", cursor)
}
}
}

View File

@ -103,7 +103,7 @@ func (s *Stream) statForRetry() ([]byte, int64, unsafe.Pointer) {
func (s *Stream) Reset() {
s.reset()
s.bufSize = initBufSize
s.bufSize = int64(len(s.buf))
}
func (s *Stream) More() bool {
@ -138,8 +138,11 @@ func (s *Stream) Token() (interface{}, error) {
s.cursor++
case '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
bytes := floatBytes(s)
s := *(*string)(unsafe.Pointer(&bytes))
f64, err := strconv.ParseFloat(s, 64)
str := *(*string)(unsafe.Pointer(&bytes))
if s.UseNumber {
return json.Number(str), nil
}
f64, err := strconv.ParseFloat(str, 64)
if err != nil {
return nil, err
}
@ -277,7 +280,7 @@ func (s *Stream) skipObject(depth int64) error {
if char(p, cursor) == nul {
s.cursor = cursor
if s.read() {
_, cursor, p = s.statForRetry()
_, cursor, p = s.stat()
continue
}
return errors.ErrUnexpectedEndOfJSON("string of object", cursor)
@ -340,7 +343,7 @@ func (s *Stream) skipArray(depth int64) error {
if char(p, cursor) == nul {
s.cursor = cursor
if s.read() {
_, cursor, p = s.statForRetry()
_, cursor, p = s.stat()
continue
}
return errors.ErrUnexpectedEndOfJSON("string of object", cursor)
@ -398,7 +401,7 @@ func (s *Stream) skipValue(depth int64) error {
if char(p, cursor) == nul {
s.cursor = cursor
if s.read() {
_, cursor, p = s.statForRetry()
_, cursor, p = s.stat()
continue
}
return errors.ErrUnexpectedEndOfJSON("value of string", s.totalOffset())
@ -423,7 +426,6 @@ func (s *Stream) skipValue(depth int64) error {
continue
} else if c == nul {
if s.read() {
s.cursor-- // for retry current character
_, cursor, p = s.stat()
continue
}

View File

@ -1,6 +1,8 @@
package decoder
import (
"bytes"
"fmt"
"reflect"
"unicode"
"unicode/utf16"
@ -58,6 +60,17 @@ func (d *stringDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsaf
return cursor, nil
}
func (d *stringDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
bytes, c, err := d.decodeByte(ctx.Buf, cursor)
if err != nil {
return nil, 0, err
}
if bytes == nil {
return [][]byte{nullbytes}, c, nil
}
return [][]byte{bytes}, c, nil
}
var (
hexToInt = [256]int{
'0': 0,
@ -93,24 +106,30 @@ func unicodeToRune(code []byte) rune {
return r
}
func readAtLeast(s *Stream, n int64, p *unsafe.Pointer) bool {
for s.cursor+n >= s.length {
if !s.read() {
return false
}
*p = s.bufptr()
}
return true
}
func decodeUnicodeRune(s *Stream, p unsafe.Pointer) (rune, int64, unsafe.Pointer, error) {
const defaultOffset = 5
const surrogateOffset = 11
if s.cursor+defaultOffset >= s.length {
if !s.read() {
if !readAtLeast(s, defaultOffset, &p) {
return rune(0), 0, nil, errors.ErrInvalidCharacter(s.char(), "escaped string", s.totalOffset())
}
p = s.bufptr()
}
r := unicodeToRune(s.buf[s.cursor+1 : s.cursor+defaultOffset])
if utf16.IsSurrogate(r) {
if s.cursor+surrogateOffset >= s.length {
s.read()
p = s.bufptr()
if !readAtLeast(s, surrogateOffset, &p) {
return unicode.ReplacementChar, defaultOffset, p, nil
}
if s.cursor+surrogateOffset >= s.length || s.buf[s.cursor+defaultOffset] != '\\' || s.buf[s.cursor+defaultOffset+1] != 'u' {
if s.buf[s.cursor+defaultOffset] != '\\' || s.buf[s.cursor+defaultOffset+1] != 'u' {
return unicode.ReplacementChar, defaultOffset, p, nil
}
r2 := unicodeToRune(s.buf[s.cursor+defaultOffset+2 : s.cursor+surrogateOffset])
@ -163,6 +182,7 @@ RETRY:
if !s.read() {
return nil, errors.ErrInvalidCharacter(s.char(), "escaped string", s.totalOffset())
}
p = s.bufptr()
goto RETRY
default:
return nil, errors.ErrUnexpectedEndOfJSON("string", s.totalOffset())
@ -308,49 +328,36 @@ func (d *stringDecoder) decodeByte(buf []byte, cursor int64) ([]byte, int64, err
cursor++
start := cursor
b := (*sliceHeader)(unsafe.Pointer(&buf)).data
escaped := 0
for {
switch char(b, cursor) {
case '\\':
escaped++
cursor++
switch char(b, cursor) {
case '"':
buf[cursor] = '"'
buf = append(buf[:cursor-1], buf[cursor:]...)
case '\\':
buf[cursor] = '\\'
buf = append(buf[:cursor-1], buf[cursor:]...)
case '/':
buf[cursor] = '/'
buf = append(buf[:cursor-1], buf[cursor:]...)
case 'b':
buf[cursor] = '\b'
buf = append(buf[:cursor-1], buf[cursor:]...)
case 'f':
buf[cursor] = '\f'
buf = append(buf[:cursor-1], buf[cursor:]...)
case 'n':
buf[cursor] = '\n'
buf = append(buf[:cursor-1], buf[cursor:]...)
case 'r':
buf[cursor] = '\r'
buf = append(buf[:cursor-1], buf[cursor:]...)
case 't':
buf[cursor] = '\t'
buf = append(buf[:cursor-1], buf[cursor:]...)
case '"', '\\', '/', 'b', 'f', 'n', 'r', 't':
cursor++
case 'u':
buflen := int64(len(buf))
if cursor+5 >= buflen {
return nil, 0, errors.ErrUnexpectedEndOfJSON("escaped string", cursor)
}
code := unicodeToRune(buf[cursor+1 : cursor+5])
unicode := []byte(string(code))
buf = append(append(buf[:cursor-1], unicode...), buf[cursor+5:]...)
for i := int64(1); i <= 4; i++ {
c := char(b, cursor+i)
if !(('0' <= c && c <= '9') || ('a' <= c && c <= 'f') || ('A' <= c && c <= 'F')) {
return nil, 0, errors.ErrSyntax(fmt.Sprintf("json: invalid character %c in \\u hexadecimal character escape", c), cursor+i)
}
}
cursor += 5
default:
return nil, 0, errors.ErrUnexpectedEndOfJSON("escaped string", cursor)
}
continue
case '"':
literal := buf[start:cursor]
if escaped > 0 {
literal = literal[:unescapeString(literal)]
}
cursor++
return literal, cursor, nil
case nul:
@ -369,3 +376,77 @@ func (d *stringDecoder) decodeByte(buf []byte, cursor int64) ([]byte, int64, err
}
}
}
var unescapeMap = [256]byte{
'"': '"',
'\\': '\\',
'/': '/',
'b': '\b',
'f': '\f',
'n': '\n',
'r': '\r',
't': '\t',
}
func unsafeAdd(ptr unsafe.Pointer, offset int) unsafe.Pointer {
return unsafe.Pointer(uintptr(ptr) + uintptr(offset))
}
func unescapeString(buf []byte) int {
p := (*sliceHeader)(unsafe.Pointer(&buf)).data
end := unsafeAdd(p, len(buf))
src := unsafeAdd(p, bytes.IndexByte(buf, '\\'))
dst := src
for src != end {
c := char(src, 0)
if c == '\\' {
escapeChar := char(src, 1)
if escapeChar != 'u' {
*(*byte)(dst) = unescapeMap[escapeChar]
src = unsafeAdd(src, 2)
dst = unsafeAdd(dst, 1)
} else {
v1 := hexToInt[char(src, 2)]
v2 := hexToInt[char(src, 3)]
v3 := hexToInt[char(src, 4)]
v4 := hexToInt[char(src, 5)]
code := rune((v1 << 12) | (v2 << 8) | (v3 << 4) | v4)
if code >= 0xd800 && code < 0xdc00 && uintptr(unsafeAdd(src, 11)) < uintptr(end) {
if char(src, 6) == '\\' && char(src, 7) == 'u' {
v1 := hexToInt[char(src, 8)]
v2 := hexToInt[char(src, 9)]
v3 := hexToInt[char(src, 10)]
v4 := hexToInt[char(src, 11)]
lo := rune((v1 << 12) | (v2 << 8) | (v3 << 4) | v4)
if lo >= 0xdc00 && lo < 0xe000 {
code = (code-0xd800)<<10 | (lo - 0xdc00) + 0x10000
src = unsafeAdd(src, 6)
}
}
}
var b [utf8.UTFMax]byte
n := utf8.EncodeRune(b[:], code)
switch n {
case 4:
*(*byte)(unsafeAdd(dst, 3)) = b[3]
fallthrough
case 3:
*(*byte)(unsafeAdd(dst, 2)) = b[2]
fallthrough
case 2:
*(*byte)(unsafeAdd(dst, 1)) = b[1]
fallthrough
case 1:
*(*byte)(unsafeAdd(dst, 0)) = b[0]
}
src = unsafeAdd(src, 6)
dst = unsafeAdd(dst, n)
}
} else {
*(*byte)(dst) = c
src = unsafeAdd(src, 1)
dst = unsafeAdd(dst, 1)
}
}
return int(uintptr(dst) - uintptr(p))
}

View File

@ -51,6 +51,14 @@ func init() {
}
}
func toASCIILower(s string) string {
b := []byte(s)
for i := range b {
b[i] = largeToSmallTable[b[i]]
}
return string(b)
}
func newStructDecoder(structName, fieldName string, fieldMap map[string]*structFieldSet) *structDecoder {
return &structDecoder{
fieldMap: fieldMap,
@ -91,6 +99,10 @@ func (d *structDecoder) tryOptimize() {
for k, v := range d.fieldMap {
key := strings.ToLower(k)
if key != k {
if key != toASCIILower(k) {
d.isTriedOptimize = true
return
}
// already exists same key (e.g. Hello and HELLO has same lower case key
if _, exists := conflicted[key]; exists {
d.isTriedOptimize = true
@ -158,49 +170,53 @@ func (d *structDecoder) tryOptimize() {
}
// decode from '\uXXXX'
func decodeKeyCharByUnicodeRune(buf []byte, cursor int64) ([]byte, int64) {
func decodeKeyCharByUnicodeRune(buf []byte, cursor int64) ([]byte, int64, error) {
const defaultOffset = 4
const surrogateOffset = 6
if cursor+defaultOffset >= int64(len(buf)) {
return nil, 0, errors.ErrUnexpectedEndOfJSON("escaped string", cursor)
}
r := unicodeToRune(buf[cursor : cursor+defaultOffset])
if utf16.IsSurrogate(r) {
cursor += defaultOffset
if cursor+surrogateOffset >= int64(len(buf)) || buf[cursor] != '\\' || buf[cursor+1] != 'u' {
return []byte(string(unicode.ReplacementChar)), cursor + defaultOffset - 1
return []byte(string(unicode.ReplacementChar)), cursor + defaultOffset - 1, nil
}
cursor += 2
r2 := unicodeToRune(buf[cursor : cursor+defaultOffset])
if r := utf16.DecodeRune(r, r2); r != unicode.ReplacementChar {
return []byte(string(r)), cursor + defaultOffset - 1
return []byte(string(r)), cursor + defaultOffset - 1, nil
}
}
return []byte(string(r)), cursor + defaultOffset - 1
return []byte(string(r)), cursor + defaultOffset - 1, nil
}
func decodeKeyCharByEscapedChar(buf []byte, cursor int64) ([]byte, int64) {
func decodeKeyCharByEscapedChar(buf []byte, cursor int64) ([]byte, int64, error) {
c := buf[cursor]
cursor++
switch c {
case '"':
return []byte{'"'}, cursor
return []byte{'"'}, cursor, nil
case '\\':
return []byte{'\\'}, cursor
return []byte{'\\'}, cursor, nil
case '/':
return []byte{'/'}, cursor
return []byte{'/'}, cursor, nil
case 'b':
return []byte{'\b'}, cursor
return []byte{'\b'}, cursor, nil
case 'f':
return []byte{'\f'}, cursor
return []byte{'\f'}, cursor, nil
case 'n':
return []byte{'\n'}, cursor
return []byte{'\n'}, cursor, nil
case 'r':
return []byte{'\r'}, cursor
return []byte{'\r'}, cursor, nil
case 't':
return []byte{'\t'}, cursor
return []byte{'\t'}, cursor, nil
case 'u':
return decodeKeyCharByUnicodeRune(buf, cursor)
}
return nil, cursor
return nil, cursor, nil
}
func decodeKeyByBitmapUint8(d *structDecoder, buf []byte, cursor int64) (int64, *structFieldSet, error) {
@ -242,7 +258,10 @@ func decodeKeyByBitmapUint8(d *structDecoder, buf []byte, cursor int64) (int64,
return 0, nil, errors.ErrUnexpectedEndOfJSON("string", cursor)
case '\\':
cursor++
chars, nextCursor := decodeKeyCharByEscapedChar(buf, cursor)
chars, nextCursor, err := decodeKeyCharByEscapedChar(buf, cursor)
if err != nil {
return 0, nil, err
}
for _, c := range chars {
curBit &= bitmap[keyIdx][largeToSmallTable[c]]
if curBit == 0 {
@ -305,7 +324,10 @@ func decodeKeyByBitmapUint16(d *structDecoder, buf []byte, cursor int64) (int64,
return 0, nil, errors.ErrUnexpectedEndOfJSON("string", cursor)
case '\\':
cursor++
chars, nextCursor := decodeKeyCharByEscapedChar(buf, cursor)
chars, nextCursor, err := decodeKeyCharByEscapedChar(buf, cursor)
if err != nil {
return 0, nil, err
}
for _, c := range chars {
curBit &= bitmap[keyIdx][largeToSmallTable[c]]
if curBit == 0 {
@ -817,3 +839,7 @@ func (d *structDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsaf
cursor++
}
}
func (d *structDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: struct decoder does not support decode path")
}

View File

@ -10,6 +10,7 @@ import (
type Decoder interface {
Decode(*RuntimeContext, int64, int64, unsafe.Pointer) (int64, error)
DecodePath(*RuntimeContext, int64, int64) ([][]byte, int64, error)
DecodeStream(*Stream, int64, unsafe.Pointer) error
}

View File

@ -188,3 +188,7 @@ func (d *uintDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe.
d.op(p, u64)
return cursor, nil
}
func (d *uintDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: uint decoder does not support decode path")
}

View File

@ -1,7 +1,9 @@
package decoder
import (
"context"
"encoding/json"
"fmt"
"unsafe"
"github.com/goccy/go-json/internal/errors"
@ -46,13 +48,20 @@ func (d *unmarshalJSONDecoder) DecodeStream(s *Stream, depth int64, p unsafe.Poi
typ: d.typ,
ptr: p,
}))
switch v := v.(type) {
case unmarshalerContext:
var ctx context.Context
if (s.Option.Flags & ContextOption) != 0 {
if err := v.(unmarshalerContext).UnmarshalJSON(s.Option.Context, dst); err != nil {
ctx = s.Option.Context
} else {
ctx = context.Background()
}
if err := v.UnmarshalJSON(ctx, dst); err != nil {
d.annotateError(s.cursor, err)
return err
}
} else {
if err := v.(json.Unmarshaler).UnmarshalJSON(dst); err != nil {
case json.Unmarshaler:
if err := v.UnmarshalJSON(dst); err != nil {
d.annotateError(s.cursor, err)
return err
}
@ -89,3 +98,7 @@ func (d *unmarshalJSONDecoder) Decode(ctx *RuntimeContext, cursor, depth int64,
}
return end, nil
}
func (d *unmarshalJSONDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: unmarshal json decoder does not support decode path")
}

View File

@ -3,6 +3,7 @@ package decoder
import (
"bytes"
"encoding"
"fmt"
"unicode"
"unicode/utf16"
"unicode/utf8"
@ -142,7 +143,11 @@ func (d *unmarshalTextDecoder) Decode(ctx *RuntimeContext, cursor, depth int64,
return end, nil
}
func unquoteBytes(s []byte) (t []byte, ok bool) {
func (d *unmarshalTextDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: unmarshal text decoder does not support decode path")
}
func unquoteBytes(s []byte) (t []byte, ok bool) { //nolint: nonamedreturns
length := len(s)
if length < 2 || s[0] != '"' || s[length-1] != '"' {
return

View File

@ -1,6 +1,7 @@
package decoder
import (
"fmt"
"reflect"
"unsafe"
@ -66,3 +67,7 @@ func (d *wrappedStringDecoder) Decode(ctx *RuntimeContext, cursor, depth int64,
ctx.Buf = oldBuf
return c, nil
}
func (d *wrappedStringDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int64) ([][]byte, int64, error) {
return nil, 0, fmt.Errorf("json: wrapped string decoder does not support decode path")
}

View File

@ -2,6 +2,7 @@ package encoder
import (
"fmt"
"reflect"
"unsafe"
"github.com/goccy/go-json/internal/runtime"
@ -10,6 +11,7 @@ import (
type Code interface {
Kind() CodeKind
ToOpcode(*compileContext) Opcodes
Filter(*FieldQuery) Code
}
type AnonymousCode interface {
@ -82,6 +84,10 @@ func (c *IntCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{code}
}
func (c *IntCode) Filter(_ *FieldQuery) Code {
return c
}
type UintCode struct {
typ *runtime.Type
bitSize uint8
@ -108,6 +114,10 @@ func (c *UintCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{code}
}
func (c *UintCode) Filter(_ *FieldQuery) Code {
return c
}
type FloatCode struct {
typ *runtime.Type
bitSize uint8
@ -140,6 +150,10 @@ func (c *FloatCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{code}
}
func (c *FloatCode) Filter(_ *FieldQuery) Code {
return c
}
type StringCode struct {
typ *runtime.Type
isPtr bool
@ -169,6 +183,10 @@ func (c *StringCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{code}
}
func (c *StringCode) Filter(_ *FieldQuery) Code {
return c
}
type BoolCode struct {
typ *runtime.Type
isPtr bool
@ -190,6 +208,10 @@ func (c *BoolCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{code}
}
func (c *BoolCode) Filter(_ *FieldQuery) Code {
return c
}
type BytesCode struct {
typ *runtime.Type
isPtr bool
@ -211,6 +233,10 @@ func (c *BytesCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{code}
}
func (c *BytesCode) Filter(_ *FieldQuery) Code {
return c
}
type SliceCode struct {
typ *runtime.Type
value Code
@ -245,6 +271,10 @@ func (c *SliceCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{header}.Add(codes...).Add(elemCode).Add(end)
}
func (c *SliceCode) Filter(_ *FieldQuery) Code {
return c
}
type ArrayCode struct {
typ *runtime.Type
value Code
@ -286,6 +316,10 @@ func (c *ArrayCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{header}.Add(codes...).Add(elemCode).Add(end)
}
func (c *ArrayCode) Filter(_ *FieldQuery) Code {
return c
}
type MapCode struct {
typ *runtime.Type
key Code
@ -332,6 +366,10 @@ func (c *MapCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{header}.Add(keyCodes...).Add(value).Add(valueCodes...).Add(key).Add(end)
}
func (c *MapCode) Filter(_ *FieldQuery) Code {
return c
}
type StructCode struct {
typ *runtime.Type
fields []*StructFieldCode
@ -346,7 +384,7 @@ func (c *StructCode) Kind() CodeKind {
}
func (c *StructCode) lastFieldCode(field *StructFieldCode, firstField *Opcode) *Opcode {
if field.isAnonymous {
if isEmbeddedStruct(field) {
return c.lastAnonymousFieldCode(firstField)
}
lastField := firstField
@ -359,7 +397,10 @@ func (c *StructCode) lastFieldCode(field *StructFieldCode, firstField *Opcode) *
func (c *StructCode) lastAnonymousFieldCode(firstField *Opcode) *Opcode {
// firstField is special StructHead operation for anonymous structure.
// So, StructHead's next operation is truly struct head operation.
lastField := firstField.Next
for firstField.Op == OpStructHead || firstField.Op == OpStructField {
firstField = firstField.Next
}
lastField := firstField
for lastField.NextField != nil {
lastField = lastField.NextField
}
@ -401,10 +442,6 @@ func (c *StructCode) ToOpcode(ctx *compileContext) Opcodes {
endField := fieldCodes.Last()
if len(codes) > 0 {
codes.First().End = endField
} else if field.isAnonymous {
firstField.End = endField
lastField := c.lastAnonymousFieldCode(firstField)
lastField.NextField = endField
} else {
firstField.End = endField
}
@ -520,6 +557,45 @@ func (c *StructCode) enableIndirect() {
structCode.enableIndirect()
}
func (c *StructCode) Filter(query *FieldQuery) Code {
fieldMap := map[string]*FieldQuery{}
for _, field := range query.Fields {
fieldMap[field.Name] = field
}
fields := make([]*StructFieldCode, 0, len(c.fields))
for _, field := range c.fields {
query, exists := fieldMap[field.key]
if !exists {
continue
}
fieldCode := &StructFieldCode{
typ: field.typ,
key: field.key,
tag: field.tag,
value: field.value,
offset: field.offset,
isAnonymous: field.isAnonymous,
isTaggedKey: field.isTaggedKey,
isNilableType: field.isNilableType,
isNilCheck: field.isNilCheck,
isAddrForMarshaler: field.isAddrForMarshaler,
isNextOpPtrType: field.isNextOpPtrType,
}
if len(query.Fields) > 0 {
fieldCode.value = fieldCode.value.Filter(query)
}
fields = append(fields, fieldCode)
}
return &StructCode{
typ: c.typ,
fields: fields,
isPtr: c.isPtr,
disableIndirectConversion: c.disableIndirectConversion,
isIndirect: c.isIndirect,
isRecursive: c.isRecursive,
}
}
type StructFieldCode struct {
typ *runtime.Type
key string
@ -532,6 +608,7 @@ type StructFieldCode struct {
isNilCheck bool
isAddrForMarshaler bool
isNextOpPtrType bool
isMarshalerContext bool
}
func (c *StructFieldCode) getStruct() *StructCode {
@ -574,8 +651,12 @@ func (c *StructFieldCode) headerOpcodes(ctx *compileContext, field *Opcode, valu
value := valueCodes.First()
op := optimizeStructHeader(value, c.tag)
field.Op = op
if value.Flags&MarshalerContextFlags != 0 {
field.Flags |= MarshalerContextFlags
}
field.NumBitSize = value.NumBitSize
field.PtrNum = value.PtrNum
field.FieldQuery = value.FieldQuery
fieldCodes := Opcodes{field}
if op.IsMultipleOpHead() {
field.Next = value
@ -590,8 +671,12 @@ func (c *StructFieldCode) fieldOpcodes(ctx *compileContext, field *Opcode, value
value := valueCodes.First()
op := optimizeStructField(value, c.tag)
field.Op = op
if value.Flags&MarshalerContextFlags != 0 {
field.Flags |= MarshalerContextFlags
}
field.NumBitSize = value.NumBitSize
field.PtrNum = value.PtrNum
field.FieldQuery = value.FieldQuery
fieldCodes := Opcodes{field}
if op.IsMultipleOpField() {
@ -611,7 +696,15 @@ func (c *StructFieldCode) addStructEndCode(ctx *compileContext, codes Opcodes) O
Indent: ctx.indent,
}
codes.Last().Next = end
codes.First().NextField = end
code := codes.First()
for code.Op == OpStructField || code.Op == OpStructHead {
code = code.Next
}
for code.NextField != nil {
code = code.NextField
}
code.NextField = end
codes = codes.Add(end)
ctx.incOpcodeIndex()
return codes
@ -645,6 +738,9 @@ func (c *StructFieldCode) flags() OpFlags {
if c.isAnonymous {
flags |= AnonymousKeyFlags
}
if c.isMarshalerContext {
flags |= MarshalerContextFlags
}
return flags
}
@ -726,6 +822,7 @@ func isEnableStructEndOptimization(value Code) bool {
type InterfaceCode struct {
typ *runtime.Type
fieldQuery *FieldQuery
isPtr bool
}
@ -741,6 +838,7 @@ func (c *InterfaceCode) ToOpcode(ctx *compileContext) Opcodes {
default:
code = newOpCode(ctx, c.typ, OpInterface)
}
code.FieldQuery = c.fieldQuery
if c.typ.NumMethod() > 0 {
code.Flags |= NonEmptyInterfaceFlags
}
@ -748,8 +846,17 @@ func (c *InterfaceCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{code}
}
func (c *InterfaceCode) Filter(query *FieldQuery) Code {
return &InterfaceCode{
typ: c.typ,
fieldQuery: query,
isPtr: c.isPtr,
}
}
type MarshalJSONCode struct {
typ *runtime.Type
fieldQuery *FieldQuery
isAddrForMarshaler bool
isNilableType bool
isMarshalerContext bool
@ -761,6 +868,7 @@ func (c *MarshalJSONCode) Kind() CodeKind {
func (c *MarshalJSONCode) ToOpcode(ctx *compileContext) Opcodes {
code := newOpCode(ctx, c.typ, OpMarshalJSON)
code.FieldQuery = c.fieldQuery
if c.isAddrForMarshaler {
code.Flags |= AddrForMarshalerFlags
}
@ -776,8 +884,19 @@ func (c *MarshalJSONCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{code}
}
func (c *MarshalJSONCode) Filter(query *FieldQuery) Code {
return &MarshalJSONCode{
typ: c.typ,
fieldQuery: query,
isAddrForMarshaler: c.isAddrForMarshaler,
isNilableType: c.isNilableType,
isMarshalerContext: c.isMarshalerContext,
}
}
type MarshalTextCode struct {
typ *runtime.Type
fieldQuery *FieldQuery
isAddrForMarshaler bool
isNilableType bool
}
@ -788,6 +907,7 @@ func (c *MarshalTextCode) Kind() CodeKind {
func (c *MarshalTextCode) ToOpcode(ctx *compileContext) Opcodes {
code := newOpCode(ctx, c.typ, OpMarshalText)
code.FieldQuery = c.fieldQuery
if c.isAddrForMarshaler {
code.Flags |= AddrForMarshalerFlags
}
@ -800,6 +920,15 @@ func (c *MarshalTextCode) ToOpcode(ctx *compileContext) Opcodes {
return Opcodes{code}
}
func (c *MarshalTextCode) Filter(query *FieldQuery) Code {
return &MarshalTextCode{
typ: c.typ,
fieldQuery: query,
isAddrForMarshaler: c.isAddrForMarshaler,
isNilableType: c.isNilableType,
}
}
type PtrCode struct {
typ *runtime.Type
value Code
@ -830,6 +959,14 @@ func (c *PtrCode) ToAnonymousOpcode(ctx *compileContext) Opcodes {
return codes
}
func (c *PtrCode) Filter(query *FieldQuery) Code {
return &PtrCode{
typ: c.typ,
value: c.value.Filter(query),
ptrNum: c.ptrNum,
}
}
func convertPtrOp(code *Opcode) OpType {
ptrHeadOp := code.Op.HeadToPtrHead()
if code.Op != ptrHeadOp {
@ -873,3 +1010,14 @@ func convertPtrOp(code *Opcode) OpType {
}
return code.Op
}
func isEmbeddedStruct(field *StructFieldCode) bool {
if !field.isAnonymous {
return false
}
t := field.typ
if t.Kind() == reflect.Ptr {
t = t.Elem()
}
return t.Kind() == reflect.Struct
}

View File

@ -213,8 +213,8 @@ func compactString(dst, src []byte, cursor int64, escape bool) ([]byte, int64, e
dst = append(dst, src[start:cursor]...)
dst = append(dst, `\u202`...)
dst = append(dst, hex[src[cursor+2]&0xF])
cursor += 2
start = cursor + 3
cursor += 2
}
}
switch c {

View File

@ -31,7 +31,7 @@ func init() {
if typeAddr == nil {
typeAddr = &runtime.TypeAddr{}
}
cachedOpcodeSets = make([]*OpcodeSet, typeAddr.AddrRange>>typeAddr.AddrShift)
cachedOpcodeSets = make([]*OpcodeSet, typeAddr.AddrRange>>typeAddr.AddrShift+1)
}
func loadOpcodeMap() map[uintptr]*OpcodeSet {
@ -63,6 +63,27 @@ func compileToGetCodeSetSlowPath(typeptr uintptr) (*OpcodeSet, error) {
return codeSet, nil
}
func getFilteredCodeSetIfNeeded(ctx *RuntimeContext, codeSet *OpcodeSet) (*OpcodeSet, error) {
if (ctx.Option.Flag & ContextOption) == 0 {
return codeSet, nil
}
query := FieldQueryFromContext(ctx.Option.Context)
if query == nil {
return codeSet, nil
}
ctx.Option.Flag |= FieldQueryOption
cacheCodeSet := codeSet.getQueryCache(query.Hash())
if cacheCodeSet != nil {
return cacheCodeSet, nil
}
queryCodeSet, err := newCompiler().codeToOpcodeSet(codeSet.Type, codeSet.Code.Filter(query))
if err != nil {
return nil, err
}
codeSet.setQueryCache(query.Hash(), queryCodeSet)
return queryCodeSet, nil
}
type Compiler struct {
structTypeToCode map[uintptr]*StructCode
}
@ -80,6 +101,10 @@ func (c *Compiler) compile(typeptr uintptr) (*OpcodeSet, error) {
if err != nil {
return nil, err
}
return c.codeToOpcodeSet(typ, code)
}
func (c *Compiler) codeToOpcodeSet(typ *runtime.Type, code Code) (*OpcodeSet, error) {
noescapeKeyCode := c.codeToOpcode(&compileContext{
structTypeToCodes: map[uintptr]Opcodes{},
recursiveCodes: &Opcodes{},
@ -107,6 +132,8 @@ func (c *Compiler) compile(typeptr uintptr) (*OpcodeSet, error) {
InterfaceEscapeKeyCode: interfaceEscapeKeyCode,
CodeLength: codeLength,
EndCode: ToEndCode(interfaceNoescapeKeyCode),
Code: code,
QueryCache: map[string]*OpcodeSet{},
}, nil
}
@ -453,14 +480,17 @@ func (c *Compiler) mapCode(typ *runtime.Type) (*MapCode, error) {
func (c *Compiler) listElemCode(typ *runtime.Type) (Code, error) {
switch {
case c.isPtrMarshalJSONType(typ):
case c.implementsMarshalJSONType(typ) || c.implementsMarshalJSONType(runtime.PtrTo(typ)):
return c.marshalJSONCode(typ)
case !typ.Implements(marshalTextType) && runtime.PtrTo(typ).Implements(marshalTextType):
return c.marshalTextCode(typ)
case typ.Kind() == reflect.Map:
return c.ptrCode(runtime.PtrTo(typ))
default:
code, err := c.typeToCodeWithPtr(typ, false)
// isPtr was originally used to indicate whether the type of top level is pointer.
// However, since the slice/array element is a specification that can get the pointer address, explicitly set isPtr to true.
// See here for related issues: https://github.com/goccy/go-json/issues/370
code, err := c.typeToCodeWithPtr(typ, true)
if err != nil {
return nil, err
}
@ -476,8 +506,6 @@ func (c *Compiler) listElemCode(typ *runtime.Type) (Code, error) {
func (c *Compiler) mapKeyCode(typ *runtime.Type) (Code, error) {
switch {
case c.implementsMarshalJSON(typ):
return c.marshalJSONCode(typ)
case c.implementsMarshalText(typ):
return c.marshalTextCode(typ)
}
@ -589,6 +617,13 @@ func (c *Compiler) structCode(typ *runtime.Type, isPtr bool) (*StructCode, error
return code, nil
}
func toElemType(t *runtime.Type) *runtime.Type {
for t.Kind() == reflect.Ptr {
t = t.Elem()
}
return t
}
func (c *Compiler) structFieldCode(structCode *StructCode, tag *runtime.StructTag, isPtr, isOnlyOneFirstField bool) (*StructFieldCode, error) {
field := tag.Field
fieldType := runtime.Type2RType(field.Type)
@ -598,7 +633,7 @@ func (c *Compiler) structFieldCode(structCode *StructCode, tag *runtime.StructTa
key: tag.Key,
tag: tag,
offset: field.Offset,
isAnonymous: field.Anonymous && !tag.IsTaggedKey,
isAnonymous: field.Anonymous && !tag.IsTaggedKey && toElemType(fieldType).Kind() == reflect.Struct,
isTaggedKey: tag.IsTaggedKey,
isNilableType: c.isNilableType(fieldType),
isNilCheck: true,
@ -826,6 +861,9 @@ func (c *Compiler) implementsMarshalText(typ *runtime.Type) bool {
}
func (c *Compiler) isNilableType(typ *runtime.Type) bool {
if !runtime.IfaceIndir(typ) {
return true
}
switch typ.Kind() {
case reflect.Ptr:
return true
@ -858,29 +896,40 @@ func (c *Compiler) codeToOpcode(ctx *compileContext, typ *runtime.Type, code Cod
}
func (c *Compiler) linkRecursiveCode(ctx *compileContext) {
recursiveCodes := map[uintptr]*CompiledCode{}
for _, recursive := range *ctx.recursiveCodes {
typeptr := uintptr(unsafe.Pointer(recursive.Type))
codes := ctx.structTypeToCodes[typeptr]
compiled := recursive.Jmp
compiled.Code = copyOpcode(codes.First())
code := compiled.Code
code.End.Next = newEndOp(&compileContext{}, recursive.Type)
code.Op = code.Op.PtrHeadToHead()
if recursiveCode, ok := recursiveCodes[typeptr]; ok {
*recursive.Jmp = *recursiveCode
continue
}
beforeLastCode := code.End
lastCode := beforeLastCode.Next
code := copyOpcode(codes.First())
code.Op = code.Op.PtrHeadToHead()
lastCode := newEndOp(&compileContext{}, recursive.Type)
lastCode.Op = OpRecursiveEnd
// OpRecursiveEnd must set before call TotalLength
code.End.Next = lastCode
totalLength := code.TotalLength()
// Idx, ElemIdx, Length must set after call TotalLength
lastCode.Idx = uint32((totalLength + 1) * uintptrSize)
lastCode.ElemIdx = lastCode.Idx + uintptrSize
lastCode.Length = lastCode.Idx + 2*uintptrSize
code.End.Next.Op = OpRecursiveEnd
// extend length to alloc slot for elemIdx + length
curTotalLength := uintptr(recursive.TotalLength()) + 3
nextTotalLength := uintptr(totalLength) + 3
compiled := recursive.Jmp
compiled.Code = code
compiled.CurLen = curTotalLength
compiled.NextLen = nextTotalLength
compiled.Linked = true
recursiveCodes[typeptr] = compiled
}
}

View File

@ -3,18 +3,30 @@
package encoder
func CompileToGetCodeSet(typeptr uintptr) (*OpcodeSet, error) {
if typeptr > typeAddr.MaxTypeAddr {
return compileToGetCodeSetSlowPath(typeptr)
func CompileToGetCodeSet(ctx *RuntimeContext, typeptr uintptr) (*OpcodeSet, error) {
if typeptr > typeAddr.MaxTypeAddr || typeptr < typeAddr.BaseTypeAddr {
codeSet, err := compileToGetCodeSetSlowPath(typeptr)
if err != nil {
return nil, err
}
return getFilteredCodeSetIfNeeded(ctx, codeSet)
}
index := (typeptr - typeAddr.BaseTypeAddr) >> typeAddr.AddrShift
if codeSet := cachedOpcodeSets[index]; codeSet != nil {
return codeSet, nil
filtered, err := getFilteredCodeSetIfNeeded(ctx, codeSet)
if err != nil {
return nil, err
}
return filtered, nil
}
codeSet, err := newCompiler().compile(typeptr)
if err != nil {
return nil, err
}
cachedOpcodeSets[index] = codeSet
return codeSet, nil
filtered, err := getFilteredCodeSetIfNeeded(ctx, codeSet)
if err != nil {
return nil, err
}
cachedOpcodeSets[index] = codeSet
return filtered, nil
}

View File

@ -9,15 +9,24 @@ import (
var setsMu sync.RWMutex
func CompileToGetCodeSet(typeptr uintptr) (*OpcodeSet, error) {
if typeptr > typeAddr.MaxTypeAddr {
return compileToGetCodeSetSlowPath(typeptr)
func CompileToGetCodeSet(ctx *RuntimeContext, typeptr uintptr) (*OpcodeSet, error) {
if typeptr > typeAddr.MaxTypeAddr || typeptr < typeAddr.BaseTypeAddr {
codeSet, err := compileToGetCodeSetSlowPath(typeptr)
if err != nil {
return nil, err
}
return getFilteredCodeSetIfNeeded(ctx, codeSet)
}
index := (typeptr - typeAddr.BaseTypeAddr) >> typeAddr.AddrShift
setsMu.RLock()
if codeSet := cachedOpcodeSets[index]; codeSet != nil {
filtered, err := getFilteredCodeSetIfNeeded(ctx, codeSet)
if err != nil {
setsMu.RUnlock()
return codeSet, nil
return nil, err
}
setsMu.RUnlock()
return filtered, nil
}
setsMu.RUnlock()
@ -25,8 +34,12 @@ func CompileToGetCodeSet(typeptr uintptr) (*OpcodeSet, error) {
if err != nil {
return nil, err
}
filtered, err := getFilteredCodeSetIfNeeded(ctx, codeSet)
if err != nil {
return nil, err
}
setsMu.Lock()
cachedOpcodeSets[index] = codeSet
setsMu.Unlock()
return codeSet, nil
return filtered, nil
}

View File

@ -0,0 +1,126 @@
package encoder
import "unicode/utf8"
const (
// The default lowest and highest continuation byte.
locb = 128 //0b10000000
hicb = 191 //0b10111111
// These names of these constants are chosen to give nice alignment in the
// table below. The first nibble is an index into acceptRanges or F for
// special one-byte cases. The second nibble is the Rune length or the
// Status for the special one-byte case.
xx = 0xF1 // invalid: size 1
as = 0xF0 // ASCII: size 1
s1 = 0x02 // accept 0, size 2
s2 = 0x13 // accept 1, size 3
s3 = 0x03 // accept 0, size 3
s4 = 0x23 // accept 2, size 3
s5 = 0x34 // accept 3, size 4
s6 = 0x04 // accept 0, size 4
s7 = 0x44 // accept 4, size 4
)
// first is information about the first byte in a UTF-8 sequence.
var first = [256]uint8{
// 1 2 3 4 5 6 7 8 9 A B C D E F
as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, // 0x00-0x0F
as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, // 0x10-0x1F
as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, // 0x20-0x2F
as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, // 0x30-0x3F
as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, // 0x40-0x4F
as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, // 0x50-0x5F
as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, // 0x60-0x6F
as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, as, // 0x70-0x7F
// 1 2 3 4 5 6 7 8 9 A B C D E F
xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, // 0x80-0x8F
xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, // 0x90-0x9F
xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, // 0xA0-0xAF
xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, // 0xB0-0xBF
xx, xx, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, // 0xC0-0xCF
s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, s1, // 0xD0-0xDF
s2, s3, s3, s3, s3, s3, s3, s3, s3, s3, s3, s3, s3, s4, s3, s3, // 0xE0-0xEF
s5, s6, s6, s6, s7, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, xx, // 0xF0-0xFF
}
const (
lineSep = byte(168) //'\u2028'
paragraphSep = byte(169) //'\u2029'
)
type decodeRuneState int
const (
validUTF8State decodeRuneState = iota
runeErrorState
lineSepState
paragraphSepState
)
func decodeRuneInString(s string) (decodeRuneState, int) {
n := len(s)
s0 := s[0]
x := first[s0]
if x >= as {
// The following code simulates an additional check for x == xx and
// handling the ASCII and invalid cases accordingly. This mask-and-or
// approach prevents an additional branch.
mask := rune(x) << 31 >> 31 // Create 0x0000 or 0xFFFF.
if rune(s[0])&^mask|utf8.RuneError&mask == utf8.RuneError {
return runeErrorState, 1
}
return validUTF8State, 1
}
sz := int(x & 7)
if n < sz {
return runeErrorState, 1
}
s1 := s[1]
switch x >> 4 {
case 0:
if s1 < locb || hicb < s1 {
return runeErrorState, 1
}
case 1:
if s1 < 0xA0 || hicb < s1 {
return runeErrorState, 1
}
case 2:
if s1 < locb || 0x9F < s1 {
return runeErrorState, 1
}
case 3:
if s1 < 0x90 || hicb < s1 {
return runeErrorState, 1
}
case 4:
if s1 < locb || 0x8F < s1 {
return runeErrorState, 1
}
}
if sz <= 2 {
return validUTF8State, 2
}
s2 := s[2]
if s2 < locb || hicb < s2 {
return runeErrorState, 1
}
if sz <= 3 {
// separator character prefixes: [2]byte{226, 128}
if s0 == 226 && s1 == 128 {
switch s2 {
case lineSep:
return lineSepState, 3
case paragraphSep:
return paragraphSepState, 3
}
}
return validUTF8State, 3
}
s3 := s[3]
if s3 < locb || hicb < s3 {
return runeErrorState, 1
}
return validUTF8State, 4
}

View File

@ -101,6 +101,22 @@ type OpcodeSet struct {
InterfaceEscapeKeyCode *Opcode
CodeLength int
EndCode *Opcode
Code Code
QueryCache map[string]*OpcodeSet
cacheMu sync.RWMutex
}
func (s *OpcodeSet) getQueryCache(hash string) *OpcodeSet {
s.cacheMu.RLock()
codeSet := s.QueryCache[hash]
s.cacheMu.RUnlock()
return codeSet
}
func (s *OpcodeSet) setQueryCache(hash string, codeSet *OpcodeSet) {
s.cacheMu.Lock()
s.QueryCache[hash] = codeSet
s.cacheMu.Unlock()
}
type CompiledCode struct {
@ -222,33 +238,56 @@ func (m *Mapslice) Swap(i, j int) {
m.Items[i], m.Items[j] = m.Items[j], m.Items[i]
}
//nolint:structcheck,unused
type mapIter struct {
key unsafe.Pointer
elem unsafe.Pointer
t unsafe.Pointer
h unsafe.Pointer
buckets unsafe.Pointer
bptr unsafe.Pointer
overflow unsafe.Pointer
oldoverflow unsafe.Pointer
startBucket uintptr
offset uint8
wrapped bool
B uint8
i uint8
bucket uintptr
checkBucket uintptr
}
type MapContext struct {
Pos []int
Start int
First int
Idx int
Slice *Mapslice
Buf []byte
Len int
Iter mapIter
}
var mapContextPool = sync.Pool{
New: func() interface{} {
return &MapContext{}
return &MapContext{
Slice: &Mapslice{},
}
},
}
func NewMapContext(mapLen int) *MapContext {
func NewMapContext(mapLen int, unorderedMap bool) *MapContext {
ctx := mapContextPool.Get().(*MapContext)
if ctx.Slice == nil {
ctx.Slice = &Mapslice{
Items: make([]MapItem, 0, mapLen),
}
}
if cap(ctx.Pos) < (mapLen*2 + 1) {
ctx.Pos = make([]int, 0, mapLen*2+1)
ctx.Slice.Items = make([]MapItem, 0, mapLen)
if !unorderedMap {
if len(ctx.Slice.Items) < mapLen {
ctx.Slice.Items = make([]MapItem, mapLen)
} else {
ctx.Pos = ctx.Pos[:0]
ctx.Slice.Items = ctx.Slice.Items[:0]
ctx.Slice.Items = ctx.Slice.Items[:mapLen]
}
}
ctx.Buf = ctx.Buf[:0]
ctx.Iter = mapIter{}
ctx.Idx = 0
ctx.Len = mapLen
return ctx
}
@ -256,17 +295,17 @@ func ReleaseMapContext(c *MapContext) {
mapContextPool.Put(c)
}
//go:linkname MapIterInit reflect.mapiterinit
//go:linkname MapIterInit runtime.mapiterinit
//go:noescape
func MapIterInit(mapType *runtime.Type, m unsafe.Pointer) unsafe.Pointer
func MapIterInit(mapType *runtime.Type, m unsafe.Pointer, it *mapIter)
//go:linkname MapIterKey reflect.mapiterkey
//go:noescape
func MapIterKey(it unsafe.Pointer) unsafe.Pointer
func MapIterKey(it *mapIter) unsafe.Pointer
//go:linkname MapIterNext reflect.mapiternext
//go:noescape
func MapIterNext(it unsafe.Pointer)
func MapIterNext(it *mapIter)
//go:linkname MapLen reflect.maplen
//go:noescape
@ -374,7 +413,11 @@ func AppendMarshalJSON(ctx *RuntimeContext, code *Opcode, b []byte, v interface{
if !ok {
return AppendNull(ctx, b), nil
}
b, err := marshaler.MarshalJSON(ctx.Option.Context)
stdctx := ctx.Option.Context
if ctx.Option.Flag&FieldQueryOption != 0 {
stdctx = SetFieldQueryToContext(stdctx, code.FieldQuery)
}
b, err := marshaler.MarshalJSON(stdctx)
if err != nil {
return nil, &errors.MarshalerError{Type: reflect.TypeOf(v), Err: err}
}
@ -546,6 +589,8 @@ func IsNilForMarshaler(v interface{}) bool {
return rv.IsNil()
case reflect.Slice:
return rv.IsNil() || rv.Len() == 0
case reflect.String:
return rv.Len() == 0
}
return false
}

View File

@ -1,3 +1,27 @@
// This files's processing codes are inspired by https://github.com/segmentio/encoding.
// The license notation is as follows.
//
// # MIT License
//
// Copyright (c) 2019 Segment.io, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package encoder
import (

View File

@ -1,3 +1,4 @@
//go:build !go1.13
// +build !go1.13
package encoder
@ -5,4 +6,4 @@ package encoder
import "unsafe"
//go:linkname MapIterValue reflect.mapitervalue
func MapIterValue(it unsafe.Pointer) unsafe.Pointer
func MapIterValue(it *mapIter) unsafe.Pointer

View File

@ -1,3 +1,4 @@
//go:build go1.13
// +build go1.13
package encoder
@ -5,4 +6,4 @@ package encoder
import "unsafe"
//go:linkname MapIterValue reflect.mapiterelem
func MapIterValue(it unsafe.Pointer) unsafe.Pointer
func MapIterValue(it *mapIter) unsafe.Pointer

View File

@ -1,7 +1,9 @@
package encoder
import (
"bytes"
"fmt"
"sort"
"strings"
"unsafe"
@ -39,10 +41,9 @@ type Opcode struct {
Type *runtime.Type // go type
Jmp *CompiledCode // for recursive call
ElemIdx uint32 // offset to access array/slice/map elem
Length uint32 // offset to access slice/map length or array length
MapIter uint32 // offset to access map iterator
MapPos uint32 // offset to access position list for sorted map
FieldQuery *FieldQuery // field query for Interface / MarshalJSON / MarshalText
ElemIdx uint32 // offset to access array/slice elem
Length uint32 // offset to access slice length or array length
Indent uint32 // indent number
Size uint32 // array/slice elem size
DisplayIdx uint32 // opcode index
@ -91,8 +92,6 @@ func (c *Opcode) MaxIdx() uint32 {
c.Idx,
c.ElemIdx,
c.Length,
c.MapIter,
c.MapPos,
c.Size,
} {
if max < value {
@ -337,12 +336,11 @@ func copyOpcode(code *Opcode) *Opcode {
Idx: c.Idx,
Offset: c.Offset,
Type: c.Type,
FieldQuery: c.FieldQuery,
DisplayIdx: c.DisplayIdx,
DisplayKey: c.DisplayKey,
ElemIdx: c.ElemIdx,
Length: c.Length,
MapIter: c.MapIter,
MapPos: c.MapPos,
Size: c.Size,
Indent: c.Indent,
Jmp: c.Jmp,
@ -367,7 +365,7 @@ func copyOpcode(code *Opcode) *Opcode {
func setTotalLengthToInterfaceOp(code *Opcode) {
for c := code; !c.IsEnd(); {
if c.Op == OpInterface {
if c.Op == OpInterface || c.Op == OpInterfacePtr {
c.Length = uint32(code.TotalLength())
}
c = c.IterNext()
@ -448,26 +446,21 @@ func (c *Opcode) dumpHead(code *Opcode) string {
func (c *Opcode) dumpMapHead(code *Opcode) string {
return fmt.Sprintf(
`[%03d]%s%s ([idx:%d][elemIdx:%d][length:%d][mapIter:%d])`,
`[%03d]%s%s ([idx:%d])`,
code.DisplayIdx,
strings.Repeat("-", int(code.Indent)),
code.Op,
code.Idx/uintptrSize,
code.ElemIdx/uintptrSize,
code.Length/uintptrSize,
code.MapIter/uintptrSize,
)
}
func (c *Opcode) dumpMapEnd(code *Opcode) string {
return fmt.Sprintf(
`[%03d]%s%s ([idx:%d][mapPos:%d][length:%d])`,
`[%03d]%s%s ([idx:%d])`,
code.DisplayIdx,
strings.Repeat("-", int(code.Indent)),
code.Op,
code.Idx/uintptrSize,
code.MapPos/uintptrSize,
code.Length/uintptrSize,
)
}
@ -504,25 +497,21 @@ func (c *Opcode) dumpField(code *Opcode) string {
func (c *Opcode) dumpKey(code *Opcode) string {
return fmt.Sprintf(
`[%03d]%s%s ([idx:%d][elemIdx:%d][length:%d][mapIter:%d])`,
`[%03d]%s%s ([idx:%d])`,
code.DisplayIdx,
strings.Repeat("-", int(code.Indent)),
code.Op,
code.Idx/uintptrSize,
code.ElemIdx/uintptrSize,
code.Length/uintptrSize,
code.MapIter/uintptrSize,
)
}
func (c *Opcode) dumpValue(code *Opcode) string {
return fmt.Sprintf(
`[%03d]%s%s ([idx:%d][mapIter:%d])`,
`[%03d]%s%s ([idx:%d])`,
code.DisplayIdx,
strings.Repeat("-", int(code.Indent)),
code.Op,
code.Idx/uintptrSize,
code.MapIter/uintptrSize,
)
}
@ -568,6 +557,87 @@ func (c *Opcode) Dump() string {
return strings.Join(codes, "\n")
}
func (c *Opcode) DumpDOT() string {
type edge struct {
from, to *Opcode
label string
weight int
}
var edges []edge
b := &bytes.Buffer{}
fmt.Fprintf(b, "digraph \"%p\" {\n", c.Type)
fmt.Fprintln(b, "mclimit=1.5;\nrankdir=TD;\nordering=out;\nnode[shape=box];")
for code := c; !code.IsEnd(); {
label := code.Op.String()
fmt.Fprintf(b, "\"%p\" [label=%q];\n", code, label)
if p := code.Next; p != nil {
edges = append(edges, edge{
from: code,
to: p,
label: "Next",
weight: 10,
})
}
if p := code.NextField; p != nil {
edges = append(edges, edge{
from: code,
to: p,
label: "NextField",
weight: 2,
})
}
if p := code.End; p != nil {
edges = append(edges, edge{
from: code,
to: p,
label: "End",
weight: 1,
})
}
if p := code.Jmp; p != nil {
edges = append(edges, edge{
from: code,
to: p.Code,
label: "Jmp",
weight: 1,
})
}
switch code.Op.CodeType() {
case CodeSliceHead:
code = code.Next
case CodeMapHead:
code = code.Next
case CodeArrayElem, CodeSliceElem:
code = code.End
case CodeMapKey:
code = code.End
case CodeMapValue:
code = code.Next
case CodeMapEnd:
code = code.Next
case CodeStructField:
code = code.Next
case CodeStructEnd:
code = code.Next
default:
code = code.Next
}
if code.IsEnd() {
fmt.Fprintf(b, "\"%p\" [label=%q];\n", code, code.Op.String())
}
}
sort.Slice(edges, func(i, j int) bool {
return edges[i].to.DisplayIdx < edges[j].to.DisplayIdx
})
for _, e := range edges {
fmt.Fprintf(b, "\"%p\" -> \"%p\" [label=%q][weight=%d];\n", e.from, e.to, e.label, e.weight)
}
fmt.Fprint(b, "}")
return b.String()
}
func newSliceHeaderCode(ctx *compileContext, typ *runtime.Type) *Opcode {
idx := opcodeOffset(ctx.ptrIndex)
ctx.incPtrIndex()
@ -629,19 +699,11 @@ func newArrayElemCode(ctx *compileContext, typ *runtime.Type, head *Opcode, leng
func newMapHeaderCode(ctx *compileContext, typ *runtime.Type) *Opcode {
idx := opcodeOffset(ctx.ptrIndex)
ctx.incPtrIndex()
elemIdx := opcodeOffset(ctx.ptrIndex)
ctx.incPtrIndex()
length := opcodeOffset(ctx.ptrIndex)
ctx.incPtrIndex()
mapIter := opcodeOffset(ctx.ptrIndex)
return &Opcode{
Op: OpMap,
Type: typ,
Idx: idx,
DisplayIdx: ctx.opcodeIndex,
ElemIdx: elemIdx,
Length: length,
MapIter: mapIter,
Indent: ctx.indent,
}
}
@ -650,11 +712,8 @@ func newMapKeyCode(ctx *compileContext, typ *runtime.Type, head *Opcode) *Opcode
return &Opcode{
Op: OpMapKey,
Type: typ,
Idx: opcodeOffset(ctx.ptrIndex),
Idx: head.Idx,
DisplayIdx: ctx.opcodeIndex,
ElemIdx: head.ElemIdx,
Length: head.Length,
MapIter: head.MapIter,
Indent: ctx.indent,
}
}
@ -663,28 +722,20 @@ func newMapValueCode(ctx *compileContext, typ *runtime.Type, head *Opcode) *Opco
return &Opcode{
Op: OpMapValue,
Type: typ,
Idx: opcodeOffset(ctx.ptrIndex),
Idx: head.Idx,
DisplayIdx: ctx.opcodeIndex,
ElemIdx: head.ElemIdx,
Length: head.Length,
MapIter: head.MapIter,
Indent: ctx.indent,
}
}
func newMapEndCode(ctx *compileContext, typ *runtime.Type, head *Opcode) *Opcode {
mapPos := opcodeOffset(ctx.ptrIndex)
ctx.incPtrIndex()
idx := opcodeOffset(ctx.ptrIndex)
return &Opcode{
Op: OpMapEnd,
Type: typ,
Idx: idx,
Next: newEndOp(ctx, typ),
Idx: head.Idx,
DisplayIdx: ctx.opcodeIndex,
Length: head.Length,
MapPos: mapPos,
Indent: ctx.indent,
Next: newEndOp(ctx, typ),
}
}

View File

@ -1,6 +1,9 @@
package encoder
import "context"
import (
"context"
"io"
)
type OptionFlag uint8
@ -11,12 +14,16 @@ const (
DebugOption
ColorizeOption
ContextOption
NormalizeUTF8Option
FieldQueryOption
)
type Option struct {
Flag OptionFlag
ColorScheme *ColorScheme
Context context.Context
DebugOut io.Writer
DebugDOTOut io.WriteCloser
}
type EncodeFormat struct {

View File

@ -0,0 +1,135 @@
package encoder
import (
"context"
"fmt"
"reflect"
)
var (
Marshal func(interface{}) ([]byte, error)
Unmarshal func([]byte, interface{}) error
)
type FieldQuery struct {
Name string
Fields []*FieldQuery
hash string
}
func (q *FieldQuery) Hash() string {
if q.hash != "" {
return q.hash
}
b, _ := Marshal(q)
q.hash = string(b)
return q.hash
}
func (q *FieldQuery) MarshalJSON() ([]byte, error) {
if q.Name != "" {
if len(q.Fields) > 0 {
return Marshal(map[string][]*FieldQuery{q.Name: q.Fields})
}
return Marshal(q.Name)
}
return Marshal(q.Fields)
}
func (q *FieldQuery) QueryString() (FieldQueryString, error) {
b, err := Marshal(q)
if err != nil {
return "", err
}
return FieldQueryString(b), nil
}
type FieldQueryString string
func (s FieldQueryString) Build() (*FieldQuery, error) {
var query interface{}
if err := Unmarshal([]byte(s), &query); err != nil {
return nil, err
}
return s.build(reflect.ValueOf(query))
}
func (s FieldQueryString) build(v reflect.Value) (*FieldQuery, error) {
switch v.Type().Kind() {
case reflect.String:
return s.buildString(v)
case reflect.Map:
return s.buildMap(v)
case reflect.Slice:
return s.buildSlice(v)
case reflect.Interface:
return s.build(reflect.ValueOf(v.Interface()))
}
return nil, fmt.Errorf("failed to build field query")
}
func (s FieldQueryString) buildString(v reflect.Value) (*FieldQuery, error) {
b := []byte(v.String())
switch b[0] {
case '[', '{':
var query interface{}
if err := Unmarshal(b, &query); err != nil {
return nil, err
}
if str, ok := query.(string); ok {
return &FieldQuery{Name: str}, nil
}
return s.build(reflect.ValueOf(query))
}
return &FieldQuery{Name: string(b)}, nil
}
func (s FieldQueryString) buildSlice(v reflect.Value) (*FieldQuery, error) {
fields := make([]*FieldQuery, 0, v.Len())
for i := 0; i < v.Len(); i++ {
def, err := s.build(v.Index(i))
if err != nil {
return nil, err
}
fields = append(fields, def)
}
return &FieldQuery{Fields: fields}, nil
}
func (s FieldQueryString) buildMap(v reflect.Value) (*FieldQuery, error) {
keys := v.MapKeys()
if len(keys) != 1 {
return nil, fmt.Errorf("failed to build field query object")
}
key := keys[0]
if key.Type().Kind() != reflect.String {
return nil, fmt.Errorf("failed to build field query. invalid object key type")
}
name := key.String()
def, err := s.build(v.MapIndex(key))
if err != nil {
return nil, err
}
return &FieldQuery{
Name: name,
Fields: def.Fields,
}, nil
}
type queryKey struct{}
func FieldQueryFromContext(ctx context.Context) *FieldQuery {
query := ctx.Value(queryKey{})
if query == nil {
return nil
}
q, ok := query.(*FieldQuery)
if !ok {
return nil
}
return q
}
func SetFieldQueryToContext(ctx context.Context, query *FieldQuery) context.Context {
return context.WithValue(ctx, queryKey{}, query)
}

View File

@ -1,9 +1,32 @@
// This files's string processing codes are inspired by https://github.com/segmentio/encoding.
// The license notation is as follows.
//
// # MIT License
//
// Copyright (c) 2019 Segment.io, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package encoder
import (
"math/bits"
"reflect"
"unicode/utf8"
"unsafe"
)
@ -12,390 +35,8 @@ const (
msb = 0x8080808080808080
)
var needEscapeWithHTML = [256]bool{
'"': true,
'&': true,
'<': true,
'>': true,
'\\': true,
0x00: true,
0x01: true,
0x02: true,
0x03: true,
0x04: true,
0x05: true,
0x06: true,
0x07: true,
0x08: true,
0x09: true,
0x0a: true,
0x0b: true,
0x0c: true,
0x0d: true,
0x0e: true,
0x0f: true,
0x10: true,
0x11: true,
0x12: true,
0x13: true,
0x14: true,
0x15: true,
0x16: true,
0x17: true,
0x18: true,
0x19: true,
0x1a: true,
0x1b: true,
0x1c: true,
0x1d: true,
0x1e: true,
0x1f: true,
/* 0x20 - 0x7f */
0x80: true,
0x81: true,
0x82: true,
0x83: true,
0x84: true,
0x85: true,
0x86: true,
0x87: true,
0x88: true,
0x89: true,
0x8a: true,
0x8b: true,
0x8c: true,
0x8d: true,
0x8e: true,
0x8f: true,
0x90: true,
0x91: true,
0x92: true,
0x93: true,
0x94: true,
0x95: true,
0x96: true,
0x97: true,
0x98: true,
0x99: true,
0x9a: true,
0x9b: true,
0x9c: true,
0x9d: true,
0x9e: true,
0x9f: true,
0xa0: true,
0xa1: true,
0xa2: true,
0xa3: true,
0xa4: true,
0xa5: true,
0xa6: true,
0xa7: true,
0xa8: true,
0xa9: true,
0xaa: true,
0xab: true,
0xac: true,
0xad: true,
0xae: true,
0xaf: true,
0xb0: true,
0xb1: true,
0xb2: true,
0xb3: true,
0xb4: true,
0xb5: true,
0xb6: true,
0xb7: true,
0xb8: true,
0xb9: true,
0xba: true,
0xbb: true,
0xbc: true,
0xbd: true,
0xbe: true,
0xbf: true,
0xc0: true,
0xc1: true,
0xc2: true,
0xc3: true,
0xc4: true,
0xc5: true,
0xc6: true,
0xc7: true,
0xc8: true,
0xc9: true,
0xca: true,
0xcb: true,
0xcc: true,
0xcd: true,
0xce: true,
0xcf: true,
0xd0: true,
0xd1: true,
0xd2: true,
0xd3: true,
0xd4: true,
0xd5: true,
0xd6: true,
0xd7: true,
0xd8: true,
0xd9: true,
0xda: true,
0xdb: true,
0xdc: true,
0xdd: true,
0xde: true,
0xdf: true,
0xe0: true,
0xe1: true,
0xe2: true,
0xe3: true,
0xe4: true,
0xe5: true,
0xe6: true,
0xe7: true,
0xe8: true,
0xe9: true,
0xea: true,
0xeb: true,
0xec: true,
0xed: true,
0xee: true,
0xef: true,
0xf0: true,
0xf1: true,
0xf2: true,
0xf3: true,
0xf4: true,
0xf5: true,
0xf6: true,
0xf7: true,
0xf8: true,
0xf9: true,
0xfa: true,
0xfb: true,
0xfc: true,
0xfd: true,
0xfe: true,
0xff: true,
}
var needEscape = [256]bool{
'"': true,
'\\': true,
0x00: true,
0x01: true,
0x02: true,
0x03: true,
0x04: true,
0x05: true,
0x06: true,
0x07: true,
0x08: true,
0x09: true,
0x0a: true,
0x0b: true,
0x0c: true,
0x0d: true,
0x0e: true,
0x0f: true,
0x10: true,
0x11: true,
0x12: true,
0x13: true,
0x14: true,
0x15: true,
0x16: true,
0x17: true,
0x18: true,
0x19: true,
0x1a: true,
0x1b: true,
0x1c: true,
0x1d: true,
0x1e: true,
0x1f: true,
/* 0x20 - 0x7f */
0x80: true,
0x81: true,
0x82: true,
0x83: true,
0x84: true,
0x85: true,
0x86: true,
0x87: true,
0x88: true,
0x89: true,
0x8a: true,
0x8b: true,
0x8c: true,
0x8d: true,
0x8e: true,
0x8f: true,
0x90: true,
0x91: true,
0x92: true,
0x93: true,
0x94: true,
0x95: true,
0x96: true,
0x97: true,
0x98: true,
0x99: true,
0x9a: true,
0x9b: true,
0x9c: true,
0x9d: true,
0x9e: true,
0x9f: true,
0xa0: true,
0xa1: true,
0xa2: true,
0xa3: true,
0xa4: true,
0xa5: true,
0xa6: true,
0xa7: true,
0xa8: true,
0xa9: true,
0xaa: true,
0xab: true,
0xac: true,
0xad: true,
0xae: true,
0xaf: true,
0xb0: true,
0xb1: true,
0xb2: true,
0xb3: true,
0xb4: true,
0xb5: true,
0xb6: true,
0xb7: true,
0xb8: true,
0xb9: true,
0xba: true,
0xbb: true,
0xbc: true,
0xbd: true,
0xbe: true,
0xbf: true,
0xc0: true,
0xc1: true,
0xc2: true,
0xc3: true,
0xc4: true,
0xc5: true,
0xc6: true,
0xc7: true,
0xc8: true,
0xc9: true,
0xca: true,
0xcb: true,
0xcc: true,
0xcd: true,
0xce: true,
0xcf: true,
0xd0: true,
0xd1: true,
0xd2: true,
0xd3: true,
0xd4: true,
0xd5: true,
0xd6: true,
0xd7: true,
0xd8: true,
0xd9: true,
0xda: true,
0xdb: true,
0xdc: true,
0xdd: true,
0xde: true,
0xdf: true,
0xe0: true,
0xe1: true,
0xe2: true,
0xe3: true,
0xe4: true,
0xe5: true,
0xe6: true,
0xe7: true,
0xe8: true,
0xe9: true,
0xea: true,
0xeb: true,
0xec: true,
0xed: true,
0xee: true,
0xef: true,
0xf0: true,
0xf1: true,
0xf2: true,
0xf3: true,
0xf4: true,
0xf5: true,
0xf6: true,
0xf7: true,
0xf8: true,
0xf9: true,
0xfa: true,
0xfb: true,
0xfc: true,
0xfd: true,
0xfe: true,
0xff: true,
}
var hex = "0123456789abcdef"
// escapeIndex finds the index of the first char in `s` that requires escaping.
// A char requires escaping if it's outside of the range of [0x20, 0x7F] or if
// it includes a double quote or backslash.
// If no chars in `s` require escaping, the return value is -1.
func escapeIndex(s string) int {
chunks := stringToUint64Slice(s)
for _, n := range chunks {
// combine masks before checking for the MSB of each byte. We include
// `n` in the mask to check whether any of the *input* byte MSBs were
// set (i.e. the byte was outside the ASCII range).
mask := n | below(n, 0x20) | contains(n, '"') | contains(n, '\\')
if (mask & msb) != 0 {
return bits.TrailingZeros64(mask&msb) / 8
}
}
valLen := len(s)
for i := len(chunks) * 8; i < valLen; i++ {
if needEscape[s[i]] {
return i
}
}
return -1
}
// below return a mask that can be used to determine if any of the bytes
// in `n` are below `b`. If a byte's MSB is set in the mask then that byte was
// below `b`. The result is only valid if `b`, and each byte in `n`, is below
// 0x80.
func below(n uint64, b byte) uint64 {
return n - expand(b)
}
// contains returns a mask that can be used to determine if any of the
// bytes in `n` are equal to `b`. If a byte's MSB is set in the mask then
// that byte is equal to `b`. The result is only valid if `b`, and each
// byte in `n`, is below 0x80.
func contains(n uint64, b byte) uint64 {
return (n ^ expand(b)) - lsb
}
// expand puts the specified byte into each of the 8 bytes of a uint64.
func expand(b byte) uint64 {
return lsb * uint64(b)
}
//nolint:govet
func stringToUint64Slice(s string) []uint64 {
return *(*[]uint64)(unsafe.Pointer(&reflect.SliceHeader{
@ -406,9 +47,19 @@ func stringToUint64Slice(s string) []uint64 {
}
func AppendString(ctx *RuntimeContext, buf []byte, s string) []byte {
if ctx.Option.Flag&HTMLEscapeOption == 0 {
if ctx.Option.Flag&HTMLEscapeOption != 0 {
if ctx.Option.Flag&NormalizeUTF8Option != 0 {
return appendNormalizedHTMLString(buf, s)
}
return appendHTMLString(buf, s)
}
if ctx.Option.Flag&NormalizeUTF8Option != 0 {
return appendNormalizedString(buf, s)
}
return appendString(buf, s)
}
func appendNormalizedHTMLString(buf []byte, s string) []byte {
valLen := len(s)
if valLen == 0 {
return append(buf, `""`...)
@ -435,7 +86,7 @@ func AppendString(ctx *RuntimeContext, buf []byte, s string) []byte {
}
}
for i := len(chunks) * 8; i < valLen; i++ {
if needEscapeWithHTML[s[i]] {
if needEscapeHTMLNormalizeUTF8[s[i]] {
j = i
goto ESCAPE_END
}
@ -447,7 +98,7 @@ ESCAPE_END:
for j < valLen {
c := s[j]
if !needEscapeWithHTML[c] {
if !needEscapeHTMLNormalizeUTF8[c] {
// fast path: most of the time, printable ascii characters are used
j++
continue
@ -489,10 +140,220 @@ ESCAPE_END:
i = j + 1
j = j + 1
continue
case 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x0B, 0x0C, 0x0E, 0x0F, // 0x00-0x0F
0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F: // 0x10-0x1F
buf = append(buf, s[i:j]...)
buf = append(buf, `\u00`...)
buf = append(buf, hex[c>>4], hex[c&0xF])
i = j + 1
j = j + 1
continue
}
state, size := decodeRuneInString(s[j:])
switch state {
case runeErrorState:
buf = append(buf, s[i:j]...)
buf = append(buf, `\ufffd`...)
i = j + 1
j = j + 1
continue
// U+2028 is LINE SEPARATOR.
// U+2029 is PARAGRAPH SEPARATOR.
// They are both technically valid characters in JSON strings,
// but don't work in JSONP, which has to be evaluated as JavaScript,
// and can lead to security holes there. It is valid JSON to
// escape them, so we do so unconditionally.
// See http://timelessrepo.com/json-isnt-a-javascript-subset for discussion.
case lineSepState:
buf = append(buf, s[i:j]...)
buf = append(buf, `\u2028`...)
i = j + 3
j = j + 3
continue
case paragraphSepState:
buf = append(buf, s[i:j]...)
buf = append(buf, `\u2029`...)
i = j + 3
j = j + 3
continue
}
j += size
}
// This encodes bytes < 0x20 except for \t, \n and \r.
if c < 0x20 {
return append(append(buf, s[i:]...), '"')
}
func appendHTMLString(buf []byte, s string) []byte {
valLen := len(s)
if valLen == 0 {
return append(buf, `""`...)
}
buf = append(buf, '"')
var (
i, j int
)
if valLen >= 8 {
chunks := stringToUint64Slice(s)
for _, n := range chunks {
// combine masks before checking for the MSB of each byte. We include
// `n` in the mask to check whether any of the *input* byte MSBs were
// set (i.e. the byte was outside the ASCII range).
mask := n | (n - (lsb * 0x20)) |
((n ^ (lsb * '"')) - lsb) |
((n ^ (lsb * '\\')) - lsb) |
((n ^ (lsb * '<')) - lsb) |
((n ^ (lsb * '>')) - lsb) |
((n ^ (lsb * '&')) - lsb)
if (mask & msb) != 0 {
j = bits.TrailingZeros64(mask&msb) / 8
goto ESCAPE_END
}
}
for i := len(chunks) * 8; i < valLen; i++ {
if needEscapeHTML[s[i]] {
j = i
goto ESCAPE_END
}
}
// no found any escape characters.
return append(append(buf, s...), '"')
}
ESCAPE_END:
for j < valLen {
c := s[j]
if !needEscapeHTML[c] {
// fast path: most of the time, printable ascii characters are used
j++
continue
}
switch c {
case '\\', '"':
buf = append(buf, s[i:j]...)
buf = append(buf, '\\', c)
i = j + 1
j = j + 1
continue
case '\n':
buf = append(buf, s[i:j]...)
buf = append(buf, '\\', 'n')
i = j + 1
j = j + 1
continue
case '\r':
buf = append(buf, s[i:j]...)
buf = append(buf, '\\', 'r')
i = j + 1
j = j + 1
continue
case '\t':
buf = append(buf, s[i:j]...)
buf = append(buf, '\\', 't')
i = j + 1
j = j + 1
continue
case '<', '>', '&':
buf = append(buf, s[i:j]...)
buf = append(buf, `\u00`...)
buf = append(buf, hex[c>>4], hex[c&0xF])
i = j + 1
j = j + 1
continue
case 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x0B, 0x0C, 0x0E, 0x0F, // 0x00-0x0F
0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F: // 0x10-0x1F
buf = append(buf, s[i:j]...)
buf = append(buf, `\u00`...)
buf = append(buf, hex[c>>4], hex[c&0xF])
i = j + 1
j = j + 1
continue
}
j++
}
return append(append(buf, s[i:]...), '"')
}
func appendNormalizedString(buf []byte, s string) []byte {
valLen := len(s)
if valLen == 0 {
return append(buf, `""`...)
}
buf = append(buf, '"')
var (
i, j int
)
if valLen >= 8 {
chunks := stringToUint64Slice(s)
for _, n := range chunks {
// combine masks before checking for the MSB of each byte. We include
// `n` in the mask to check whether any of the *input* byte MSBs were
// set (i.e. the byte was outside the ASCII range).
mask := n | (n - (lsb * 0x20)) |
((n ^ (lsb * '"')) - lsb) |
((n ^ (lsb * '\\')) - lsb)
if (mask & msb) != 0 {
j = bits.TrailingZeros64(mask&msb) / 8
goto ESCAPE_END
}
}
valLen := len(s)
for i := len(chunks) * 8; i < valLen; i++ {
if needEscapeNormalizeUTF8[s[i]] {
j = i
goto ESCAPE_END
}
}
return append(append(buf, s...), '"')
}
ESCAPE_END:
for j < valLen {
c := s[j]
if !needEscapeNormalizeUTF8[c] {
// fast path: most of the time, printable ascii characters are used
j++
continue
}
switch c {
case '\\', '"':
buf = append(buf, s[i:j]...)
buf = append(buf, '\\', c)
i = j + 1
j = j + 1
continue
case '\n':
buf = append(buf, s[i:j]...)
buf = append(buf, '\\', 'n')
i = j + 1
j = j + 1
continue
case '\r':
buf = append(buf, s[i:j]...)
buf = append(buf, '\\', 'r')
i = j + 1
j = j + 1
continue
case '\t':
buf = append(buf, s[i:j]...)
buf = append(buf, '\\', 't')
i = j + 1
j = j + 1
continue
case 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x0B, 0x0C, 0x0E, 0x0F, // 0x00-0x0F
0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F: // 0x10-0x1F
buf = append(buf, s[i:j]...)
buf = append(buf, `\u00`...)
buf = append(buf, hex[c>>4], hex[c&0xF])
@ -501,18 +362,14 @@ ESCAPE_END:
continue
}
r, size := utf8.DecodeRuneInString(s[j:])
if r == utf8.RuneError && size == 1 {
state, size := decodeRuneInString(s[j:])
switch state {
case runeErrorState:
buf = append(buf, s[i:j]...)
buf = append(buf, `\ufffd`...)
i = j + size
j = j + size
i = j + 1
j = j + 1
continue
}
switch r {
case '\u2028', '\u2029':
// U+2028 is LINE SEPARATOR.
// U+2029 is PARAGRAPH SEPARATOR.
// They are both technically valid characters in JSON strings,
@ -520,14 +377,19 @@ ESCAPE_END:
// and can lead to security holes there. It is valid JSON to
// escape them, so we do so unconditionally.
// See http://timelessrepo.com/json-isnt-a-javascript-subset for discussion.
case lineSepState:
buf = append(buf, s[i:j]...)
buf = append(buf, `\u202`...)
buf = append(buf, hex[r&0xF])
i = j + size
j = j + size
buf = append(buf, `\u2028`...)
i = j + 3
j = j + 3
continue
case paragraphSepState:
buf = append(buf, s[i:j]...)
buf = append(buf, `\u2029`...)
i = j + 3
j = j + 3
continue
}
j += size
}
@ -540,19 +402,37 @@ func appendString(buf []byte, s string) []byte {
return append(buf, `""`...)
}
buf = append(buf, '"')
var escapeIdx int
var (
i, j int
)
if valLen >= 8 {
if escapeIdx = escapeIndex(s); escapeIdx < 0 {
chunks := stringToUint64Slice(s)
for _, n := range chunks {
// combine masks before checking for the MSB of each byte. We include
// `n` in the mask to check whether any of the *input* byte MSBs were
// set (i.e. the byte was outside the ASCII range).
mask := n | (n - (lsb * 0x20)) |
((n ^ (lsb * '"')) - lsb) |
((n ^ (lsb * '\\')) - lsb)
if (mask & msb) != 0 {
j = bits.TrailingZeros64(mask&msb) / 8
goto ESCAPE_END
}
}
valLen := len(s)
for i := len(chunks) * 8; i < valLen; i++ {
if needEscape[s[i]] {
j = i
goto ESCAPE_END
}
}
return append(append(buf, s...), '"')
}
}
i := 0
j := escapeIdx
ESCAPE_END:
for j < valLen {
c := s[j]
if c >= 0x20 && c <= 0x7f && c != '\\' && c != '"' {
if !needEscape[c] {
// fast path: most of the time, printable ascii characters are used
j++
continue
@ -587,7 +467,8 @@ func appendString(buf []byte, s string) []byte {
j = j + 1
continue
case '<', '>', '&':
case 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x0B, 0x0C, 0x0E, 0x0F, // 0x00-0x0F
0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F: // 0x10-0x1F
buf = append(buf, s[i:j]...)
buf = append(buf, `\u00`...)
buf = append(buf, hex[c>>4], hex[c&0xF])
@ -595,45 +476,7 @@ func appendString(buf []byte, s string) []byte {
j = j + 1
continue
}
// This encodes bytes < 0x20 except for \t, \n and \r.
if c < 0x20 {
buf = append(buf, s[i:j]...)
buf = append(buf, `\u00`...)
buf = append(buf, hex[c>>4], hex[c&0xF])
i = j + 1
j = j + 1
continue
}
r, size := utf8.DecodeRuneInString(s[j:])
if r == utf8.RuneError && size == 1 {
buf = append(buf, s[i:j]...)
buf = append(buf, `\ufffd`...)
i = j + size
j = j + size
continue
}
switch r {
case '\u2028', '\u2029':
// U+2028 is LINE SEPARATOR.
// U+2029 is PARAGRAPH SEPARATOR.
// They are both technically valid characters in JSON strings,
// but don't work in JSONP, which has to be evaluated as JavaScript,
// and can lead to security holes there. It is valid JSON to
// escape them, so we do so unconditionally.
// See http://timelessrepo.com/json-isnt-a-javascript-subset for discussion.
buf = append(buf, s[i:j]...)
buf = append(buf, `\u202`...)
buf = append(buf, hex[r&0xF])
i = j + size
j = j + size
continue
}
j += size
j++
}
return append(append(buf, s[i:]...), '"')

View File

@ -0,0 +1,415 @@
package encoder
var needEscapeHTMLNormalizeUTF8 = [256]bool{
'"': true,
'&': true,
'<': true,
'>': true,
'\\': true,
0x00: true,
0x01: true,
0x02: true,
0x03: true,
0x04: true,
0x05: true,
0x06: true,
0x07: true,
0x08: true,
0x09: true,
0x0a: true,
0x0b: true,
0x0c: true,
0x0d: true,
0x0e: true,
0x0f: true,
0x10: true,
0x11: true,
0x12: true,
0x13: true,
0x14: true,
0x15: true,
0x16: true,
0x17: true,
0x18: true,
0x19: true,
0x1a: true,
0x1b: true,
0x1c: true,
0x1d: true,
0x1e: true,
0x1f: true,
/* 0x20 - 0x7f */
0x80: true,
0x81: true,
0x82: true,
0x83: true,
0x84: true,
0x85: true,
0x86: true,
0x87: true,
0x88: true,
0x89: true,
0x8a: true,
0x8b: true,
0x8c: true,
0x8d: true,
0x8e: true,
0x8f: true,
0x90: true,
0x91: true,
0x92: true,
0x93: true,
0x94: true,
0x95: true,
0x96: true,
0x97: true,
0x98: true,
0x99: true,
0x9a: true,
0x9b: true,
0x9c: true,
0x9d: true,
0x9e: true,
0x9f: true,
0xa0: true,
0xa1: true,
0xa2: true,
0xa3: true,
0xa4: true,
0xa5: true,
0xa6: true,
0xa7: true,
0xa8: true,
0xa9: true,
0xaa: true,
0xab: true,
0xac: true,
0xad: true,
0xae: true,
0xaf: true,
0xb0: true,
0xb1: true,
0xb2: true,
0xb3: true,
0xb4: true,
0xb5: true,
0xb6: true,
0xb7: true,
0xb8: true,
0xb9: true,
0xba: true,
0xbb: true,
0xbc: true,
0xbd: true,
0xbe: true,
0xbf: true,
0xc0: true,
0xc1: true,
0xc2: true,
0xc3: true,
0xc4: true,
0xc5: true,
0xc6: true,
0xc7: true,
0xc8: true,
0xc9: true,
0xca: true,
0xcb: true,
0xcc: true,
0xcd: true,
0xce: true,
0xcf: true,
0xd0: true,
0xd1: true,
0xd2: true,
0xd3: true,
0xd4: true,
0xd5: true,
0xd6: true,
0xd7: true,
0xd8: true,
0xd9: true,
0xda: true,
0xdb: true,
0xdc: true,
0xdd: true,
0xde: true,
0xdf: true,
0xe0: true,
0xe1: true,
0xe2: true,
0xe3: true,
0xe4: true,
0xe5: true,
0xe6: true,
0xe7: true,
0xe8: true,
0xe9: true,
0xea: true,
0xeb: true,
0xec: true,
0xed: true,
0xee: true,
0xef: true,
0xf0: true,
0xf1: true,
0xf2: true,
0xf3: true,
0xf4: true,
0xf5: true,
0xf6: true,
0xf7: true,
0xf8: true,
0xf9: true,
0xfa: true,
0xfb: true,
0xfc: true,
0xfd: true,
0xfe: true,
0xff: true,
}
var needEscapeNormalizeUTF8 = [256]bool{
'"': true,
'\\': true,
0x00: true,
0x01: true,
0x02: true,
0x03: true,
0x04: true,
0x05: true,
0x06: true,
0x07: true,
0x08: true,
0x09: true,
0x0a: true,
0x0b: true,
0x0c: true,
0x0d: true,
0x0e: true,
0x0f: true,
0x10: true,
0x11: true,
0x12: true,
0x13: true,
0x14: true,
0x15: true,
0x16: true,
0x17: true,
0x18: true,
0x19: true,
0x1a: true,
0x1b: true,
0x1c: true,
0x1d: true,
0x1e: true,
0x1f: true,
/* 0x20 - 0x7f */
0x80: true,
0x81: true,
0x82: true,
0x83: true,
0x84: true,
0x85: true,
0x86: true,
0x87: true,
0x88: true,
0x89: true,
0x8a: true,
0x8b: true,
0x8c: true,
0x8d: true,
0x8e: true,
0x8f: true,
0x90: true,
0x91: true,
0x92: true,
0x93: true,
0x94: true,
0x95: true,
0x96: true,
0x97: true,
0x98: true,
0x99: true,
0x9a: true,
0x9b: true,
0x9c: true,
0x9d: true,
0x9e: true,
0x9f: true,
0xa0: true,
0xa1: true,
0xa2: true,
0xa3: true,
0xa4: true,
0xa5: true,
0xa6: true,
0xa7: true,
0xa8: true,
0xa9: true,
0xaa: true,
0xab: true,
0xac: true,
0xad: true,
0xae: true,
0xaf: true,
0xb0: true,
0xb1: true,
0xb2: true,
0xb3: true,
0xb4: true,
0xb5: true,
0xb6: true,
0xb7: true,
0xb8: true,
0xb9: true,
0xba: true,
0xbb: true,
0xbc: true,
0xbd: true,
0xbe: true,
0xbf: true,
0xc0: true,
0xc1: true,
0xc2: true,
0xc3: true,
0xc4: true,
0xc5: true,
0xc6: true,
0xc7: true,
0xc8: true,
0xc9: true,
0xca: true,
0xcb: true,
0xcc: true,
0xcd: true,
0xce: true,
0xcf: true,
0xd0: true,
0xd1: true,
0xd2: true,
0xd3: true,
0xd4: true,
0xd5: true,
0xd6: true,
0xd7: true,
0xd8: true,
0xd9: true,
0xda: true,
0xdb: true,
0xdc: true,
0xdd: true,
0xde: true,
0xdf: true,
0xe0: true,
0xe1: true,
0xe2: true,
0xe3: true,
0xe4: true,
0xe5: true,
0xe6: true,
0xe7: true,
0xe8: true,
0xe9: true,
0xea: true,
0xeb: true,
0xec: true,
0xed: true,
0xee: true,
0xef: true,
0xf0: true,
0xf1: true,
0xf2: true,
0xf3: true,
0xf4: true,
0xf5: true,
0xf6: true,
0xf7: true,
0xf8: true,
0xf9: true,
0xfa: true,
0xfb: true,
0xfc: true,
0xfd: true,
0xfe: true,
0xff: true,
}
var needEscapeHTML = [256]bool{
'"': true,
'&': true,
'<': true,
'>': true,
'\\': true,
0x00: true,
0x01: true,
0x02: true,
0x03: true,
0x04: true,
0x05: true,
0x06: true,
0x07: true,
0x08: true,
0x09: true,
0x0a: true,
0x0b: true,
0x0c: true,
0x0d: true,
0x0e: true,
0x0f: true,
0x10: true,
0x11: true,
0x12: true,
0x13: true,
0x14: true,
0x15: true,
0x16: true,
0x17: true,
0x18: true,
0x19: true,
0x1a: true,
0x1b: true,
0x1c: true,
0x1d: true,
0x1e: true,
0x1f: true,
/* 0x20 - 0xff */
}
var needEscape = [256]bool{
'"': true,
'\\': true,
0x00: true,
0x01: true,
0x02: true,
0x03: true,
0x04: true,
0x05: true,
0x06: true,
0x07: true,
0x08: true,
0x09: true,
0x0a: true,
0x0b: true,
0x0c: true,
0x0d: true,
0x0e: true,
0x0f: true,
0x10: true,
0x11: true,
0x12: true,
0x13: true,
0x14: true,
0x15: true,
0x16: true,
0x17: true,
0x18: true,
0x19: true,
0x1a: true,
0x1b: true,
0x1c: true,
0x1d: true,
0x1e: true,
0x1f: true,
/* 0x20 - 0xff */
}

View File

@ -2,6 +2,7 @@ package vm
import (
"fmt"
"io"
"github.com/goccy/go-json/internal/encoder"
)
@ -14,18 +15,24 @@ func DebugRun(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet)
} else {
code = codeSet.NoescapeKeyCode
}
if wc := ctx.Option.DebugDOTOut; wc != nil {
_, _ = io.WriteString(wc, code.DumpDOT())
wc.Close()
ctx.Option.DebugDOTOut = nil
}
if err := recover(); err != nil {
fmt.Println("=============[DEBUG]===============")
fmt.Println("* [TYPE]")
fmt.Println(codeSet.Type)
fmt.Printf("\n")
fmt.Println("* [ALL OPCODE]")
fmt.Println(code.Dump())
fmt.Printf("\n")
fmt.Println("* [CONTEXT]")
fmt.Printf("%+v\n", ctx)
fmt.Println("===================================")
w := ctx.Option.DebugOut
fmt.Fprintln(w, "=============[DEBUG]===============")
fmt.Fprintln(w, "* [TYPE]")
fmt.Fprintln(w, codeSet.Type)
fmt.Fprintf(w, "\n")
fmt.Fprintln(w, "* [ALL OPCODE]")
fmt.Fprintln(w, code.Dump())
fmt.Fprintf(w, "\n")
fmt.Fprintln(w, "* [CONTEXT]")
fmt.Fprintf(w, "%+v\n", ctx)
fmt.Fprintln(w, "===================================")
panic(err)
}
}()

View File

@ -3,6 +3,7 @@ package vm
import (
"math"
"reflect"
"sort"
"unsafe"
@ -194,12 +195,15 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
typ = iface.typ
}
if ifacePtr == nil {
isDirectedNil := typ != nil && typ.Kind() == reflect.Struct && !runtime.IfaceIndir(typ)
if !isDirectedNil {
b = appendNullComma(ctx, b)
code = code.Next
break
}
}
ctx.KeepRefs = append(ctx.KeepRefs, up)
ifaceCodeSet, err := encoder.CompileToGetCodeSet(uintptr(unsafe.Pointer(typ)))
ifaceCodeSet, err := encoder.CompileToGetCodeSet(ctx, uintptr(unsafe.Pointer(typ)))
if err != nil {
return nil, err
}
@ -218,8 +222,7 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
oldOffset := ptrOffset
ptrOffset += totalLength * uintptrSize
oldBaseIndent := ctx.BaseIndent
indentDiffFromTop := c.Indent - 1
ctx.BaseIndent += code.Indent - indentDiffFromTop
ctx.BaseIndent += code.Indent
newLen := offsetNum + totalLength + nextTotalLength
if curlen < newLen {
@ -403,48 +406,42 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
break
}
b = appendStructHead(ctx, b)
iter := mapiterinit(code.Type, uptr)
ctx.KeepRefs = append(ctx.KeepRefs, iter)
store(ctxptr, code.ElemIdx, 0)
store(ctxptr, code.Length, uintptr(mlen))
store(ctxptr, code.MapIter, uintptr(iter))
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
unorderedMap := (ctx.Option.Flag & encoder.UnorderedMapOption) != 0
mapCtx := encoder.NewMapContext(mlen, unorderedMap)
mapiterinit(code.Type, uptr, &mapCtx.Iter)
store(ctxptr, code.Idx, uintptr(unsafe.Pointer(mapCtx)))
ctx.KeepRefs = append(ctx.KeepRefs, unsafe.Pointer(mapCtx))
if unorderedMap {
b = appendMapKeyIndent(ctx, code.Next, b)
} else {
mapCtx := encoder.NewMapContext(mlen)
mapCtx.Pos = append(mapCtx.Pos, len(b))
ctx.KeepRefs = append(ctx.KeepRefs, unsafe.Pointer(mapCtx))
store(ctxptr, code.End.MapPos, uintptr(unsafe.Pointer(mapCtx)))
mapCtx.Start = len(b)
mapCtx.First = len(b)
}
key := mapiterkey(iter)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
case encoder.OpMapKey:
idx := load(ctxptr, code.ElemIdx)
length := load(ctxptr, code.Length)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
idx := mapCtx.Idx
idx++
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
if idx < length {
if idx < mapCtx.Len {
b = appendMapKeyIndent(ctx, code, b)
store(ctxptr, code.ElemIdx, idx)
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
key := mapiterkey(iter)
mapCtx.Idx = int(idx)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
} else {
b = appendObjectEnd(ctx, code, b)
encoder.ReleaseMapContext(mapCtx)
code = code.End.Next
}
} else {
ptr := load(ctxptr, code.End.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
mapCtx.Pos = append(mapCtx.Pos, len(b))
if idx < length {
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
store(ctxptr, code.ElemIdx, idx)
key := mapiterkey(iter)
mapCtx.Slice.Items[mapCtx.Idx].Value = b[mapCtx.Start:len(b)]
if idx < mapCtx.Len {
mapCtx.Idx = int(idx)
mapCtx.Start = len(b)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
} else {
@ -452,46 +449,27 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
}
}
case encoder.OpMapValue:
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
b = appendColon(ctx, b)
} else {
ptr := load(ctxptr, code.End.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
mapCtx.Pos = append(mapCtx.Pos, len(b))
mapCtx.Slice.Items[mapCtx.Idx].Key = b[mapCtx.Start:len(b)]
mapCtx.Start = len(b)
}
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
value := mapitervalue(iter)
value := mapitervalue(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(value))
mapiternext(iter)
mapiternext(&mapCtx.Iter)
code = code.Next
case encoder.OpMapEnd:
// this operation only used by sorted map.
length := int(load(ctxptr, code.Length))
ptr := load(ctxptr, code.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
pos := mapCtx.Pos
for i := 0; i < length; i++ {
startKey := pos[i*2]
startValue := pos[i*2+1]
var endValue int
if i+1 < length {
endValue = pos[i*2+2]
} else {
endValue = len(b)
}
mapCtx.Slice.Items = append(mapCtx.Slice.Items, encoder.MapItem{
Key: b[startKey:startValue],
Value: b[startValue:endValue],
})
}
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
sort.Sort(mapCtx.Slice)
buf := mapCtx.Buf
for _, item := range mapCtx.Slice.Items {
buf = appendMapKeyValue(ctx, code, buf, item.Key, item.Value)
}
buf = appendMapEnd(ctx, code, buf)
b = b[:pos[0]]
b = b[:mapCtx.First]
b = append(b, buf...)
mapCtx.Buf = buf
encoder.ReleaseMapContext(mapCtx)
@ -731,14 +709,15 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
u64 := ptrToUint64(p+uintptr(code.Offset), code.NumBitSize)
p += uintptr(code.Offset)
u64 := ptrToUint64(p, code.NumBitSize)
v := u64 & ((1 << code.NumBitSize) - 1)
if v == 0 {
code = code.NextField
} else {
b = appendStructKey(ctx, code, b)
b = append(b, '"')
b = appendInt(ctx, b, p+uintptr(code.Offset), code)
b = appendInt(ctx, b, p, code)
b = append(b, '"')
b = appendComma(ctx, b)
code = code.Next
@ -2979,9 +2958,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
b = appendStructHead(ctx, b)
}
b = appendStructKey(ctx, code, b)
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadMarshalJSON {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {
@ -3020,9 +3000,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadOmitEmptyMarshalJSON {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
iface := ptrToInterface(code, p)
@ -3140,9 +3121,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
b = appendStructHead(ctx, b)
}
b = appendStructKey(ctx, code, b)
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadMarshalText {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {
@ -3181,9 +3163,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadOmitEmptyMarshalText {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {

View File

@ -16,16 +16,17 @@ func DebugRun(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet)
defer func() {
if err := recover(); err != nil {
fmt.Println("=============[DEBUG]===============")
fmt.Println("* [TYPE]")
fmt.Println(codeSet.Type)
fmt.Printf("\n")
fmt.Println("* [ALL OPCODE]")
fmt.Println(code.Dump())
fmt.Printf("\n")
fmt.Println("* [CONTEXT]")
fmt.Printf("%+v\n", ctx)
fmt.Println("===================================")
w := ctx.Option.DebugOut
fmt.Fprintln(w, "=============[DEBUG]===============")
fmt.Fprintln(w, "* [TYPE]")
fmt.Fprintln(w, codeSet.Type)
fmt.Fprintf(w, "\n")
fmt.Fprintln(w, "* [ALL OPCODE]")
fmt.Fprintln(w, code.Dump())
fmt.Fprintf(w, "\n")
fmt.Fprintln(w, "* [CONTEXT]")
fmt.Fprintf(w, "%+v\n", ctx)
fmt.Fprintln(w, "===================================")
panic(err)
}
}()

View File

@ -3,6 +3,7 @@ package vm_color
import (
"math"
"reflect"
"sort"
"unsafe"
@ -194,12 +195,15 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
typ = iface.typ
}
if ifacePtr == nil {
isDirectedNil := typ != nil && typ.Kind() == reflect.Struct && !runtime.IfaceIndir(typ)
if !isDirectedNil {
b = appendNullComma(ctx, b)
code = code.Next
break
}
}
ctx.KeepRefs = append(ctx.KeepRefs, up)
ifaceCodeSet, err := encoder.CompileToGetCodeSet(uintptr(unsafe.Pointer(typ)))
ifaceCodeSet, err := encoder.CompileToGetCodeSet(ctx, uintptr(unsafe.Pointer(typ)))
if err != nil {
return nil, err
}
@ -218,8 +222,7 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
oldOffset := ptrOffset
ptrOffset += totalLength * uintptrSize
oldBaseIndent := ctx.BaseIndent
indentDiffFromTop := c.Indent - 1
ctx.BaseIndent += code.Indent - indentDiffFromTop
ctx.BaseIndent += code.Indent
newLen := offsetNum + totalLength + nextTotalLength
if curlen < newLen {
@ -403,48 +406,42 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
break
}
b = appendStructHead(ctx, b)
iter := mapiterinit(code.Type, uptr)
ctx.KeepRefs = append(ctx.KeepRefs, iter)
store(ctxptr, code.ElemIdx, 0)
store(ctxptr, code.Length, uintptr(mlen))
store(ctxptr, code.MapIter, uintptr(iter))
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
unorderedMap := (ctx.Option.Flag & encoder.UnorderedMapOption) != 0
mapCtx := encoder.NewMapContext(mlen, unorderedMap)
mapiterinit(code.Type, uptr, &mapCtx.Iter)
store(ctxptr, code.Idx, uintptr(unsafe.Pointer(mapCtx)))
ctx.KeepRefs = append(ctx.KeepRefs, unsafe.Pointer(mapCtx))
if unorderedMap {
b = appendMapKeyIndent(ctx, code.Next, b)
} else {
mapCtx := encoder.NewMapContext(mlen)
mapCtx.Pos = append(mapCtx.Pos, len(b))
ctx.KeepRefs = append(ctx.KeepRefs, unsafe.Pointer(mapCtx))
store(ctxptr, code.End.MapPos, uintptr(unsafe.Pointer(mapCtx)))
mapCtx.Start = len(b)
mapCtx.First = len(b)
}
key := mapiterkey(iter)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
case encoder.OpMapKey:
idx := load(ctxptr, code.ElemIdx)
length := load(ctxptr, code.Length)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
idx := mapCtx.Idx
idx++
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
if idx < length {
if idx < mapCtx.Len {
b = appendMapKeyIndent(ctx, code, b)
store(ctxptr, code.ElemIdx, idx)
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
key := mapiterkey(iter)
mapCtx.Idx = int(idx)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
} else {
b = appendObjectEnd(ctx, code, b)
encoder.ReleaseMapContext(mapCtx)
code = code.End.Next
}
} else {
ptr := load(ctxptr, code.End.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
mapCtx.Pos = append(mapCtx.Pos, len(b))
if idx < length {
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
store(ctxptr, code.ElemIdx, idx)
key := mapiterkey(iter)
mapCtx.Slice.Items[mapCtx.Idx].Value = b[mapCtx.Start:len(b)]
if idx < mapCtx.Len {
mapCtx.Idx = int(idx)
mapCtx.Start = len(b)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
} else {
@ -452,46 +449,27 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
}
}
case encoder.OpMapValue:
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
b = appendColon(ctx, b)
} else {
ptr := load(ctxptr, code.End.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
mapCtx.Pos = append(mapCtx.Pos, len(b))
mapCtx.Slice.Items[mapCtx.Idx].Key = b[mapCtx.Start:len(b)]
mapCtx.Start = len(b)
}
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
value := mapitervalue(iter)
value := mapitervalue(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(value))
mapiternext(iter)
mapiternext(&mapCtx.Iter)
code = code.Next
case encoder.OpMapEnd:
// this operation only used by sorted map.
length := int(load(ctxptr, code.Length))
ptr := load(ctxptr, code.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
pos := mapCtx.Pos
for i := 0; i < length; i++ {
startKey := pos[i*2]
startValue := pos[i*2+1]
var endValue int
if i+1 < length {
endValue = pos[i*2+2]
} else {
endValue = len(b)
}
mapCtx.Slice.Items = append(mapCtx.Slice.Items, encoder.MapItem{
Key: b[startKey:startValue],
Value: b[startValue:endValue],
})
}
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
sort.Sort(mapCtx.Slice)
buf := mapCtx.Buf
for _, item := range mapCtx.Slice.Items {
buf = appendMapKeyValue(ctx, code, buf, item.Key, item.Value)
}
buf = appendMapEnd(ctx, code, buf)
b = b[:pos[0]]
b = b[:mapCtx.First]
b = append(b, buf...)
mapCtx.Buf = buf
encoder.ReleaseMapContext(mapCtx)
@ -731,14 +709,15 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
u64 := ptrToUint64(p+uintptr(code.Offset), code.NumBitSize)
p += uintptr(code.Offset)
u64 := ptrToUint64(p, code.NumBitSize)
v := u64 & ((1 << code.NumBitSize) - 1)
if v == 0 {
code = code.NextField
} else {
b = appendStructKey(ctx, code, b)
b = append(b, '"')
b = appendInt(ctx, b, p+uintptr(code.Offset), code)
b = appendInt(ctx, b, p, code)
b = append(b, '"')
b = appendComma(ctx, b)
code = code.Next
@ -2979,9 +2958,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
b = appendStructHead(ctx, b)
}
b = appendStructKey(ctx, code, b)
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadMarshalJSON {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {
@ -3020,9 +3000,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadOmitEmptyMarshalJSON {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
iface := ptrToInterface(code, p)
@ -3140,9 +3121,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
b = appendStructHead(ctx, b)
}
b = appendStructKey(ctx, code, b)
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadMarshalText {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {
@ -3181,9 +3163,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadOmitEmptyMarshalText {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {

View File

@ -16,16 +16,17 @@ func DebugRun(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet)
defer func() {
if err := recover(); err != nil {
fmt.Println("=============[DEBUG]===============")
fmt.Println("* [TYPE]")
fmt.Println(codeSet.Type)
fmt.Printf("\n")
fmt.Println("* [ALL OPCODE]")
fmt.Println(code.Dump())
fmt.Printf("\n")
fmt.Println("* [CONTEXT]")
fmt.Printf("%+v\n", ctx)
fmt.Println("===================================")
w := ctx.Option.DebugOut
fmt.Fprintln(w, "=============[DEBUG]===============")
fmt.Fprintln(w, "* [TYPE]")
fmt.Fprintln(w, codeSet.Type)
fmt.Fprintf(w, "\n")
fmt.Fprintln(w, "* [ALL OPCODE]")
fmt.Fprintln(w, code.Dump())
fmt.Fprintf(w, "\n")
fmt.Fprintln(w, "* [CONTEXT]")
fmt.Fprintf(w, "%+v\n", ctx)
fmt.Fprintln(w, "===================================")
panic(err)
}
}()

View File

@ -189,7 +189,7 @@ func appendNullComma(ctx *encoder.RuntimeContext, b []byte) []byte {
}
func appendColon(_ *encoder.RuntimeContext, b []byte) []byte {
return append(b, ':', ' ')
return append(b[:len(b)-2], ':', ' ')
}
func appendMapKeyValue(ctx *encoder.RuntimeContext, code *encoder.Opcode, b, key, value []byte) []byte {
@ -229,8 +229,9 @@ func appendEmptyObject(_ *encoder.RuntimeContext, b []byte) []byte {
func appendObjectEnd(ctx *encoder.RuntimeContext, code *encoder.Opcode, b []byte) []byte {
last := len(b) - 1
b[last] = '\n'
b = appendIndent(ctx, b, code.Indent-1)
// replace comma to newline
b[last-1] = '\n'
b = appendIndent(ctx, b[:last], code.Indent)
return append(b, '}', ',', '\n')
}

View File

@ -3,6 +3,7 @@ package vm_color_indent
import (
"math"
"reflect"
"sort"
"unsafe"
@ -194,12 +195,15 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
typ = iface.typ
}
if ifacePtr == nil {
isDirectedNil := typ != nil && typ.Kind() == reflect.Struct && !runtime.IfaceIndir(typ)
if !isDirectedNil {
b = appendNullComma(ctx, b)
code = code.Next
break
}
}
ctx.KeepRefs = append(ctx.KeepRefs, up)
ifaceCodeSet, err := encoder.CompileToGetCodeSet(uintptr(unsafe.Pointer(typ)))
ifaceCodeSet, err := encoder.CompileToGetCodeSet(ctx, uintptr(unsafe.Pointer(typ)))
if err != nil {
return nil, err
}
@ -218,8 +222,7 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
oldOffset := ptrOffset
ptrOffset += totalLength * uintptrSize
oldBaseIndent := ctx.BaseIndent
indentDiffFromTop := c.Indent - 1
ctx.BaseIndent += code.Indent - indentDiffFromTop
ctx.BaseIndent += code.Indent
newLen := offsetNum + totalLength + nextTotalLength
if curlen < newLen {
@ -403,48 +406,42 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
break
}
b = appendStructHead(ctx, b)
iter := mapiterinit(code.Type, uptr)
ctx.KeepRefs = append(ctx.KeepRefs, iter)
store(ctxptr, code.ElemIdx, 0)
store(ctxptr, code.Length, uintptr(mlen))
store(ctxptr, code.MapIter, uintptr(iter))
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
unorderedMap := (ctx.Option.Flag & encoder.UnorderedMapOption) != 0
mapCtx := encoder.NewMapContext(mlen, unorderedMap)
mapiterinit(code.Type, uptr, &mapCtx.Iter)
store(ctxptr, code.Idx, uintptr(unsafe.Pointer(mapCtx)))
ctx.KeepRefs = append(ctx.KeepRefs, unsafe.Pointer(mapCtx))
if unorderedMap {
b = appendMapKeyIndent(ctx, code.Next, b)
} else {
mapCtx := encoder.NewMapContext(mlen)
mapCtx.Pos = append(mapCtx.Pos, len(b))
ctx.KeepRefs = append(ctx.KeepRefs, unsafe.Pointer(mapCtx))
store(ctxptr, code.End.MapPos, uintptr(unsafe.Pointer(mapCtx)))
mapCtx.Start = len(b)
mapCtx.First = len(b)
}
key := mapiterkey(iter)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
case encoder.OpMapKey:
idx := load(ctxptr, code.ElemIdx)
length := load(ctxptr, code.Length)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
idx := mapCtx.Idx
idx++
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
if idx < length {
if idx < mapCtx.Len {
b = appendMapKeyIndent(ctx, code, b)
store(ctxptr, code.ElemIdx, idx)
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
key := mapiterkey(iter)
mapCtx.Idx = int(idx)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
} else {
b = appendObjectEnd(ctx, code, b)
encoder.ReleaseMapContext(mapCtx)
code = code.End.Next
}
} else {
ptr := load(ctxptr, code.End.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
mapCtx.Pos = append(mapCtx.Pos, len(b))
if idx < length {
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
store(ctxptr, code.ElemIdx, idx)
key := mapiterkey(iter)
mapCtx.Slice.Items[mapCtx.Idx].Value = b[mapCtx.Start:len(b)]
if idx < mapCtx.Len {
mapCtx.Idx = int(idx)
mapCtx.Start = len(b)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
} else {
@ -452,46 +449,27 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
}
}
case encoder.OpMapValue:
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
b = appendColon(ctx, b)
} else {
ptr := load(ctxptr, code.End.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
mapCtx.Pos = append(mapCtx.Pos, len(b))
mapCtx.Slice.Items[mapCtx.Idx].Key = b[mapCtx.Start:len(b)]
mapCtx.Start = len(b)
}
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
value := mapitervalue(iter)
value := mapitervalue(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(value))
mapiternext(iter)
mapiternext(&mapCtx.Iter)
code = code.Next
case encoder.OpMapEnd:
// this operation only used by sorted map.
length := int(load(ctxptr, code.Length))
ptr := load(ctxptr, code.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
pos := mapCtx.Pos
for i := 0; i < length; i++ {
startKey := pos[i*2]
startValue := pos[i*2+1]
var endValue int
if i+1 < length {
endValue = pos[i*2+2]
} else {
endValue = len(b)
}
mapCtx.Slice.Items = append(mapCtx.Slice.Items, encoder.MapItem{
Key: b[startKey:startValue],
Value: b[startValue:endValue],
})
}
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
sort.Sort(mapCtx.Slice)
buf := mapCtx.Buf
for _, item := range mapCtx.Slice.Items {
buf = appendMapKeyValue(ctx, code, buf, item.Key, item.Value)
}
buf = appendMapEnd(ctx, code, buf)
b = b[:pos[0]]
b = b[:mapCtx.First]
b = append(b, buf...)
mapCtx.Buf = buf
encoder.ReleaseMapContext(mapCtx)
@ -731,14 +709,15 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
u64 := ptrToUint64(p+uintptr(code.Offset), code.NumBitSize)
p += uintptr(code.Offset)
u64 := ptrToUint64(p, code.NumBitSize)
v := u64 & ((1 << code.NumBitSize) - 1)
if v == 0 {
code = code.NextField
} else {
b = appendStructKey(ctx, code, b)
b = append(b, '"')
b = appendInt(ctx, b, p+uintptr(code.Offset), code)
b = appendInt(ctx, b, p, code)
b = append(b, '"')
b = appendComma(ctx, b)
code = code.Next
@ -2979,9 +2958,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
b = appendStructHead(ctx, b)
}
b = appendStructKey(ctx, code, b)
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadMarshalJSON {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {
@ -3020,9 +3000,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadOmitEmptyMarshalJSON {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
iface := ptrToInterface(code, p)
@ -3140,9 +3121,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
b = appendStructHead(ctx, b)
}
b = appendStructKey(ctx, code, b)
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadMarshalText {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {
@ -3181,9 +3163,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadOmitEmptyMarshalText {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {

View File

@ -16,16 +16,17 @@ func DebugRun(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet)
defer func() {
if err := recover(); err != nil {
fmt.Println("=============[DEBUG]===============")
fmt.Println("* [TYPE]")
fmt.Println(codeSet.Type)
fmt.Printf("\n")
fmt.Println("* [ALL OPCODE]")
fmt.Println(code.Dump())
fmt.Printf("\n")
fmt.Println("* [CONTEXT]")
fmt.Printf("%+v\n", ctx)
fmt.Println("===================================")
w := ctx.Option.DebugOut
fmt.Fprintln(w, "=============[DEBUG]===============")
fmt.Fprintln(w, "* [TYPE]")
fmt.Fprintln(w, codeSet.Type)
fmt.Fprintf(w, "\n")
fmt.Fprintln(w, "* [ALL OPCODE]")
fmt.Fprintln(w, code.Dump())
fmt.Fprintf(w, "\n")
fmt.Fprintln(w, "* [CONTEXT]")
fmt.Fprintf(w, "%+v\n", ctx)
fmt.Fprintln(w, "===================================")
panic(err)
}
}()

View File

@ -133,7 +133,7 @@ func appendNullComma(_ *encoder.RuntimeContext, b []byte) []byte {
}
func appendColon(_ *encoder.RuntimeContext, b []byte) []byte {
return append(b, ':', ' ')
return append(b[:len(b)-2], ':', ' ')
}
func appendMapKeyValue(ctx *encoder.RuntimeContext, code *encoder.Opcode, b, key, value []byte) []byte {
@ -173,8 +173,9 @@ func appendEmptyObject(_ *encoder.RuntimeContext, b []byte) []byte {
func appendObjectEnd(ctx *encoder.RuntimeContext, code *encoder.Opcode, b []byte) []byte {
last := len(b) - 1
b[last] = '\n'
b = appendIndent(ctx, b, code.Indent-1)
// replace comma to newline
b[last-1] = '\n'
b = appendIndent(ctx, b[:last], code.Indent)
return append(b, '}', ',', '\n')
}

View File

@ -3,6 +3,7 @@ package vm_indent
import (
"math"
"reflect"
"sort"
"unsafe"
@ -194,12 +195,15 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
typ = iface.typ
}
if ifacePtr == nil {
isDirectedNil := typ != nil && typ.Kind() == reflect.Struct && !runtime.IfaceIndir(typ)
if !isDirectedNil {
b = appendNullComma(ctx, b)
code = code.Next
break
}
}
ctx.KeepRefs = append(ctx.KeepRefs, up)
ifaceCodeSet, err := encoder.CompileToGetCodeSet(uintptr(unsafe.Pointer(typ)))
ifaceCodeSet, err := encoder.CompileToGetCodeSet(ctx, uintptr(unsafe.Pointer(typ)))
if err != nil {
return nil, err
}
@ -218,8 +222,7 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
oldOffset := ptrOffset
ptrOffset += totalLength * uintptrSize
oldBaseIndent := ctx.BaseIndent
indentDiffFromTop := c.Indent - 1
ctx.BaseIndent += code.Indent - indentDiffFromTop
ctx.BaseIndent += code.Indent
newLen := offsetNum + totalLength + nextTotalLength
if curlen < newLen {
@ -403,48 +406,42 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
break
}
b = appendStructHead(ctx, b)
iter := mapiterinit(code.Type, uptr)
ctx.KeepRefs = append(ctx.KeepRefs, iter)
store(ctxptr, code.ElemIdx, 0)
store(ctxptr, code.Length, uintptr(mlen))
store(ctxptr, code.MapIter, uintptr(iter))
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
unorderedMap := (ctx.Option.Flag & encoder.UnorderedMapOption) != 0
mapCtx := encoder.NewMapContext(mlen, unorderedMap)
mapiterinit(code.Type, uptr, &mapCtx.Iter)
store(ctxptr, code.Idx, uintptr(unsafe.Pointer(mapCtx)))
ctx.KeepRefs = append(ctx.KeepRefs, unsafe.Pointer(mapCtx))
if unorderedMap {
b = appendMapKeyIndent(ctx, code.Next, b)
} else {
mapCtx := encoder.NewMapContext(mlen)
mapCtx.Pos = append(mapCtx.Pos, len(b))
ctx.KeepRefs = append(ctx.KeepRefs, unsafe.Pointer(mapCtx))
store(ctxptr, code.End.MapPos, uintptr(unsafe.Pointer(mapCtx)))
mapCtx.Start = len(b)
mapCtx.First = len(b)
}
key := mapiterkey(iter)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
case encoder.OpMapKey:
idx := load(ctxptr, code.ElemIdx)
length := load(ctxptr, code.Length)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
idx := mapCtx.Idx
idx++
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
if idx < length {
if idx < mapCtx.Len {
b = appendMapKeyIndent(ctx, code, b)
store(ctxptr, code.ElemIdx, idx)
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
key := mapiterkey(iter)
mapCtx.Idx = int(idx)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
} else {
b = appendObjectEnd(ctx, code, b)
encoder.ReleaseMapContext(mapCtx)
code = code.End.Next
}
} else {
ptr := load(ctxptr, code.End.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
mapCtx.Pos = append(mapCtx.Pos, len(b))
if idx < length {
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
store(ctxptr, code.ElemIdx, idx)
key := mapiterkey(iter)
mapCtx.Slice.Items[mapCtx.Idx].Value = b[mapCtx.Start:len(b)]
if idx < mapCtx.Len {
mapCtx.Idx = int(idx)
mapCtx.Start = len(b)
key := mapiterkey(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(key))
code = code.Next
} else {
@ -452,46 +449,27 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
}
}
case encoder.OpMapValue:
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
if (ctx.Option.Flag & encoder.UnorderedMapOption) != 0 {
b = appendColon(ctx, b)
} else {
ptr := load(ctxptr, code.End.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
mapCtx.Pos = append(mapCtx.Pos, len(b))
mapCtx.Slice.Items[mapCtx.Idx].Key = b[mapCtx.Start:len(b)]
mapCtx.Start = len(b)
}
ptr := load(ctxptr, code.MapIter)
iter := ptrToUnsafePtr(ptr)
value := mapitervalue(iter)
value := mapitervalue(&mapCtx.Iter)
store(ctxptr, code.Next.Idx, uintptr(value))
mapiternext(iter)
mapiternext(&mapCtx.Iter)
code = code.Next
case encoder.OpMapEnd:
// this operation only used by sorted map.
length := int(load(ctxptr, code.Length))
ptr := load(ctxptr, code.MapPos)
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(ptr))
pos := mapCtx.Pos
for i := 0; i < length; i++ {
startKey := pos[i*2]
startValue := pos[i*2+1]
var endValue int
if i+1 < length {
endValue = pos[i*2+2]
} else {
endValue = len(b)
}
mapCtx.Slice.Items = append(mapCtx.Slice.Items, encoder.MapItem{
Key: b[startKey:startValue],
Value: b[startValue:endValue],
})
}
mapCtx := (*encoder.MapContext)(ptrToUnsafePtr(load(ctxptr, code.Idx)))
sort.Sort(mapCtx.Slice)
buf := mapCtx.Buf
for _, item := range mapCtx.Slice.Items {
buf = appendMapKeyValue(ctx, code, buf, item.Key, item.Value)
}
buf = appendMapEnd(ctx, code, buf)
b = b[:pos[0]]
b = b[:mapCtx.First]
b = append(b, buf...)
mapCtx.Buf = buf
encoder.ReleaseMapContext(mapCtx)
@ -731,14 +709,15 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
u64 := ptrToUint64(p+uintptr(code.Offset), code.NumBitSize)
p += uintptr(code.Offset)
u64 := ptrToUint64(p, code.NumBitSize)
v := u64 & ((1 << code.NumBitSize) - 1)
if v == 0 {
code = code.NextField
} else {
b = appendStructKey(ctx, code, b)
b = append(b, '"')
b = appendInt(ctx, b, p+uintptr(code.Offset), code)
b = appendInt(ctx, b, p, code)
b = append(b, '"')
b = appendComma(ctx, b)
code = code.Next
@ -2979,9 +2958,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
b = appendStructHead(ctx, b)
}
b = appendStructKey(ctx, code, b)
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadMarshalJSON {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {
@ -3020,9 +3000,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadOmitEmptyMarshalJSON {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
iface := ptrToInterface(code, p)
@ -3140,9 +3121,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
b = appendStructHead(ctx, b)
}
b = appendStructKey(ctx, code, b)
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadMarshalText {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {
@ -3181,9 +3163,10 @@ func Run(ctx *encoder.RuntimeContext, b []byte, codeSet *encoder.OpcodeSet) ([]b
if code.Flags&encoder.AnonymousHeadFlags == 0 {
b = appendStructHead(ctx, b)
}
p += uintptr(code.Offset)
if (code.Flags & encoder.IsNilableTypeFlags) != 0 {
if (code.Flags&encoder.IndirectFlags) != 0 || code.Op == encoder.OpStructPtrHeadOmitEmptyMarshalText {
p = ptrToPtr(p + uintptr(code.Offset))
p = ptrToPtr(p)
}
}
if p == 0 && (code.Flags&encoder.NilCheckFlags) != 0 {

View File

@ -162,3 +162,22 @@ func ErrInvalidBeginningOfValue(c byte, cursor int64) *SyntaxError {
Offset: cursor,
}
}
type PathError struct {
msg string
}
func (e *PathError) Error() string {
return fmt.Sprintf("json: invalid path format: %s", e.msg)
}
func ErrInvalidPath(msg string, args ...interface{}) *PathError {
if len(args) != 0 {
return &PathError{msg: fmt.Sprintf(msg, args...)}
}
return &PathError{msg: msg}
}
func ErrEmptyPath() *PathError {
return &PathError{msg: "path is empty"}
}

View File

@ -252,7 +252,6 @@ func IfaceIndir(*Type) bool
//go:noescape
func RType2Type(t *Type) reflect.Type
//go:nolint structcheck
type emptyInterface struct {
_ *Type
ptr unsafe.Pointer

View File

@ -13,7 +13,11 @@ func getTag(field reflect.StructField) string {
func IsIgnoredStructField(field reflect.StructField) bool {
if field.PkgPath != "" {
if field.Anonymous {
if !(field.Type.Kind() == reflect.Ptr && field.Type.Elem().Kind() == reflect.Struct) && field.Type.Kind() != reflect.Struct {
t := field.Type
if t.Kind() == reflect.Ptr {
t = t.Elem()
}
if t.Kind() != reflect.Struct {
return true
}
} else {

View File

@ -166,7 +166,6 @@ type UnmarshalerContext interface {
// JSON cannot represent cyclic data structures and Marshal does not
// handle them. Passing cyclic structures to Marshal will result in
// an infinite recursion.
//
func Marshal(v interface{}) ([]byte, error) {
return MarshalWithOption(v)
}
@ -264,14 +263,13 @@ func MarshalIndentWithOption(v interface{}, prefix, indent string, optFuncs ...E
//
// The JSON null value unmarshals into an interface, map, pointer, or slice
// by setting that Go value to nil. Because null is often used in JSON to mean
// ``not present,'' unmarshaling a JSON null into any other Go type has no effect
// “not present,” unmarshaling a JSON null into any other Go type has no effect
// on the value and produces no error.
//
// When unmarshaling quoted strings, invalid UTF-8 or
// invalid UTF-16 surrogate pairs are not treated as an error.
// Instead, they are replaced by the Unicode replacement
// character U+FFFD.
//
func Unmarshal(data []byte, v interface{}) error {
return unmarshal(data, v)
}
@ -299,7 +297,6 @@ func UnmarshalNoEscape(data []byte, v interface{}, optFuncs ...DecodeOptionFunc)
// Number, for JSON numbers
// string, for JSON string literals
// nil, for JSON null
//
type Token = json.Token
// A Number represents a JSON number literal.
@ -364,3 +361,8 @@ func Valid(data []byte) bool {
}
return decoder.InputOffset() >= int64(len(data))
}
func init() {
encoder.Marshal = Marshal
encoder.Unmarshal = Unmarshal
}

View File

@ -1,6 +1,8 @@
package json
import (
"io"
"github.com/goccy/go-json/internal/decoder"
"github.com/goccy/go-json/internal/encoder"
)
@ -15,6 +17,23 @@ func UnorderedMap() EncodeOptionFunc {
}
}
// DisableHTMLEscape disables escaping of HTML characters ( '&', '<', '>' ) when encoding string.
func DisableHTMLEscape() EncodeOptionFunc {
return func(opt *EncodeOption) {
opt.Flag &= ^encoder.HTMLEscapeOption
}
}
// DisableNormalizeUTF8
// By default, when encoding string, UTF8 characters in the range of 0x80 - 0xFF are processed by applying \ufffd for invalid code and escaping for \u2028 and \u2029.
// This option disables this behaviour. You can expect faster speeds by applying this option, but be careful.
// encoding/json implements here: https://github.com/golang/go/blob/6178d25fc0b28724b1b5aec2b1b74fc06d9294c7/src/encoding/json/encode.go#L1067-L1093.
func DisableNormalizeUTF8() EncodeOptionFunc {
return func(opt *EncodeOption) {
opt.Flag &= ^encoder.NormalizeUTF8Option
}
}
// Debug outputs debug information when panic occurs during encoding.
func Debug() EncodeOptionFunc {
return func(opt *EncodeOption) {
@ -22,6 +41,20 @@ func Debug() EncodeOptionFunc {
}
}
// DebugWith sets the destination to write debug messages.
func DebugWith(w io.Writer) EncodeOptionFunc {
return func(opt *EncodeOption) {
opt.DebugOut = w
}
}
// DebugDOT sets the destination to write opcodes graph.
func DebugDOT(w io.WriteCloser) EncodeOptionFunc {
return func(opt *EncodeOption) {
opt.DebugDOTOut = w
}
}
// Colorize add an identifier for coloring to the string of the encoded result.
func Colorize(scheme *ColorScheme) EncodeOptionFunc {
return func(opt *EncodeOption) {

84
vendor/github.com/goccy/go-json/path.go generated vendored Normal file
View File

@ -0,0 +1,84 @@
package json
import (
"reflect"
"github.com/goccy/go-json/internal/decoder"
)
// CreatePath creates JSON Path.
//
// JSON Path rule
// $ : root object or element. The JSON Path format must start with this operator, which refers to the outermost level of the JSON-formatted string.
// . : child operator. You can identify child values using dot-notation.
// .. : recursive descent.
// [] : subscript operator. If the JSON object is an array, you can use brackets to specify the array index.
// [*] : all objects/elements for array.
//
// Reserved words must be properly escaped when included in Path.
//
// Escape Rule
// single quote style escape: e.g.) `$['a.b'].c`
// double quote style escape: e.g.) `$."a.b".c`
func CreatePath(p string) (*Path, error) {
path, err := decoder.PathString(p).Build()
if err != nil {
return nil, err
}
return &Path{path: path}, nil
}
// Path represents JSON Path.
type Path struct {
path *decoder.Path
}
// RootSelectorOnly whether only the root selector ($) is used.
func (p *Path) RootSelectorOnly() bool {
return p.path.RootSelectorOnly
}
// UsedSingleQuotePathSelector whether single quote-based escaping was done when building the JSON Path.
func (p *Path) UsedSingleQuotePathSelector() bool {
return p.path.SingleQuotePathSelector
}
// UsedSingleQuotePathSelector whether double quote-based escaping was done when building the JSON Path.
func (p *Path) UsedDoubleQuotePathSelector() bool {
return p.path.DoubleQuotePathSelector
}
// Extract extracts a specific JSON string.
func (p *Path) Extract(data []byte, optFuncs ...DecodeOptionFunc) ([][]byte, error) {
return extractFromPath(p, data, optFuncs...)
}
// PathString returns original JSON Path string.
func (p *Path) PathString() string {
return p.path.String()
}
// Unmarshal extract and decode the value of the part corresponding to JSON Path from the input data.
func (p *Path) Unmarshal(data []byte, v interface{}, optFuncs ...DecodeOptionFunc) error {
contents, err := extractFromPath(p, data, optFuncs...)
if err != nil {
return err
}
results := make([]interface{}, 0, len(contents))
for _, content := range contents {
var result interface{}
if err := Unmarshal(content, &result); err != nil {
return err
}
results = append(results, result)
}
if err := decoder.AssignValue(reflect.ValueOf(results), reflect.ValueOf(v)); err != nil {
return err
}
return nil
}
// Get extract and substitute the value of the part corresponding to JSON Path from the input value.
func (p *Path) Get(src, dst interface{}) error {
return p.path.Get(reflect.ValueOf(src), reflect.ValueOf(dst))
}

47
vendor/github.com/goccy/go-json/query.go generated vendored Normal file
View File

@ -0,0 +1,47 @@
package json
import (
"github.com/goccy/go-json/internal/encoder"
)
type (
// FieldQuery you can dynamically filter the fields in the structure by creating a FieldQuery,
// adding it to context.Context using SetFieldQueryToContext and then passing it to MarshalContext.
// This is a type-safe operation, so it is faster than filtering using map[string]interface{}.
FieldQuery = encoder.FieldQuery
FieldQueryString = encoder.FieldQueryString
)
var (
// FieldQueryFromContext get current FieldQuery from context.Context.
FieldQueryFromContext = encoder.FieldQueryFromContext
// SetFieldQueryToContext set current FieldQuery to context.Context.
SetFieldQueryToContext = encoder.SetFieldQueryToContext
)
// BuildFieldQuery builds FieldQuery by fieldName or sub field query.
// First, specify the field name that you want to keep in structure type.
// If the field you want to keep is a structure type, by creating a sub field query using BuildSubFieldQuery,
// you can select the fields you want to keep in the structure.
// This description can be written recursively.
func BuildFieldQuery(fields ...FieldQueryString) (*FieldQuery, error) {
query, err := Marshal(fields)
if err != nil {
return nil, err
}
return FieldQueryString(query).Build()
}
// BuildSubFieldQuery builds sub field query.
func BuildSubFieldQuery(name string) *SubFieldQuery {
return &SubFieldQuery{name: name}
}
type SubFieldQuery struct {
name string
}
func (q *SubFieldQuery) Fields(fields ...FieldQueryString) FieldQueryString {
query, _ := Marshal(map[string][]FieldQueryString{q.name: fields})
return FieldQueryString(query)
}

View File

@ -1,21 +0,0 @@
arch:
- amd64
- ppc64le
language: go
go:
- 1.14.x
- 1.15.x
- tip
env:
- GO111MODULE=on
install:
- go get -v golang.org/x/lint/golint
script:
- golint -set_exit_status ./...
- go test -race -coverprofile=coverage.txt -covermode=atomic ./...
after_success:
- bash <(curl -s https://codecov.io/bash)
matrix:
allow_failures:
- go: tip

View File

@ -1,5 +1,299 @@
# Changelog
## v4.12.0 - 2024-04-15
**Security**
* Update golang.org/x/net dep because of [GO-2024-2687](https://pkg.go.dev/vuln/GO-2024-2687) by @aldas in https://github.com/labstack/echo/pull/2625
**Enhancements**
* binder: make binding to Map work better with string destinations by @aldas in https://github.com/labstack/echo/pull/2554
* README.md: add Encore as sponsor by @marcuskohlberg in https://github.com/labstack/echo/pull/2579
* Reorder paragraphs in README.md by @aldas in https://github.com/labstack/echo/pull/2581
* CI: upgrade actions/checkout to v4 by @aldas in https://github.com/labstack/echo/pull/2584
* Remove default charset from 'application/json' Content-Type header by @doortts in https://github.com/labstack/echo/pull/2568
* CI: Use Go 1.22 by @aldas in https://github.com/labstack/echo/pull/2588
* binder: allow binding to a nil map by @georgmu in https://github.com/labstack/echo/pull/2574
* Add Skipper Unit Test In BasicBasicAuthConfig and Add More Detail Explanation regarding BasicAuthValidator by @RyoKusnadi in https://github.com/labstack/echo/pull/2461
* fix some typos by @teslaedison in https://github.com/labstack/echo/pull/2603
* fix: some typos by @pomadev in https://github.com/labstack/echo/pull/2596
* Allow ResponseWriters to unwrap writers when flushing/hijacking by @aldas in https://github.com/labstack/echo/pull/2595
* Add SPDX licence comments to files. by @aldas in https://github.com/labstack/echo/pull/2604
* Upgrade deps by @aldas in https://github.com/labstack/echo/pull/2605
* Change type definition blocks to single declarations. This helps copy… by @aldas in https://github.com/labstack/echo/pull/2606
* Fix Real IP logic by @cl-bvl in https://github.com/labstack/echo/pull/2550
* Default binder can use `UnmarshalParams(params []string) error` inter… by @aldas in https://github.com/labstack/echo/pull/2607
* Default binder can bind pointer to slice as struct field. For example `*[]string` by @aldas in https://github.com/labstack/echo/pull/2608
* Remove maxparam dependence from Context by @aldas in https://github.com/labstack/echo/pull/2611
* When route is registered with empty path it is normalized to `/`. by @aldas in https://github.com/labstack/echo/pull/2616
* proxy middleware should use httputil.ReverseProxy for SSE requests by @aldas in https://github.com/labstack/echo/pull/2624
## v4.11.4 - 2023-12-20
**Security**
* Upgrade golang.org/x/crypto to v0.17.0 to fix vulnerability [issue](https://pkg.go.dev/vuln/GO-2023-2402) [#2562](https://github.com/labstack/echo/pull/2562)
**Enhancements**
* Update deps and mark Go version to 1.18 as this is what golang.org/x/* use [#2563](https://github.com/labstack/echo/pull/2563)
* Request logger: add example for Slog https://pkg.go.dev/log/slog [#2543](https://github.com/labstack/echo/pull/2543)
## v4.11.3 - 2023-11-07
**Security**
* 'c.Attachment' and 'c.Inline' should escape filename in 'Content-Disposition' header to avoid 'Reflect File Download' vulnerability. [#2541](https://github.com/labstack/echo/pull/2541)
**Enhancements**
* Tests: refactor context tests to be separate functions [#2540](https://github.com/labstack/echo/pull/2540)
* Proxy middleware: reuse echo request context [#2537](https://github.com/labstack/echo/pull/2537)
* Mark unmarshallable yaml struct tags as ignored [#2536](https://github.com/labstack/echo/pull/2536)
## v4.11.2 - 2023-10-11
**Security**
* Bump golang.org/x/net to prevent CVE-2023-39325 / CVE-2023-44487 HTTP/2 Rapid Reset Attack [#2527](https://github.com/labstack/echo/pull/2527)
* fix(sec): randomString bias introduced by #2490 [#2492](https://github.com/labstack/echo/pull/2492)
* CSRF/RequestID mw: switch math/random usage to crypto/random [#2490](https://github.com/labstack/echo/pull/2490)
**Enhancements**
* Delete unused context in body_limit.go [#2483](https://github.com/labstack/echo/pull/2483)
* Use Go 1.21 in CI [#2505](https://github.com/labstack/echo/pull/2505)
* Fix some typos [#2511](https://github.com/labstack/echo/pull/2511)
* Allow CORS middleware to send Access-Control-Max-Age: 0 [#2518](https://github.com/labstack/echo/pull/2518)
* Bump dependancies [#2522](https://github.com/labstack/echo/pull/2522)
## v4.11.1 - 2023-07-16
**Fixes**
* Fix `Gzip` middleware not sending response code for no content responses (404, 301/302 redirects etc) [#2481](https://github.com/labstack/echo/pull/2481)
## v4.11.0 - 2023-07-14
**Fixes**
* Fixes the proxy middleware concurrency issue of calling the Next() proxy target on Round Robin Balancer [#2409](https://github.com/labstack/echo/pull/2409)
* Fix `group.RouteNotFound` not working when group has attached middlewares [#2411](https://github.com/labstack/echo/pull/2411)
* Fix global error handler return error message when message is an error [#2456](https://github.com/labstack/echo/pull/2456)
* Do not use global timeNow variables [#2477](https://github.com/labstack/echo/pull/2477)
**Enhancements**
* Added a optional config variable to disable centralized error handler in recovery middleware [#2410](https://github.com/labstack/echo/pull/2410)
* refactor: use `strings.ReplaceAll` directly [#2424](https://github.com/labstack/echo/pull/2424)
* Add support for Go1.20 `http.rwUnwrapper` to Response struct [#2425](https://github.com/labstack/echo/pull/2425)
* Check whether is nil before invoking centralized error handling [#2429](https://github.com/labstack/echo/pull/2429)
* Proper colon support in `echo.Reverse` method [#2416](https://github.com/labstack/echo/pull/2416)
* Fix misuses of a vs an in documentation comments [#2436](https://github.com/labstack/echo/pull/2436)
* Add link to slog.Handler library for Echo logging into README.md [#2444](https://github.com/labstack/echo/pull/2444)
* In proxy middleware Support retries of failed proxy requests [#2414](https://github.com/labstack/echo/pull/2414)
* gofmt fixes to comments [#2452](https://github.com/labstack/echo/pull/2452)
* gzip response only if it exceeds a minimal length [#2267](https://github.com/labstack/echo/pull/2267)
* Upgrade packages [#2475](https://github.com/labstack/echo/pull/2475)
## v4.10.2 - 2023-02-22
**Security**
* `filepath.Clean` behaviour has changed in Go 1.20 - adapt to it [#2406](https://github.com/labstack/echo/pull/2406)
* Add `middleware.CORSConfig.UnsafeWildcardOriginWithAllowCredentials` to make UNSAFE usages of wildcard origin + allow cretentials less likely [#2405](https://github.com/labstack/echo/pull/2405)
**Enhancements**
* Add more HTTP error values [#2277](https://github.com/labstack/echo/pull/2277)
## v4.10.1 - 2023-02-19
**Security**
* Upgrade deps due to the latest golang.org/x/net vulnerability [#2402](https://github.com/labstack/echo/pull/2402)
**Enhancements**
* Add new JWT repository to the README [#2377](https://github.com/labstack/echo/pull/2377)
* Return an empty string for ctx.path if there is no registered path [#2385](https://github.com/labstack/echo/pull/2385)
* Add context timeout middleware [#2380](https://github.com/labstack/echo/pull/2380)
* Update link to jaegertracing [#2394](https://github.com/labstack/echo/pull/2394)
## v4.10.0 - 2022-12-27
**Security**
* We are deprecating JWT middleware in this repository. Please use https://github.com/labstack/echo-jwt instead.
JWT middleware is moved to separate repository to allow us to bump/upgrade version of JWT implementation (`github.com/golang-jwt/jwt`) we are using
which we can not do in Echo core because this would break backwards compatibility guarantees we try to maintain.
* This minor version bumps minimum Go version to 1.17 (from 1.16) due `golang.org/x/` packages we depend on. There are
several vulnerabilities fixed in these libraries.
Echo still tries to support last 4 Go versions but there are occasions we can not guarantee this promise.
**Enhancements**
* Bump x/text to 0.3.8 [#2305](https://github.com/labstack/echo/pull/2305)
* Bump dependencies and add notes about Go releases we support [#2336](https://github.com/labstack/echo/pull/2336)
* Add helper interface for ProxyBalancer interface [#2316](https://github.com/labstack/echo/pull/2316)
* Expose `middleware.CreateExtractors` function so we can use it from echo-contrib repository [#2338](https://github.com/labstack/echo/pull/2338)
* Refactor func(Context) error to HandlerFunc [#2315](https://github.com/labstack/echo/pull/2315)
* Improve function comments [#2329](https://github.com/labstack/echo/pull/2329)
* Add new method HTTPError.WithInternal [#2340](https://github.com/labstack/echo/pull/2340)
* Replace io/ioutil package usages [#2342](https://github.com/labstack/echo/pull/2342)
* Add staticcheck to CI flow [#2343](https://github.com/labstack/echo/pull/2343)
* Replace relative path determination from proprietary to std [#2345](https://github.com/labstack/echo/pull/2345)
* Remove square brackets from ipv6 addresses in XFF (X-Forwarded-For header) [#2182](https://github.com/labstack/echo/pull/2182)
* Add testcases for some BodyLimit middleware configuration options [#2350](https://github.com/labstack/echo/pull/2350)
* Additional configuration options for RequestLogger and Logger middleware [#2341](https://github.com/labstack/echo/pull/2341)
* Add route to request log [#2162](https://github.com/labstack/echo/pull/2162)
* GitHub Workflows security hardening [#2358](https://github.com/labstack/echo/pull/2358)
* Add govulncheck to CI and bump dependencies [#2362](https://github.com/labstack/echo/pull/2362)
* Fix rate limiter docs [#2366](https://github.com/labstack/echo/pull/2366)
* Refactor how `e.Routes()` work and introduce `e.OnAddRouteHandler` callback [#2337](https://github.com/labstack/echo/pull/2337)
## v4.9.1 - 2022-10-12
**Fixes**
* Fix logger panicing (when template is set to empty) by bumping dependency version [#2295](https://github.com/labstack/echo/issues/2295)
**Enhancements**
* Improve CORS documentation [#2272](https://github.com/labstack/echo/pull/2272)
* Update readme about supported Go versions [#2291](https://github.com/labstack/echo/pull/2291)
* Tests: improve error handling on closing body [#2254](https://github.com/labstack/echo/pull/2254)
* Tests: refactor some of the assertions in tests [#2275](https://github.com/labstack/echo/pull/2275)
* Tests: refactor assertions [#2301](https://github.com/labstack/echo/pull/2301)
## v4.9.0 - 2022-09-04
**Security**
* Fix open redirect vulnerability in handlers serving static directories (e.Static, e.StaticFs, echo.StaticDirectoryHandler) [#2260](https://github.com/labstack/echo/pull/2260)
**Enhancements**
* Allow configuring ErrorHandler in CSRF middleware [#2257](https://github.com/labstack/echo/pull/2257)
* Replace HTTP method constants in tests with stdlib constants [#2247](https://github.com/labstack/echo/pull/2247)
## v4.8.0 - 2022-08-10
**Most notable things**
You can now add any arbitrary HTTP method type as a route [#2237](https://github.com/labstack/echo/pull/2237)
```go
e.Add("COPY", "/*", func(c echo.Context) error
return c.String(http.StatusOK, "OK COPY")
})
```
You can add custom 404 handler for specific paths [#2217](https://github.com/labstack/echo/pull/2217)
```go
e.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })
g := e.Group("/images")
g.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })
```
**Enhancements**
* Add new value binding methods (UnixTimeMilli,TextUnmarshaler,JSONUnmarshaler) to Valuebinder [#2127](https://github.com/labstack/echo/pull/2127)
* Refactor: body_limit middleware unit test [#2145](https://github.com/labstack/echo/pull/2145)
* Refactor: Timeout mw: rework how test waits for timeout. [#2187](https://github.com/labstack/echo/pull/2187)
* BasicAuth middleware returns 500 InternalServerError on invalid base64 strings but should return 400 [#2191](https://github.com/labstack/echo/pull/2191)
* Refactor: duplicated findStaticChild process at findChildWithLabel [#2176](https://github.com/labstack/echo/pull/2176)
* Allow different param names in different methods with same path scheme [#2209](https://github.com/labstack/echo/pull/2209)
* Add support for registering handlers for different 404 routes [#2217](https://github.com/labstack/echo/pull/2217)
* Middlewares should use errors.As() instead of type assertion on HTTPError [#2227](https://github.com/labstack/echo/pull/2227)
* Allow arbitrary HTTP method types to be added as routes [#2237](https://github.com/labstack/echo/pull/2237)
## v4.7.2 - 2022-03-16
**Fixes**
* Fix nil pointer exception when calling Start again after address binding error [#2131](https://github.com/labstack/echo/pull/2131)
* Fix CSRF middleware not being able to extract token from multipart/form-data form [#2136](https://github.com/labstack/echo/pull/2136)
* Fix Timeout middleware write race [#2126](https://github.com/labstack/echo/pull/2126)
**Enhancements**
* Recover middleware should not log panic for aborted handler [#2134](https://github.com/labstack/echo/pull/2134)
## v4.7.1 - 2022-03-13
**Fixes**
* Fix `e.Static`, `.File()`, `c.Attachment()` being picky with paths starting with `./`, `../` and `/` after 4.7.0 introduced echo.Filesystem support (Go1.16+) [#2123](https://github.com/labstack/echo/pull/2123)
**Enhancements**
* Remove some unused code [#2116](https://github.com/labstack/echo/pull/2116)
## v4.7.0 - 2022-03-01
**Enhancements**
* Add JWT, KeyAuth, CSRF multivalue extractors [#2060](https://github.com/labstack/echo/pull/2060)
* Add LogErrorFunc to recover middleware [#2072](https://github.com/labstack/echo/pull/2072)
* Add support for HEAD method query params binding [#2027](https://github.com/labstack/echo/pull/2027)
* Improve filesystem support with echo.FileFS, echo.StaticFS, group.FileFS, group.StaticFS [#2064](https://github.com/labstack/echo/pull/2064)
**Fixes**
* Fix X-Real-IP bug, improve tests [#2007](https://github.com/labstack/echo/pull/2007)
* Minor syntax fixes [#1994](https://github.com/labstack/echo/pull/1994), [#2102](https://github.com/labstack/echo/pull/2102), [#2102](https://github.com/labstack/echo/pull/2102)
**General**
* Add cache-control and connection headers [#2103](https://github.com/labstack/echo/pull/2103)
* Add Retry-After header constant [#2078](https://github.com/labstack/echo/pull/2078)
* Upgrade `go` directive in `go.mod` to 1.17 [#2049](https://github.com/labstack/echo/pull/2049)
* Add Pagoda [#2077](https://github.com/labstack/echo/pull/2077) and Souin [#2069](https://github.com/labstack/echo/pull/2069) to 3rd-party middlewares in README
## v4.6.3 - 2022-01-10
**Fixes**
* Fixed Echo version number in greeting message which was not incremented to `4.6.2` [#2066](https://github.com/labstack/echo/issues/2066)
## v4.6.2 - 2022-01-08
**Fixes**
* Fixed route containing escaped colon should be matchable but is not matched to request path [#2047](https://github.com/labstack/echo/pull/2047)
* Fixed a problem that returned wrong content-encoding when the gzip compressed content was empty. [#1921](https://github.com/labstack/echo/pull/1921)
* Update (test) dependencies [#2021](https://github.com/labstack/echo/pull/2021)
**Enhancements**
* Add support for configurable target header for the request_id middleware [#2040](https://github.com/labstack/echo/pull/2040)
* Change decompress middleware to use stream decompression instead of buffering [#2018](https://github.com/labstack/echo/pull/2018)
* Documentation updates
## v4.6.1 - 2021-09-26
**Enhancements**

View File

@ -9,9 +9,11 @@ tag:
check: lint vet race ## Check project
init:
@go get -u golang.org/x/lint/golint
@go install golang.org/x/lint/golint@latest
@go install honnef.co/go/tools/cmd/staticcheck@latest
lint: ## Lint the files
@staticcheck ${PKG_LIST}
@golint -set_exit_status ${PKG_LIST}
vet: ## Vet the files
@ -29,6 +31,6 @@ benchmark: ## Run benchmarks
help: ## Display this help screen
@grep -h -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
goversion ?= "1.15"
test_version: ## Run tests inside Docker with given version (defaults to 1.15 oldest supported). Example: make test_version goversion=1.15
goversion ?= "1.19"
test_version: ## Run tests inside Docker with given version (defaults to 1.19 oldest supported). Example: make test_version goversion=1.19
@docker run --rm -it -v $(shell pwd):/project golang:$(goversion) /bin/sh -c "cd /project && make init check"

Some files were not shown because too many files have changed in this diff Show More