Skip to content

[gRPC][discovery] panic: JSON decoder out of sync - data changing underfoot? #306

Closed
@kittaakos

Description

@kittaakos
Contributor

Bug Report

Current behavior

I am using the CLI via gRPC from a JS-based application. My client code performs mainly the following requests:

  • PlatformListReq,
  • PlatformSearchReq, and
  • BoardListReq.

Due to the async nature of the client, I am working on, and these requests might run in parallel which from time to time results in a panic error from the discovery code. After the failure, the CLI crashes, and the daemon process terminates.

panic: JSON decoder out of sync - data changing underfoot?
goroutine 42 [running]:
encoding/json.(*decodeState).value(0xc000130188, 0x46e4da0, 0xc001ffa390, 0x16, 0xa, 0xa)
    /usr/local/go/src/encoding/json/decode.go:367 +0x23f
encoding/json.(*decodeState).unmarshal(0xc000130188, 0x46e4da0, 0xc001ffa390, 0x0, 0xc0004b2480)
    /usr/local/go/src/encoding/json/decode.go:179 +0x209
encoding/json.(*Decoder).Decode(0xc000130160, 0x46e4da0, 0xc001ffa390, 0xc001ff6055, 0xc000eba9c0)
    /usr/local/go/src/encoding/json/stream.go:73 +0x187
github.com/arduino/arduino-cli/arduino/discovery.(*Discovery).List(0xc000eba9c0, 0xc001ffa330, 0xc0000cc1c0, 0x1, 0x1, 0x4e8a7c0)
    /Users/akos.kitta/git/arduino-cli/arduino/discovery/discovery.go:105 +0x230
github.com/arduino/arduino-cli/commands/board.List(0x4933f80, 0xc001ffa300, 0xc001ffa330, 0x48064e0, 0xc001ff8100, 0x49412c0)
    /Users/akos.kitta/git/arduino-cli/commands/board/list.go:38 +0x609
github.com/arduino/arduino-cli/commands/daemon.(*ArduinoCoreServerImpl).BoardList(0xc000208200, 0x4933f80, 0xc001ffa300, 0xc001ffa330, 0xc000208200, 0xc001ffa300, 0xc000075ba8)
    /Users/akos.kitta/git/arduino-cli/commands/daemon/daemon.go:52 +0x3f
github.com/arduino/arduino-cli/rpc/commands._ArduinoCore_BoardList_Handler(0x48064e0, 0xc000208200, 0x4933f80, 0xc001ffa300, 0xc001ff81e0, 0x0, 0x4933f80, 0xc001ffa300, 0xc001ff604c, 0x4)
    /Users/akos.kitta/git/arduino-cli/rpc/commands/commands.pb.go:1551 +0x23e
google.golang.org/grpc.(*Server).processUnaryRPC(0xc00009a780, 0x493bd00, 0xc00009ad80, 0xc00021a800, 0xc0001fc900, 0x4e90ea0, 0x0, 0x0, 0x0)
    /Users/akos.kitta/go/pkg/mod/google.golang.org/grpc@v1.21.1/server.go:998 +0x470
google.golang.org/grpc.(*Server).handleStream(0xc00009a780, 0x493bd00, 0xc00009ad80, 0xc00021a800, 0x0)
    /Users/akos.kitta/go/pkg/mod/google.golang.org/grpc@v1.21.1/server.go:1278 +0xda6
google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0000e9320, 0xc00009a780, 0x493bd00, 0xc00009ad80, 0xc00021a800)
    /Users/akos.kitta/go/pkg/mod/google.golang.org/grpc@v1.21.1/server.go:717 +0x9f
created by google.golang.org/grpc.(*Server).serveStreams.func1
    /Users/akos.kitta/go/pkg/mod/google.golang.org/grpc@v1.21.1/server.go:715 +0xa1

After making my client code a bit more relaxed, the error happens less frequently, but the daemon produces the following error:

Error getting port list from discovery /Users/akos.kitta/xxx/data/packages/builtin/tools/serial-discovery/0.5.0/serial-discovery: sending LIST command to discovery: write |1: file already closed
Error getting port list from discovery /Users/akos.kitta/xxx/data/packages/builtin/tools/serial-discovery/0.5.0/serial-discovery: sending LIST command to discovery: write |1: file already closed

This could lead to unpredictable behavior.

Question: is the CLI capable of processing concurrent requests via gRPC?

Expected behavior

The CLI does not crash when it has to process multiple gRPC requests concurrently.

Environment

  • CLI version (output of arduino-cli version):
  • Go version (if building from sources):
  • OS version:
    macOS 10.14.4 (18E226)

Additional context

Activity

self-assigned this
on Jul 24, 2019
masci

masci commented on Jul 24, 2019

@masci
Contributor

I was able to reproduce, the problematic command is BoardList, any concurrent execution (even with just 2 goroutines) provides the same output you get

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    Participants

    @masci@kittaakos

    Issue actions

      [gRPC][discovery] panic: JSON decoder out of sync - data changing underfoot? · Issue #306 · arduino/arduino-cli