Mismatching results when executing platform search in daemon mode

[kittaakos]

Bug Report

Current behavior

I am experiencing some oddities when executing a platform search via the CLI.

When I call the CLI directly, it works. So for instance, when I run ./arduino-cli core list, everything is fine. Here in the code, both the Boards and BoardsManifest are populated on the PlatformRelease instance as expected; hence mapping to the RPC format is correct.

However, when I start the CLI in daemon mode and connect from my client, the Boards is empty when I perform the same platform search.

Most likely, either my configuration is incomplete, or my InitRequest is bogus.

I have noticed, the CLI always adds the default package index URL to the board manager additional URLs. I have corrected it; still, the Bords array is empty.

My client code looks like this:

const config = new Configuration()
config.setSketchbookdir('~/Documents/Arduino');
config.setDatadir('~/Library/Arduino15/staging');
config.setDownloadsdir('~/Library/Arduino15/staging');
config.setBoardmanageradditionalurlsList(['https://downloads.arduino.cc/packages/package_index.json']);

const initReq = new InitReq();
initReq.setConfiguration(config);
const initResp = await new Promise<InitResp>(resolve => {
    let resp: InitResp | undefined = undefined;
    const stream = client.init(initReq);
    stream.on('data', (data: InitResp) => {
        if (!resp) {
            resp = data;
        }
    })
    stream.on('end', () => {
        resolve(resp);
    })
});

const instance = initResp.getInstance();
// Trigger an update index via `UpdateIndexReq`.

Expected behavior

The Boards are populated just like BoardsManifest on the cores.PlatformRelease instances when calling:

arduino-cli/commands/core/core.go

Line 28 in 1628b39

func platformReleaseToRPC(platformRelease *cores.PlatformRelease) *rpc.Platform {

Environment

• CLI version (output of arduino-cli version):
  arduino-cli Version: nightly-21 Commit: 1628b39 BuildDate: 2019-07-15 09:31:26.332485 +0000 UTC
• Go version (if building from sources):
• OS version:
  macOS 10.14.4 (18E226)

Additional context

[kittaakos]

I have found that the Boards are set only for the most recent version of the PlatformRelease even when running the CLI in non-daemon mode. It is available for 1.6.23 but missing for all other previous release versions. I do not know if it is related, though.

[kittaakos]

I think it is broken for core search as well. I have checked out the source, modified a bit by adding some logging, and ran the core search "" command; the Boards were an empty array. BoardsManifest were correctly set.

Commit: 1628b39
My local changes:

diff --git a/commands/core/core.go b/commands/core/core.go
index 86845ce..24e1580 100644
--- a/commands/core/core.go
+++ b/commands/core/core.go
@@ -18,6 +18,8 @@
 package core
 
 import (
+       "fmt"
+
        "github.com/arduino/arduino-cli/arduino/cores"
        rpc "github.com/arduino/arduino-cli/rpc/commands"
 )
@@ -27,6 +29,8 @@ import (
 // platformRelease we're currently converting is actually installed.
 func platformReleaseToRPC(platformRelease *cores.PlatformRelease) *rpc.Platform {
        boards := make([]*rpc.Board, len(platformRelease.Boards))
+       fmt.Printf("@@@ platformRelease.Boards: %v\n\n", platformRelease.Boards)
+       fmt.Printf("### platformRelease.BoardsManifest: %v\n\n", platformRelease.BoardsManifest)
        i := 0
        for _, b := range platformRelease.Boards {
                boards[i] = &rpc.Board{

Executed command: go run main.go core search ""
Output:

Searching for platforms matching ''
@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f4180]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f4210]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f4690]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f49f0 0xc0003f4a20 0xc0003f4a50]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f4a80]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f4ba0]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f4fc0]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f5170]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003ed9e0]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f3770 0xc0003f37a0 0xc0003f37d0 0xc0003f3800 0xc0003f3830 0xc0003f3860 0xc0003f3890 0xc0003f38c0 0xc0003f38f0 0xc0003f3920 0xc0003f3950 0xc0003f3980 0xc0003f39b0 0xc0003f39e0]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f2e10]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f3bf0]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f3e90 0xc0003f3ec0]

@@@ platformRelease.Boards: map[LilyPadUSB:arduino:avr:LilyPadUSB atmegang:arduino:avr:atmegang bt:arduino:avr:bt chiwawa:arduino:avr:chiwawa circuitplay32u4cat:arduino:avr:circuitplay32u4cat diecimila:arduino:avr:diecimila esplora:arduino:avr:esplora ethernet:arduino:avr:ethernet fio:arduino:avr:fio gemma:arduino:avr:gemma leonardo:arduino:avr:leonardo leonardoeth:arduino:avr:leonardoeth lilypad:arduino:avr:lilypad mega:arduino:avr:mega megaADK:arduino:avr:megaADK micro:arduino:avr:micro mini:arduino:avr:mini nano:arduino:avr:nano one:arduino:avr:one pro:arduino:avr:pro robotControl:arduino:avr:robotControl robotMotor:arduino:avr:robotMotor uno:arduino:avr:uno unowifi:arduino:avr:unowifi yun:arduino:avr:yun yunmini:arduino:avr:yunmini]

### platformRelease.BoardsManifest: [0xc0003ec900 0xc0003ec930 0xc0003ec960 0xc0003ec990 0xc0003ec9c0 0xc0003ec9f0 0xc0003eca20 0xc0003eca50 0xc0003eca80 0xc0003ecab0 0xc0003ecae0 0xc0003ecb10 0xc0003ecb40 0xc0003ecb70 0xc0003ecba0 0xc0003ecbd0 0xc0003ecc00 0xc0003ecc30 0xc0003ecc60 0xc0003ecc90 0xc0003eccc0 0xc0003eccf0 0xc0003ecd20 0xc0003ecd50 0xc0003ecd80 0xc0003ecdb0]

@@@ platformRelease.Boards: map[nona4809:arduino:megaavr:nona4809 uno2018:arduino:megaavr:uno2018]

### platformRelease.BoardsManifest: [0xc0003ed260 0xc0003ed290]

@@@ platformRelease.Boards: map[]

### platformRelease.BoardsManifest: [0xc0003f4e70 0xc0003f4ea0 0xc0003f4ed0 0xc0003f4f00 0xc0003f4f30 0xc0003f4f60]

ID                   Version   Name                                             
Arrow:samd           2.1.0     Arrow Boards                                     
Intel:arc32          2.0.4     Intel Curie Boards                               
Intel:i586           1.6.7+1.0 Intel i586 Boards                                
Intel:i686           1.6.7+1.0 Intel i686 Boards                                
Microsoft:win10      1.1.2     Windows 10 Iot Core                              
arduino:avr          1.6.23    Arduino AVR Boards                               
arduino:megaavr      1.8.2     Arduino megaAVR Boards                           
arduino:nrf52        1.0.2     Arduino nRF52 Boards                             
arduino:sam          1.6.12    Arduino SAM Boards (32-bits ARM Cortex-M3)       
arduino:samd         1.8.3     Arduino SAMD Boards (32-bits ARM Cortex-M0+)     
arduino:samd_beta    1.6.25    Arduino SAMD Beta Boards (32-bits ARM Cortex-M0+)
arduino:stm32f4      1.0.1     Arduino STM32F4 Boards                           
atmel-avr-xminis:avr 0.6.0     Atmel AVR Xplained-minis                         
emoro:avr            3.2.2     EMORO 2560                                       
industruino:samd     1.0.1     Industruino SAMD Boards (32-bits ARM Cortex-M0+) 
littleBits:avr       1.0.0     littleBits Arduino AVR Modules

I would have expected, when the BoardsManifest is populated, the Boards are too. Since the Boards are not set the gRPC client cannot get back the Boards for a particular Platform.

[kittaakos]

I am still trying to catch up with the codebase but isn't it a problem that we set the BoardsManifest but we do not set the Boards here

arduino-cli/arduino/cores/packageindex/index.go

Line 142 in 296c872

outPlatformRelease.BoardsManifest = inPlatformRelease.extractBoardsManifest()

when we load the package index from https:/downloads.arduino.cc/packages/package_index.json?

[cmaglie]

Fixed by #294