Could SSH‐Remote into Ubuntu 18.04 HPC via VSCode but not Positron

[KGZaker]

First, thank you so much for maintaining Positron—it’s been a huge help!

Background:
I’m trying to use Positron’s SSH feature to connect to an HPC cluster running Ubuntu 18.04, but out of the box neither Positron nor VS Code’s Remote–SSH work on that older distro. Following the instructions here: can-i-run-vs-code-server-on-older-linux-distributions, I configured a sysroot and patchelf. After that, I can successfully connect to the Ubuntu 18.04 host from the latest VS Code via the built-in Remote–SSH extension. However, Positron still fails to connect via SSH.

Expected Behavior:
Maybe Positron can leverage the same patched server approach (or an equivalent workaround) to establish an SSH‐Remote session on Ubuntu 18.04 HPC hosts.

Questions:
Are there alternative methods you’d recommend for getting Positron’s SSH support working on older Linux distributions?

Any guidance would be much appreciated! Thank you again for your work on Positron.

[austin3dickey]

Hey @KGZaker, nice find with those crosstool-NG instructions! It's interesting because we do technically look for those environment variables, as seen in the main server script. For some reason, though, the way that our open-remote-ssh extension calls it means they're not provided. So the patch to the bundled node is not happening before you try to start up the server.

I was able to get around this by manually patching node. After you've tried to connect, it should have downloaded the server files to the remote host at a location that looks like $HOME/.positron-server/bin/$DISTRO_COMMIT/. On the remote host, I ran the same commands that are defined in that startup script:

ROOT=$HOME/.positron-server/bin/<insert your commit here>
"$VSCODE_SERVER_PATCHELF_PATH" --set-rpath "$VSCODE_SERVER_CUSTOM_GLIBC_PATH" "$ROOT/node"
"$VSCODE_SERVER_PATCHELF_PATH" --set-interpreter "$VSCODE_SERVER_CUSTOM_GLIBC_LINKER" "$ROOT/node"

Then I tried to connect again and it worked. As the VS Code docs say, "this approach is a technical workaround and is not an officially supported usage scenario," so I can't guarantee it'll remain working this way. But for now, hopefully this helps.

[KGZaker]

Hi @austin3dickey, thank you soooo much for your tutorial. Unfortunately, I’m still running into an issue on my cluster.

I first tried to connect to the remote server with Positron, and got an error (which is expected). I then got the $DISTRO_COMMIT (d3aebd077b1b5a1ad2073966a4cc20270da68f89 in my case ).

Then, on the remote server, I ran:

ROOT=$HOME/.positron-server/bin/d3aebd077b1b5a1ad2073966a4cc20270da68f89
"$VSCODE_SERVER_PATCHELF_PATH" --set-rpath "$VSCODE_SERVER_CUSTOM_GLIBC_PATH" "$ROOT/node"
"$VSCODE_SERVER_PATCHELF_PATH" --set-interpreter "$VSCODE_SERVER_CUSTOM_GLIBC_LINKER" "$ROOT/node"

After that, I click the retry button in positron, however, the connection still fails. The log shows:

/home/XXX/.positron-server/bin/d3aebd077b1b5a1ad2073966a4cc20270da68f89/bin/positron-server: line 22: 177010 Segmentation fault      (core dumped) "$ROOT/node" ${INSPECT:-} "$ROOT/out/server-main.js" "$@"

[austin3dickey]

Yikes, segfaults are tricky to debug. It's a long shot but could you remove ~/.positron-server, restart the remote host, and try again? Unfortunately I can't reproduce this, and since it's probably highly system-dependent I'm not sure what else to do. It's interesting that it works for VS Code still.

[KGZaker]

Restarting the remote host still hasn’t worked. It might be due to a setting in my cluster that I’m not aware of🥹. But thanks for the suggestion, it seems promising.

[jmcphers]

It looks like the node process is crashing, which could happen for any number of reasons but is probably due to glibc incompatibility.

Did your patchelf commands succeed? I'd start by checking the rpath and making sure it was updated to the correct location.

If that looks fine, try running the node process under gdb, then getting a stack trace with bt when it crashes. Paste that here and we'll see if we can ascertain anything further!

[KGZaker]

Hi @jmcphers, thanks for your kind help. I was able to successfully apply the patchelf commands. Below is the stack trace I obtained:

gdb --args .positron-server/bin/d3aebd077b1b5a1ad2073966a4cc20270da68f89/node 
GNU gdb (GDB) 15.2
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-conda-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from .positron-server/bin/d3aebd077b1b5a1ad2073966a4cc20270da68f89/node...
(gdb) run
Starting program: /home/zeyuzhu/.positron-server/bin/d3aebd077b1b5a1ad2073966a4cc20270da68f89/node 

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7dd988b in dl_main (phdr=<optimized out>, phnum=<optimized out>, user_entry=<optimized out>, 
    auxv=<optimized out>) at rtld.c:1586
warning: 1586	rtld.c: No such file or directory
(gdb) bt
#0  0x00007ffff7dd988b in dl_main (phdr=<optimized out>, phnum=<optimized out>, user_entry=<optimized out>, 
    auxv=<optimized out>) at rtld.c:1586
#1  0x00007ffff7dee320 in _dl_sysdep_start (start_argptr=start_argptr@entry=0x7fffffffbc20, 
    dl_main=dl_main@entry=0x7ffff7dd8390 <dl_main>) at ../elf/dl-sysdep.c:253
#2  0x00007ffff7dd7f58 in _dl_start_final (arg=0x7fffffffbc20) at rtld.c:415
#3  _dl_start (arg=0x7fffffffbc20) at rtld.c:522
#4  0x00007ffff7dd6ed8 in _start () from /home/zeyuzhu/tools/vscode-sysroot/lib/ld-linux-x86-64.so.2
#5  0x0000000000000001 in ?? ()
#6  0x00007fffffffc13c in ?? ()
#7  0x0000000000000000 in ?? ()