Implement cursor movement in unix_formatting

[jcubic]

I have idea for a new feature for jQuery Terminal

I've already stared in unix_formatting.js#L517 but the parser have issues. This one is first version of the code, since then it was updated. Was not able to fix it myself and I've asked on StackOveflow: How to parse cursor ANSI escape codes?

• Add basic cursor movement
• Working SIXEL example
• Add H escape code

[jerch]

Any chance that you are running the ervy lib directly without a pty in between? If so, you prolly have to replace every LF ('\n') with CR-LF ('\r\n'). (Thats one of the transformations a tty typically does.)

If this doesnt help, maybe post the stream data as JSON string here.

[jcubic]

Here is the output of the plot:
" Y Axis A: \u001b[31m*\u001b[0m | B: \u001b[34m# \u001b[0m | C: \u001b[36m@ \u001b[0m\n\n\u001b[0C\u001b[0A \n\u001b[0C \n\u001b[0C \n\u001b[0C \n\u001b[0C \n\u001b[0C \n\u001b[0C \n\u001b[0C \n\u001b[0C \n\u001b[0C \n\u001b[0C \n\u001b[5C\u001b[1A\u001b[31m*\u001b[0m\n\u001b[1B\u001b[7C\u001b[2A\u001b[31m*\u001b[0m\n\u001b[2B\u001b[9C\u001b[3A\u001b[31m*\u001b[0m\n\u001b[3B\u001b[11C\u001b[4A\u001b[31m*\u001b[0m\n\u001b[4B\u001b[13C\u001b[5A\u001b[31m*\u001b[0m\n\u001b[5B\u001b[15C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[17C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[19C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[21C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[23C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[25C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[27C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[29C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[31C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[33C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[35C\u001b[6A\u001b[31m*\u001b[0m\n\u001b[6B\u001b[7C\u001b[6A\u001b[34m# \u001b[0m\u001b[34m# \u001b[0m\n\u001b[7C\u001b[34m# \u001b[0m\u001b[34m# \u001b[0m\n\u001b[6B\u001b[3C\u001b[0A\u001b[36m@ \u001b[0m\n\u001b[0B\u001b[30D\u001b[11A A\n 10|\n |\n 8|\n |\n 6|\n |\n 4|\n |\n 2|\n |\n |\u001b[1D+---+---+---+---+---+----+----+----+-> X Axis\n 0 2 4 6 8 10 12 14 "

\r is ignored by my library and \n is working like on Linux.

[jcubic]

I think I know what the problem is:

{s: "   A", x: -26, y: 2, line: "   A                                  "}
``
what should negative cursor do? I've tried to set it to 0 and y--.

[jerch]

Ah ok, well those cursor movement sequences always stick to 0 (left/top) or (cols/rows)-1 (given that your coords are 0 based). They also would never flip over to a next/previous line, they simply stop at the border they would pass:

• CSI 99999 ; 99999 H would stop at the rightmost cell in the bottom line (given the terminal is smaller)
• CSI 99999 D would stop at the leftmost cell, no line switch

[jcubic]

This is my latest code also did mistake in if (!line) case where cursor.x > 0

var terminal = {
    inst_p: function(s) {
        var line = result[cursor.y];
        if (!line) {
            if (cursor.x > 0) {
                result[cursor.y] = new Array(cursor.x + 1).join(' ') + s;
            } else {
                result[cursor.y] = s;
            }
        } else if (cursor.x === 0) {
            result[cursor.y] = s + line.substring(s.length);
        } else if (line.length < cursor.x) {
            var len = cursor.x - (line.length - 1);
            result[cursor.y] += new Array(len).join(' ') + s;
        } else if (line.length === cursor.x) {
            result[cursor.y] += s;
        } else {
            var before = line.substring(0, cursor.x);
            var after = line.substring(cursor.x + s.length);
            result[cursor.y] = before + s + after;
        }
        cursor.x += s.length;
        console.log({s, ...cursor, line: result[cursor.y]});
    },
    inst_o: function(s) {console.log('osc', s);},
    inst_x: function(flag) {
        var code = flag.charCodeAt(0);
        if (code === 10) {
            cursor.y++;
            cursor.x = 0;
        }
        console.log('newline');
    },
    inst_c: function(collected, params, flag) {
        console.log({collected, params, flag});
        var value = params[0] === 0 ? 1 : params[0];
        switch(flag) {
            case 'A': // UP
                cursor.y -= value;
                break;
            case 'B': // Down
                cursor.y += value - 1;
                break;
            case 'C': // forward
                cursor.x += value;
                break;
            case 'D': // Back
                cursor.x -= value;
                break;
            case 'E': // Cursor Next Line
                cursor.x = 0;
                cursor.y += value;
                break;
            case 'F': // Cursor Previous Line
                cursor.x = 0;
                cursor.y -= value;
                break;
        }
        if (cursor.x < 0) {
            cursor.x = 0;
        }
    },
    inst_e: function(collected, flag) {console.log('esc', collected, flag);},
    inst_H: function(collected, params, flag) {console.log('dcs-Hook', collected, params, flag);},
    inst_P: function(dcs) {console.log('dcs-Put', dcs);},
    inst_U: function() {console.log('dcs-Unhook');}
};

if I remove that -1 the plot is broken completely. looks weird that's there.

[jerch]

I dont see where you limit the cursor access to the bottom, I think those cursorDown sequences with high params will keep enlarging result to the bottom. The -1 seems to partly fix that (but in a wrong way). Does it work if you replace that line with cursor.y = Math.min(cursor.y + value, result.length - 1);? (Assuming the lib expects to write its plot at the bottom, the cursor can never go beyond the last line).

Edit: Yepp, this seems to be a bug in the ervy lib - if I remove the other examples from their demo, the scatter plot gets screwed up like this a maximized and cleared terminal (tested with xterm):

So there are two issue behind this:

• ervy lib expects always to write at the bottom of a terminal viewport, but thats not always the case (so your code is right about that behavior)
• you should limit cursor access also to the right and bottom (kinda like spanning a canvas where the cursor can be moved in), only overflows at the bottom line in inst_p and '\n' are allowed to scroll (means your "canvas frame" would move one line down in result, thus the very first line is not accessible anymore by cursor movements and so on)

[jcubic]

It seems that this works, cursor.y = Math.min(cursor.y + value, result.length); so it can move to next line but not more.

[jerch]

Eww moving to the next line to the bottom should not work by cursor movement sequences, only overflow+wrap in PRINT or explicit NL at the bottom line should scroll the terminal viewport. I am not sure if ervy is a good test candidate here as it seems to have this bug above.

Note that some apps using the terminal more as text canvas might do exessive CUD and CUF just to get the cursor at the bottom line or rightmost cell without caring for the actual viewport size and printing from there backwards. Thus I think you should declare a cursor adressible area (viewport in COLS/ROWS), that would scroll relative to result offset on print wrapping or NL in the bottom line.

[jcubic]

But why XTerm work properly, is it detecting malformed output? What about xterm.js? You said that with TTY it work correctly with ervy plot.

[jerch]

But why XTerm work properly, is it detecting malformed output? What about xterm.js?

No, ervy does not work correctly in xterm if the terminal window is maximized and cleared before running a scatter plot. As far as I can tell from the sequences sent by ervy they kinda draw the plot with excessive CUD sequences, which moves the cursor too far to the bottom screwing up the plot (see the image above). Thats a bug in ervy, xterm does no special error correction here, it just draws what it was told. Same with xterm.js (xterm and xterm.js show the same output).

You said that with TTY it work correctly with ervy plot.

Not sure where I said that. Regarding the sequences involved in the plot output the TTY only would do the LR --> CRLF transformation, which is not needed if you do it in the terminal yourself. I dont see any other control codes, that would be affected by the TTY. Note that a TTY does not deal with multibyte sequences at all, it simply forwards those to the terminal. (TTYs are getting complicated, when the user inputs data, but thats not the case here.)

I dont know your internal terminal design, most likely you already have a row/cols limited "viewport" (+ some scrollback). Isn't it easier to just reposition the cursor within that area and let the print happen by your already established output/print methods?

[jcubic]

Uh, sorry I was not using Xterm but xfce4-terminal and there it works even if I render single scatter plot if taller terminal. But that terminal is fixing the errors. I need to check this terminal source code. Maybe I'll find some clue how it's handled.

Not sure where I said that.

You said that xterm works the same as in xterm.js I've assume that it was correct. I've maybe I've interpreted this wrong.

[jcubic]

dont know your internal terminal design

In my terminal the cursor is independent of the output, so I have cursor inside string to format ANSI escapes I need single string as output.

Also my terminal is not TTY it have nothing to do with Linux, the library is mainly used to create command line apps purely in the browser. If someone need tty I suggest xterm.js.

[jerch]

Uh, sorry I was not using Xterm but xfce4-terminal and there it works even if I render single scatter plot if taller terminal.

Well I only tested their demo scatter plot with other outputs disabled . xfce-terminal looks exactly the same as the xterm output above. Again, thats an ervy bug, nothing a terminal can do about it.

You said that xterm works the same as in xterm.js I've assume that it was correct. I've maybe I've interpreted this wrong.

Ah yes, in the other post. Well that was meant for the screwed up image you posted there, which I wasnt able to repro. If I run it in xterm.js demo (with a pty in between), it outputs the same as xterm and xfce-terminal (including the buggy plot in maximized and cleared terminal).

In my terminal the cursor is independent of the output, so I have cursor inside string to format ANSI escapes I need single string as output.

Well I guess I cant help you here, seems to be quite different to the classical text grid + cursor movements semantics of other terminals.

Also my terminal is not TTY it have nothing to do with Linux, the library is mainly used to create command line apps purely in the browser. If someone need tty I suggest xterm.js.

Btw a TTY as a hardware device is just what a typewriter would do - advance the cursor with input and handle basic control codes like CR + LF, BS, TAB and such. In that sense xterm.js is a TTY, as it also resembles that functionality. And your terminal doesnt?

Note that the meaning of TTY goes beyond that in Unix/POSIX - it originally denoted a piece of hardware (a real tty or a video terminal) + process I/O and controlling functions from the OS. Thus a process would not write/read directly from a terminal buffer, but always stream STDIN and STDOUT as bytes from/to the terminal (The early devices had not even anything like a buffer beside a piece of paper or punchcards). The kernel driver sits in between to enable interrupt handling / signalling (+ some byte transformations). When I refer to pty/tty I typically mean only the kernel driver part, due to the pseudoterminal interface (PTY/PTS) the terminal part is not important anymore as any process can act as such. (Im pretty sure that your terminal could as well with only minor modifications).

[jcubic]

Thanks, for explanation, your correct the ervy is broken if run the code in clear terminal. I tested with lot of output in terminal with that it was working. But on clear terminal without any output it give the same output as your screen. Thanks for your help, will file a bug in ervy.