Slow clear_output() performance in native notebooks

[jflam]

Issue Type: Bug

1. Create a notebook and add this code to it:

from IPython.display import clear_output

for i in range(10000):
    clear_output()
    print(i)

2. Run the cell. You'll see flickering in the output of the cell as it seems to go from zero height (from the call to clear_output()? to 1 line in height. If you run the same code in Jupyter classic notebooks, you'll see that the height of the output cell doesn't change due to the call to clear_output().

This is tremendously impacts performance. In native notebooks (in an a run that is different from the simplified case below) it took 1800s vs. 24s in Jupyter on the same machine.

This kind of code is pretty common to show progress during a long training run.

VS Code version: Code 1.57.1 (507ce72a4466fbb27b715c3722558bb15afa9f48, 2021-06-17T13:28:07.755Z)
OS version: Windows_NT x64 10.0.22000
Restricted Mode: No
Remote OS version: Linux x64 5.10.16.3-microsoft-standard-WSL2

System Info

Item	Value
CPUs	AMD Ryzen Threadripper 1950X 16-Core Processor (32 x 3394)
GPU Status	2d_canvas: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: enabled opengl: enabled_on rasterization: enabled skia_renderer: enabled_on video_decode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	63.88GB (37.39GB free)
Process Argv	--folder-uri=vscode-remote://wsl+Ubuntu-20.04/home/jlam/src/ML-For-Beginners --remote=wsl+Ubuntu-20.04 --crash-reporter-id 219e6300-3bad-4c93-9f05-4147417751e1
Screen Reader	no
VM	0%

Item	Value
Remote	WSL: Ubuntu-20.04
OS	Linux x64 5.10.16.3-microsoft-standard-WSL2
CPUs	AMD Ryzen Threadripper 1950X 16-Core Processor (32 x 3393)
Memory (System)	31.29GB (22.24GB free)
VM	0%

Extensions (19)

Extension	Author (truncated)	Version
vsc-material-theme	Equ	33.2.2
vsc-material-theme-icons	equ	2.0.10
remote-containers	ms-	0.183.0
remote-ssh	ms-	0.65.7
remote-ssh-edit	ms-	0.65.7
remote-wsl	ms-	0.56.5
rewrap	stk	1.14.0
vim	vsc	1.21.3
gitlens	eam	11.5.1
vscode-pull-request-github	Git	0.27.1
go	gol	0.26.0
file-downloader	min	1.0.11
vscode-docker	ms-	1.14.0
vscode-kubernetes-tools	ms-	1.3.3
python	ms-	2021.6.944021595
vscode-pylance	ms-	2021.6.3
jupyter	ms-	2021.6.999662501
azurecli	ms-	0.5.0
vscode-yaml	red	0.20.0

(5 theme extensions excluded)

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
pythonvspyt700cf:30270857
pythonvspyt602:30300191
vspor879:30202332
vspor708:30202333
vspor363:30204092
pythonvspyt639:30300192
pythontb:30283811
vspre833:30321513
pythonptprofiler:30281270
vsdfh931:30280409
vshan820:30294714
pythondataviewer:30285071
vscus158:30321503
pythonvsuse255:30323308
vscorehov:30309549
vscod805cf:30301675
pythonvspyt200cf:30331938
binariesv615:30325510
vsccppwtct:30329789
aatestflightcf:30331712

[rebornix]

@jflam thanks for reporting this. This is an interesting scenario where a ton of appendOutput and clearOutput requests are queued and the core always try to apply these requests on the model.

Screen.Recording.2021-07-05.at.1.38.41.PM.mov

The challenge for the text model is, we don't know if there is a clear output requests coming next. I'm wondering if we can do the same optimization as Jupyter classic in the Jupyter ext side.

[DonJayamanne]

@rebornix this would be a problem for anyone writing to ipynb (anyone building a Python kernel, or even with .NET, julia & the like).
We could go a few ways, my initial plan was to have an API in core that allows users to write to ipynb (on top of the VS Code API). lets will discuss.

[rebornix]

This is probably due to #5662

[DonJayamanne]

@rebornix Moved into Jupyter repo & the issue is in Jupyter extension.