Skip to content

Set PenSkin silhouette data directly #475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

adroitwhiz
Copy link
Contributor

@adroitwhiz adroitwhiz commented Jul 8, 2019

Proposed Changes

  • Pre-allocate ImageData and an array for gl.readPixels for PenSkins.
  • Set PenSkin's Silhouette directly from the skin's ImageData.

Reason for Changes

The previous texture > silhouette pipeline for PenSkin was:

  1. gl.readPixels of the PenSkin's framebuffer into a newly created Uint8Array.
  2. Create a new ImageData object from the array.
  3. Draw the ImageData to the PenSkin's canvas.
  4. Pass the canvas to the Silhouette, which:
  5. Draws the passed canvas to its own canvas, then
  6. Reads its canvas' ImageData and stores it.

The new pipeline is:

  1. gl.readPixels of the PenSkin's framebuffer into a previously allocated Uint8Array.
  2. Read the array into a previously created ImageData object.
  3. Pass the ImageData to the Silhouette, which uses it directly.

By only creating a new array and ImageData object when the PenSkin's canvas size is changed, and passing the Silhouette the ImageData directly, we can greatly improve the speed of silhouette updates as shown in this benchmark:

Latest develop This PR
image image

@thisandagain
Copy link
Contributor

/cc @mzgoddard @cwillisf @kchadha

Copy link
Contributor

@mzgoddard mzgoddard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome.

During loading performance optimizations we found that passing an ImageData to a webgl texture instead of a canvas was both much faster and used much less memory. It made sense to add ImageData as an accepted type to update since we were always producing the ImageData outside Silhouette. But we overlooked updating PenSkin when those changes were made.

@@ -612,6 +618,9 @@ class PenSkin extends Skin {
gl.clearColor(0, 0, 0, 0);
gl.clear(gl.COLOR_BUFFER_BIT);

this._silhouettePixels = new Uint8Array(Math.floor(width * height * 4));
this._silhouetteImageData = this._canvas.getContext('2d').createImageData(width, height);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We should definitely reuse these.

src/PenSkin.js Outdated
);

this._silhouetteImageData.data.set(this._silhouettePixels);
this._silhouette.update(this._silhouetteImageData);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its great to see this updated to use the ImageData api change.

@adroitwhiz adroitwhiz force-pushed the penskin-silhouette-from-data branch from 357d684 to b194394 Compare January 29, 2020 02:07
@adroitwhiz adroitwhiz mentioned this pull request Feb 20, 2020
Copy link
Contributor

@cwillisf cwillisf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😎

@@ -597,6 +603,9 @@ class PenSkin extends Skin {
gl.clearColor(0, 0, 0, 0);
gl.clear(gl.COLOR_BUFFER_BIT);

this._silhouettePixels = new Uint8Array(Math.floor(width * height * 4));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the math.floor isn't necessary here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to be used in every other place in the codebase that constructs a Uint8Array, specifically to prevent issues with Edge.

If I removed it here, I don't think it would cause any trouble (width and height should be integers), and newer versions of Edge (I tested it on Microsoft Edge 44.18362.449.0, Microsoft EdgeHTML 18.18363) have no trouble with floating-point values in typed array constructors, but it'd break the convention that the rest of the codebase is using. Is that okay?

(P.S. it may be worth revisiting this in the future by either testing whether Edge 15, the minimum version Scratch officially supports, supports floating-point values in TypedArray constructors, or by bumping the minimum supported Edge version in the FAQ to something newer.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops - thanks for the reminder! I had completely forgotten about that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants