Builtin extensions

[cwillisf]

Proposed Changes

This change converts the pen and WeDo 2.0 blocks into "internal extensions" -- extensions allowed to run in the main thread instead of in a worker -- and provides a mechanism for loading those two extensions by ID.

To load a built-in extension, call extensionManager.loadExtensionURL(extensionId). The valid IDs are 'pen' and 'wedo2'; any other ID will be treated as a URL.

Reason for Changes

This will make it easier to experiment with the extensions system before we have all the technical details settled.

Future Work

The extension system doesn't yet have support for drop-down menus, so although the WeDo 2.0 blocks contain the data necessary to generate menus those fields will appear as text inputs for now. Extension menus will be implemented as a followup to this change. (Tracking in #689)

This change also contains a partial fix for #580, though I now believe a complete fix will require changes in the scratch-blocks repository.

The message dispatch system, which is the foundation of the extension system, returns a promise for any call. Unfortunately the runtime is set up such that a call which returns a promise will always yield. I put in a hack to work around this limitation for internal extensions, but we need to do some work in the runtime if we want worker-sandboxed extensions to perform well. (Tracking in #688)

[cwillisf]

Oops - integration tests are failing because the opcode for pen blocks has changed. Will fix soon...

[paulkaplan]

@cwillisf did you have a plan for how to load extensions that are used when loading the project? For example, using this I can't load projects that use pen blocks (I get a number of errors). I think that this change should include some kind of auto-loader for at least the built-in extensions.

[ericrosenbaum]

@cwillisf Cool! Two small changes to get the Wedo to start working:

• Remove the scratch3_wedo2 require from runtime
• Add this.connect() to the Scratch3WeDo2Blocks constructor

[cwillisf]

@paulkaplan that's definitely something we'll need to handle. In Scratch 2.0 we record the extension ID/URL along with enough information to reconstruct the shape of the blocks if the extension can't be loaded. I think we should do the same in 3.0, but obviously it's not there yet. (Tracking in #692)

You should be able to get pen blocks working if you load a Scratch 2.0 project and also load a pen extension. Unfortunately(?) the Scratch 3.0 pen opcodes changed due to extension-ification, so pen blocks in a Scratch 3.0 JSON saved before this change will no longer work correctly after this change.

[rschamp]

This looks like what we want to me. Although process-wise, I think it would be good to either include the auto-loading of the pen extension for 2.0 projects in this PR, or to have the PR that accomplishes that ready when we merge this PR. Curious what @thisandagain thinks about that.

[cwillisf]

Yeah - that's one interesting point that we haven't really discussed yet. It's one thing to auto-load an extension explicitly mentioned in the extensions list of a project, but for pen -- and any other blocks we decide to convert to extensions -- we may have to inspect the opcodes in the project in order to decide whether we should load the pen extension. That, or we just always load the pen extension when importing a Scratch 2.0 project.

[cwillisf]

Not that inspecting the opcodes would be difficult or anything. I think it would make sense to add an optional field in sb2_specmap.js indicating which extension a block lives in, and collect/dedupe/load extensions as part of the SB2 import process.

[cwillisf]

I'm going to go ahead and merge this so I can file clean PRs for some of my other extension work. I'm also working on auto-loading the pen extension and expect to have that ready today.