HTML Modules

[justinfagnani]

Now that JavaScript modules are on the verge of widespread browser support, we should think about an HTML module system that plays well with it. HTML Imports doesn't have a mechanism for exporting symbols, nor can they be imported by JavaScript, however its loading behavior is quite compatible with JavaScript modules.

@dglazkov sketched out a proposal for HTML modules here: https://github.com/dglazkov/webcomponents/blob/html-modules/proposals/HTML-Imports-and-ES-Modules.md

The main points are that, using the JavaScript modules plumbing, you can import HTML.

Either in HTML:

<script type="module" url="foo.html">

or JavaScript:

import * as foo from "foo.html";

But within the scope of that sketch there are still several questions about the specifics of exporting symbols from HTML and importing to JavaScript.

[justinfagnani]

@TakayoshiKochi made a polyfill (based on the <script type-module> polyfill by @matthewp) that implements one set of choices to the questions, and shows JS and HTML modules working together: https://github.com/TakayoshiKochi/script-type-module/tree/html-module-experiment

[domenic]

Thanks for starting the discussion, @justinfagnani!

For context for others, I've been involved with @TakayoshiKochi and others trying to firm up what HTML modules means before we present it widely. There's still a variety of possibilities. Here's my high-level overview:

---

Writing a custom element that authors can include using a single file is a bit tricky. In particular, how do you get your template HTML? (Which is often literally a <template>.)

• If you want to stay self-contained, and thus friendly to consumers, you're currently forced to inline your HTML into your JavaScript as a template string, then later use innerHTML. This is costly; it requires first parsing it in JavaScript, then later as HTML, and using up extra memory for both representations. It requires all-at-once parsing, instead of streaming, and requires it to be done on the main thread (I think). And it defeats the preload scanner for any subresources.
• If you want to "use the platform", you'll have to ask your consumers to include the <template> in their index.html, with some well-known ID. Then all of the above disadvantages disappear, but consuming your library is very un-ergonomic.

At a high level, HTML modules wants to solve this problem. I think the case for it being a worthwhile problem would be stronger if we had data on the costs of the HTML-in-a-JS-string solution, but just by inspection it seems pretty bad. Still, I hope we can provide some data.

---

The next big question then is about developer ergonomics: what should the authoring and consumption experience for web components be?

• At one extreme, we could just allow importing of inert DocumentFragments via the JS module system. This solves the above problem in a minimal, simple fashion. But, it means you cannot collocate your HTML and JS into a single file. Most modern frameworks seem to aim for a developer experience of collation, from React's JSX, to Vue's .vue files, to Polymer's HTML imports usage, to Angular's HTML-in-a-JS-string approach.
• At another, we could work to add some opinionated semantics to imported HTML files that make them good as a "container" format for a web component. This is the approach @TakayoshiKochi has gone with, and I think it's a good illustration. It's much less of a primitive, and requires adding new unusual semantics (e.g. executing scripts outside the main document), but that might be OK.

---

Hope this helps set the stage!

[addyosmani]

Thanks for getting a public discussion going, @justinfagnani!

I see HTML Modules as being a good container format for achieving generic subresource dependency specification. They solve a need for platform-level colocated <template>, <style> and <script>/Module in a similar style to what I would get with something like Vue's .vue files or Polymer's existing HTML Imports. I'd love for them to sufficiently export a JS module - I found this to be a shortcoming of HTML Imports when I last used them.

Some questions:

• What it is about HTML Modules that makes them specific to Web Components? If a third-party decides to just use HTML + CSS + JS in this container format, that should just work too right? I think we should cater for a few different styles of component to work well with HTML Modules.

• Are HTML Modules primarily a DX convenience for containment? or do we also foresee trying to bake in a performance carrot (e.g preload scanner benefits) to the design?

• How do we see HTML Modules fitting with respect to the broader Web Packaging efforts?

[justinfagnani]

I also want to point out that HTML modules would not prescribe how component developers package their libraries, or even prescribe Web Components for that matter.

There are a number of styles that components could be written and/or packaged:

1. Pure JS, using either template strings, DOM builder libraries, JSX, etc. (no HTML modules)
2. Logic in JS, template/CSS in HTML: import template from 'template.html'; in the component's module. Maybe event import style from 'styles.css'; is we include CSS modules.
3. Single-file, like Vue and Polymer: JavaScript, HTML and CSS all in one file.

These styles could all be compatible.

[domenic]

These styles could all be compatible.

That's a good point. Even if we go with an opinionated container format with new capabilities, like @TakayoshiKochi's prototype, it will likely be possible to opt-out of those new capabilities, e.g. by just making your main element a <template> so that everything inside it is inert.

[matthewp]

It looks like @TakayoshiKochi's experiment doesn't create a new document for the imported module. Not sure if this is an intentional omission or not, as the @dglazkov sketch does mention there being a new document like in html imports. Here's an example module for those interested.

In the interest of listing problems and not solutions I would say some things I want (in addition to other things mentioned by people above) are:

1. I want to be able to export JavaScript modules from an HTML module.
2. I want to be able to query for elements within the HTML module.
3. I want to be able to "reach in" to an HTML module some how from the outside and use stuff other than the HTML modules within. But maybe if (1) and (2) are satisfied that would mean that an HTML module could make the other parts (like the <template>) accessible?

[TakayoshiKochi]

Thanks @justinfagnani for starting this thread! Thanks @domenic for the summary.
I was wondering if this is a good place as @addyosmani mentioned if HTML Modules may not be
specifically for web components. But as it is, this is good starting here in public:)

We're open to any questions / problems for HTML modules.
Feel free to pour your thoughts here.

FYI, here's slide deck I presented in BlinkOn back in February:
https://docs.google.com/presentation/d/1ksnC9Qr3c8RwbDyo1G8ZZSVOEfXpnfQsTHhR5ny9Wk4/edit

[TakayoshiKochi]

Re @matthewp (#645 (comment))

It looks like @TakayoshiKochi's experiment doesn't create a new document for the imported module.

Yes, it was intentional. Imported HTML is treated as DocumentFragment (as opposed to Document), so we don't have to bother about various semantics to be satisfied, although it yields some weirdness (e.g. why/how can <script> inside be executed?). I'm interested if you prefer Document (other than that was described in Dimitri's doc) to DocumentFragment.

If you're interested in history, here's record of Document vs DocumentFramgent for HTML Imports:
https://www.w3.org/Bugs/Public/show_bug.cgi?id=22305

[TakayoshiKochi]

We are interested in hearing requirements from framework authors, as HTML modules is not (should not be?) specific to web components definition (= container for custom elements and shadow DOM definition).

I admit that my experimental polyfill was biased towards HTML Imports + web components, and would be a good fit for Polymer or Vue use cases, but I would be more excited if the resulting HTML modules would be usable for more than these frameworks or libraries.

In my blink-dev thread, a developer from Scirra introduced me his very inspiring blog post:
https://www.scirra.com/blog/ashley/34/html-imports-are-the-best-web-component
The post is long, but I recommend reading it from top to bottom - my interpretation is that HTML Imports fit their product development requirement quite well, even though they do not use Shadow DOM and do use only little Custom Elements. That exemplifies the need for something that supports scalable development on web platform, and HTML Imports is one key primitive for it.

[rniwa]

Here's my take on this. In today's world, developers are putting HTML inside their JS, not JS inside their HTML. The fact that we don't currently preload resources in a string literal, or we end up two string representations of HTML if you used string literal to include HTML in a JS file is a pure implementation detail. There is nothing preventing us from implementing an optimization if there was a well known symbol which was used to tag a HTML in a JS.

Furthermore, for various performance reasons, we really need to have a mechanism to define an in-file ES6 module; a mechanism which allows multiple modules to be defined within a single JS file. Once we have that, then it's very natural to define three modules: one with HTML, one with CSS, and one with JS in a single JS file.

So I'm not all convinced that we need to put JS into HTML as done in the HTML import. In fact, we'd likely oppose to such an approach on the basis that popular JS frameworks such as React have gone to the completely other direction of embedding HTML inside JS.

[matthewp]

@rniwa I find this opinion confusing given your interest in template parts. Are you not interested in that feature either, now? Or are you saying you'd prefer to define your <template> (with parts) as a string literal in JS rather than in HTML. Why?

[web-padawan]

the basis that popular JS frameworks such as React have gone to the completely other direction of embedding HTML inside JS.

While not being a framework author nor a browser developer here, just an end-user, I'd like to say that, although it is a widely adopted approach, it still requires tooling and you'll not be able to serve vanilla HTML, CSS and JS unprocessed by stuff like Webpack and so on.

I'd also like to remind you that React was initially built with PHP where including HTML was once widely used. But is such a mixing really considered as a good practice by PHP developers now?

Furthermore, how shall HTML modules work in browsers with disabled JavaScript? If markup stays in HTML, I can imagine them still working somehow. But what if it is placed in string literals?

[AshleyScirra]

I wrote this blog post. I'm interested in participating in work for HTML modules, and I think our experience is valuable: we have published a commercial PWA using ~300 HTML imports.

As I touched on in the blog, I quite like the idea of being able to import a HTML file and get a Document, much like XHR for type "document". So this:

import * as doc from "./myimport.html"

would be similar to assigning doc the Document resulting from fetching myimport.html. You can then querySelector etc. on doc and use its DOM content in the JS module. (IMHO, this is far better than pasting chunks of markup in to strings in JS)

Then all you need to do is add a HTML tag that can propagate to further dependencies. I suggested changing

<link rel="import" href="import.html">

to be equivalent to

<script type="module">
import * as doc from "./myimport.html"
</script>

It could equally be <script type="module" src="myimport.html"></script> to do the same. This means further scripts/imports/modules are loaded and run, etc.

I have to say style application from imports is a useful feature for us - see my example of a dialog definition for an example. It's possible a framework could work around this (e.g. applyStylesFrom(doc)), but it seems a shame if the dialog definition example can't work natively.

If HTML modules covered that, I'd guess that largely covers everything we use imports for. In our case they're largely just the architectural scaffolding that brings together all the components of our web app, and as I tried to emphasise in my blog, they excel at that, particularly since it covers all three aspects of a web app (markup, style, script, at least until Google roll back style application 😢).

I am strongly against trying to use script for any markup or style content. I know some frameworks are having success with that approach, and so we shouldn't do anything that makes that harder. But if you have 1000 lines of markup, surely nobody wants to paste that in to a script as a giant string? I strongly believe that there should at least be the provision for leaving markup in HTML files, and not orient the spec around doing what one framework does today. Remember frameworks come and go over the years. In our case, our PWA is notable for not using VDOM, has almost no use of markup-in-script, uses plain markup (no Angular-style directives), no templating (like {{this}} in markup), and no two-way data binding. I don't think any other framework on the web today takes this approach, but I believe our PWA proves that you can still build scalable, complex web apps that way. I do worry that people have framework-blindness where they think things should be done like the frameworks do, rather than thinking in terms of providing general-purpose capabilities that frameworks of different designs and approaches can make use of, even in the future when everything likely changes.

Anyway, here's a couple of extra ideas that may be useful to extend the utility of HTML modules as I described them. Perhaps it'd be useful to add a selector to pick out specific elements, e.g.:

import "#elem" as elem from "./myimport.html"

Now elem is the DOM element returned by matching the selector #elem on myimport.html. Not sure how to handle multiple results (e.g. using ".myclass" as a selector) - maybe import ".myclass" as [resultsArray] from ".myimport.html"?

Finally I don't know if it makes sense to export from a HTML module, but perhaps you could have something like:

import myElem from "./myimport.html"

where myimport.html has a special tag like

<export name="myElem">
    <div>This div is an exported element!</div>
</export>

Perhaps that could also be reconfigured to re-export a JS module that the HTML module imports, e.g:

<export name="submodule">
    <script type="module" src="a.js"></script>
</export>

...then allowing import submodule from "./myimport.html", which propagates the exports from a.js through the HTML import and in to another JS module. I feel like I'm shooting in the dark there though!

[domenic]

Furthermore, how shall HTML modules work in browsers with disabled JavaScript? If markup stays in HTML, I can imagine them still working somehow. But what if it is placed in string literals?

I can answer that, at least: they won't. HTML modules are part of the module system, which is entirely dependent on JavaScript.

Even HTML imports never worked in browsers without JavaScript. (The document would be "imported", but useless, since you need script to access imported documents.)

[justinfagnani]

Even HTML imports never worked in browsers without JavaScript. (The document would be "imported", but useless, since you need script to access imported documents.)

Depending on future developments of course. Declarative custom elements (which we'll talk about at TPAC) presumably wouldn't need script enabled.