[p5.js 2.0 RFC Proposal]: Async/Await setup()

[limzykenneth]

Increasing access

Adopting async/await can help with sketch authors' adoption of modern JavaScript syntax and best practices, making their skills more generally transferrable. The overall syntax may be considered to be simpler than the need to use the preload() function as well.

Which types of changes would be made?

• Breaking change (Add-on libraries or sketches will work differently even if their code stays the same.)
• Systemic change (Many features or contributor workflows will be affected.)
• Overdue change (Modifications will be made that have been desirable for a long time.)
• Unsure (The community can help to determine the type of change.)

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build process
• Unit testing
• Internationalization
• Friendly errors
• Other (specify if possible)

What's the problem?

Taking directly from the current RFC

The async setup() function eliminates the need to have the preload() function. As such the data loading codebase will be refactored to remove preload() related code, while documentation around data loading should be updated accordingly.

What's the solution?

Data loading functions will all be updated to be fully async using promises (with async/await) while keeping the callback syntax if necessary. setup() will be awaited in the runtime and if the user define it as async functions, the promisified data loading functions can be awaited in them. With an async setup() function, the draw() loop will only start after the setup() function has resolved.

draw() can be defined as an async function as well, although this is not recommended because of the possible timing conflict it may have with requestAnimationFrame. (need more testing to confirm)

---

In terms of example code, in global mode, to load an image and display it:

let img;
async function setup(){
  createCanvas(400, 400);
  img = await loadImage("./Assets/cat.jpg");
  console.log(img);
}

function draw(){
  background(200);
  image(img, 300, 300, 100, 100);
}

In instance mode:

const sketch = function(p){
  let img;
  p.setup = async function(){
    p.createCanvas(400, 400);
    img = await p.loadImage("./Assets/cat.jpg");
  }

  p.draw = function(){
    p.background(200);
    p.image(img, 300, 300, 100, 100);
  }
};

new p5(sketch);

This will extend to all loading functions and can be used by addon libraries simply by making any asynchronous operation async.

For internal loading functions, the callback functions may be retained, eg. loadImage(imgSrc, successCallback, failureCallback), although a case can be made to not have this and just use the underlying promise instead, eg. loadimage(imgSrc).then(successCallback).catch(failureCallback).

A working example for this can be seen here.

Pros (updated based on community comments)

• Semantic JavaScript syntax for asynchronous operations
• Simpler and potentially more intuitive data loading
• Better compatibility with other promises based API in JavaScript ecosystem

Cons (updated based on community comments)

• Requiring some level of understanding of asynchronicity to start using
• Existing addons with reliance on older preload() behaviour will likely stop working

Proposal status

Under review

[nickmcintyre]

@limzykenneth this looks good to me. Most people have written dozens of sketches by the time they load external media. Introducing async / await syntax shouldn't be a big leap for beginners by that point.

[davepagurek]

One other point to mention: if one is loading multiple files, the behaviour of current p5's preload is more equivalent to if the user awaits Promise.all(...) for all their assets. We probably still want that behaviour if we can, for faster load times. Do you think this is something we want to make users do manually?

An option we can consider that would introduce some magic under the hood is to internally keep track of all the promises created by load* functions, and only run setup after awaiting Promise.all(...) of all of those. Then users could still await things within preload if some assets have dependencies on other assets (e.g. load a text file, and then load an image for each line in the file) but by default still have everything load in parallel. The downside of something like that is that anything users load, they'd still want to assign to a variable so they can use it in draw, so this would re-introduce the issue where we have to return an object immediately, and then modify it on load, preventing us from loadJSONing both objects and arrays.

Another option is to add a helper or something with a slightly nicer API. Maybe:

let myJSON, myStrings;

async function preload() {
  // Like Promise.all but working on object values instead of array items
  { myJSON, myStrings } = loadAll({
    myJSON: loadJSON('something.json'),
    myStrings: loadStrings('something.txt')
  });
}

...but that could use some work since it's relying on destructuring, which is sort of advances javascript for newcomers. Hopefully other people have some other API ideas?

[limzykenneth]

@davepagurek Parallel loading is indeed a potential concern for this. I'm initially thinking just leaning into promises if the syntax don't feel too complicated.

let img1, img2, img3;
async function setup(){
  [img1, img2, img3] = await Promise.all([
    loadImage("cat1.jpg"),
    loadImage("cat2.jpg"),
    loadImage("cat3.jpg")
  ]);
}

There's also the potential issue of error handling which with async/await we'll need to use try...catch.

[nickmcintyre]

Thanks for pointing this out @davepagurek. It feels like we should hide this complexity if at all possible. Loading a few images quickly (in parallel) should be as straightforward as possible.

Unless teachers do a lot of hand waving, the current suggestions would force beginners to learn some mix of JavaScript objects/arrays, destructuring, async / await, Promise objects, and dot notation in order to draw a few cats.

[davepagurek]

I'm not sure that this is a good idea, but what if we do something like this:

let data
async function setup() {
  data = await preload({
    img1: loadImage('cat1.jpg'),
    img2: loadImage('cat2.jpg'),
    img3: loadImage('cat3.jpg'),
  })
}

function draw() {
  image(data.img1, 0, 0)
  image(data.img2, 100, 0)
  image(data.img3, 200, 0)
}

Here, each load* method still returns a promise, and the preload method is something like this, which loads all the values in parallel but then returns an object with the same keys mapped to the loaded values:

async function preload(data) {
  const result = {}
  await Promise.all(Object.keys(data).map(async (key) => {
    result[key] = await data[key]
  })
  return result
}

More advanced users could destructure this if they want, but otherwise, they can just leave it as an object. The benefit of using this thing that works like Promise.all but for object values is that you can refer to the result using more readable keys, like data.img1, rather than just by index. I'm not sure if that's better enough to be a solution though.

[nickmcintyre]

Neat, that's definitely getting simpler. Objects (or arrays) filled with function calls seem a little complex, though.

How about overloading the load* functions so that they use Promise.all() when an object is passed? Doing so would mean that users only get partial parallelism (per load* function). I believe the code would be easier to reason about, so maybe that's an OK tradeoff.

// Single image.
let img;

async function setup() {
  img = await loadImage("cat.jpg");
}

function draw() {
  image(img, 0, 0);
}

// Multiple images.
let data;

async function setup() {
  data = await loadImage({
    img1: "cat1.jpg",
    img2: "cat2.jpg",
    img3: "cat3.jpg",
  });
}

function draw() {
  image(data.img1, 0, 0)
  image(data.img2, 100, 0)
  image(data.img3, 200, 0)
}

Compared to the current preload() function, the only new concepts and syntax would be JavaScript objects {} and async / await.

Could we get full parallelism in a loadAll() function by inferring file types and calling the appropriate load* function?

// Image and JSON.
let data;

async function setup() {
  createCanvas(400, 400);

  data = await loadAll({
    img: "cat.jpg",
    pos: "position.json",
  });
}

function draw() {
  background(200);

  // Get current position.
  let pos = data.pos[frameCount];

  // Draw cat.
  image(data.img, pos.x, pos.y);
}

[GregStanton]

Hi all. There's a lot of creative discussion going on here! I have a different view of this, but I could be wrong. Basically, I think the native syntax is significantly less semantic than the current p5 syntax, and more complex for non-experts. But there might be benefits to this proposal that I don't fully understand yet.

On semantic code

With preload(), the user can guess the meaning without documentation: it loads things in advance. With the combination of setup(), async, and await, guessing the meaning is quite difficult. It's not too hard to write down a plausible train of thought that would lead a beginner to incorrectly guess the meaning.

In any case, they're likely to try to look up the meaning instead of spending a long time guessing, and this is equally fraught. As noted on MDN, "Asynchronous JavaScript is a fairly advanced topic." Tutorials tend to be long and assume multiple prerequisites beginners won't have. And it's not hard to find forums where people say they've read all the articles and videos but still can't understand it. Here's an example, so that we have some direct evidence.

Leaving the user to rely on these kinds of articles and tutorials seems problematic for p5, and explaining async/await in a reference page seems like it'd be quite difficult. It's also not clear where we'd do that. Since these are features of JavaScript itself, the natural place seems to be the Foundation section of the p5.js reference, but async/await is not a beginner-level concept like the other entries in that section. To me, that's a red flag. If a feature is not a basic aspect of JavaScript, and it's essential for making sketches, then p5.js generally provides a more friendly API on top of it. That's a big part of what sets p5.js apart.

On simple syntax

Syntactically, preload() also seems much simpler for non-experts, since it's a simple function call like the rest of the p5.js API, and it takes no parameters like most of the other functions in the Structure section of the reference.

With async/await, we preface function declarations with unfamiliar keywords, and we need to pass in an object literal. To an experienced JavaScript programmer, this use of object literals is a great simplification because we can basically simulate named parameters. But for a beginning p5.js user, an object literal as a function argument is likely to raise a lot of questions:

What's this syntax with curly braces inside of parentheses? Is that a special thing for load functions? Why can't I just pass in separate values like usual? I can't remember, was it a square brace or a curly brace? Was there a colon or a semi-colon between the two things? Between the other two things, was there a semicolon or a comma?"

Learning new syntax is beneficial for users, but as a rule, I think it's probably good to introduce new syntax when it becomes necessary, and no sooner. The current API with preload() seems to show these complications aren't necessary.

On modern syntax

I still need to study this more, but my understanding is that JavaScript originally used nested callbacks to accomplish the programming tasks we're discussing here. But, nesting lots of functions inside other functions resulted in code that was hard to look at (so-called "callback hell"). So, in ECMAScript 2015 (ES6), promises were introduced, and these supported chaining with dot notation. That was more readable but still involved connecting a lot of instructions together. Later, in ECMAScript 2017, async/await was introduced, which made it possible to replace chaining with a sequence of separate instructions (just like the most basic JavaScript syntax). That was a great simplification, so people were excited about it.

Based on all this, using callbacks for these sorts of tasks may be viewed as outdated. But old isn't necessarily bad. The original problem that needed to be solved was that deeply nested functions aren't very readable. That doesn't seem to be a problem with the current API. Am I wrong? I haven't worked with this area of p5 much.

Benefits?

I wouldn't think that removing preload(), by itself, is enough of a justification. But if the other benefits of this proposal are significant enough, then the points I made above may not matter, even if they're correct. Could anyone give examples of the benefits that better compatibility with other promise-based APIs would bring? Are there other benefits, such as extra flexibility in cases that are likely to be encountered? Are there specific benefits for add-on libraries over the current API?

[nickmcintyre]

I'm definitely in the preload() is simpler camp. But I wonder if we can find the simplest possible approach to async/await for discussion's sake. If we can reduce the complexity enough for beginners, then I'm all for pulling the curtain back a bit.

freeCodeCamp's explanation of the await keyword seems like a reasonable starting point:

The await keyword basically makes JavaScript wait until the Promise object is resolved or rejected.

would become something like

The await keyword basically makes p5.js wait until the data loads or an error is detected.

For parallelism, we could simplify a bit further by working with arguments and Promise.all() behind the scenes. This version would return an array:

// Single image.
let img;

async function setup() {
  img = await loadImage("cat.jpg");
}

function draw() {
  image(img, 0, 0);
}

// Multiple images.
let img1;
let img2;
let img3;

async function setup() {
  img1 = await loadImage("cat1.jpg");
  img2 = await loadImage("cat2.jpg");
  img3 = await loadImage("cat3.jpg");
}

function draw() {
  image(img1, 0, 0);
  image(img2, 100, 0);
  image(img3, 200, 0);
}

// Multiple images in parallel.
let img1;
let img2;
let img3;

async function setup() {
  let data = await loadImage("cat1.jpg", "cat2.jpg", "cat3.jpg");
  img1 = data[0];
  img2 = data[1];
  img3 = data[2];
}

function draw() {
  image(img1, 0, 0);
  image(img2, 100, 0);
  image(img3, 200, 0);
}

// Image and JSON in parallel.
let img;
let pos;

async function setup() {
  let data = await loadAll("cat.jpg", "position.json");
  img = data[0];
  pos = data[1];
}

function draw() {
  image(img, pos.x, pos.y);
}

[davepagurek]

One benefit of the async syntax is that it means there are less states where you could be looking at a variable in an incomplete form. e.g.:

let data
function preload() {
  data = loadJSON('data.json')
  console.log(data) // just logs {}
} 

function setup() {
  console.log(data) // logs the actual data now
}

vs:

let data
async function setup() {
  data = await loadJSON('data.json')
  console.log(data) // logs the data
} 

Our load* functions have to return something right away and then update the object when it has data. await acts like a barrier, so we go from a state of having a promise to having real data, without that odd in-between of having something but where you have to know it's not actually complete yet.

Another side effect of having to immediately return an object is that it has to stay an object. This means when you loadJSON, we return an object immediately, but if the data turns out to be an array, we're forced to convert it to an object. There are a number of issues opened in the past about this, e.g. #2154, but we're technically unable to solve it with the current preload setup. With async functions, we can return a promise that resolves to either an object or an array. (Another solution to this might be to just implement a loadJSONArray function.)

Not sure if these alone are enough to tilt the scales, but I wanted to mention them to paint a complete picture!

[GregStanton]

Thanks @nickmcintyre! You make a great point. I totally agree that it's best to seek out the simplest version of asyc/await for the sake of discussion. And wow. Your latest version is looking quite good.

The way you handle the second case, where await is used multiple times, might actually be simpler than using p5's load* functions with callbacks. I'm glad you realized I was wrong about the need to pass in an object literal; I didn't think carefully enough about that.

As far as documentation goes, maybe we could have one short explanation of async/await that just explains the basic idea in relation to this particular use, in a combined async/await entry under the Foundation section of the reference. And the reference pages for each of the load* functions could link to that. This way, we wouldn't need to maintain explanations in the reference pages for each load* function.

Also, @davepagurek, the first benefit that you mentioned seems like it could be pretty significant, just by itself. I need to look into the second one more to understand the options there, but it definitely seems relevant too.

I'm hoping to come back to this when I have more time so that I can understand all the nuances better, but I might actually be in favor of the proposal now. I just need to think more.