Merge Accessibility Add-On into p5.js

contributor_docs/project_wrapups/README.md

@@ -3,6 +3,8 @@ This folder contains wrapup reports for projects from p5.js related [Google Summ
 
 *Note for contributors: Embedded images and media are welcome. Please host these files externally rather than placing in this repo to avoid adding growing the repository filesize too much.*
 
+### Google Summer of Code 2020
+* [p5.js accessibility and canvas descriptions](https://github.com/processing/p5.js/blob/main/contributor_docs/project_wrapups/luismn_gsoc_2020.md) by Luis Morales-Navarro, 2020
 
 ### Google Summer of Code 2019
 * [Search Bar for Sketches in the p5.js Web Editor](https://github.com/processing/p5.js/blob/main/contributor_docs/project_wrapups/rachellim_gsoc_2019.md) by Rachel Lim, 2019

contributor_docs/project_wrapups/luismn_gsoc_2020.md

@@ -0,0 +1,53 @@
+# p5.js accessibility and canvas descriptions
+GSoC 2020 | [Luis Morales-Navarro](https://luismn.com/)
+
+### Overview:
+During this Google Summer of Code, I worked with [Kate Hollenbach](https://github.com/kjhollen)
+to improve the accessibility features of p5.js. We focused on merging the
+text output and table output functionalities of [p5.accessibility](https://github.com/processing/p5.accessibility)
+into p5.js and created functions that support p5.js users in writing their own screen reader accessible canvas descriptions.
+
+### Background:
+#### p5.js and Accessibility: from an editor feature to an add-on to the library
+The work done during this summer is part of the project's [efforts to make p5.js more accessible for persons with dissabilities](https://contributors-zine.p5js.org/#reflection-claire-kearney-volpe).
+Early work by Claire Kearney-Volpe, Taeyoon Choi, and Atul Varma identified the need to make p5.js sketches
+and the canvas accessible to screen readers and people who are blind. I met Claire in late 2016 when
+they were working with Mathura Govindarajan to add accessibility features to the p5.js editor. I joined them
+and together with the support of dedicated contributors and advisors (including Cassie Tarajakan, Lauren McCarthy,
+Josh Mielle, Sina Bahram, and Chancey Fleet) we implemented three accessible canvas outputs (a text output, a grid
+output and a sound output) on the alpha editor.
+
+Later on, through a 2018 Processing Foundation Fellowship Claire, Mathura and I developed p5.accessibility.js a p5.js add-on. 
+p5.accessibility.js (developed with contributions from Antonio Guimaraes, Elizabeth G Betts, Mithru Vigneshwara, and Yossi Spira)
+helped us bring the work we had done with accessible outputs in the editor to any p5.js sketch that included the add-on.
+However, the add-on was still an add-on that required users to include an extra file and edit their html.
+
+At the 2019 p5.js Contributors Conference, as a community, we reinforced the project's commitment to access and inclussion.
+Together with Claire, Sina, Lauren, Kate, Olivia McKayla Ross and Evelyn Masso we outlined the pathway forward.
+Among short-term actions, we identified the need for functions that allow users to write their own descriptions
+and the importance of merging the add-on into the p5.js library.
+
+### Contributions:
+During the course of Summer of Code, my work focused on creating library generated screen reader accessible outputs
+for basic shapes on the canvas and functions to support user-generated screen reader accessible descriptions of canvas content.
+I worked on the following PRs:
+- [Add describe() and describeElement() #4654](https://github.com/processing/p5.js/pull/4654): This PR adds the functions describe() and describeElement(), tests for these functions, documentation and examples.
+- [Merge Accessibility Add-On into p5.js #4703](https://github.com/processing/p5.js/pull/4703): This PR adds the functions textOutput() and gridOutput(), helper functions to create and update, the outputs and tests, documentation and examples. At first the plan was to update the add-on and prepare it for merging it with p5.js in the near future. However, we realized it was more time effective to recreate the functionality of the text output and grid output in p5.js than upgrading the add-on which relied on ["monkey patching," entities and interceptors](https://medium.com/processing-foundation/making-p5-js-accessible-e2ce366e05a0). Now, the outputs are fully integrated to the library.
+
+More information on how these accessibility features work is available in the [web accessibility contributor docs](https://github.com/processing/p5.js/blob/main/contributor_docs/web_accessibility.md).
+
+### Future
+- There is a lot of work that can be done to improve the accessibility of p5.js sketches. In the [Web accessibility next steps conversation #4721 Issue](https://github.com/processing/p5.js/issues/4721) we have outlined some ideas and questions.
+- The work done during the summer focused on code and code issues but it is important to iteratively test these features with members of the community, particularly novices and learners who are blind. It is also important to create more resources for learning and teaching that support accessibility.
+- Immediate next steps include:
+  - A tutorial on how to describe things on the canvas.
+  - Changes in the way screen-reader descriptions are created in the reference. Using the describe() function instead of relying on @alt
+  - Maybe adding describe() to the templates on the website and editor
+  - Upgrading the tutorial on using p5 with a screen reader
+  - Changing the way the accessibility settings work on the editor
+
+### Acknowledgements
+I am grateful to Kate Hollenbach for their guidance, feedback and assistance, to Lauren McCarthy for their feedback and to Claire Kearney-Volpe for helping me come up with this project. Thanks to Sina Bahram for their input —our conversations at 2019 p5.js Contributors Conference inspired the describe() and describeElement() functions—, and to Akshay Padte for their advice on unit testing. This GSoC project would not have been possible without Chancey Fleet and Claire (who started thinking of ways to make p5.js sketches screen reader accessible in late 2015), the work of Mathura Govindarajan, and of many other contributors and supporters in the p5.js community.
+
+:heart: 
+

contributor_docs/web_accessibility.md

@@ -1,6 +1,6 @@
 # p5.js Web Accessibility
 
-This document describes the structure of the web accessibility features of p5.js for contributors and maintainers—and any other interested parties. If you're interested in making your sketches [screen reader](https://en.wikipedia.org/wiki/Screen_reader) accessible, visit the [tutorial]() or if you want to use p5.js with a screen reader visit the [Using p5.js with a Screen Reader tutorial]().
+This document describes the structure of the web accessibility features of p5.js for contributors and maintainers—and any other interested parties. If you're interested in making your sketches [screen reader](https://en.wikipedia.org/wiki/Screen_reader) accessible, visit the [tutorial](https://p5js.org/learn) or if you want to use p5.js with a screen reader visit the [Using p5.js with a Screen Reader tutorial](https://p5js.org/learn/p5-screen-reader.html).
 
 ## Overview
 
@@ -19,10 +19,10 @@ Supported accessible outputs for basic shapes include text and grid outputs.
 If a user passes `LABEL` as a parameter in either of these functions, an additional div with the output adjacent to the canvas is created, this is useful for non-screen reader users that might want to display the output outside of the canvas' sub DOM as they code. However, using `LABEL` will create unnecessary redundancy for screen reader users. We recommend using `LABEL` only as part of the development process of a sketch and removing it before publishing or sharing your sketch with screen reader users.
 
 ### Outputs structure
-Although `textOutput()` and `gridOutput()` are located in [src/accessibility/outputs.js](), the outputs are created and updated using functions distributed across the library. This section details the different functions that support the accessible outputs.
+Although `textOutput()` and `gridOutput()` are located in [src/accessibility/outputs.js](https://github.com/processing/p5.js/blob/main/src/accessibility/outputs.js), the outputs are created and updated using functions distributed across the library. This section details the different functions that support the accessible outputs.
 
 #### outputs.js
-[src/accessibility/outputs.js]() includes the core functions that create the accessible outputs:
+[src/accessibility/outputs.js](https://github.com/processing/p5.js/blob/main/src/accessibility/outputs.js) includes the core functions that create the accessible outputs:
 * `textOutput()`: This function activates the text output by setting `this._accessibleOutputs.text` to `true` and calling `_createOutput('textOutput', 'Fallback')`. If `LABEL` is passed as a parameter the function also activates the text output label by setting `this._accessibleOutputs.textLabel` as `true` and calls `_createOutput('textOutput', 'Label')` for the label.
 * `gridOutput()`: This function activates the grid output by setting `this._accessibleOutputs.grid` to `true` and calling `_createOutput('gridOutput', 'Fallback')`. If `LABEL` is passed as a parameter the function also activates the grid output label by setting `this._accessibleOutputs.textLabel` as `true` and calls `_createOutput('gridOutput', 'Label')` for the label.
 * `_createOutput()`: This function creates the HTML structure for all accessible outputs. Depending on the type and display of the outputs the HTML structure created varies. The function also initializes `this.ingredients` which stores all the data for the outputs including: shapes, colors, and pShapes (which stores a string of the previous shapes of the canvas). It also creates `this.dummyDOM` if it doesn't exist. `this.dummyDOM` stores the HTMLCollection of DOM elements inside of `<body>`.
@@ -56,23 +56,23 @@ When `this._accessibleOutputs.text` or `this._accessibleOutputs.text` are `true`
   * `p5.Renderer2D.prototype.background()`
 
 #### textOutput.js
-[src/accessibility/textOutput.js]() contains all functions that update the text output. The main function in this file is `_updateTextOutput()` which is called by `_updateAccsOutput()` in [src/accessibility/outputs.js]() when `this._accessibleOutputs.text` or `this._accessibleOutputs.textLabel` are `true.`
+[src/accessibility/textOutput.js](https://github.com/processing/p5.js/blob/main/src/accessibility/textOutput.js) contains all functions that update the text output. The main function in this file is `_updateTextOutput()` which is called by `_updateAccsOutput()` in [src/accessibility/outputs.js](https://github.com/processing/p5.js/blob/main/src/accessibility/outputs.js) when `this._accessibleOutputs.text` or `this._accessibleOutputs.textLabel` are `true.`
 
 `_updateTextOutput()` uses `this.ingredients` to build the content of the text output and text output label which include a summary, a list of shapes, and a table of shapes details. If these are different from the current outputs it updates them. Building the output content is supported by several helper functions in the file that are not part of the prototype: 
 *  `_textSummary()`: Builds the content of the text output summary.
 *  `_shapeDetails()`: Builds the text output table that contains shape details.
 *  `_shapeList()`: Builds the list of shapes of the text output.
 
 #### gridOutput.js
-[src/accessibility/gridOutput.js]() contains all functions that update the grid output. The main function in this file is `_updateGridOutput()` which is called by `_updateAccsOutput()` in [src/accessibility/outputs.js]() when `this._accessibleOutputs.grid` or `this._accessibleOutputs.gridLabel` are `true.`
+[src/accessibility/gridOutput.js](https://github.com/processing/p5.js/blob/main/src/accessibility/gridOutput.js) contains all functions that update the grid output. The main function in this file is `_updateGridOutput()` which is called by `_updateAccsOutput()` in [src/accessibility/outputs.js](https://github.com/processing/p5.js/blob/main/src/accessibility/outputs.js) when `this._accessibleOutputs.grid` or `this._accessibleOutputs.gridLabel` are `true.`
 
 `_updateGridOutput()` uses `this.ingredients` to build the content of the grid output and grid output label which include a summary, a grid that maps the location of shapes and a list of shapes. If these are different from the current outputs it updates them. Building the output content is supported by several helper functions in the file that are not part of the prototype: 
 *  `_gridSummary()`: Builds the content of the grid output summary.
 *  `_gridMap()`: Builds a grid that maps the location of shapes on the canvas.
 *  `_gridShapeDetails()`: Builds the list of shapes of the grid output, each line of the list includes details about the shape.
 
 #### color_namer.js
-When creating screen reader accessible outputs, naming the colors used in the canvas is important. [src/accessibility/color_namer.js]() contains `_rgbColorName()` a function that receives rgba values and returns a color name. This function is called by `_accsBackground()` and `_accsCanvasColors` in `src/accessibility/outputs.js`. 
+When creating screen reader accessible outputs, naming the colors used in the canvas is important. [src/accessibility/color_namer.js](https://github.com/processing/p5.js/blob/main/src/accessibility/color_namer.js) contains `_rgbColorName()` a function that receives rgba values and returns a color name. This function is called by `_accsBackground()` and `_accsCanvasColors` in [src/accessibility/outputs.js](https://github.com/processing/p5.js/blob/main/src/accessibility/outputs.js). 
 
 `_rgbColorName()` uses `color_conversion._rgbaToHSBA()` to get the hsv values of the color and then uses `_calculateColor()` to get the color name. The function `_calculateColor()` in this file comes from [colorNamer.js](https://github.com/MathuraMG/color-namer) which was developed as part of a [2018 Processing Foundation fellowship](https://medium.com/processing-foundation/making-p5-js-accessible-e2ce366e05a0) and in consultation with blind screen reader expert users. This function returns color names by comparing hsv values to those stored in the `colorLookUp` array. The function should be updated as some shades of gray are not named correctly. When updating it, it is also important to ensure contributor readability by including comments that explain what each line of code does.
 
@@ -83,14 +83,12 @@ The `describe()` function creates a screen reader accessible description for the
 
 `describe()` is supported by several functions in [src/accessibility/describe.js](https://github.com/processing/p5.js/blob/main/src/accessibility/describe.js):
 * `_descriptionText()`: Checks that text is not `LABEL` or `FALLBACK` and ensures text ends with a punctuation mark. If the text does not end with '.', ',', ';', '?', '!', this function adds a '.' at the end of the string. Returns text.
-* `_describeFallbackHTML()`: Creates fallback HTML structure for the canvas. 
-* `_describeLabelHTML()`: This function is only called when the second parameter of `_describe()` is `LABEL`. It creates a div adjacent to the canvas element for the description text.
+* `_describeHTML()`: Creates fallback HTML structure for the canvas. If the second parameter of `_describe()` is `LABEL`, this function creates a div adjacent to the canvas element for the description text.
 
 ### describeElement()
 The `describeElement()` function creates a screen reader accessible description for sketch elements or groups of shapes that create meaning together. The first parameter should be a string with the name of the element, the second parameter should be a string with the description of the element. The third parameter is optional. If specified, it determines how the description is displayed. All element descriptions become part of the sub DOM of the canvas element. If a user passes `LABEL` as a third parameter, an additional div with the element description adjacent to the canvas is created. 
 
 `describeElement()` is supported by several functions in [src/accessibility/describe.js](https://github.com/processing/p5.js/blob/main/src/accessibility/describe.js):
 * `_elementName()`: Checks that element name is not `LABEL` or `FALLBACK` and ensures text ends with a colon. Returns element name.
 * `_descriptionText()`: Checks that text is not `LABEL` or `FALLBACK` and ensures text ends with a punctuation mark. If the text does not end with '.', ',', ';', '?', '!', this function adds a '.' at the end of the string. Returns text.
-* `_describeElementFallbackHTML()`: Creates fallback HTML structure for the canvas. 
-* `_describeElementLabelHTML()`: This function is only called when the second parameter of `_describeElement()` is `LABEL`. It creates a div adjacent to the canvas element for the description text.
+* `_describeElementHTML()`: Creates fallback HTML structure for the canvas. When the second parameter of `_describeElement()` is `LABEL`, this function creates a div adjacent to the canvas element for the descriptions.

src/accessibility/color_namer.js

@@ -622,19 +622,25 @@ const colorLookUp = [
 //returns text with color name
 function _calculateColor(hsb) {
   let colortext;
+  //round hue
   if (hsb[0] !== 0) {
     hsb[0] = Math.round(hsb[0] * 100);
     let hue = hsb[0].toString().split('');
     const last = hue.length - 1;
     hue[last] = parseInt(hue[last]);
+    //if last digit of hue is < 2.5 make it 0
     if (hue[last] < 2.5) {
       hue[last] = 0;
+      //if last digit of hue is >= 2.5 and less than 7.5 make it 5
     } else if (hue[last] >= 2.5 && hue[last] < 7.5) {
       hue[last] = 5;
     }
+    //if hue only has two digits
     if (hue.length === 2) {
       hue[0] = parseInt(hue[0]);
+      //if last is greater than 7.5
       if (hue[last] >= 7.5) {
+        //add one to the tens
         hue[last] = 0;
         hue[0] = hue[0] + 1;
       }
@@ -647,6 +653,9 @@ function _calculateColor(hsb) {
       }
     }
   }
+  //map brightness from 0 to 1
+  hsb[2] = hsb[2] / 255;
+  //round saturation and brightness
   for (let i = hsb.length - 1; i >= 1; i--) {
     if (hsb[i] <= 0.25) {
       hsb[i] = 0;
@@ -656,11 +665,14 @@ function _calculateColor(hsb) {
       hsb[i] = 1;
     }
   }
+  //after rounding, if the values are hue 0, saturation 0 and brightness 1
+  //look at color exceptions which includes several tones from white to gray
   if (hsb[0] === 0 && hsb[1] === 0 && hsb[2] === 1) {
+    //round original hsb values
     for (let i = 2; i >= 0; i--) {
-      //for (let i = originalHSB.length - 1; i >= 0; i--) {
       originalHSB[i] = Math.round(originalHSB[i] * 10000) / 10000;
     }
+    //compare with the values in the colorExceptions array
     for (let e = 0; e < colorExceptions.length; e++) {
       if (
         colorExceptions[e].h === originalHSB[0] &&
@@ -670,10 +682,12 @@ function _calculateColor(hsb) {
         colortext = colorExceptions[e].name;
         break;
       } else {
+        //if there is no match return white
         colortext = 'white';
       }
     }
   } else {
+    //otherwise, compare with values in colorLookUp
     for (let i = 0; i < colorLookUp.length; i++) {
       if (
         colorLookUp[i].h === hsb[0] &&