Skip to content

Support big endian #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
guybedford merged 5 commits into from
Oct 13, 2020
Merged

Support big endian #13

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
98e485a
support big endian
guybedford Oct 13, 2020
d3320af
second attempt
guybedford Oct 13, 2020
43d35ba
fixup iterator
guybedford Oct 13, 2020
4e16c06
fixes
guybedford Oct 13, 2020
7c86b91
finally working
guybedford Oct 13, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 18 additions & 4 deletions src/lexer.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,21 @@ const strictReserved = new Set(['implements', 'interface', 'let', 'package', 'pr

let wasm;

const isLE = new Uint8Array(new Uint16Array([1]).buffer)[0] === 1;

export function parse (source, name = '@') {
if (!wasm)
throw new Error('Not initialized');

const len = source.length + 1 + (source.length + 1) % 2;

// need 2 bytes per code point plus analysis space so we double again
const extraMem = (wasm.__heap_base.value || wasm.__heap_base) + source.length * 4 - wasm.memory.buffer.byteLength;
const extraMem = (wasm.__heap_base.value || wasm.__heap_base) + len * 4 - wasm.memory.buffer.byteLength;
if (extraMem > 0)
wasm.memory.grow(Math.ceil(extraMem / 65536));

const addr = wasm.sa(source.length);
copy(source, new Uint16Array(wasm.memory.buffer, addr, source.length + 1));
const addr = wasm.sa(len);
(isLE ? copyLE : copyBE)(source, new Uint16Array(wasm.memory.buffer, addr, len));

if (!wasm.parseCJS(addr, source.length, 0, 0))
throw Object.assign(new Error(`Parse error ${name}${wasm.e()}:${source.slice(0, wasm.e()).split('\n').length}:${wasm.e() - source.lastIndexOf('\n', wasm.e() - 1)}`), { idx: wasm.e() });
Expand All @@ -29,7 +33,17 @@ export function parse (source, name = '@') {
return { exports: [...exports], reexports: [...reexports] };
}

function copy (src, outBuf16) {
function copyBE (src, outBuf16) {
const len = src.length / 2;
let i = 0;
while (i < len) {
outBuf16[i] = src.charCodeAt(i * 2 + 1);
outBuf16[i + 1] = src.charCodeAt(i * 2);
Copy link
Copy Markdown
Member

@richardlau richardlau Oct 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't appear to work as

  • outBuf16[i + 1] gets overwritten the next time around the loop by outBuf16[i]
  • it swaps pairs of 16-bit values rather than the octets within the 16-bit value.

I've hacked something together that appears to work on AIX (tests pass). I'll need to tidy up a bit (I've almost certainly broken little endian in what I currently have) -- I'll try to get a PR in either later tonight or tomorrow.

Copy link
Copy Markdown
Collaborator Author

@guybedford guybedford Oct 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @richardlau, I've pushed up some fixes here, running on nodejs/node#35634 to see if it works, but if it's still failing your PR would be a huge help.

i++;
}
}

function copyLE (src, outBuf16) {
const len = src.length;
let i = 0;
while (i < len)
Expand Down