You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Mar 10, 2020. It is now read-only.
Uploading a file > 1MB, as a pull-stream fails, with the error:
Uncaught Error: stream.push() after EOF
at new NodeError (errors-browser.js:30)
at readableAddChunk (_stream_readable.js:269)
at PullDuplexStream.push../node_modules/readable-stream/lib/_stream_readable.js.Readable.push (_stream_readable.js:240)
at next (index.js:55)
at FileReader.loaded (index.js:22)
The issue is occuring in master on webui.ipfs.io since upgrading v30.1.2 from v29.1.0
There is nothing in pull-stream-reader to stop it from emitting the cb(true) for when the offset exeeds the filesize, while it is still waiting for a data callback from the FileReader...
in the above, with a 2Mb file, we see it trigger the offset > filesize callback before the final data chunk.
It sure did! I think something else must have been dealing with it, when it should always have been an error. I just added a quick, "count the pending chunks" check to pull-stream-reader and it fixes it! Will PR a cleaner fix now.
@hugomrdias oh no wait, you are right, this is super weird, the pull-stream sink should not be asking for more chunks before it gets a callback from the previous one... that's the weirdness that is causing this. I could guard against it in pull-file-reader, but it seems like we have a rogue pull-stream sink in ipfs now that is just pulling as fast as it can, rather than waiting for each chunk
Activity
olizilla commentedon Apr 15, 2019
Also of note, in ipfs-webui we use https://github.com/tableflip/pull-file-reader
which has a default chunk size of 1MB https://github.com/tableflip/pull-file-reader/blob/master/index.js#L8 which suspiciously conincides with the threshold for triggering this issue.
olizilla commentedon Apr 18, 2019
@hugomrdias this is blocking a relase of webui and desktop. Are you digging in to it or should I jump on it?
olizilla commentedon Apr 18, 2019
investigating this some more...
ipfs.add
in the browser due to ipfs shutdown errors out on Windows ipfs/kubo#4078upload-file-via-browser
example in this repohugomrdias commentedon Apr 18, 2019
im on it, will report back asap
olizilla commentedon Apr 18, 2019
@hugomrdias I'm on it. The problem seems to be in pull-file-reader
olizilla commentedon Apr 18, 2019
There is nothing in pull-stream-reader to stop it from emitting the
cb(true)
for when the offset exeeds the filesize, while it is still waiting for a data callback from the FileReader...in the above, with a 2Mb file, we see it trigger the offset > filesize callback before the final data chunk.
hugomrdias commentedon Apr 18, 2019
But this worked before right? what changed in ipfs to trigger this problem?
olizilla commentedon Apr 18, 2019
It sure did! I think something else must have been dealing with it, when it should always have been an error. I just added a quick, "count the pending chunks" check to pull-stream-reader and it fixes it! Will PR a cleaner fix now.
olizilla commentedon Apr 18, 2019
olizilla commentedon Apr 18, 2019
@hugomrdias oh no wait, you are right, this is super weird, the pull-stream sink should not be asking for more chunks before it gets a callback from the previous one... that's the weirdness that is causing this. I could guard against it in pull-file-reader, but it seems like we have a rogue pull-stream sink in ipfs now that is just pulling as fast as it can, rather than waiting for each chunk
3 remaining items
fix: Uncaught Error: stream.push() after EOF
fix: uncaught error: stream.push() after EOF
fix: uncaught error: stream.push() after EOF (#980)
fix: file upload error in http client
fix: file upload error in http client (#1033)