-
Notifications
You must be signed in to change notification settings - Fork 22
Description
Proposal
Add an as_slice
on IoSlice
to go from &IoSlice<'a>
to &'a [u8]
.
Add an into_slice
on IoSliceMut
to go from IoSliceMut<'a>
to &'a mut [u8]
.
Problem statement
Using vectored IO (eg. write_vectored
) is close to impossible to do correctly due to the difficulty of slicing/advancing IoSlice
Consider something like this:
fn test(mut out: impl Write) -> std::io::Result<()> {
let data1 = [1; 8];
let data2 = [15; 8];
let io_slice1 = IoSlice::new(&data1);
let io_slice2 = IoSlice::new(&data2);
out.write_vectored(&[io_slice1, io_slice2])?;
Ok(())
}
The issue here is that write_vectored
may only do a partial write of a slices. To ensure we write all data we need to advance the slices. Dealing with this is a little bit more annoying than ordinary writes, but we might try something like this:
fn test(mut out: impl Write) -> std::io::Result<()> {
let data1 = [1; 8];
let data2 = [15; 8];
let io_slice1 = IoSlice::new(&data1);
let io_slice2 = IoSlice::new(&data2);
let mut buf = [io_slice1, io_slice2];
let mut buf = &mut buf[..];
let mut written = 0;
while !buf.is_empty() {
if buf[0].len() < written {
written -= buf[0].len();
buf = &mut buf[1..];
} else {
buf[0] = IoSlice::new(&buf[0][written..]);
written = out.write_vectored(buf)?;
}
}
Ok(())
}
But this fails to compile! The problem is that in IoSlice::new(&buf[0][written..])
we try to slice the IoSlice
. This uses the Deref
trait implemented on the IoSlice
, but this ends up borrowing the IO slice. Thus we cannot modify buf
as we are also borrowing data stored in it. Instead, if we simply had an as_slice
as above, we could simply write that as IoSlice::new(&buf[0].as_slice()[written..])
instead.
There has been a few similar solutions to this problem, such as rust-lang/rust#62726 and rust-lang/rust#70436. These have sort of stalled, in part due to the resulting signatures ending up being sort of odd in cases where we want to advance a slice of IoSlice
s. I think the above proposal is simpler and more basic as it allows the user to construct functions like write_all_vectored
, advance
, or advance_slices
in ordinary safe Rust.
Motivation, use-cases
None found - I haven't actually been able to find any applications doing vectored IO in Rust. Only vectored IO I have found has been wrapping write_vectored
and the like: https://github.com/search?l=Rust&p=2&q=%22write_vectored%22&type=Code
Solution sketches
Already have it implemented here: https://github.com/rust-lang/rust/compare/master...mpdn:rust:master?expand=1
Activity
nathaniel-bennett commentedon Nov 20, 2022
I've been trying to use vectored I/O for some performant networking use cases, and having a feature like this would really help--there's just no good way to update slice indices right now.
m-ou-se commentedon May 18, 2023
We discussed this in the libs meetup. This looks fine, except that
IoSlice::as_slice()
should just takeself
by value, since it is Copy anyway.Feel free to open a tracking issue and open a PR to rust-lang/rust to add this as an unstable feature.
std::io::IoSlice::as_slice
method rust-lang/rust#124680Auto merge of rust-lang#132790 - aDotInTheVoid:ioslice-asslice-rides-…
Auto merge of rust-lang#132790 - aDotInTheVoid:ioslice-asslice-rides-…
io_slice_as_bytes
rust-lang/rust#132818Rollup merge of rust-lang#132790 - aDotInTheVoid:ioslice-asslice-ride…
Unrolled build for rust-lang#132790