Under the surface of WASI in Rust
In the recent years WebAssembly (Wasm) got more and more popular: For different codebases you have one runtime in the web. So you can bring your C or Rust code into the web browser!
Few years ago WebAssembly broke out of the web and started to aim the system as a runtime environment. With the WebAssembly System Interface you can run WebAssembly directly on the machine - no browser needed. This enables a wider range of possibilities.
One of the first languages that implemented the WASI is Rust. Just out of interest I was looking into how the implementation of the WASI calls happened in Rust. I want to do write down here the things I found out - probably some of them have gapes, but some things may be interesting though. Important note: Many things in the Wasm and the Rust implementation is in an experimental state. So it is possible that these words will be outdated in a few months or years.
What is WASI
WASI is defined by the work of the bytecodealliance. It defines a set of standardized modules for system calls - and in the future more. This shall give the user one runtime for multiple languages.
WASI is defined in a modular way. The first draft of WASI defines the most important host calls for file reading/writing, sockets, clocks and so on. The definitions can be found in the GitHub repository.
For example the fd_write interface looks like the following:
fd_write(fd: fd, iovs: ciovec_array) -> Result<size, errno>
Expressing WASI interfaces in .witx
A machine-consumable file with the definitions is served via a .witx file - an experimental file format for the expression of the API definitions.
There you can find the fd_write interface:
;;; Write to a file descriptor.
;;; Note: This is similar to `writev` in POSIX.
(@interface func (export "fd_write")
(param $fd $fd)
;;; List of scatter/gather vectors from which to retrieve data.
(param $iovs $ciovec_array)
(result $error (expected $size (error $errno)))
)
You can see in the comment that it has a binding to the POSIX call which is due to the nature of the first version of WASI: Provide a way to execute system calls.
Executing WASI based files
Before we dive in to the usage of WASI in Rust, lets see how WASI is executed.
For the execution is a runtime needed. There already are many different runtimes like wasmtime, wasmer, wasmedge. Sometimes you may read of lucet. But this project is not maintained anymore and the devs are actively developing wasmtime.
Basically you can just execute the wasm file by handing it over to the runtime:
wasmtime hello_world.wasm
The runtime itself executes the wasm file and passes the used wasi functions in as imports.
So the conclusion is that the language itself is not responsible for the correct implementation of the wasi function - it just calls it and the runtime passes the implementation.
At this place I recommend this link from Mozilla Hacks if you want to know more about this.
Using WASI in Rust
There are two ways to use WASI in Rust. One elegant way and another way which will give a more understanding of how WASI is build. I’ll start with the latter one.
The wasi crate
The wasi crate binds Rust calls to Wasm.
It makes use of the .witx file which I mentioned above and generates the API bindings.
So, when you checkout the repository of the wasi crate and run the generator, you’ll get a lib_generated.rs file. This contains methods which in return call the WASI API.
The fd_read method is generated as the following one:
/// Write to a file descriptor.
/// Note: This is similar to `writev` in POSIX.
///
/// ## Parameters
///
/// * `iovs` - List of scatter/gather vectors from which to retrieve data.
pub unsafe fn fd_write(fd: Fd, iovs: CiovecArray<'_>) -> Result<Size, Errno> {
let mut rp0 = MaybeUninit::<Size>::uninit();
let ret = wasi_snapshot_preview1::fd_write(
fd as i32,
iovs.as_ptr() as i32,
iovs.len() as i32,
rp0.as_mut_ptr() as i32,
);
match ret {
0 => Ok(core::ptr::read(rp0.as_mut_ptr() as i32 as *const Size)),
_ => Err(Errno(ret as u16)),
}
}
You can see the call to wasi_snapshot_preview1::fd_write which is defined as an external code in the generated file:
/// Write to a file descriptor.
/// Note: This is similar to `writev` in POSIX.
pub fn fd_write(arg0: i32, arg1: i32, arg2: i32, arg3: i32) -> i32;
Note: For C there exists the wasi-libc project that does the same thing.
A code sample with the wasi-crate
Let’s have a look at a sample which uses the wasi-crate.
First generate a new project:
cargo new --bin sample_wasi_crate
Now, let’s use the wasi crate by adding it as an dependency in the Cargo.toml.
[dependencies]
wasi = "0.11.0"
In the main.rs we implement the same example as in the crate:
fn main() {
let stdout = 1;
let message = "Hello, World!\n";
let data = [wasi::Ciovec {
buf: message.as_ptr(),
buf_len: message.len(),
}];
unsafe { wasi::fd_write(stdout, &data).unwrap(); }
}
This writes Hello, World! into stdout.
There are different ways to build a Rust project to WASI. One is to use the cargo wasi command:
cargo wasi run
This compiles and executes the code.
You can find the compiled code in the target/wasm32-wasi/debug directory.
When you open it with a Wasm viewer or generate it to a wat file you can have a look into the file and see that there is an import statement for the fd_write function:
(import "wasi_snapshot_preview1" "fd_write" (func (;0;) (type 10)))
This fd_write function is served by the runtime. But where can we find this implementation?
The runtime implementation of WASI
The wasi-common crate is a library which implements the WASI hostcalls. This library can be used by runtimes to provide the imports for the services that use WASI. Like it is used by wasmtime!
You can see the implementation of the fd_write call in the src/snapshots/preview_1.rs file:
async fn fd_write<'a>(
&mut self,
fd: types::Fd,
ciovs: &types::CiovecArray<'a>,
) -> Result<types::Size, Error> {
let f = self
.table()
.get_file_mut(u32::from(fd))?
.get_cap_mut(FileCaps::WRITE)?;
let guest_slices: Vec<wiggle::GuestSlice<u8>> = ciovs
.iter()
.map(|iov_ptr| {
let iov_ptr = iov_ptr?;
let iov: types::Ciovec = iov_ptr.read()?;
Ok(iov
.buf
.as_array(iov.buf_len)
.as_slice()?
.expect("cannot use with shared memories; see https://github.com/bytecodealliance/wasmtime/issues/5235 (TODO)"))
})
.collect::<Result<_, Error>>()?;
let ioslices: Vec<IoSlice> = guest_slices
.iter()
.map(|s| IoSlice::new(s.deref()))
.collect();
let bytes_written = f.write_vectored(&ioslices).await?;
Ok(types::Size::try_from(bytes_written)?)
}
There it is! The implementation of the fd_write function for a runtime like wasmtime.
Using the standard-library of Rust
If you look at the code of the sample of the wasi crate you notice that this code is bulky and low-level. This also states the README of the wasi crate.
Fortunately Rust supports WASI as a target and has an implementation of the std library for it.
So you can use your Rust code as usually and just compile to WASI. You can find an example of it on the wasmbyexample.com page.
The interesting part is that Rust itself is using the wasi crate for its std library. Which makes totally sense and is great as it gives us end users more abstraction. The write function from our example is the following one:
pub fn write(&self, bufs: &[IoSlice<'_>]) -> io::Result<usize> {
unsafe { wasi::fd_write(self.as_raw_fd() as wasi::Fd, ciovec(bufs)).map_err(err2io) }
}
When we use this the compiler will do it’s magic things with LLVM and so on and provide us the Wasm file as soon as we compile to this target!
Visualization of the dependencies
Well, there are a lot of dependencies, libraries and code samples here. You can visualize the post as the following:
Conclusion
This post dove into the definition of the WASI interface and how it got implemented in Rust. Some parts of the implementation, specification and tools are in experimental status and can change drastically in the future. For example, the witx file format will be dropped in favor of the wit format. But the current state shows that the ideas around WASI work.