Creating an own WASI function
One of the core concepts of WASI is to make the execution of Wasm code portable. This means: Every WASI runtime needs to implement the WASI specification and serve the needed methods for the Wasm module.
When you followed along my post about the implementation of WASI in Rust you might have seen the following import of the fd_write function in the compiled Wasm module:
(import "wasi_snapshot_preview1" "fd_write" (func (;0;) (type 10)))
The function fd_write will be imported from the wasi_snapshot_preview1 module and needs implement the interface which is defined as type 10.
The type can also be found in the compiled module. In this case it is the type at index 10 and looks like the following:
(type (;10;) (func (param i32 i32 i32 i32) (result i32)))
So, to execute the Wasm module successfully, the runtime (e.g. wasmtime, the browser, …) needs to handover a function with four params and the return of one value.
The interface for fd_write can be found in the specification. Here is the reference C-Header for it:
__wasi_errno_t __wasi_fd_write(
__wasi_fd_t fd,
/**
* List of scatter/gather vectors from which to retrieve data.
*/
const __wasi_ciovec_t *iovs,
/**
* The length of the array pointed to by `iovs`.
*/
size_t iovs_len,
__wasi_size_t *retptr0
) __attribute__((__warn_unused_result__));
This article shows how to implement this function in JavaScript and provide it to a Wasm module.
The browser will act as the runtime: During the execution of the module it will provide the needed function.
Description of the C-Header
A lot is going on in the C-Header, let’s go through it step-by-step:
Return value:
__wasi_errno_t- This is the return value which represents the success or failure of the operation. There’s a bunch of error codes which can be seen in the specification, but in this sample just the code0(= success) will be used.
Parameters:
__wasi_fd_t fd- That’s the file descriptor which tells the function where the write operation shall happen.__wasi_ciovec_t *iovs- Pointer to the first iovector. An iovector is a pair ofi32values and contains the memory offset and the length of the data to read. So extracting one iovector will tell where to start reading and how much data to read.
You can see this in the definition of the __wasi_ciovec_t type in C:
typedef struct __wasi_ciovec_t {
const void *buf;
size_t buf_len;
} __wasi_ciovec_t;
size_t iovs_len- How many iovectors shall be extracted.__wasi_size_t *retptr0- A pointer to a place in memory. When the write operation completes, the number of bytes written will be stored there.
Parts to ignore:
The __attribute__((__warn_unused_result__)) part can be ignored. It is a C/C++ compiler attribute which indicates that the compiler should throw a warning when the return result is not used.
The Wasm module
First, let’s write a Wasm module which contains the import and type for fd_write and one function hello, that calls the fd_write function.
Check the comments in the following module:
;; hello.wat
(module
;; define the expected type for fd_write
(type $write_type (func (param i32 i32 i32 i32) (result i32)))
(import "wasi_snapshot_preview1" "fd_write" (func $write (type $write_type)))
;; Define a memory that is one page in size (64KiB).
(memory (export "memory") 1)
;; At offset 66 and 80 in the memory, we store the data.
(data (i32.const 66) "Hello, World!\n")
(data (i32.const 80) "Howdy!\n")
(func $main (export "hello") (result i32)
;; Store the iovs
(i32.store (i32.const 0) (i32.const 66)) ;; Start of data (= *buf)
(i32.store (i32.const 4) (i32.const 14)) ;; Length of data (= buf_len)
(i32.store (i32.const 8) (i32.const 80)) ;; Start of data (= *buf)
(i32.store (i32.const 12) (i32.const 8)) ;; Length of data (= buf_len)
i32.const 1 ;; fd param. In Unix this means: 0 = stdin, 1 = stdout, 2 = stderr
;; here are different combinations possible:
;; 0,1 would read only "Hello, World!"
;; 8,1 would read only 'Howdy'
i32.const 0 ;; iovs -> Offset. Points to first iovec
i32.const 2 ;; iovs_len -> How many iovs should be read, set it to 2 to read also Howdy
i32.const 92 ;; Pointer to the place in memory where the number of written bytes shall be placed.
call $write ;; call fd_write and drop the result code
drop
(i32.load (i32.const 92)) ;; output the number of bytes written
)
)
Code explanation
The first lines of the code define and import the fd_write function.
Then the memory is initialized and two strings are stored into it.
In the code you can see that at position 66 the string "Hello, World!\n" is stored.
(data (i32.const 66) "Hello, World!\n")
Later in the hello function the iovs are defined. The iovs contain the start of the data (position 66) and the length (14 characters).
They must be stored side-by-side, as the WASI run time needs to fetch them from there as a pair:
(i32.store (i32.const 0) (i32.const 66)) ;; Start of data (= *buf)
(i32.store (i32.const 4) (i32.const 14)) ;; Length of data (= buf_len)
The function passes the beginning of this vector (= position 0) as an argument for iovs to fd_write.
The parameter iovs_len tells how many of these iovs shall be read (= 2).
I recommend to play with this parameters and see which effect it has on the output!
i32.const 0 ;; iovs -> Offset. Points to first iovec
i32.const 2 ;; iovs_len -> How many iovs should be read, set it to 2 to read also Howdy
Store, compile… and test!
Store this file as hello.wat and compile it to Wasm with wat2wasm:
wat2wasm hello.wat
You can also verify the code by running it with a WASI-runtime like wasmtime:
wasmtime hello.wasm --invoke hello
The ‘runtime’
The browser shall act as the WASI runtime. To make this possible, it is needed to provide an implementation of the fd_write function.
But.. in the browser you don’t have any access to the filesystem, so we will make use of the DOM and write everything in there. It’s like a simulation of the file system, but also shows the portability aspect of WASI, as you can provide a function that fits best for the runtime. See the following code:
fd_write(fd, iovs, iovs_len, ret_ptr) {
const memory = new Uint32Array(instance.exports.memory.buffer);
let nwritten = 0;
for (let i = 0; i < iovs_len; i++) {
const offset = i * 8; // = jump over 2 i32 values per iteration
const iov = new Uint32Array(memory.buffer, iovs + offset, 2);
// use the iovs to read the data from the memory
const bytes = new Uint8Array(memory.buffer, iov[0], iov[1]);
const data = new TextDecoder("utf8").decode(bytes);
out(fd, data);
nwritten += iov[1];
}
// Set the nwritten in ret_ptr
const bytes_written = new Uint32Array(memory.buffer, ret_ptr, 1);
bytes_written[0] = nwritten;
out(fd, `bytes written: ${memory[ret_ptr]}`)
return 0;
}
This implements the specified fd_write fucntion from before.
First, the memory is initialized as an Uint32Array.
Then the for-loop starts to read the iovs as many times as iovs_len said.
Within this loop an offset is calculated which points in every iteration to the next iovec.
Remember: One iovec contains two i32 values, which are 8 bytes. That’s why per iteration 8 bytes are jumped over.
The iov stores this pair and uses it to read the data from the memory:
const bytes = new Uint8Array(memory.buffer, iov[0], iov[1]);
With that it can be passed to the out function, which prints the data either to the DOM or console (depending on the fd).
In the last step the number of the written bytes is stored in the place given by the ret_ptr.
Full working code
The following listing shows a full working code which loads and initializes the Wasm module.
<!doctype html>
<html>
<head>
<title>WASI fd_write demo</title>
</head>
<body>
<button id="call-hello">Write</button>
<pre><code id="out"></code></pre>
<script>
function out(stream, data) {
if (stream === 1) {
const textNode = document.createTextNode(data);
document.getElementById("out").appendChild(textNode);
}
if (stream === 2) {
console.error(data);
}
}
async function main() {
// Define the imports passed as imports (= WASI polyfill).
const imports = {
"wasi_snapshot_preview1": {
fd_write(fd, iovs, iovs_len, ret_ptr) {
const memory = new Uint32Array(instance.exports.memory.buffer);
let nwritten = 0;
for (let i = 0; i < iovs_len; i++) {
const offset = i * 8; // = jump over 2 i32 values per iteration
const iov = new Uint32Array(memory.buffer, iovs + offset, 2);
// use the iovs to read the data from the memory
const bytes = new Uint8Array(memory.buffer, iov[0], iov[1]);
const data = new TextDecoder("utf8").decode(bytes);
out(fd, data);
nwritten += iov[1];
}
// Set the nwritten in ret_ptr
const bytes_written = new Uint32Array(memory.buffer, ret_ptr, 1);
bytes_written[0] = nwritten;
out(fd, `bytes written: ${memory[ret_ptr]}`)
return 0;
}
}
};
// Fetch and instantiate our Wasm module.
const response = await fetch("./hello.wasm");
const wasmBytes = await response.arrayBuffer();
const { instance } = await WebAssembly.instantiate(wasmBytes, imports);
// Call its exported `hello` function with the reference to the DOM node.
instance.exports.hello();
// Every time the button is clicked, call the exported
// `hello` function again.
const callHello = document.getElementById("call-hello");
callHello.addEventListener("click", () => {
instance.exports.hello();
});
}
main().catch(e => {
out.textContent = `${e}\n\nStack:\n${e.stack}`;
});
</script>
</body>
</html>
When you serve the page and open it, you see the successfully written output:
If you want to test it fast, use the following python server:
#!/usr/bin/env python3
import http.server
import socketserver
PORT = 8080
Handler = http.server.SimpleHTTPRequestHandler
Handler.extensions_map[".wasm"] = "application/wasm"
with socketserver.TCPServer(("", PORT), Handler) as httpd:
print("serving at port", PORT)
httpd.serve_forever()
Conclusion
By writing an own WASI fd_write function, this post gives an example of how WASI runtimes and Wasm modules can work and interact together.
This is a good extension to the post about the implementation of WASI in Rust and gives a better idea and abstraction of WASI and its interfaces.
Main parts of this post come from the reference types in wasmtime post, which introduced the idea of WASI.
On blog.ttulka.com you can also see an example for fd_write and fd_read. Check this post for more technical details.s
A special thank you goes to Ashton Meuser for his valuable contribution in pointing out errors in the original version of this blog post. His insights and information were crucial in making important fixes regarding the iovecs and implementation of the fd_write function.