Yoshua Wuyts's blog
Introduction The goal of in-place initialization is to enable the construction of types directly into a memory location without any additional moves or copies. When working with big types this can be more efficient and even prevent stack overflows. But some types are what we call address sensitive and so cannot be moved for correctness reasons. There is some disagreement about how we should encode…
Introduction In my post on Hoisting Expressions I discussed the move($expr) feature and how it works much like an inverse of the defer feature many languages have. Instead of creating expressions which run after the scope ends, move($expr) runs code before the scope is entered 1 . The goal of this code is to make it so clone -heavy ecosystems like bevy have a much better experience using Rust. 1…
Introduction There is an RFC open on Rust which proposes what I’m calling hoisting expressions into the language. These are expressions which can be introduced inside of closures-only (for now), and are hoisted by the compiler to run before the rest of the closure does. To illustrate how they work, consider this example: rust print! ( "hello " ); hoist { print! ( "world!" ) }; Copy Even though in…
In my last post I described what an effect notation might look like using with -clauses and blocks. When discussing effects over dinner with lcnr a couple of days ago they asked a really good question that I’ll paraphrase here: “Why are effects the right answer? What do they provide that extensions to the trait system cannot?” The answer to ”Why effects?” is not one of capabilities. It is true…
A bad habit of mine is that I’ll write 10.000 or 20.000 words on a topic, and then never publish that. A lot of my longer-form writing never sees the light of day because it gets stuck in editing purgatory. Editing small pieces is easy and quick, editing long pieces is not. I’ve been wanting to write more about effect notations for a while now and have been noodling on something I hope people…
Yesterday Making WebAssembly a First-Class Language on the Web was making the rounds on social media, and I saw some people express confusion about this Wasm Component business. Why would the folks working on WebAssembly be going through all this trouble to implement something on top of WebAssembly? And why is that interesting for browser vendors like Firefox to begin implementing? The way I’ve…
I don’t think I’ve ever quite articulated my “grand vision” for Rust, despite having written a fair bit about the language and its features. There is a lot I could say here, but currently there are three directions of development which I find particularly interesting: Improving Rust’s support for effects Improving Rust’s support for substructural rules Adding support for refinement types in Rust…
I believe it must have been about three or four years ago when Rust added the unstable yeet keyword on nightly which can be used to return new errors in try -functions. More recently Rust has added the unstable bikeshed keyword on nightly which gives try -blocks and closures the ability to express which kind of error they operate on. We’ve been deferring making decisions of error handling syntax…
Introduction In my last post I introduced placing functions , which are functions that can “return” values without copying them. This is useful not just because it’s more efficient (fewer copies), but also because it guarantees addresses remain stable - which is something we need for types that have internal borrows (self-referential types). Here is a little reminder of what placing functions look…
What are placing functions? About a year ago I observed that in-place construction seems surprisingly simple . By separating the creating of the place in memory from writing the value to memory, it’s not that hard to see how we can turn that into a language feature. So about six months ago, that’s what I went ahead and did and created the placing crate : a proc-macro-based prototype for “placing…
Structured concurrency (Tree-)Structured Concurrency is neat because it greatly simplifies concurrent programs. It greatly reduces, if not outright eliminates the possibility of logical races due to concurrency issues. And conceptually it’s not that hard either, as we can encode structured concurrency with just two rules: Every child computation (except the root) must have exactly one logical…
Introduction There’s a fun little tidbit that most people don’t seem to be aware of when writing async functions in traits (AFITs) and that is: they allow you to directly return futures from methods. Sounds confusing? Let me illustrate what I mean with a simple example. Assume we have an a trait AsyncIterator which has an async fn next method that is defined as follows: rust trait AsyncIterator {…
Introduction The entire point of futures and Rust’s async/.await system is to introduce two new capabilities: arbitrary concurrent execution of computations, and arbitrary cancellation of computations. The difference between blocking and non-blocking computation is unimportant if we then don’t also capitalize on it by scheduling work concurrently. But there’s a problem - In order to schedule work…
When discussing effect systems in the context of production programming languages, I believe that the most important distinction is whether effects are purely defined by the language, or whether users are also able to define them. When Magnus Madsen from the Flix programming language joined the Rust T-Lang call for two sessions last December, he used the terms “open” and “closed” to describe this…
Introduction When working with low-level concurrency (atomics), programming languages are generally quite eager for compilers to reorder operations if it leads to better performance. Information about whether it's ok to reorder operations is encoded using Atomic Orderings , Fences , and Operations . It's strange that most programming languages that support semantic-aware reordering of low-level…
Introduction One of the things that stands out to me is how similar match and if..else are semantically , while being diverging a fair bit syntactically . The reasons for that seem mostly accidental, and I have a sneaking suspicion we could make things a little easier by making both constructs look more similar. In this post I'll be showing some control flow examples based on a queue that can…
Here's a silly little insight I had the other day: if you squint, both View Types and Pattern Types seem like lightweight forms of Refinement Types 1 . Both will enable constraining types, but in slightly different and complementary ways. Let's take a look at this using an example of an RGB struct containing fields for individual Red, Green, and Blue channels stored as usize 1 : 1 I like the term…
Introduction Oh yeah, you like iterators? Name all of them. — I'm increasingly of the belief that before we can meaningfully discuss the solution space, we have to first make an effort to build consensus on the problem space. It's hard to plan for a journey together if we don't know what the trip will be like along the way. Or worse: if we don't agree in advance where we want the journey to take…
At the end of my last post I mentioned that one of the main issues with the IntoIterator trait is that it’s kind of a pain to write. I wasn’t around when it was first introduced, but it’s not hard to tell that the original authors intended for Iterator to be the primary interface with IntoIterator being an additional convenience. This didn’t quite turn out to be the case though, and it’s common…
One of the open questions surrounding the unstable gen {} feature is whether it should return Iterator or IntoIterator . People have had a feeling there might be good reasons for it to return IntoIterator , but couldn't necessarily articulate why . Which is why it was included in the "unresolved questions" section on the gen blocks RFC. Because I'd like to see gen {} blocks stabilize sooner, I…
The lovely folks working on security over at Google have recently been writing about "temporal (memory) safety" and "spatial (memory) safety" . When I first saw these terms it took me a minute to figure out what they meant, as searching for it online didn't yield immediate answers. So I figured it might be helpful to write it down for others to find: Spatial memory safety: describes violations…
I've been wondering for a little while why the Future::poll methods takes self: Pin<&mut Self> in its signature. I assumed there must have been a good reason, but when I asked my fellow WG Async members nobody seemed to know off hand why that was exactly. Or maybe they did, and I just had some trouble following. Either way, I think I've figured it out, and I want to spell it out for posterity so…
For the past decade I've struggled with wrist injuries on and off. It's a common occupational injury for us keyboard users, and after my most recent bout this year I decided it was time to actually do something about it. Permanently. I like systemic solutions - they tickle a good part of my brain. And I figured I could apply some of that to my keyboard habits too. In this post I want to share a…
In my last post I discussed how we might be able to introduce ergonomic self-referential types (SRTs) to Rust, mostly by introducing features we know we want in some form anyway. The features listed were: Some form of 'unsafe and 'self lifetimes. A safe out-pointer notation for Rust ( super let / -> super Type ). A way to introduce out-pointers without breaking backwards-compat. A new Move…
I've been thinking a little about self-referential types recently, and while it's technically possible to write them today using Pin (limits apply), they're not at all convenient. So what would it take to make it convenient? Well, as far as I can tell there are four components involved in making it work: The ability to write 'self lifetimes. The ability to construct types from functions in fixed…
introduction I've been thinking a little bit about self-referential types recently, and one of its requirements is that a type, when constructed, has a fixed location in memory 1 . That's needed, because a reference is pointer to a memory address - and if the memory address it points to changes, that would invalidate the pointer, which can lead to undefined behavior. 1 Eric Holk and I toyed around…
In program language design I'm a big fan of features which fall out of other, more general features. People occasionally talk about both "unleakable" and "undroppable" types. I see a lot of value in "unleakable" types because if a type is "unleakable" we can guarantee the type will always have its destructor called. This would allow us to use Drop to be used to uphold safety invariants, which will…
Okay, so you've probably read the title and are thinking: "Hey is this clickbait?" - and no dear reader, it is not. I'm fairly convinced at this point that tasks, as an abstraction, are not right for Rust and we would do better to achieve parallel execution through composition. However when it comes to task scheduling things seem less clear, and I would like to have more data available. Because…
On this blog I usually post about technical explorations in programming languages, API design, asynchronous computing, and so on. More recently I've re-started my work on WebAssembly, WASI, and their integration with Rust. I expect that work to eventually become more technical again; but my involvement there for the past year has mostly been focused on process and people. And I wanted to take a…
In 2019 Stjepan Glavina and I developed the async-std runtime . That was an off-shoot from the runtime project , which itself was an attempt to make it easier to abstract over different async runtimes. One of the things I'm most proud of in the work I did on async-std is the core IO abstraction which Stjepan later factored out into the polling and async-io libraries as part of the smol project ,…
This is the transcript of my RustConf 2023 talk: "Extending Rust's Effect System", presented on September 13th 2023 in Albuquerque, New Mexico and streamed online. Introduction Rust has continuously evolved since version 1.0 was released in 2015. We've added major features such as the try operator ( ? ), const generics, generic associated types (GATs), and of course: async/.await . Out of those…
Will this function block? rust fn work ( x : u64 , y : u64 ) -> u64 { x + y } Copy How about this function? rust fn work ( x : u32 ) -> u32 { let mut prev = 0 ; let mut curr = 1 ; for _ in 0 ..x { let next = prev + curr ; prev = curr ; curr = next ; } curr } Copy Or this one? rust use sha256 :: digest ; fn work ( input : & str ) -> String { digest ( input ) } Copy Will this function block? rust…
This is a "short" post. Unlike my usual (lengthy) research pieces, this is mostly a train-of-thought, low-edit post intended to convey an idea or perspective I thought would be useful to write down. edit(2023-11-19): I've changed the title of this post from "Rethinking WIT" to "Reframing WIT". I strongly hold that WIT being human-readable is a good thing we shouldn't change, even though in this…
Note: this post is not my usual research posts; but more a loose collection of thoughts I've been thinking of recently. I might start tagging these as "short", since I'm writing and publishing before fully validating. Not long after I started programming I developed an intuition for what APIs are: they are the interfaces we use in our applications to communicate between distinct components. But…
update (2023-11-08): I've swapped the usage of "horizontal" and "vertical" in this post based on feedback. This is going to be an ultra-short post to share an idea I've been mulling over in my head for a while. Yesterday I raised the question again about WG Async getting checkboxes for approval on decisions. For the past five or so years we've had a version of the async working group driving the…
This is another short post covering another short idea: I want to explain the mechanics required to make Iterator an alias for the Coroutine trait. Making that an alias is something Eric Holk brought up yesterday. We then talked it through and mutually decided it probably wasn't practical. But I thought about it some more today, and I might have just figured out a way we could make work? In this…
This post is part of the Async Iteration series: Async Iteration I: Async Iteration Semantics Async Iteration II: The Async Iterator Crate Async Iteration III: The Async Iterator Trait (this post) Async Functions in Traits (AFIT) are in the process of being stabilized and I figured it would be a good time to look more closely at the properties they provide. In this post I want to compare…
I regularly point people to Tmandry's 2021 post "Contexts and Capabilities in Rust" . While nobody is actively working on the design at the moment, it's more because of prioritization than anything else. It seems like it could make certain usage patterns a lot nicer in Rust, and one of those is probably working with custom allocators. I've been working a lot with slab allocators recently, and I'd…
Recently Eric and I have been chatting with Daan Leijen , of Koka fame, discussing among other things the distribution of effects. Daan estimated that for a typical program written in Koka the distribution of effects would roughly be something like: ~70% of a program could be "total" (as in: has no effects) ~15% of a program would have to deal with exceptions or divergence (may error, may panic,…
It's been over three years since Fitzgen published: "Announcing Better Support for Fuzzing with Structured Inputs in Rust" , and a little over two years since arbitrary 1.0 was released. A few years agoI wrote a property-testing crate based on arbitrary , but never ended up writing about it. So I wanted to take a moment to change that. We'll start by taking a look at automated testing, then cover…
For a while now I've been trying to find a good way to explain what structured concurrency is, and how it applies to Rust. I've come up with zingers such as: "Structured concurrency is structured programming as applied to concurrent control-flow primitives" . But that requires me to start explaining what structured programming is, and suddenly I find myself 2000 words deep into a concept which…
Introduction Now that the final touches are being put on the second version of WASI (WASI Preview 2), I figured now might be a good time to write a few words on WASI, WebAssembly, and how to think about them. In recent months I've been driving the WASI Preview 2 work in the Rust compiler. So in order to do that I've had to familiarize where things are currently, where things are headed, and how…
Microsoft Build is happening this week, and with it come new announcements for the C# language and dotnet runtime. I was watching the video: "What's new in C#12 and beyond" , and the C# team talked about pattern matching and pattern constructors. I've been thinking a bit about patterns in Rust recently, and I wanted to share some of my doodles. Disclaimer time: I'm not on the Rust language team,…
This post represents an overview of an MVP "linear types" design which we could probably start implementing and validating today if we wanted to. What I'm sharing here is a combination of conversations I've had with Gankra and Jonas Sheevink . Prior Reading The Pain Of Real Linear Types in Rust Must move types Linearity and Control What are linear types? We typically frame linear types as: "Real…
Introduction Update 2023-03-28 : I've published a follow-up to this post which presents an alternate design, reframing linearity as: "Drop is guaranteed to run". To get a full overview of the design space, read this post to get a sense of the problem domain, and the follow-up post for a concrete solution. A week ago Niko published a post on linear types , introducing the idea of "must move" types,…
If you've been around an industry for long enough you end up with Opinions™ about things. Here's one of mine: I think one of the biggest missing aspects of web programming is the lack of support for treating HTML as a compile-target. Most folks reading this blog may not be aware of this, but I started my career working with web technologies, which I kept at for the better part of a decade. I…
Introduction In 2021 Tmandry published the post: "Contexts and Capabilities" . In it they present the idea of with -clauses, providing a means of passing types such as allocators around in a more convenient way. The post goes into detail about the feature, but you can roughly think of it as a way to declare types which are live within a certain scope, and are automatically passed into functions…
When I write custom errors in a project, I also like to write a few small error macros to accompany them. In my opinion this can make error handling just a little nicer to use. In this post I briefly want to talk about domain-specific error macros such as ensure! , what they're useful for, and the io-ensure prototype crate I've written which I'm propose for inclusion in the stdlib next time I get…
Introduction 40% of Rust Developers believe Rust's debugging experience could use improvement. And that's not surprising: when we write code we make assumptions, and sometimes we assume wrong. This leads to bugs, which we then track down and fix. This is what we call "debugging", and is a core part of programming. The purpose-built tools which help us with debugging are called "debuggers". Unlike…
I've been writing a bit about state machines recently. In my last post I explained how we could potentially leverage arbitrary self-types and anonymous enums to support type-level state machines as a first-class construct. Which is cool, but it had some readers asking: "How does this improve on the existing type state pattern?" Which is fair! Well, to start off by answering that question: it's…