In my previous article, I have discussed panicking in Rust from the perspective of the Rust compiler.
I mostly talked about implementing unwinding(the compiler side of panicking) in my Rust to .NET compiler.
This article will deal with panics from the perspective of the Rust standard library.
I will go over the entire implementation of panics, step by step, explaining it along the way.
So, fasten your seatbelts - we are in for a ride!
First of all, what is a panic? Well, a Rust panic is more or less equivalent to a C++ exception.
The main difference is that, by convention, panics are usually used to indicate errors that you can't recover from.
if 2 + 2 != 4{
panic!("Math is not matching :(!");
}For errors that you can recover from(eg. a parsing error, or a network error), you should use a Result instead.
fn fallible(arg:Param)->Result<Data, Error> To raise a panic, you use the aptly named panic! macro.
Most people familiar with Rust already know that. But, have you ever wondered how it works under the hood?
Obviously, it is not magic or powered by unicorns: it is doing something.
It certainly calls some functions - but what kind of functions? It is time to discover that.
Let us look at this simple example:
fn main(){
panic!("Oops! something went wrong...");
}In order to see the guts of this macro, we can expand it.
Expanding macros in Rust is super easy, barely an inconvenience.
All we need to do is use cargo-expand, or the expand tool in the Rust playground. link.
cargo expandAnd, viola! We have spilled the guts of the panic macro!
#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2024::*;
#[macro_use]
extern crate std;
fn main() {
{
::core::panicking::panic_fmt(format_args!("Oops! something went wrong..."));
};
}...
That is quite chunk of code, don't you think?
All of this - just to panic?
Well, most of this is just a side-effect of macro expansion.
Let us first get the less relevant things out of the way.
#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2024::*;The first 3 lines import the Rust prelude. What is the Rust prelude?
The prelude is a set of macros, types and functions that are included in every Rust file.
This is why you can use the Rust String or Vec type without explicitly importing them - they are a part of the prelude.
Normally, the Rust compiler will import the prelude for us, before the macro expansion happens.
Since we are looking at our code after macro expansion, the prelude is now explicitly imported.
use std::prelude::rust_2024::*;You might have noticed that we are not just importing prelude - we are importing the 2024 version of it. Why?
Well, Rust has a mechanism called "editions". Each Rust crate specifies an edition, and any breaking change requires introducing a new one.
This way, code created in 2015 will still compile, even with the newest compiler. That code signals it is using the 2015 edition of Rust, so the breaking changes introduced in later editions of Rust don't affect it.
Different editions have different preludes, with a slightly different set of functions. So, our code using the 2024 version of Rust will import the 2024 prelude.
Interestingly, the panic macro itself has changed between editions. This is why something like std::panic::panic2015 exists.
However, we are interested in the 2024 version of panicking.
#[macro_use]
extern crate std;The next 2 lines just explicitly import all the macros from std.
Once again, this is normally done for us by the compiler. Since we want to look at the expanded versions of macros, this step is now explicit.
Now that all of this is out of the way, we can take a look at our main function.
fn main() {
{
::core::panicking::panic_fmt(format_args!("Oops! something went wrong..."));
};
}All of the code from the expanded macro is inside a Rust block {}.
This is because Rust macros are hygienic, and can’t introduce anything(new variables, functions) outside their internal scope.
This newly introduced block is responsible for just that: it is the scope of the expanded macro.
Let us now look at the guts of the panic macro.
Here, we can see a call to panic_fmt, and...
::core::panicking::panic_fmt(format_args!("Oops! something went wrong..."));Hang on a minute! Something looks very wrong here!
You might have noticed that the format_args! macro is... not expanded?
Is this a bug? I could have sworn I clicked "expand" macros...
Maybe we need to try again?
::core::panicking::panic_fmt(format_args!("Oops! something went wrong..."));Nope! It still has not expanded.
There is a good reason we can’t expand this macro: it is not a macro at all!
format_args is actually... a compiler built-in. It looks like an ordinary macro, behaves like an ordinary macro, but it is not one.
Currently, there is no way to implement all the features of format_args! using just the standard macro syntax.
So, the Rust language cheats a tiny bit, and implements format_args! directly within the compiler source.
It is neat to know that format_args is a compiler builtin, but what does it do?
Well, format_args! takes in a format string, and set of arguments, with formatting options specified.
format_args!("a:{a}, dec:{}, hex:{:x}, bin:{:b}", hex_num, bin = 6)Later, it packs them all into a single data structure(core::fmt::Arguments).
This structure contains the information necessary for the formatting to happen.
Now that I explained format_args, we can start getting into the meat of the panicking machinery.
panic_fmtThe panic_fmt function is responsible for beginning the panicking process.
::core::panicking::panic_fmt(format_args!("Oops! something went wrong..."));We know that this function is passed core::fmt::Arguments(a data structure representing the panic message), and somehow starts a panic - but how? Let's take a glance at its implementation.
#[track_caller]
pub const fn panic_fmt(fmt: fmt::Arguments<'_>) -> ! {
if cfg!(feature = "panic_immediate_abort") {
super::intrinsics::abort()
}
// NOTE This function never crosses the FFI boundary; it's a Rust-to-Rust call
// that gets resolved to the `#[panic_handler]` function.
extern "Rust" {
#[lang = "panic_impl"]
fn panic_impl(pi: &PanicInfo<'_>) -> !;
}
let pi = PanicInfo::internal_constructor(Some(&fmt), Location::caller(), true);
// SAFETY: `panic_impl` is defined in safe Rust code and thus is safe to call.
unsafe { panic_impl(&pi) }
}panic_fmt may look a bit complex at first, but it is easy to understand, if you look at it part by part.
Its signature seems like a good place to start.
pub const fn panic_fmt(fmt: fmt::Arguments<'_>) -> !As I mentioned before, panic_fmt accepts formatting arguments, which contain our panic message.
That seems pretty self-explanatory. It is also marked with const, since it can be called in a const context.
const ASSERT:() = if size_of::<usize>() != 8{
// Returns a compile-time error if usize is not 64 bits in size.
panic!("Only 64 bit systems supported!");
}else{};This allows it to be used for things like static assertions, and a whole lot more.
Another interesting thing about panic_fmt is its return type - '!'
The never type has a lot of really neat properties.
The main thing you need to know is that this means that panic_fmt can never return.
This is something the compiler ensures, and something it can use to allow for some pretty neat syntax.
You see, since you never can obtain a value of type never(since functions that return never never return), the compiler can coerce those it to any other type.
let name = match animal{
Animal::Dog => "dog",
Animal::Cat => "cat",
// We panic when this arm gets executed, so the
// execution can't continue from this point.
// The compiler then coreces that never value
// to a string type. This has no effect at runtime, but
// it allows for the compiler to typecheck divergent paths
// like this.
_=> panic!("Unknown animal!"),
}The main purpose of the never type is just that - type checking divergent paths.
It has other interesting properties, but treating it as a "this function never returns" marker should suffice.
I will gloss over the "track_caller" attribute for now.
#[track_caller]The next thing you might notice is the if statement, which checks the value of cfg!(feature = "panic_immediate_abort"). This is just Rust syntax for checking if a feature was enabled at build time.
if cfg!(feature = "panic_immediate_abort") {
super::intrinsics::abort()
}If this feature is enabled, all functions that panic immediately stop the execution of the program, without printing any messages. Calling abort will do just that - terminate the execution of the program.
Next, we see an extern block - What does it do? Well, it is a bit special. It has the #[lang_item] attribute, meaning it is a language item.
// NOTE This function never crosses the FFI boundary; it's a Rust-to-Rust call
// that gets resolved to the `#[panic_handler]` function.
extern "Rust" {
#[lang = "panic_impl"]
fn panic_impl(pi: &PanicInfo<'_>) -> !;
}Language items are functions / types / statics, which require special handling on the compiler side.
In this case, the attribute tells the compiler to resolve this function to the panic handler.
By default, this will be the function begin_panic_handler in std.
However, panic handlers can also be defined outside the Rust standard library.
This is exceptionally useful in embedded scenarios, where you might want to do something special(eg. reset the microcontroller) when a panic occurs.
#[panic_handler]
pub fn my_panic_handler(info: &core::panic::PanicInfo<'_>) -> ! {
writeln(UART, "panic:{info}");
cpu::reset();
}So, this extern block just tells the compiler what the signature of the panic_impl function is.
On the very next line, we construct a `PanicInfo` - a data structure describing the source of a panic, and the message it contains.
let pi = PanicInfo::internal_constructor(Some(&fmt), Location::caller(), true);Its constructor accepts two arguments: an optional panic message(in the form of core::fmt::Arguments), and a `Location`. The panic message is pretty self explanatory: this is what the panic! macro generated, and it allows us to describe the cause of the panic in detail.
The Location::caller() is a bit harder to understand. This part is responsible for detecting where the panic originally occurred.
Imagine a piece of code like this:
fn main(){
let a:Option<i32> = None;
a.unwrap();
}even tough the panicking process starts in panic_fmt(called by unwrap), the error is reported as coming from main.rs.
thread 'main' panicked at src/main.rs:3:3:
called `Option::unwrap()` on a `None` valueHow? How does the unwrap function know who called it?
This is where the #[track_caller] attribute comes in. This attribute tells the compiler to keep track of the caller of this function. We can then use Location::caller to retrieve the information about this caller.
Currently, #[track_caller] is implemented in a rather simple and straightforward way: the compiler just introduces a single, hidden argument, to all functions annotated with this attribute.
So, in reality, our function looks a bit more like this:
pub const fn panic_fmt(fmt: fmt::Arguments<’_>,caller:&’static Location) -> !When we want to check who called us, we just simply use this hidden argument. Neat.
PanicInfo::internal_constructor(Some(&fmt), caller, true)Additionally, track_caller is transitive. If multiple functions are marked with track_caller, we will retrieve the "original" caller.
#[track_caller]
fn a() {
panic!(); // The panic message will contain the place
// in source code which originally called b(which then
// called a). In this case, that would be
// "fn c(): some file.rs 32:64"
}
#[track_caller]
fn b() {
// Since a has the `track_caller` attribute too, it will report the same caller that `b` reports.
a();
}
fn c() {
b(); // Panics in `b` & `a` will be reported as coming from here.
}Knowing all of this should give you a rough idea about what #[track_caller] is, and how to use it.
For example, if you have a function that can only fail if the caller gave it the wrong arguments(eg. Passing -1 to a square root function, or passing a wrong key to a map), you can use this attribute to make the panic appear at the site of the caller.
#[track_caller]
fn sqrt(i:i32)->i32{
assert!(i >= 0, "Attempted to compute the square root of a negative number.");
/* Implementation of square root*/
}
fn stupid(){
sqrt(-1); // The panic message will contain the source file, line and column of this invalid call to `sqrt`.
}There is a lot more complexity behind the scenes(eg. when dealing with function pointers), but all of it is hidden from the user.
Overall, #[track_caller] is an important part of Rust, which gives you more control over how panics are reported.
Things like Option::unwrap use this function to give better information about where a panic really comes from.
With all of this out of the way, we can finally look at the last line of panic_fmt, and then go even deeper, into the bowels of panic_impl.
// SAFETY: `panic_impl` is defined in safe Rust code and thus is safe to call.
unsafe { panic_impl(&pi) }As I mentioned before, panic_impl is a language item, and is resolved in a bit of a funky way.
In std, this will get resolved to the function begin_panic_handler, in std/src/panicking.rs.
#[panic_handler]
pub fn begin_panic_handler(info: &core::panic::PanicInfo<'_>) -> ! {
let loc = info.location().unwrap(); // The current implementation always returns Some
let msg = info.message();
crate::sys::backtrace::__rust_end_short_backtrace(move || {
if let Some(s) = msg.as_str() {
rust_panic_with_hook(
&mut StaticStrPayload(s),
loc,
info.can_unwind(),
info.force_no_backtrace(),
);
} else {
rust_panic_with_hook(
&mut FormatStringPayload { inner: &msg, string: None },
loc,
info.can_unwind(),
info.force_no_backtrace(),
);
}
})
}The first 2 lines just extract some information from PanicInfo. They retrieve the panic location, and the panic message.
let loc = info.location().unwrap(); // The current implementation always returns Some
let msg = info.message();The next line is a bit interesting. We call a very odd function __rust_end_short_backtrace with a closure. Why?
__rust_end_short_backtrace and its brother __rust_begin_short_backtrace are used to display better backtraces.
You probably don't care all that much about anything that happens before main when diagnosing a panic. Who cares what the standard library did to initialize your program, if the error happens waaay after that?
This part of the backtrace will almost always stay the same, so it is just clutter.
27: 0x557f0d3c91c4 - std::sys::backtrace::__rust_begin_short_backtrace::h51d47f9917bfc9c9
at /rustc/9ffde4b089fe8e43d5891eb517001df27a8443ff/library/std/src/sys/backtrace.rs:152:18
28: 0x557f0d3cccca - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h111315bb86404840
at /rustc/9ffde4b089fe8e43d5891eb517001df27a8443ff/library/std/src/thread/mod.rs:559:17
29: 0x557f0d3cccca - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h998ce8172649aff7
at /rustc/9ffde4b089fe8e43d5891eb517001df27a8443ff/library/core/src/panic/unwind_safe.rs:272:9
30: 0x557f0d3cccca - std::panicking::try::do_call::hf04bd389dff187f3
at /rustc/9ffde4b089fe8e43d5891eb517001df27a8443ff/library/std/src/panicking.rs:589:40
31: 0x557f0d3cccca - std::panicking::try::h6f702a4728de202d
at /rustc/9ffde4b089fe8e43d5891eb517001df27a8443ff/library/std/src/panicking.rs:552:19
32: 0x557f0d3cccca - std::panic::catch_unwind::h60677702e4e21dfd
at /rustc/9ffde4b089fe8e43d5891eb517001df27a8443ff/library/std/src/panic.rs:359:14
33: 0x557f0d3cccca - std::thread::Builder::spawn_unchecked_::{{closure}}::h3bfcc3f4d1f835f6
at /rustc/9ffde4b089fe8e43d5891eb517001df27a8443ff/library/std/src/thread/mod.rs:557:30
34: 0x557f0d3cccca - core::ops::function::FnOnce::call_once{{vtable.shim}}::hdbd61a047ce6fc0f
at /rustc/9ffde4b089fe8e43d5891eb517001df27a8443ff/library/core/src/ops/function.rs:250:5
35: 0x7fe5cbfcb477 - std::sys::pal::unix::thread::Thread::new::thread_start::hfae4f2cea0a780e5
36: 0x7fe5c5e4eba8 - start_thread
37: 0x7fe5c5ed2b8c - __GI___clone3Ideally, we'd like to display the "interesting" parts of the backtrace by default, and only show it in its entirety when need.
__rust_end_short_backtrace and __rust_begin_short_backtrace mark the beginning and end of this "short backtrace".
By calling __rust_end_short_backtrace, we are marking anything after this call as not important to display.
So, unless we request otherwise(RUST_BACKTRACE=full), our program will not display the platform-specific guts of the panicking machinery.
Let us now look at the closure we passed to __rust_end_short_backtrace.
if let Some(s) = msg.as_str() {
rust_panic_with_hook(
&mut StaticStrPayload(s),
loc,
info.can_unwind(),
info.force_no_backtrace(),
);
} else {
rust_panic_with_hook(
&mut FormatStringPayload { inner: &msg, string: None },
loc,
info.can_unwind(),
info.force_no_backtrace(),
);
}The two arms of this if expression don't differ too much - the main difference is in what kind of "payload" they pass.
Consider this panic:
panic!("Woopise doopsie!");The message it contains never changes, and does not need any formatting. We can simply store a pointer to the static data containing our message, and be on our merry way. This panic, on the other hand:
panic!("{n} is not even!");requires formatting, and an allocation. msg.as_str() checks if a given instance of core::fmt::Arguments is just a static string.
If we know it is a static string, we can avoid all the formatting and the allocation, and just pack a pointer to the const data in our panic message.
Now that I explained the difference between those branches, let's take a look at all the arguments we pass to panic_with_hook:
rust_panic_with_hook(
&mut StaticStrPayload(s),
loc,
info.can_unwind(),
info.force_no_backtrace(),
); Besides the payload, and the panic location, we pass 2 more arguments, extracted from PanicInfo.
can_unwind is the most interesting of those arguments. As you might recall, "unwinding" is the mechanism by which panics can be thrown and caught. When an unwind occurs, our program will traverse its call stack, dropping data along the way. It will stop when it finds the intrinsic catch_unwind - this allows us to recover from some panics.
This is quite useful - for example, Rust test harness uses this mechanism to report when a test panics.
However, not all panics can(or should) unwind. For example, you can't unwind across the "C" function ABI:
extern "C" fn test(){
panic!(); // This will abort, instead of panicking.
}When you attempt to unwind across a C function, the program will call the function panic_cannot_unwind.
This function will then panic with the panic in a function that cannot unwind message.
thread 'main' panicked at src/main.rs:2:5:
explicit panic
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at library/core/src/panicking.rs:218:5:
panic in a function that cannot unwindIf panic_cannot_unwind is called, we already know that we will not be able to unwind.
So, the can_unwind field of PanicInfo will be set to false, and and the program will terminate instead of attempting to unwind.
fn panic_cannot_unwind() -> ! {
panic_nounwind("panic in a function that cannot unwind")
}force_no_backtraceIf this argument is set, the panic backtrace will never be printed - even if you set RUST_BACKTRACE=full.
Why? What uses does this have?
Well, we can look at this issue to figure out why.
Turns out, force_no_backtrace is mostly needed because of an odd corner case. Consider a snippet of code like this:
struct Foo;
impl Drop for Foo {
fn drop(&mut self) {
panic!("drop");
}
}
fn main() {
let f = Foo;
panic!("main");
}When we trigger the original panic, the unwinding process will start, and attempt to drop the variable f.
That will then trigger yet another panic. In such a case force_no_backtrace will be set, to avoid printing the backtrace twice.
rust_panic_with_hookNow that I discussed all arguments of rust_panic_with_hook, let us look at the function in question. It is quite big, so I'll explain it piece by piece.
#[optimize(size)]
fn rust_panic_with_hook(
payload: &mut dyn PanicPayload,
location: &Location<'_>,
can_unwind: bool,
force_no_backtrace: bool,
) -> ! {The first interesting thing about this function is the #[optimize(size)] attribute. Rust programs rarely panic, so optimizing this function for performance makes little sense.
We can instead optimize it to reduce its size. You probably will not notice the minuscule difference in performance of panics this will cause.
However, the reduction in size of the panicking machinery has a lot of benefits.
Remember, this function is used in pretty much all Rust code: reducing its size will reduce the size of all Rust programs.
rust_panic_with_hook starts doing its magic by performing a call to panic_count::increase:
let must_abort = panic_count::increase(true);Let us stop here for a moment, and look a bit closer at `panic_count::increase`. It is responsible for a lot of interesting things.
panic_count::increasepub fn increase(run_panic_hook: bool) -> Option<MustAbort> {
let global_count = GLOBAL_PANIC_COUNT.fetch_add(1, Ordering::Relaxed);
if global_count & ALWAYS_ABORT_FLAG != 0 {
// Do *not* access thread-local state, we might be after a `fork`.
return Some(MustAbort::AlwaysAbort);
}
LOCAL_PANIC_COUNT.with(|c| {
let (count, in_panic_hook) = c.get();
if in_panic_hook {
return Some(MustAbort::PanicInHook);
}
c.set((count + 1, run_panic_hook));
None
})
}First of all, it increments the global panic count, and checks if the ALWAYS_ABORT_FLAG is set. If this flag is set, it returns the value AlwaysAbort, signalling that the program should immediately terminate. This may seem quite strange - when would this be needed?
ALWAYS_ABORT_FLAG has a very interesting purpose, related to the fork function. You see, on many platforms, allocating memory after a call to fork is undefined behaviour. This means that, after we use fork to spawn a child thread, that thread can't allocate memory. Period.
Since printing backtraces requires allocation, we can't do any of that.
But, why is calling malloc after fork in the child thread UB?
forking mallocThe fork function duplicates the calling process, along with its memory space.
Seems fine, until you consider things like Mutexes. Suppose we have a thread A and a thread B. When thread A is holding a lock, thread B forks.
// thread A
let guard = mutex.lock();
// thread B
fork();Now, a new process C holds the exact copy of the memory space shared by A and B.
In the process, the locked mutex was also copied. But, A and C do not share a memory space. So, while B can "see" that A has released the lock, the copy of the lock C has will never be released.
This means process C will spend an eternity, waiting for a lock nobody is holding anymore.
Now imagine what could happen if our implementation of malloc is using any locks.
This is just one example of the way fork can mess things up.
If the panicking machinery allocates any memory at all, we would have a problem.
To prevent this sort of issues in Rust, the child thread will call the function painc_count::set_always_abort. This sets ALWAYS_ABORT_FLAG, ensuring panic-king will not allocate any memory.
After checking ALWAYS_ABORT_FLAG is not set, we can now safely take a look at our thread-local state(thread locals can allocate).
LOCAL_PANIC_COUNT.with(|c| {
let (count, in_panic_hook) = c.get();
if in_panic_hook {
return Some(MustAbort::PanicInHook);
}
c.set((count + 1, run_panic_hook));
None
})First, we check if the field in_panic_hook of LOCAL_PANIC_COUNT is set. If it is set, that means a panic has occurred in the panic hook, and we should terminate the process.
If this flag is not set, we will increment the local panic counter, and optionally set in_panic_hook.
You may wonder: what is the purpose of the panic counter?
It used for a couple of things: for example, for checking if a thread is currently panicking.
#[inline]
pub fn panicking() -> bool {
!panic_count::count_is_zero()
}We can't implement such a check using just a flag: panics can nest.
If we just used a simple flag, and a double-panic occurred, that flag would get cleared early.
Using a panic counter guarantees we will handle nested panics correctly.
Anyway: if neither ALWAYS_ABORT_FLAG nor in_panic_hook is set, that means everything is in order. There is no need to terminate the process, so panic_count::increase will return None.
Explaining this one line of rust_panic_with_hook took quite some time, huh?
let must_abort = panic_count::increase(true);Well, nobody said that the panicking process is simple. Thankfully, the rest of this function is a bit easier to understand. Consider the next few lines:
// Check if we need to abort immediately.
if let Some(must_abort) = must_abort {
match must_abort {
panic_count::MustAbort::PanicInHook => {
// Don't try to format the message in this case, perhaps that is causing the
// recursive panics. However if the message is just a string, no user-defined
// code is involved in printing it, so that is risk-free.
let message: &str = payload.as_str().unwrap_or_default();
rtprintpanic!(
"panicked at {location}:\n{message}\nthread panicked while processing panic. aborting.\n"
);
}
panic_count::MustAbort::AlwaysAbort => {
// Unfortunately, this does not print a backtrace, because creating
// a `Backtrace` will allocate, which we must avoid here.
rtprintpanic!("aborting due to panic at {location}:\n{payload}\n");
}
}
crate::sys::abort_internal();
}They are kind of self explanatory, once you learned what panic_count::increase does. We check if it returned Some(indicating we need to abort), and print different messages depending on the cause of the abort.
rtprintpanic!(
"panicked at {location}:\n{message}\nthread panicked while processing panic. aborting.\n"
);If we have panicked in the panic hook, we probably should not call that hook again, or even attempt formatting the panic message. Think about it: if rust_panic_with_hook panicked before(because the hook panicked), it is likely to panic again. That could lead us to a loop of panics. This loop would overflow the stack, and lead to very confusing error messages. Since such a case is unlikely to be recoverable, we ought to abort instead.
rtprintpanic!("aborting due to panic at {location}:\n{payload}\n");If we have panicked after a call to fork, we can't print any backtraces, since that allocates memory. We still can print the cause of the panic, by having the formatting machinery write directly to stderr. This is exactly what the rtprintpanic macro does.
if let Some(mut out) = crate::sys::stdio::panic_output() {
let _ = crate::io::Write::write_fmt(&mut out, format_args!($($t)*));
}No matter the case of the termination, we use the function abort_internal to stop the execution of the program.
crate::sys::abort_internal();However, if panic_count::increase returns None, we don't need to terminate. In such a case, we will simply pass right trough to the rest of the panicking machinery.
We have now incremented the panic counter, and check that we don't need to terminate.
Now, we will now retrieve the panic hook. What is a panic hook? Glad you asked.
let hook = HOOK.read().unwrap_or_else(PoisonError::into_inner);A panic hook is, at its core, nothing more than a callback, invoked during each panic. Changing this hook allows us to control what a panic will print.
We can, for example, write information about the panic to a log file:
std::panic::set_hook(Box::new(|info| {
eprintln!("panic:{info}");
writeln!(log_file, "panicked at {date:?}:{info}", date = Instant::now(););
}));We can also filter out some more interesting errors:
std::panic::set_hook(Box::new(|info| {
eprintln!("panic:{info}");
let Some(location) = info.location();
if location.file_name().contains("important_crate"){
writeln!(log_file, "panicked at {date:?} in `important_crate`:{info}", date = Instant::now(););
}
}));Really, the only limit to what a panic hook can do is your imagination.
After garbing the hook, rust_panic_with_hook will then call it if it is set. Otherwise, it will just call the default one.
match *hook {
Hook::Default if panic_output().is_none() => {}
Hook::Default => {
info.set_payload(payload.get());
default_hook(&info);
}
Hook::Custom(ref hook) => {
info.set_payload(payload.get());
hook(&info);
}
};As an added optimization, if it is known that printing on a target(eg. WASM) does nothing, we can skip calling the default hook altogether.
After the hook finishes executing, we will perform some cleanup, preparing to drop down to platform-specific implementation of panicking.
First, we will call panic_count::finished_panic_hook to:
panic_count::finished_panic_hook();Before we drop down to the unwinding code, we need to check if we even can unwind. Remember, certain kinds of panics are not recoverable.
if !can_unwind {
// If a thread panics while running destructors or tries to unwind
// through a nounwind function (e.g. extern "C") then we cannot continue
// unwinding and have to abort immediately.
rtprintpanic!("thread caused non-unwinding panic. aborting.\n");
crate::sys::abort_internal();
}If we can't unwind(recover from this panic), we will have to terminate the execution of this thread.
At this point, all of the code responsible for reporting panics has finished execution. We ensured that we can safely panic, called the panic hook(which prints the panic message and backtrace). Now, it is the time to finally trigger the panic, and jump to code responsible for unwinding the stack.
rust_panic(payload)I have lied a tiny bit. There is one more function we need to discuss. rust_panic is just a thin wrapper around __rust_start_panic. Why is it needed?
#[inline(never)]
fn rust_panic(msg: &mut dyn PanicPayload) -> ! {
let code = unsafe { __rust_start_panic(msg) };
rtabort!("failed to initiate panic, error {code}")
}Well, the unwinding process can still fail for a variety of reasons. For example, stack corruption. If you somehow manage to overwrite the return pointer in a function, we no longer know who called your function. Without that knowledge, we can't unwind. In such a case, we will print an error message instead, and abort execution.
Somewhat confusingly, there are 2 different implementations of __rust_start_panic.
If our code is compiled with -Cpanic=abort, __rust_start_panic will abort execution.
pub unsafe fn __rust_start_panic(_payload: &mut dyn PanicPayload) -> u32 {
unsafe {
abort();
}
}If we compile with unwinding support, `__rust_start_panic` is just a thin wrapper around the platform-specific implementation of panics.
pub unsafe fn __rust_start_panic(payload: &mut dyn PanicPayload) -> u32 {
unsafe {
let payload = Box::from_raw(payload.take_box());
imp::panic(payload)
}
}First, it will convert the panic payload to a box, using the take_box function. You may wonder: why are we using this function? Boxing a value is super easy: could we not just use Box::new?
let payload = Box::new(payload);Well, take_box is a bit special. If the panic payload is already boxed it will take a pointer to that box. Otherwise, it will box our value up. This allows us to save a needless allocation in a lot of cases.
That boxed payload will then be passed to imp::panic. Implementation of that function is platform-specific. On Windows, it uses SEH(Structured Exception Handling). On most other platforms, it uses a library called libunwind.
In this article, I will talk about panicking on GCC Linux - so I'll be looking at the implementation using libunwind.
The libunwind-based implementation of panicking starts by assembling an Exception structure.
pub(crate) unsafe fn panic(data: Box<dyn Any + Send>) -> u32 {
let exception = Box::new(Exception {
_uwe: uw::_Unwind_Exception {
exception_class: RUST_EXCEPTION_CLASS,
exception_cleanup: Some(exception_cleanup),
private: [core::ptr::null(); uw::unwinder_private_data_size],
},
canary: &CANARY,
cause: data,
});There is a lot going on here - so, let me break this down, step by step. First of all, the _Unwind_Exception type.
_Unwind_ExceptionThe layout of this type is mandated by libunwind, and contains a few interesting pieces of data.
First of all, it begins with an exception class. This is a 64 bit identifier, specifying the type of this exception. In the case of C++ exceptions, this will be "XXXXC++\0", where the "XXXX" part of this string is replaced with vendor-specific data.
The Rust exception class is a little throwback to the origin of the language:
const RUST_EXCEPTION_CLASS: uw::_Unwind_Exception_Class = u64::from_ne_bytes(*b"MOZ\0RUST");This value allows us to distinguish C++ exceptions and Rust panics from each other. If the exception_class is MOZ\0RUST, we know that an unwind was caused by a Rust panic.
The next field, exception_cleanup, is a pointer to a cleanup function.
exception_cleanup: Some(exception_cleanup),This function is a part of the libunwind API, and can be used to dispose of an exception object.
However, catching Rust panics in outside catch_unwind (eg. in foreign code, like C++) code is not supported.
This is when `exception_cleanup` comes in.
extern "C" fn exception_cleanup(
_unwind_code: uw::_Unwind_Reason_Code,
exception: *mut uw::_Unwind_Exception,
) {
unsafe {
let _: Box<Exception> = Box::from_raw(exception as *mut Exception);
super::__rust_drop_panic();
}
}This function will first convert the payload pointer back to a box, and then immediately drop that box, disposing of the panic object. After that, exception_cleanup will call __rust_drop_panic.
extern "C" fn __rust_drop_panic() -> ! {
rtabort!("Rust panics must be rethrown");
}This will print the error message "Rust panics must be rethrown". Since Rust code will never call exception_cleanup by itself, we know that this panic was caught in foreign code.
#include <exception>
#include <iostream>
int rust_fn();
int square(int num) {
try{
rust_fn();
}
catch (std::exception e){
std::cout<<"Ignoring Rust panics like a boss!"<<std::endl;
}
}If we don't re-throw this exception, its exception_cleanup function will be called(to dispose of the exception object). We will then see the message "Rust panics must be rethrown", and know exactly what is wrong.
Finally, we can take a look at the last field of _Unwind_Exception.
What does this field does? Well, it is none of your business!
private: [core::ptr::null(); uw::unwinder_private_data_size],Seriously, we don't care about this field at all. libunwind uses it to store some of its internal state.
We zero-initialize them here, but never touch them again.
Now that I explained all the fields of _Unwind_Exception, let us take a look at the type wrapping it: the creatively named Exception.
The Exception type has a fixed, C-style layout. As I already mentioned, its first field is _Unwind_Exception - that type contains all the data libunwind needs to work.
After that lies Rust-specific data, used by the Rust panicking machinery.
canary: &CANARY,
cause: data,I will gloss over the canary field for now. It serves quite an important purpose, but it is hard to explain without context.
In the cause field, we store the panic payload. This is what catch_unwind will return, once it catches our panic.
Ok, now we have created our exception object. What is next?
let exception_param = Box::into_raw(exception) as *mut uw::_Unwind_Exception;
return unsafe { uw::_Unwind_RaiseException(exception_param) as u32 };We take a pointer to that exception, and pass it to _Unwind_RaiseException. If unwinding succeeds, we will never return to this function. If it fails, _Unwind_RaiseException will return an error code, which imp::panic will return for __rust_start_panic to handle.
In most cases(save for things like stack corruption), _Unwind_RaiseException will succeed.
So, what happens now?
...
Now, the libunwind library will start its treacherous walk up the call stack, dropping(running destructors of) data along the way.
Its journey will continue, until it either encounters the catch_unwind intrinsic, or something that prevents the unwind from happening(eg. a function with an ABI not supporting unwinding).
I describe the whole process in more detail in my article about the compiler-side implementation of unwinding.
The details of the process are quite complex, but, at its basic level, it is fairly easy to understand.
Just like a C++ exception travels up the call stack, so does a Rust panic.
Both of them will call the destructors of data, and both of them will eventually find a "catch".
catch_unwindSomewhat confusingly, the name catch_unwind refers to 2 different functions.
The function std::panic::catch_unwind is a convenient wrapper around the whole unwinding machinery.
let result = panic::catch_unwind(|| {
println!("hello!");
});
assert!(result.is_ok());It calls a closure, returning Ok if that closure does not panic, or Err(payload) if it does.
The core intrinsic catch_unwind is a whole lot more complex. It takes 2 function pointers, and a data pointer. It returns an integer value, indicating if a panic has occurred.
pub unsafe fn catch_unwind(
_try_fn: fn(*mut u8),
_data: *mut u8,
_catch_fn: fn(*mut u8, *mut u8),
) -> i32While the entire catch-ing machinery in Rust is built on top of this intrinsic, but, it is not too relevant for now. I'll start by talking about `std::panic::catch_unwind`.
std::panic::catch_unwindYou might be getting deja-vu. Like a lot of functions mentioned before, catch_unwind is just a thin wrapper around another function: std::panicking::try.
pub fn catch_unwind<F: FnOnce() -> R + UnwindSafe, R>(f: F) -> Result<R> {
unsafe { panicking::r#try(f) }
}Let me first explain its odd name: in the newer versions of Rust, try became a keyword. You can escape Rust keywords using the r# prefix:
fn r#use(){}
fn r#mod(){}That allows you to use keywords as function names. So, this is why this function is written as r#try in code.
Now, why is this wrapper needed?
Well, there exist 2 distinct versions of the try function.
#[cfg(feature = "panic_immediate_abort")]
pub unsafe fn r#try<R, F: FnOnce() -> R>(f: F) -> Result<R, Box<dyn Any + Send>> {
Ok(f())
}This one might look kind of weird, until you take a second to understand it a bit better.
It does not actually perform any catch-ing, and simply calls the passed closure directly. And, it always returns Ok... this does not make any sense.
Until you notice this line:
#[cfg(feature = "panic_immediate_abort")]Yep - this is a specialized version of the try function, which is used when you disable all the unwinding machinery. If your program can't ever unwind, catching unwinds makes little sense. So, we simply... don't do that. As easy as that.
The other variant of this function is used when unwinds are enabled. For performance reasons, it contains a lot of very, very weird code. I'll try my best to explain what the hell is going on.
pub unsafe fn r#try<R, F: FnOnce() -> R>(f: F) -> Result<R, Box<dyn Any + Send>>
union Data<F, R> {
f: ManuallyDrop<F>,
r: ManuallyDrop<R>,
p: ManuallyDrop<Box<dyn Any + Send>>,
}This function starts by declaring an union of 3 different types: F(the closure to call), R(the return value of that closure), and Box<dyn Any + Send>, which will contain the panic payload.
All of them are wrapped in ManuallyDrop - a type that inhibits the normal Rust dropping behaviour. This is needed, because later on, this function will do a lot of really odd things.
Next, we will pack our closure into this data structure, and get a pointer to that.
let mut data = Data { f: ManuallyDrop::new(f) };
let data_ptr = (&raw mut data) as *mut u8;The next few lines are written in a quite confusing way. I'll show you the original code here, and then rewrite them a bit to easier show what is going on.
unsafe {
return if intrinsics::catch_unwind(do_call::<F, R>, data_ptr, do_catch::<F, R>) == 0 {
Ok(ManuallyDrop::into_inner(data.r))
} else {
Err(ManuallyDrop::into_inner(data.p))
};
}The most important step here is the call to the catch_unwind intrinsic I mentioned before. Let us recap how that intrinsic works.
0 if no unwind was caught, or a non-zero value if an unwind was caught.Let us now look at the refactored version of the code I showed above.
let has_panicked = intrinsics::catch_unwind(do_call::<F, R>, data_ptr, do_catch::<F, R>);
if has_panicked == 0 {
// has_panicked is false, extract the result from the data
return Ok(ManuallyDrop::into_inner(data.r))
} else {
// has_panicked is true, extract the panic from the data
return Err(ManuallyDrop::into_inner(data.p))
};You can quite clearly see what is going on: the `do_call` function will call our closure(provide via the data pointer). If that call succeeds, it will then store the result of that call in the data pointer.
fn do_call<F: FnOnce() -> R, R>(data: *mut u8) {
// SAFETY: this is the responsibility of the caller, see above.
unsafe {
let data = data as *mut Data<F, R>;
let data = &mut (*data);
let f = ManuallyDrop::take(&mut data.f);
data.r = ManuallyDrop::new(f());
}
}If an unwind occurs, the intrinsic will call do_catch. That function will retrieve the panic payload, and store it in the data pointer.
Later, we simply check the return value of catch_unwind to see if an unwind occurred, and behave accordingly.
if has_panicked == 0 {
// has_panicked is false, extract the result from the data
return Ok(ManuallyDrop::into_inner(data.r))
} else {
// has_panicked is true, extract the panic from the data
return Err(ManuallyDrop::into_inner(data.p))
};We extract either the result of our closure, or the panic payload.
We then return that, packed in a Result. That Result will then be passed back to the user code. It is the value returned by the std::panic::catch_unwind function.
You might now understand most of the process of raising and catching panics, but there is one detail I committed.
How does the function do_catch work?
do_catchI quickly glossed over this function. I said that it "retrieves the panic payload, and stores it in the data pointer" - but how?.
intrinsics::catch_unwind(do_call::<F, R>, data_ptr, do_catch::<F, R>)Let us now look at its implementation!
unsafe {
let data = data as *mut Data<F, R>;
let data = &mut (*data);
let obj = cleanup(payload);
data.p = ManuallyDrop::new(obj);
}It is quite similar to do_call - with a few differences. First of all, it sets the p(payload) value in the data pointer - but it also calls a very interesting function: cleanup.
let obj = cleanup(payload);cleanup is responsible for extracting the payload from a platform-specific exception object, but it also does a whole bunch more.
unsafe fn cleanup(payload: *mut u8) -> Box<dyn Any + Send + 'static> {
let obj = unsafe { Box::from_raw(__rust_panic_cleanup(payload)) };
panic_count::decrease();
obj
}To extract the payload, it calls a function named __rust_panic_cleanup - more on that later.
It also decrements the panic counter, since the panic has been caught.
Now, let us go back to __rust_panic_cleanup. It is yet another case of a "magic" symbol that gets resolved to something else entirely.
What can you do. In reality, this will call the `cleanup` function within the platform-specific panic runtime.
We once again drop down to code interacting with libunwind. This is what is responsible for panics on GCC Linux, and a bunch of other platforms. On Windows, this code will look a bit different.
Recall the _Unwind_Exception object we assembled when throwing a panic. Here, we get back that object.
let exception = ptr as *mut uw::_Unwind_Exception;
if (*exception).exception_class != RUST_EXCEPTION_CLASS {
uw::_Unwind_DeleteException(exception);
super::__rust_foreign_exception();
}The very first thing we do is check the exception_class field of the exception. This is used to check if an exception is coming from Rust(MOZ\0RUS), or something like C++(XXXXC++\0).
If this is not a Rust panic, we dispose of that panic, and call __rust_foreign_exception.
That function will print an error message, and abort the execution of the program.
Next, once we know this is a Rust exception, we will cast the pointer to the Exception type. This type contains the libunwind exception, and some Rust specific data.
let exception = exception.cast::<Exception>();After we get the pointer to the Exception type, we will access one of its field: the canary. You may recall that I skipped explaining the purpose of that field. Now, I will go into that in detail.
let canary = (&raw const (*exception).canary).read();
if !ptr::eq(canary, &CANARY) {
super::__rust_foreign_exception();
}The canary field, like its feathered namesake, protects us against a rare, but vicious danger.
Consider the following. Somebody wrote a very useful library, libgreat, in C++. That library depends on libawsome, which just so happens to be written in Rust.
libawsome is compiled with Rust 1.66. Now, we call libgreat from a Rust program, compiled with rustc 1.78.
Due to an oversight, libawsome panics. libgreat does not catch that panic - after all, Rust panic's can't be caught in C++. So, it allows the panic to ride on trough, right to a catch_unwind inside our Rust program.
The program checks the exception class, and sees MOZ\0RUST. Great, this panic was thrown from Rust, so we can catch it, right?
if (*exception).exception_class != RUST_EXCEPTION_CLASS{}
// Let's catch this bad boy!Well, not so fast. Recall, our two pieces of Rust code were compiled with different compilers.
This was not an issue before - neither libawsome nor our Rust program were using the Rust ABI, and they did not interact directly.
Who says that trait objects(used by the payload) haven't changed between Rust versions? Maybe their layout is slightly different?
And now, our panicking machinery will call a function with the wrong version of the Rust ABI to retrieve the panic payload.
That would mean instant UB! Even tough the exception class matches, one version of Rust can't catch a panic from a different one!
This is where the canary field comes into play.
static CANARY: u8 = 0;This field is private to the standard library, and each copy of the standard library will have its own canary - at a different address.
If we detect a panic with a different canary(different address), that means we are not catching "our" panic.
So, we print an error message, and stop the program.
if !ptr::eq(canary, &CANARY) {
super::__rust_foreign_exception();
}You may wonder: would having some kind of version number not be better? Well, the canary approach is much more robust, and can always detect when a panic is coming from a different copy of std. Even if both copies are compiled with the same compiler.
Fun fact: catching a panic from a different std can still cause issues, even if it is compiled with the same compiler.
Explaining exactly how that can cause problems is a bit outside the scope of this article, tough.
However unlikely such an occurrence may be, the Rust library still guards against it. I fell like that demonstrates the philosophy of Rust quite well.
As the very last step, cleanup will turn the Exception pointer to a Box, extract the payload, and free that box.
let exception = Box::from_raw(exception as *mut Exception);
exception.causeAnd with this, I have described the entire Rust panicking machinery. From the original macro, trough the guts of std, till the bowels of catch_unwind.
I am finally...
I have been working on this article since October 2024. At first, I kind of underestimated just how much things to talk about there is in the rust panic runtime.
I certainly did not expect this article to be nearly as big as it got.
You know what is funny? The article originally did not end here.
Originally, this was just a prologue to an article about converting .NET exceptions to Rust panics.
That was the entire reason I started writing this. To talk about how my Rust to .NET compiler handles panics and exceptions.
That will also come... at some point.
For now, I hope you enjoyed the journey trough the guts of std, and I wish you a good day :).
I have triple-checked most of the things I talk about here, but there are bound to be mistakes that slipped trough. If you spot them, please let me know.