When I first started writing async Rust, I was honestly pretty overwhelmed. Coming from languages where async just works out of the box, I wasn’t used to the “batteries-not-included” approach. Instead of a built-in runtime, Rust just hands you a Future trait and tells you to go pick a external runtime yourself, such as tokio, async-std, or smol.
I defaulted to tokio for my past projects, but it always felt like a black box. I knew how to use it, but I never really looked under the hood. Recently, I was toying with a custom task scheduler project and struggling with fairness algorithms. Hitting this roadblock made me wonder how production-grade systems handle task scheduling. Knowing that Rust is famous for its performance, I started wondering how its async runtimes manage and schedule the tasks across a limited number of OS threads. I figured that diving into those mechanics might spark some new ideas for my own scheduler.
To strip away the magic and learn how it’s done, I decided to build my own minimal runtime from scratch. The initial goal was simple: get block_on and spawn working, build a basic RunQueue, and get two tasks to cooperatively ping-pong back and forth.
After reading the Rust Async Book and skimming tokio source code, I naively thought I had Wakers and threads figured out. My first implementation quickly proved me wrong.
Everyone’s learning path is different, but hopefully, walking through my two biggest mistakes can help you understand async Rust a little better. Here is what I got wrong about Wakers, and how I fixed it.
1. Cooperative Async Requires Actual Cooperation
To test my executor, I spawned two futures: one to print “ping” and one to print “pong”, sharing a Mutex<Option<String>>. My expectation was that they would take turns automatically. Instead, the first task printed “ping” exactly once, and then the entire program froze permanently.
I had written a standard while count < 5 loop inside the async block:
// Naive approach: assuming the executor will pause this loop
let fut1 = spawn(async move {
let mut count = 0;
while count < 5 {
let mut guard = shared_state.lock().unwrap();
if guard.as_deref() == Some("pong") || guard.is_none() {
*guard = Some("ping".to_string());
count += 1;
println!("ping");
}
// BUG: No .await point here!
// The loop spins synchronously forever, starving the executor.
}
});
I was still thinking in terms of OS threads (preemptive multitasking), assuming the executor would magically pause my loop to let the other task run.
Realization: Async Rust is cooperative. The executor cannot force a task to pause. A task will hold the CPU hostage and run synchronously until it hits an .await point that explicitly returns Poll::Pending. Because my while loop had no .await points, the first task refused to cooperate. It starved the executor, spinning infinitely in a deadlock because it never yielded the CPU to let the other task update the shared state.
To fix this without busy-spinning, I used std::future::poll_fn (a helper to yield Poll) to check if it was the task’s turn. If it wasn’t, the task would cooperatively save its Waker, return Poll::Pending, and hand the CPU back to the executor.
use std::future::poll_fn;
use std::task::Poll;
// Cooperative approach: yielding control while waiting
let fut1 = spawn(async move {
let mut count = 0;
while count < 5 {
poll_fn(|cx| {
let mut guard = shared_state.lock().unwrap();
if guard.data.as_deref() == Some("pong") || guard.data.is_none() {
// It is my turn! Take the turn.
guard.data = Some("ping".to_string());
// Wake the other task using the waker stored by other "pong" task.
if let Some(waker) = guard.waiting_task.take() {
waker.wake();
}
Poll::Ready(())
} else {
// It is NOT my turn.
// Save my waker and cooperatively yield the CPU.
guard.waiting_task = Some(cx.waker().clone());
Poll::Pending
}
}).await; // <-- The executor takes back control here!
count += 1;
println!("ping");
}
});
2. Wakers Don’t Automatically Know Who to Wake
Once my tasks could cooperatively go to sleep, I needed to test waking them up from a background OS thread. I implemented my Waker’s wake_by_ref function like this:
unsafe fn wake_by_ref(data: *const ()) {
let arc_data = unsafe { Arc::from_raw(data as *const Task) };
let cloned = arc_data.clone();
// Push the task back to the run queue
arc_data.task_sender.push(cloned);
// BUG: unpark() wakes the thread that called it!
std::thread::current().unpark();
std::mem::forget(arc_data);
}
My background thread successfully grabbed the Waker, called .wake_by_ref(), the task was pushed to the queue… and then the executor timed out and died.
Realization: std::thread::current() returns a handle to the thread currently executing the code. Because the background thread is the one calling .wake_by_ref(), it ended up unparking itself instead of the main thread. Meanwhile, the main thread running my block_on loop stayed asleep in std::thread::park() forever.
Wakers are just passive data structs. They don’t magically know which thread they belong to. To fix this, the RunQueue must explicitly capture the main thread’s handle upon creation, store it, and automatically ring that specific “phone number” whenever a new task is pushed.
struct RunQueue {
queue: Arc<Mutex<VecDeque<Arc<Task>>>>,
// Store the specific thread handle of the executor
executor_thread: std::thread::Thread,
}
impl RunQueue {
fn push(&self, task: Arc<Task>) {
if let Ok(mut queue) = self.queue.lock() {
queue.push_back(task);
}
// Fix: The queue rings the specific phone number
// of the executor thread whenever a new task arrives!
self.executor_thread.unpark();
}
}
Conclusion
Building mini-rt taught me that “single-threaded executor” is a bit of a misnomer. The polling happens on a single thread, but the wake-ups come from across the OS. Understanding how those background threads safely throw wake-up signals across the boundary is the true secret of async Rust.
If you want to poke around the actual code for this first phase, you can find it in the mini-rt repository (Tag: M1).