Asterinas: OS kernel written in Rust and providing Linux-compatible ABI

I always feel Arc is the admission that the borrow checker with different/overlapping lifetimes is too difficult, despite what many Rust developers - who liberally use Arc - claim.

It's not that the borrow checker is too difficult, it's that it's too limiting.

The _static_ borrow checker can only check what is _statically_ verifiable, which is but a subset of valid programs. There are few things more frustrating than doing something you know is correct, but that you cannot express in your language.

Lifetime tracking and ownership are very difficult. That's why languages like C and C++ don't do it. It's also why those languages needs tons of extra validation steps and analysis tools to prevent bugs.

Arc is nothing more than reference counting. C++ can do that too, and I'm sure there are C libraries for it. That's not an admission of anything, it's actually solving the problem rather than ignoring it and hoping it doesn't crash your program in fun and unexpected ways.

Using Arc also comes with a performance hit because validation needs to be done at runtime. You can go back to the faster C/C++ style data exchange by wrapping your code in unsafe {} blocks, though, but the risks of memory corruption, concurrent access, and using deallocated memory are on you if you do it, and those are generally the whole reason people pick Rust over C++ in the first place.

It's not just difficult, sometimes it's impossible to statically know a lifetime of a value, so you must dynamically track it. Arc is one of such tools.

For kernels (and I suspect database engines might be added to the list, since they seem to have similar requirements to be both scalable and deal with massive amounts of shared state, but I'm not overly familiar with them) is where it gets particularly difficult.

Several kernels for example use type-stable memory, memory that is guaranteed to only hold objects of a particular type, though perhaps only providing that guarantee for as long as you hold an RCU read-lock (this is the case in Linux with SLAB_TYPESAFE_BY_RCU). It is possible in some cases to be able to safely deal with references to objects where the "lifetime" of the referent has ended, but where by dint of it being guaranteed to be the same type of object, you can still do what you want to do.

This comes in handy when you have a problem that commonly appears in kernels where you need to invert a typical lock ordering (a classic case is that the page fault codepath might want to lock, say, VM object then page queue, but the page-replacement codepath will want to lock page-queue then VM object.)

Unfortunately it's hard to think of how the preconditions for these tricks could be formally expressed.

If tracking lifetimes is simple 90% of the time and complex 10% of the time, maybe a tool that lets you have them automatically managed (with some runtime overhead) that 10% of the time is the right way forward.

Looking at the code, it consists of long chains of get().unwrap().to_mut().unwrap().get() noise. Looks like coping with library design than ownership tacking. Also why Result<Option<T>>? Isn't Result already Option by itself? I guess that's why you need get().unwrap().to_mut() to get a value from Result<Option<T>> from an average function call?

If I ask my repository (backed by an sql db) to get a user, there might be 3 different scenarios I'm interested in:

- Technical problem (like connection problems) means I don't know what's in the db

- No technical problem, but no user entry

- No technical problem, and a user entry

You need the Result for the technical problems, and the Option for whether there's a user entry or not.

Then you can use a language and runtime like C# or Java. Or you can use patterns like Go promotes.

There are lots of options if you want.

>> Also why Result<Option<T>>? Isn't Result already Option by itself?

No. I've written code that returns Result<Option<T>>. It was a wrapper for a server's query web API.

The Result part determines whether the request succeeded and the response is valid.

The Option part is because the parameter being queried might not exist. For example, if I ask the API for the current state of the user session with a given Session ID, but that Session ID does not exist, then the Rust wrapper could return OK(None) meaning that the request succeeded, but that no such session was found.

Arc usually means you've not structured your code to have clear lifetimes where one object clearly outlives another. Typically I see c++ applications avoid it but actually suffer from bugs due to the same structural deficiencies. They said, I think it's almost always possible to to avoid it if you try hard enough. With async you need to use structured concurrency.

Wouldn't you want the relevant Rust lifetime to be that of the type-stable memory block, not the individual "object" inside? I'm not that familiar with kernel programming, but that sounds a lot like an arena, and (IIRC) that's the approach with an arena.

Presumably missing session is an alternative scenario and thus should be reported as an error, then you match and handle this error. Your design complicates the common scenario: in case of valid session you need double unwrap, cf File::open that could return Result<Option<File>> if file is not found.

Result is whether an operation returned an error or not. Option is whether you have a value or no value.

Surely Result is supposed to hold both system errors and business errors.

Exactly.

That is why a query that successfully returns no items can be represented as Ok(None).

A successful query with items returned would instead be Ok(Vec<Item>).

An error in the completing the query (for example, problem with the database), would be Err(DatabaseError) or Err(SomeOtherError).

>> Presumably missing session is an alternative scenario and thus should be reported as an error

But in this case, a query using an invalid session ID is not an error. It is asking for details about something that does not exist.

>> cf File::open that could return Result<Option<File>> if file is not found.

This type of query is not like File::open which gets a handle to a resource. Trying to get a handle to a resource that does not exist is an error.

This type of query is read-only and does not allocate any resources or prepare to do anything with the session.

It simplifies the control flow because it distinguishes between errors in completing a query versus the presence or absence of items returned from the query.

I've found the opposite. every time I attempt to subvert the borrow checker, I eventually discover that I'm attempting to write a bug.

How does the caller know whether the response was empty because of an invalid session id ? You have conflated empty response with an erroneous situation. The simplest solution is just Result<T, E>.

> Then you can use a language and runtime like C# or Java. Or you can use patterns like Go promotes.

But in that case you're stuck paying the overhead 100% of the time, even though 90% of the lifetimes are simple. (Perhaps a little less so with escape analysis etc., but doing it at compile time in a way that's understandable in the source feels a lot more reliable)

>> How does the caller know whether the response was empty because of an invalid session id ?

Documentation and how SQL database queries work.

The documentation states that a valid session id will return a SessionInfo struct (since it is an Option the type is Some(SessionInfo) ), and that an invalid session id will return None.

If the SQL query is something like "SELECT * FROM USER_SESSIONS WHERE USER_SESSION_ID = $1" then if an invalid session id is provided the database returns zero rows. The query was successful, but there were no matching sessions with that session id.

>> You have conflated empty response with an erroneous situation. The simplest solution is just Result<T, E>.

Again, an empty response is not an error in this situation. If your database query returns zero rows, is it an error? The database query succeeded. There are no sessions with the provided session id. What error occurred?

Java and C# are languages of different class. C# is perfectly capable of systems programming and manual memory management that is at least more convenient than C (but not C++ with move semantics and operator overloading abuse, otoh C#'s type system and build process are saner which at least partially pays for this).