Zlib-rs is faster than C

The usual answer is: You only need to verify the unsafe blocks, not every block. Though 'unsafe' in Rust is actually even less safe than regular C, if a bit more predictable, so there's a crossover point where you really shouldn't have bothered.

The Rust compiler is indeed better than the C one, largely because of having more information and doing full-program optimisation. A `vec_foo = vec_foo.into_iter().map(...).collect::Vec<foo>`, for example, isn't going to do any bounds checks or allocate.

I have been told that "unsafe" affects code outside of that block, but hopefully steveklabnik may explain it better (again).

> isn't going to do any bounds checks or allocate.

You need to add explicit bounds check or explicitly allocate in C though. It is not there if you do not add it yourself.

> I have been told that "unsafe" affects code outside of that block, but hopefully stevelabnik may explain it better (again).

Poorly-written unsafe code can have effects extending out into safe code. But correctly-written unsafe code does not have any effects on safe code w.r.t. memory safety. So to ensure memory safety, you just have to verify the correctness of the unsafe code (and any helper functions, etc., it depends on), rather than the entire codebase.

Also, some forms of unsafe code are far less dangeous than others in practice. E.g., most of the SIMD functions are practically safe to call in every situation, but they all have 'unsafe' slapped on them due to being intrinsics.

> You need to add explicit bounds check or explicitly allocate in C though. It is not there if you do not add it yourself.

Unfortunately, you do need to allocate a new buffer in C if you change the type of the elements. The annoying side of strict aliasing is that every buffer has a single type that's set in stone for all time. (Unless you preemptively use unions for everything.)

Buggy unsafe blocks can affect code anywhere (through Undefined Behavior, or breaking the API contract).

However, if you verify that the unsafe blocks are correct, and the safe API wrapping them rejects invalid inputs, then they won't be able to cause unsafety anywhere.

This does reduce how much code you need to review for memory safety issues. Once it's encapsulated in a safe API, the compiler ensures it can't be broken.

This encapsulation also prevents combinatorial explosion of complexity when multiple (unsafe) libraries interact.

I can take zlib-rs, and some multi-threaded job executor (also unsafe internally), but I don't need to specifically check how these two interact. zlib-rs needs to ensure they use slices and lifetimes correctly, the threading library needs to ensure it uses correct lifetimes and type bounds, and then the compiler will check all interactions between these two libraries for me. That's like (M+N) complexity to deal with instead of (M*N).

> I have been told that "unsafe" affects code outside of that block, but hopefully stevelabnik may explain it better (again).

It's due to a couple of different things interacting with each other: unsafe relies on invariants that safe code must also uphold, and that the privacy boundary in Rust is the module.

Before we get into the unsafe stuff, I want you to consider an example. Is this Rust code okay?

    struct Foo {
       bar: usize,
    }
    
    impl Foo {
        fn set_bar(&mut self, bar: usize) {
            self.bar = bar;
        }
    }

No unsafe shenanigans here. This code is perfectly safe, if a bit useless.

Let's talk about unsafe. The canonical example of unsafe code being affected outside of unsafe itself is the implementation of Vec<T>. Vecs look something like this (the real code is different for reasons that don't really matter in this context):

    struct Vec<T> {
       ptr: *mut T,
       len: usize,
       cap: usize,
    }

The pointer is to a bunch of Ts in a row, the length is the current number of Ts that are valid, and the capacity is the total number of Ts. The length and the capacity are different so that memory allocation is amortized; the capacity is always greater than or equal to the length.

That property is very important! If the length is greater than the capacity, when we try and index into the Vec, we'd be accessing random memory.

So now, this function, which is the same as Foo::set_bar, is no longer okay:

    impl<T> Vec<T> {
        fn set_len(&mut self, len: usize) {
            self.len = len;
        }
    }

This is because the unsafe code inside of other methods of Vec<T> need to be able to rely on the fact that len <= capacity. And so you'll find that Vec<T>::set_len in Rust is marked as unsafe, even though it doesn't contain unsafe code. It still requires judicious use of to not introduce memory unsafety.

And this is why the module being the privacy boundary matters: the only way to set len directly in safe Rust code is code within the same privacy boundary as the Vec<T> itself. And so, that's the same module, or its children.

C has type-changing stores. If you store to a buffer with a new type, it has the new type. Clang does not implement this correctly though, but GCC does.

Won't the final result allocate?

It won't allocate in this case because it's still a vec of foo at the end, so we know it has enough space. If it were a different type, it may or may not allocate, depending on if it had enough capacity.

> You need to add explicit bounds check or explicitly allocate in C though. It is not there if you do not add it yourself.

Yes — in C you can skip the bounds-checks and allocation, because you can convince yourself they aren't needed; the problem is you may be wrong, either immediately or after later refactoring.

In other memory-safe languages you don't risk the buffer overrun, but it's likely you'll get the bounds checks and allocation, and you have the overhead of GC.

Rust is close to alone in doing both.

Rust is not the only one, there is Ada as well. Ada without SPARK adds bounds checks (which can be disabled through a compiler option), but with SPARK, it does not have to be done at runtime, among many other things (contract-based programming (without SPARK), formal verification (with SPARK) where you need it, and so forth), everything as a breeze.

https://docs.adacore.com/spark2014-docs/html/ug/en/usage_sce...

Look at the table after this paragraph:

> SPARK builds on the strengths of Ada to provide even more guarantees statically rather than dynamically. As summarized in the following table, Ada provides strict syntax and strong typing at compile time plus dynamic checking of run-time errors and program contracts. SPARK allows such checking to be performed statically. In addition, it enforces the use of a safer language subset and detects data flow errors statically.

This is the documentation (namely SPARK User's Guide).

As for what SPARK is: https://learn.adacore.com/courses/intro-to-spark/chapters/01..., so you will be able to see (if you read further), that Ada alone may suffice for the majority of the cases, as for many things you do not even need SPARK to begin with.

Many courses for both Ada and SPARK are available here: https://learn.adacore.com/index.html

There are very good reasons for why Ada is used in critical systems, especially, but not limited to avionics and railway systems, see more at https://www.adacore.com/industries.