☑ Uncovering Rust: References and Ownership

Over the last few years I’ve become more aware of the Rust programming langauge. Slightly more than a decade old, it has consistently topped the Stack Overflow Developer Survey in the most loved langauge category for the last four years, so there’s clearly a decent core of very keen developers using it. It aims to offer performance on a par with C++ whilst considerably improving on the safety of the language, so as a long-time C++ programmer who’s all too aware of its potential for painfully opaque bugs, I thought it was definitely worth checking what Rust brings to the table.

As the first article in what I hope will become a reasonable series, I should briefly point out what these articles are not. They are certainly not meant to be a detailed discussion of Rust’s history or design principles, nor a tutorial. The official documentation and other sources already do a great job of those things.

Instead, this series is a hopefully interesting tour of some of the aspects of the language that set it apart, enough to get a flavour of it and perhaps decide if you’re interested in looking further yourself. I’m specifically going to be comparing the language to C++ and perhaps occasionally Python as the two languages with which I’m currently most familiar.

Mutability¶

Before I get going on the topic of this post, I feel it’s important to clarify one perhaps surprising detail of Rust to help understand the code examples below, and it is this: all variables are immutable by default. It’s possible to declare any variable mutable by prefixing with the mut keyword.

I could imagine some people considering this is a minor syntactic issue as it just means what would be const in C++ is non-mut in Rust, and non-const in C++ is mut in Rust. So why mention it? Well, mostly to help people understand the code examples a little easier; whilst it’s debatably not a fundamental issue, it’s also not something that’s necessarily self-evident from the syntax either.

Also, I think it’s a nice little preview of the way the language pushes you towards one of its primary goals: safety. If you forget the modifier things default to the most restrictive situation, and the compiler will prod you to add the modifier explicitly if that’s what you want. But if it isn’t what you want, you get the hint to fix a potential bug. Immutable values typically also make it much easier to take advantage of concurrency safely, but that’s a topic for a future post.

Ownership¶

Since one of the touted features of the language is safety around memory allocation, I’m going to start off outlining how ownership works in Rust.

Ownership is a concept that’s stressed many times during the Rust documentation, although in my view it’s pretty fundamental to truly understanding any language. Manipulating variables in memory is the bulk of what software does most of the time and errors around ownership are some of the most common sources of bugs across multiple langauges.

In general “owning” a value in this context means that a piece of code has a responsibility to manage the memory associated with that value. This isn’t about mutability or any other concept people might feasibly regard as forms of ownership.

Just to be clear, I’m going to skip discussion of stack-allocated variables here. Management of data on the stack is generally similar in all mainstream imperative languages and generally falls out of the language scoping rules quite neatly, so I’m going to focus this discussion on the more interesting and variable topic of managing heap allocations.

In C++ ownership is a nebulous concept and left for the programmer to define. The language provides the facility to allocate memory and it’s up to the programmer to decide when it’s safe to free it. Techniques such as RAII allow a heap allocation to be tied to a particular scope, either on the stack or linked with an owning class, but this must be manually implemented by the programmer. It’s quite easy to neglect this in some case or other, and since it’s aggressively optional then the compiler isn’t going to help you police yourself. As a result, memory mismangement is a very common class of bugs in C++ code.

Higher-level languages tend to utilise different forms of garbage collection to avoid exposing the programmer to these issues. Python’s reference counting is a simple concept and covers most cases gracefully, although it adds peformance overhead to many operations in the language and cyclic references complicate matters such that additional garbage collection algorithems are still required. Languages like Java with tracing garbage collectors impose less performance penalty on access than reference counting, but may be prone to spikes of sudden load when a garbage sweep is done. These systems are also often more complex to implement, especially as in the real world they’re often a hybrid of multiple techniques. This isn’t necessarily a direct concern for the programmer, as someone else has done all the hard work of implementing the algorithm, but it does inch up the risk of hitting unpredictable pathalogical performance behaviour. These can be the sort of intermittent bugs that we all love to hate to investigate.

All this said, Rust takes a simpler approach, which I suppose you could think of as what’s left of reference counting after a particularly aggressive assult from Ockham’s Razor.

Rust enforces three simple rules of ownership:

1. Each value has a variable which is the owner.
2. Each value has exactly one owner at a time.
3. When the owner goes out of scope the value is dropped¹.

I’m not going to go into detail on the scoping rules of Rust right now, although there are some interesting details that I’ll probably cover in another post. For now suffice to say that Rust is lexically scoped in a very similar way to C++ where variables are in scope from their definition until the end of the block in which they’re defined².

This means, therefore, that because a value has only a single owner, and because the scope of that owner is well-defined and must always exit at some point, there is no possible way for the value to not be dropped and its memory leaked. Hence achieving the promised memory safety with some very simple rules that can be validated at compile-time.

So there you go, you assign a variable and the value will be valid until such point as that variable goes out of scope. What could be simpler?

1
2
3
4
5
6
7
8
9

// Start of block.
{
    …
    // String value springs into existence.
    let my_value = String::from("hello, world");
    println!("Value: {}", my_value);
    …
}
// End of block, my_value out of scope, value dropped.

Moving right along¶

Well of course it’s not quite that simple. For example, what happens if we assign the value to another variable? I mean, that’s a pretty simple case. How hard can it be to figure out what this code will print?

1
2
3
4
5

fn main() {
    let my_value = String::from("hello, world");
    let another_value = my_value;
    println!("Values: {} {}", my_value, another_value);
}

The answer is: slightly harder than you might imagine. In fact the code above won’t even compile:

   Compiling sample v0.1.0 (/Users/andy/src/local/rust-tutorial/sample)
error[E0382]: borrow of moved value: `my_value`
 --> src/main.rs:4:31
  |
2 |     let my_value = String::from("hello, world");
  |         -------- move occurs because `my_value` has type
`std::string::String`, which does not implement the `Copy` trait
3 |     let another_value = my_value;
  |                         -------- value moved here
4 |     println!("Values: {} {}", my_value, another_value);
  |                               ^^^^^^^^ value borrowed here after move

  error: aborting due to previous error

This is because Rust implements move semantics by default on assignment. So what’s really happening in the code above is that a string value is created and ownership is assigned to the my_value variable. Then this is assigned to another_value which results in ownership being transferred to the another_value variable. At this point the my_value variable is still in scope, but it’s no longer valid.

The compiler is pretty comprehensive in explaining what’s going on here, the value is moved in the second line and then the invalidated my_value is referenced in the third line, which is what triggers the error.

This may seem unintuitive to some people, but before making any judgements you should consider the alternatives. Firstly, Rust could abandon its simple ownership rules and allow arbitrary aliasing like in C++. Except that would mean either exposing manual memory management or replacing it with a more expensive garbage collector, both of which compromise on the goals of safety and performance respectively.

Secondly, Rust could perform a deep copy of the data on the assignment, so duplicating the value and ending up with two variables each with its own copy. This is workable, but defeats the goal of performance as memory copying is pretty slow if you end up doing an awful lot of it. It also violates a basic programmer expectation that a simple action like assignment should not be expensive.

And so we’re left with the move semantics defined above. It’s worth noting, however, that this doesn’t apply to all types. Some are defined as being safe to copy: generally the simple scalar types such as integers, floats, booleans, and so on. The key property of these which make them safe is that they’re stored entirely on the stack, there’s no associated heap allocation to handle. It’s also possible to declare that new types are safe to copy by adding the Copy trait, but traits are definitely a topic for a later post.

It’s also worth noting that these move semantics are not as restrictive as they might seem due to the existence of references, which I’ll talk about later in this post. First, though, it’s interesting to look at how these semantics work with functions.

Onwnership in and out of functions¶

The ownership rules within a scope are now clear, but what about passing values into functions? In C++, for example, arguments are passed by value which means that the function essentially operates on a copy. If this value happens to be a pointer or reference then of course the original value may be modified, but as mentioned above we’re deferring discussion of references in Rust for a moment.

Argument passing would appear to suffer the same issues as the assignment example above, in that we don’t want to perform a deep copy, but neither do we want to complicate the ownership rules. So it’s probably little surprise that argument passing into functions also passes ownership in the same way as the assignment.

This code snippet will fail to compile:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

fn main() {
    let s = String::from("hello");
    my_function(s);
    // Oops, s isn't valid here any more!
    println!("Value of s: {}", s);
}

fn my_function(arg: String) {
    // Ownership passes to the 'arg' parameter.
    println!("Now I own {}", s);
    // Here 'arg' goes out of scope and the String is dropped.
}

Although this may seem superficially surprising, when you really think about it argument passing is just a fancy form of assignment into a form of nested scope, so it shouldn’t be a surprise that it follows the same semantics.

The same logic applies to function return values, and this is where things could get slightly surprising for C++ programmers who are used to returning pointers or references to stack values being a tremendous source of bugs; and returning non-referential values as a cause of potentially expensive copy operations.

In C++ when the function call ends, any pointer or reference to anything on its stack that is passed to the caller will now be invalid. These can be some pretty nasty bugs, particuarly for less experienced programmers. It doesn’t help that the compiler doesn’t stop you doing this, and also that these situations often give the appearance of working correctly initially, since the stack frame of the function has often not been reused yet so the pointer still seems to point to valid data immediately after the call returns. This clearly harms the safety of the code.

If the programmer decides to resolve this issue by returning a complex class directly by value instead of by pointer or reference, then this generally entails default construction of an instance in the caller, then execution of the function and then assignment of the returned value to the instance in the caller which might involve some expensive copying. This potentially harms the performance of the code.

I’m deliberately glossing over some subtleties here around returning temporary objects, return value optimisation and move semantics in C++ which are all well outside the scope of this post on Rust. But even though solutions to these issues exist, they require significant knowledge and experience on the part of the programmer to take advantage of correctly, particularly for user-defined classes.

In Rust things are simpler: you can return a local value and ownership passes to the caller in the obvious manner.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

fn main() {
    let my_value = create();
    // At this point 'my_value' owns a String.
    println!("Now I own {}", my_value);

    let another_value = transform(my_value);
    // At this point 'another_value' owns a string,
    // but 'my_value' is now invalid.
    println!("Now I own {}", another_value);
}

fn create() -> String {
    let new_str = String::from("hello, world");
    // Ownership will pass to the caller.
    new_str
}

fn transform(mut arg: String) -> String {
    // We've delcared the argument mutable, which is OK
    // since ownership has passed to us. We append some
    // text to it and then return it, whereupon ownership
    // passes back to the caller.
    arg.push_str("!!!");
    arg
}

For anyone puzzled by the bare expressions at the end of the functions on lines 15 and 24, suffice to say for now this is an idiomatic way to return a value in Rust. The language does have a return statement, but a bare expression also works in some cases. I’ll discuss this more in a later post.

So in the case of return values, the move semantics of ownership in Rust turn out to be pretty useful: the ownership passes to the caller safely and with no need for expensive copying, since somewhere under the hood it’s just a transfer of some reference to a value on the heap. Since the rules apply everywhere it all feels quite consistent and logical.

But as logical as it is, it may seem awfully inconvenient. There are many cases we want a value to persist after it has been operated on by a function. It would be annoying to have to deep-copy an object every time, or to constantly have to return the argument to the caller as in the example above.

Fortunately Rust provides references to resolve this inconvenience.

References¶

In Rust references provide a way to refer to a value without actually taking ownership of it. The example below demonstrates the syntax, which is quite reminiscent of C++:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

fn main() {
    let my_string = String::from("one two three");
    let num_words = count_words(&my_string);
    // 'my_string' is still valid here.
    println!("'{}' has {} words", my_string, num_words);
}

// I'm sure there are more elegant ways to implement
// this function, this is just for illustrating the point.
fn count_words(s: &String) -> usize {
    let mut words = 0;
    let mut in_word = false;
    for c in s.chars() {
        if c.is_alphanumeric() {
            if !in_word {
                words += 1;
                in_word = true;
            }
        } else {
            in_word = false;
        }
    }
    words
}

The code example above shows a value being passed by immutable reference. Note that the function signature needs to be updated to take a reference &String, but the caller must also explicitly declare the parameter to be a reference with &my_string. This is unlike in C++ where there’s no explicit hint to someone reading the code in the caller that a value might be passed by reference. For immutable references (or const refs in C++ parlance) this isn’t a big deal, but I’ve always felt that it’s always important to know for sure whether a function might modify one of its parameters in-place, and in C++ you have to go check the function signature every time to tell whether this is the case. This has always been one of my biggest annoyances with C++ syntax and it’s great to see it’s been addressed in Rust.

Taking a reference is rather quaintly known as borrowing in Rust. You can take as many references to a value as you like as long as they’re immutable.

1
2
3
4
5
6

fn main() {
    let mut my_value = String::from("hello, world");
    let ref1 = &my_value;
    let ref2 = &my_value;
    let ref3 = &my_value;
}

Of course, attempting to modify the value through any of these references will result in a compile error, since they’re immutable. As you’d expect it’s also possible to take mutable references:

1
2
3
4
5
6
7
8
9

fn main() {
    let mut my_value = String::from("world");
    prefix_hello(&mut my_value);
    println!("New value: {}", my_value);
}

fn prefix_hello(arg: &mut String) {
    arg.insert_str(0, "hello ");
}

This example also illustrates that it’s once again clear in the context of the caller that it’s specifically a mutable reference that’s being passed.

This all seems great, but there’s a couple of restrictions I haven’t mentioned yet. Firstly, it’s only valid to have a single mutable reference to a value at once. If you try to create more than one you’ll get an error at compile-time. Secondly, you can’t have both immutable and a mutable reference valid at the same time, which would also be a compile-time error.

The logic behind this is around safety when values are used concurrently. These rules do a good job of ruling out race conditions, as it’s not possible to multiple references to the same object unless they’re all immutable, and if the data doesn’t change then there can’t be a race. It’s essentially a multiple readers/single writer lock.

The compiler also protects you against creating dangling references, such as returning a reference to a stack function. That will fail to compile³.

A slice of life¶

Whilst I’m talking about references anyway, it’s worth briefly mentioning slices. These are like references, but they only refer to a subset of a collection.

1
2
3
4
5
6

fn main() {
    let my_value = String::from("hello there, world");
    // String slice 'there'.
    let there = &my_value[6..11];
    println!("<<{}>>", there);
}

The example above shows a use for an immutable string slice. Actually you may not realised it but you’ve seen one of those earlier in this post: all string literals are in fact immutable string slices.

As with slices in most languages the syntax is a half-open interval where the first index is inclusive, the second exclusive. It’s also possible to have slices of other collections that are contiguous and it’s possible to have mutable slices as well.

1
2
3
4
5
6
7

fn main() {
    let mut my_list = [1,2,3,4,5];
    let slice = &mut my_list[1..3];
    slice[1] = 99;
    // [1, 2, 99, 4, 5]
    println!("{:?}", my_list);
}

As far as I’ve been able to tell so far, however, it doesn’t seem to be possible to assign to the entirity of a mutable slice to replace it. I can understand several reasons why this might not be a good idea to implement, not least of which that it can change the size of the slice and hence necessitate moving items around in memory that aren’t even part of the slice (if you assign something of a different length). But I thought it was worth noting.

Conclusions¶

In this post I’ve summarised what I know so far about ownership and references in Rust and generally I think it’s shaping up to be a pretty sensible language. Of course it’s hard to say until you’ve put it to some serious use⁴, but I can see that there are good justifications for the quirks that I’ve discovered so far, bearing in mind the overarching goals of the language.

The ownership rules seem simple enough to keep in mind in practice, and it remains to be seen whether they will make writing non-trivial code more cumbersome than it needs to be. I like the explicit reference syntax in the caller and whilst the move semantics might seem odd at first, I think they’re simple and consistent enough to get used to pretty quickly. The fact that the compiler catches so many errors should be particularly helpful, especially as I’ve found its output format to be pleasantly detailed and particularly helpful compared to many other languages.