diff --git a/src/SUMMARY.md b/src/SUMMARY.md index dfd360edf6b4..d1793acad7fc 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -445,6 +445,11 @@ - [Serializer: implement Struct](idiomatic/leveraging-the-type-system/typestate-pattern/typestate-generics/struct.md) - [Serializer: implement Property](idiomatic/leveraging-the-type-system/typestate-pattern/typestate-generics/property.md) - [Serializer: Complete implementation](idiomatic/leveraging-the-type-system/typestate-pattern/typestate-generics/complete.md) + - [Borrow checking invariants](idiomatic/leveraging-the-type-system/borrow-checker-invariants.md) + - [Generalizing "Ownership"](idiomatic/leveraging-the-type-system/borrow-checker-invariants/generalizing-ownership.md) + - [Single-use values](idiomatic/leveraging-the-type-system/borrow-checker-invariants/single-use-values.md) + - [Aliasing XOR Mutability](idiomatic/leveraging-the-type-system/borrow-checker-invariants/aliasing-xor-mutability.md) + - [Lifetime Relationships & External Resources](idiomatic/leveraging-the-type-system/borrow-checker-invariants/lifetime-connections.md) --- diff --git a/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants.md b/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants.md new file mode 100644 index 000000000000..85128bb54429 --- /dev/null +++ b/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants.md @@ -0,0 +1,80 @@ +--- +minutes: 0 +--- + +# Using the Borrow checker to enforce Invariants + +The logic of the borrow checker, while tied to "memory ownership", can be +abstracted away from this central use case to model other problems and prevent +API misuse. + +```rust,editable +fn main() { + // Doors can be open or closed, and you need the right key to lock or unlock + // one. Modelled with a Shared key and Owned door. + pub struct DoorKey { + pub key_shape: u32, + } + pub struct LockedDoor { + lock_shape: u32, + } + pub struct OpenDoor { + lock_shape: u32, + } + + fn open_door(key: &DoorKey, door: LockedDoor) -> Result { + if door.lock_shape == key.key_shape { + Ok(OpenDoor { lock_shape: door.lock_shape }) + } else { + Err(door) + } + } + + fn close_door(key: &DoorKey, door: OpenDoor) -> Result { + if door.lock_shape == key.key_shape { + Ok(LockedDoor { lock_shape: door.lock_shape }) + } else { + Err(door) + } + } + + let key = DoorKey { key_shape: 7 }; + let closed_door = LockedDoor { lock_shape: 7 }; + let opened_door = open_door(&key, closed_door); + if let Ok(opened_door) = opened_door { + println!("Opened the door with key shape '{}'", key.key_shape); + } else { + eprintln!( + "Door wasn't opened! Your key only opens locks with shape '{}'", + key.key_shape + ); + } +} +``` + +
+ + + +- The borrow checker has been used to prevent use-after-free and multiple + mutable references up until this point, and we've used types to shape and + restrict use of APIs already using the "typestate" pattern. + +- This example uses the ownership & borrowing rules to model the locking and + unlocking of a door. We can try to open a door with a key, but if it's the + wrong key the door is still closed (here represented as an error) and the key + persists regardless. + +- The rules of the borrow checker exist to prevent developers from accessing, + changing, and holding onto data in memory in unpredictable ways without being + so restrictive that it would prevent _writing software_. The underlying + logical system does not "know" what memory is. All it does is enforce a + specific set of rules of how different operations affect what later operations + are possible. + +- Those rules can apply to many other cases: We can piggy-back onto the rules of + the borrow checker to design APIs to be harder or impossible to misuse, even + when there's little or no "memory safety" concerns in the problem domain. This + section will walk through some of those different domains. + +
diff --git a/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/aliasing-xor-mutability.md b/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/aliasing-xor-mutability.md new file mode 100644 index 000000000000..1f8fe79c1d33 --- /dev/null +++ b/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/aliasing-xor-mutability.md @@ -0,0 +1,89 @@ +--- +minutes: 0 +--- + +# Mutually Exclusive References, or "Aliasing XOR Mutability" + +We can use the mutual exclusion of `&T` and `&mut T` references for a single +value to model some constraints. + +```rust,editable,compile_fail +pub struct Transaction(/* some kind of interior state */); +pub struct QueryResult(String); + +pub struct DatabaseConnection { + transaction: Transaction, + query_results: Vec, +} + +impl DatabaseConnection { + pub fn new() -> Self { + Self { + transaction: Transaction(/* again, pretend there's some interior state */), + query_results: vec![], + } + } + pub fn get_transaction(&mut self) -> &mut Transaction { + &mut self.transaction + } + pub fn results(&self) -> &[QueryResult] { + &self.query_results + } + pub fn commit(&mut self) { + println!("Transaction committed!") + } +} + +pub fn do_something_with_transaction(transaction: &mut Transaction) {} + +fn main() { + let mut db = DatabaseConnection::new(); + let mut transaction = db.get_transaction(); + do_something_with_transaction(transaction); + let assumed_the_transactions_happened_immediately = db.results(); // ❌🔨 + do_something_with_transaction(transaction); + // Works, as the lifetime of "transaction" as a reference ended above. + let assumed_the_transactions_happened_immediately_again = db.results(); + db.commit(); +} +``` + +
+ +- Aliasing XOR Mutability means "we can have multiple immutable references, a + single mutable reference, but not both." + +- This example shows how we can use the mutual exclusion of these kinds of + references to dissuade a user from reading query results while using a + transaction API, something that might happen if the user is working under the + false assumption that the queries being written to the transaction happen + "immediately" rather than being queued up and performed together. + +- By borrowing one field of a struct under a mutable / exclusive reference we + prevent access to the other fields of that struct under a shared / + non-exclusive reference until the lifetime of that borrow ends. + +- As laid out in [generalizing ownership](generalizing-ownership.md) we can look + at the ways Mutable References and Shareable References interact to see if + they fit with the invariants we want to uphold for an API. + +- In this case, having the query results not public and placed behind a getter + function, we can enforce the invariant "users of this API are not looking at + the query results at the same time as they are writing to a transaction." + + +
+ +The "don't look at query results while building a transaction" invariant can still be circumvented, how so? + +
    +
  • + The user could access the transaction solely through `db.get_transaction()`, leaving the lifetime too temporary to prevent access to `db.results()`. +
  • +
  • + How could we avoid this by working in other concepts from "Leveraging the Type System"? +
  • +
+
+ +
diff --git a/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/generalizing-ownership.md b/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/generalizing-ownership.md new file mode 100644 index 000000000000..d8cd3b5e186f --- /dev/null +++ b/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/generalizing-ownership.md @@ -0,0 +1,69 @@ +--- +minutes: 0 +--- + +# Generalizing Ownership + +The logic of the borrow checker, while modelled off "memory ownership", can be +abstracted away from that use case to model other problems where we want to +prevent API misuse. + +```rust,editable,compile_fail +// An internal data type to have something to hold onto. +pub struct Internal; +// The "outer" data. +pub struct Data(Internal); + +fn shared_use(value: &Data) -> &Internal { + &value.0 +} +fn exclusive_use(value: &mut Data) -> &mut Internal { + &mut value.0 +} +fn deny_future_use(value: Data) {} + +fn main() { + let mut value = Data(Internal); + let deny_mut = shared_use(&value); + let try_to_deny_immutable = exclusive_use(&mut value); // ❌🔨 + let more_mut_denial = &deny_mut; + deny_future_use(value); + let even_more_mut_denial = shared_use(&value); // ❌🔨 +} +``` + +
+ +- This example re-frames the borrow checker rules away from references and + towards semantic meaning in non-memory-safety settings. Nothing is being + mutated, nothing is being sent across threads. + +- To use the borrow checker as a problem solving tool, we will need to "forget" + that the original purpose of it is to prevent mutable aliasing in the context + of concurrency & dangling pointers, instead imagining and working within + situations where the rules are the same but the meaning is slightly different. + +- In rust's borrow checker we have access to three different ways of "taking" a + value: + + + - Owned value `T`. Very permissive case, to the point where mutability can be + re-set, but demands that nothing else is using it in any context and drops + the value when scope ends (unless that scope returns this value) (see: + RAII.) + + - Mutable Reference `&mut T`. While holding onto a mutable reference we can + still "dispatch" to methods and functions that take an immutable, shared + reference of the value but only as long as we're not aliasing immutable, + shared references to related data "after" that dispatch. + + - Shared Reference `&T`. Allows aliasing but prevents mutable access while any + of these exist. We can't "dispatch" to methods and functions that take + mutable references when all we have is a shared reference. + +- Remember that every `&T` and `&mut T` has an _implicit lifetime._ We get to + avoid annotating a lot of lifetimes because the rust compiler can infer the + majority of them. See: + [Lifetime Elision](../../../lifetimes/lifetime-elision.md). + +
diff --git a/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/lifetime-connections.md b/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/lifetime-connections.md new file mode 100644 index 000000000000..03ffc3f12152 --- /dev/null +++ b/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/lifetime-connections.md @@ -0,0 +1,101 @@ +--- +minutes: 0 +--- + +# Lifetime "Connections" & External Resources + +Using `PhantomData` in conjunction with lifetimes lets us say "this value may +own its data, but it can only live as long as the value that generated it" in +rust's type system. + +```rust,editable,compile_fail +fn main() { + use std::marker::PhantomData; + pub struct Tag; + pub struct ErasedData<'a> { + data: String, + _phantom: PhantomData<&'a ()>, + } + impl<'a> ErasedData<'a> { + pub fn get(&self) -> &str { + &self.data + } + } + pub struct TaggedData { + data: String, + _phantom: PhantomData, + } + impl TaggedData { + pub fn new(data: String) -> Self { + Self { data, _phantom: PhantomData } + } + pub fn consume(self) {} + pub fn get_erased(&self) -> ErasedData<'_> { + // has an owned String, but _phantom holds onto the lifetime of the + // TaggedData that created it. + ErasedData { data: self.data.clone(), _phantom: PhantomData } + } + } + + let tagged_data: TaggedData = TaggedData::new("Real Data".to_owned()); + // Get the erased-but-still-linked data. + let erased_owned_and_linked = tagged_data.get_erased(); + tagged_data.consume(); + // Owned by `erased_owned_and_linked` but still connected to `tagged_data`. + println!("{}", erased_owned_and_linked.get()); // ❌🔨 +} +``` + +
+ +- `PhantomData` lets developers "tag" types with type and lifetime parameters + that are not "really" present in the struct or enum. + + `PhantomData` can be used with the Typestate pattern to have data with the + same structure i.e. `TaggedData` can have methods or trait + implementations that `TaggedData` doesn't. + + It can also be used to encode a connection between the lifetime of one value + and another, while both values still maintain separate owned data within them. + +- This is really useful for modelling a bunch of relationships between data, + where we want to establish that while a type has owned values within it is + still connected to another piece of data and can only live as long as it. + + Consider a case where you want to return owned data from a method, but you + don't want that data to live longer than the value that created it. + +- Lifetimes need to come from somewhere! We can't build functions of the form + `fn lifetime_shenanigans<'a>(owned: OwnedData) -> &'b Data` (without tying + `'b` to `'a` in some way). + + Lifetime elision hides where a lot of lifetimes come from, but that doesn't + mean the explicitly named lifetimes "come from nowhere." + + Suggestion: Show off un-eliding the lifetimes in `get_erased` in this example. + +- [`BorrowedFd`](https://rust-lang.github.io/rfcs/3128-io-safety.html#ownedfd-and-borrowedfdfd) + uses these captured lifetimes to enforce the invariant that "if this file + descriptor exists, the OS file descriptor is still open" because a + `BorrowedFd`'s lifetime parameter demands that there exists another value in + your program that has the same lifetime as it, and this has been encoded by + the API designer to mean _that value is what keeps the access to the file + open_. + + Its counterpart `OwnedFd` is instead a file descriptor that closes that file + on drop. + +- This way of encoding information in types is _exceptionally powerful_ when + combined with unsafe, as the ways one can manipulate lifetimes becomes almost + arbitrary. This is also dangerous, but when combined with tools like external, + mechanically-verified proofs _we can safely encode cyclic/self-referential + types while encoding lifetime & safety expectations in the relevant data + types._ + + The [GhostCell (2021)](https://plv.mpi-sws.org/rustbelt/ghostcell/) paper and + its [relevant implementation](https://gitlab.mpi-sws.org/FP/ghostcell) show + this kind of work off. While the borrow checker is restrictive, there are + still ways to use escape hatches and then _show that the ways you used those + escape hatches are consistent and safe._ + +
diff --git a/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/single-use-values.md b/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/single-use-values.md new file mode 100644 index 000000000000..cfcb1bf19dc6 --- /dev/null +++ b/src/idiomatic/leveraging-the-type-system/borrow-checker-invariants/single-use-values.md @@ -0,0 +1,62 @@ +--- +minutes: 0 +--- + +# Single-use values + +In some circumstances we want values that can be used _exactly once_. One +critical example of this is in cryptography: "Nonces." + +```rust,editable,compile_fail +fn main() { + mod cryptography { + pub struct Key; + // Pretend this is a cryptographically sound, single-use number. + pub struct Nonce(u32); + // And pretend this is cryptographically sound random generator function. + pub fn new_nonce() -> Nonce { + Nonce(std::time::UNIX_EPOCH.elapsed().unwrap_or_default().subsec_nanos()) + } + + // We consume a nonce, but not the key or the data. + pub fn encrypt(nonce: Nonce, key: &Key, data: &[u8]) {} + } + + use cryptography::*; + + let nonce = new_nonce(); + let data_1: [u8; 4] = [1, 2, 3, 4]; + let data_2: [u8; 4] = [4, 3, 2, 1]; + let key = Key; + + // The key and data can be re-used, copied, etc. but the nonce cannot. + encrypt(nonce, &key, &data_1); + encrypt(nonce, &key, &data_2); // 🛠️❌ +} +``` + +
+ +- Owned "consumption" of values lets us model things that need to be single-use. + +- By keeping constructors private and not implementing clone/copy for a type, + making the interior type opaque (as per the newtype pattern), we can prevent + multiple uses of the same, API-controlled value. + +- In the above example, a Nonce is a additional piece of random, unique data + during an encryption process that helps prevent "replay attacks". + + - In practice people have ended up re-using nonces in circumstances where + security is important, making it possible for private key information to be + derived by attackers. + + - By tying nonce creation and consumption up in rust's ownership model, and by + not implementing clone/copy on sensitive single-use data, we can prevent + this kind of dangerous misuse. + + - Cryptography Nuance: There is still the case where a nonce may be used twice + if it's created through purely a pseudo-random process with no additional + metadata, and that circumstance can't be avoided through this particular + method. This kind of API prevents one kind of misuse, but not all kinds. + +