(De)serialization

All types stored inside the database are either bytes already or are perfectly bitcast-able.

As such, they do not incur heavy (de)serialization costs when storing/fetching them from the database. The main (de)serialization used is bytemuck's traits and casting functions.

Size and layout

The size & layout of types is stable across compiler versions, as they are set and determined with #[repr(C)] and bytemuck's derive macros such as bytemuck::Pod.

Note that the data stored in the tables are still type-safe; we still refer to the key and values within our tables by the type.

How

The main deserialization trait for database storage is Storable.

When a type is casted into bytes, the reference is casted, i.e. this is zero-cost serialization.

However, it is worth noting that when bytes are casted into the type, it is copied. This is due to byte alignment guarantee issues with both backends, see:

Without this, bytemuck will panic with TargetAlignmentGreaterAndInputNotAligned when casting.

Copying the bytes fixes this problem, although it is more costly than necessary. However, in the main use-case for cuprate_database (tower::Service API) the bytes would need to be owned regardless as the Request/Response API uses owned data types (T, Vec<T>, HashMap<K, V>, etc).

Practically speaking, this means lower-level database functions that normally look like such:

#![allow(unused)]
fn main() {
fn get(key: &Key) -> &Value;
}

end up looking like this in cuprate_database:

#![allow(unused)]
fn main() {
fn get(key: &Key) -> Value;
}

Since each backend has its own (de)serialization methods, our types are wrapped in compatibility types that map our Storable functions into whatever is required for the backend, e.g:

Compatibility structs also exist for any Storable containers:

Again, it's unfortunate that these must be owned, although in the tower::Service use-case, they would have to be owned anyway.

Last change: 2024-11-18, commit: e8598a0