(De)serialization
All types stored inside the database are either bytes already or are perfectly bitcast-able.
As such, they do not incur heavy (de)serialization costs when storing/fetching them from the database. The main (de)serialization used is bytemuck
's traits and casting functions.
Size and layout
The size & layout of types is stable across compiler versions, as they are set and determined with #[repr(C)]
and bytemuck
's derive macros such as bytemuck::Pod
.
Note that the data stored in the tables are still type-safe; we still refer to the key and values within our tables by the type.
How
The main deserialization trait
for database storage is Storable
.
- Before storage, the type is simply cast into bytes
- When fetching, the bytes are simply cast into the type
When a type is casted into bytes, the reference is casted, i.e. this is zero-cost serialization.
However, it is worth noting that when bytes are casted into the type, it is copied. This is due to byte alignment guarantee issues with both backends, see:
Without this, bytemuck
will panic with TargetAlignmentGreaterAndInputNotAligned
when casting.
Copying the bytes fixes this problem, although it is more costly than necessary. However, in the main use-case for cuprate_database
(tower::Service
API) the bytes would need to be owned regardless as the Request/Response
API uses owned data types (T
, Vec<T>
, HashMap<K, V>
, etc).
Practically speaking, this means lower-level database functions that normally look like such:
#![allow(unused)] fn main() { fn get(key: &Key) -> &Value; }
end up looking like this in cuprate_database
:
#![allow(unused)] fn main() { fn get(key: &Key) -> Value; }
Since each backend has its own (de)serialization methods, our types are wrapped in compatibility types that map our Storable
functions into whatever is required for the backend, e.g:
Compatibility structs also exist for any Storable
containers:
Again, it's unfortunate that these must be owned, although in the tower::Service
use-case, they would have to be owned anyway.