Concurrency and parallelism
It is incumbent upon software like Cuprate to take advantage of today's highly parallel hardware as much as practically possible.
With that said, programs must setup guardrails when operating in a concurrent and parallel manner, for correctness and safety.
There are "synchronization primitives" that help with this, common ones being:
These tools are relatively easy to use in isolation, but trickier to do so when considering the entire system. It is not uncommon for the bottleneck to be the poor orchastration of these primitives.
Analogy
A common analogy for a parallel system is an intersection.
Like a parallel computer system, an intersection contains:
- Parallelism: multiple individual units that want to move around (cars, pedestrians, etc)
- Synchronization primitives: traffic lights, car lights, walk signals
In theory, the amount of "work" the units can do is only limited by the speed of the units themselves, but in practice, the slow cascading reaction speeds between all units, the frequent hiccups that can occur, and the synchronization primitives themselves become bottlenecks far before the maximum speed of any unit is reached.
A car that hogs the middle of the intersection on the wrong light is akin to a system thread holding onto a lock longer than it should be - it degrades total system output.
Unlike humans however, computer systems at least have the potential to move at lightning speeds, but only if the above synchronization primitives are used correctly.
Goal
To aid the long-term maintenance of highly concurrent and parallel code, this section documents:
- All system threads spawned and maintained
- All major sections where synchronization primitives are used
- The asynchronous behavior of some components
and how these compose together efficiently in Cuprate.