====================================================================

The GPU Banana Stand

Published: 20 August 2022
Tags: gpu, web

GPU programming is notorious for not being very practical. This is even more-so the case when you're trying to use it on the web. In this article, Steven Wittens presents some of the functionality and features of his "declarative/reactive WebGPU library", Use.GPU.
Some key takeaways are:

• The React derived runtime, Live, is completely self-hosted and is fully rewindable and memoized
• Compute shaders are fully supported
• All of the elements are just components
• WebGPU has made things that were impractical with WebGL practical
• By sacrificing some performance, the implementation and use of the library has become far simpler
• The library has quality-of-life components out of the box, but you can still "crack them open and use the raw parts if they're not right for you"
• Copying how popular software in one domain (this case React) works to another domain can lead to intuitive and easily adopted interfaces

====================================================================

How SQLite Scales Read Concurrency

Published: 23 August 2022
Tags: sqlite

SQLite has historically been considered unusable for anything beyond a single user hobby project. That's no longer the case though, thanks to the introduction of a write-ahead log (WAL) in 2010, as an option instead of the rollback journal. Ben Johnson's article goes deeply into how the journaling method works.
Some key takeaways are:

• Rollback journal works "by copying the old version of changed pages to another file so that they can be copied back to the main database file if the transaction rolls back"
• WAL "writes the new version of a page to another file and leaves the original page in-place in the main database file"
• WAL has to be enabled via config option PRAGMA journal_mode = wal;
• For WAL, write transactions can occur without interfering with reads as a result of pages not being overwritten
• Checkpointing (when SQLite copies the latest version of each page in the WAL back into the main database file) ensures the WAL doesn't get too long
• SQLite uses a "shared memory" file to store an index for speeding up the look up of the latest version of a page for every transaction
• Thanks to WAL, SQLite can be used for more than just hobby projects

====================================================================

Fixed Partitions

Published: 22 August 2022
Tags: distributed systems

A fairly complicated problem with distributed systems is keeping track of where all of the data is stored, while also keeping things performant. This becomes an even bigger issue when there's the potential for a lot of data to be added in the system's lifetime. Unmesh Joshi presents a method of setting up a distributed system with data mapped to node (logical) partitions, keeping the necessary copying of data to a minimum when new nodes are added.
Some key takeaways are:

• Look up keys are the hash of the data item key and its partition
• Instantiate the system with a large number of partitions (3-10x more than the number of nodes)
• Data storage/retrieval is a two step process: find partition of data item -> find cluster node storing partition
• Key hash function should be runtime independent (MD5 or Murmur)
• Redistributing data among the nodes becomes a matter of just copying relatively small partitions
• Use a "dedicated Consistent Core as a coordinator which keeps track of all nodes in the cluster and maps partitions to nodes"
• Meta data should be kept on all cluster nodes so the Consistent Core controller isn't the bottleneck for querying

Your referrals: