====================================================================
Published: 22 November 2022
Tags: data, data processing, database, distributed systems, python
Mark Veidemanis explains his work on Pathogen, a data analytics pipeline. Mark also mentions Sandstorm, which is a cool open source platform for self-hosting utility apps.
Some highlights:
- Concurrency and threads are hard
- Finding the correct tool/library is hard
- "There's always another millisecond to shave off the execution time, but how long are you going to spend doing it?"