Articles on Ai
Last updated: 2023/02/20
Top deep-dives on Ai
Sailesh Mukil uses ChatGPT to write a Redis client, documenting all of the prompts and outputs along the way.
- ChatGPT made a working Redis client
- ChatGPT has a good understanding of technical jargon
- ChatGPT has the capability to translate code it has written into many different languages with a simple prompt (allegedly, since it only did it partially at the end of the article)
Xinyu Hu, Olcay Cirit, Tanmay Binaykiya, and Ramit Hora present how Uber developed a "low-latency deep neural network architecture for global ETA prediction", focusing on "learnings and design choices".
Xavier Amatriain has created a catalog of all of the transformer models that have been created in the past few years. The catalog includes the name of the model, a brief description, the date it was created, and any relevant links.
- A timeline of the transformer models, highlighting the ones Xavier consider to be noteworthy.
- Explanation of what transformers are
- Lots of good resources for different models
Steven Pinker and Scott Aaronson discuss the future of the GPT-n language models.
Mattsi Jansky discusses some of the limitations of ChatGPT. I personally agree with his conclusion that it'll eventually become a tool used by people writing code to accelerate development, but will still require careful scrutiny. Now that I think of it, sounds like it could also have an application for helping newbies learn to review code.
- Has no access to internet (so no real-time data)
- It usually produces incorrect or buggy code that requires careful scrutiny
- It takes what you say literally and isn't great with context
A little bit old, but more relevant than ever with the recent explosion in the AI as a service industry. David Chapman discusses how there is no one agreed-upon set of criteria for what counts as progress in the field of artificial intelligence.
- AI has always borrowed criteria, approaches, and specific methods from at least six fields: science, engineering, mathematics, philosophy, design, and spectacle
- Because the criteria are incommensurable, they suggest divergent directions for research and produce sharp disagreements about what methods to apply
- "Meta-rationality means figuring out how to use technical rationality in specific situations"
Ferenc Huszár uses a paper by Sang Michael Xie, Aditi Raghunathan, Percy Liang and Tengyu Ma as a basis for a discussion on how large language models kind of learn to learn.
Laurence Tratt elaborates on how generative AI can be used for programming.
- "Programming turns specifications into software"
- "Programming is unforgiving of approximations", which makes extra work if models produce the incorrect output
- The generative models can be useful for some techniques, like fuzzing
Accelerating Queries over Unstructured Data with ML, Part 5 (Semantic Indexes for Machine Learning-based Queries over Unstructured Data)
Daniel Kang, John Guibas, Peter Bailis, Tatsunori Hashimoto, and Matei Zaharia "propose TASTI, a method for constructing indexes for unstructured data" for machine learning.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang present a novel implementation of a transformer to help make machine learning models that depend on time-series data better at forecasting.