Reading
A running list of books, papers, and articles I'm currently reading or recently read.
Recent
-
Interesting article on boundaries on copyright. Titled: This monkey selfie will protect you from AI slop
-
Improving the underlying architecture of training models. “The most significant leap in inference efficiency comes from the transition from Single-Input, Single-Output (SISO) to Multi-Input, Multi-Output (MIMO) SSMs.”
-
Technique to train smaller models from large models
-
It is becoming clear that Small Language Models are marginally better when the underlying parameters are about the same. Results from this paper claim that the models generally have 1% deviation from each other in results.
-
Bringing long-term memory to inference
-
Great lectures by Andrew Ng and the Stanford team. Thank you to Stanford for making this available to the public.
-
There is a general notion that LLMs are nondeterministic due to concurrency and floating points roundoff on the GPU. This paper revisits that idea that LLMs can become deterministic.