RAG with LLMs: Easy to POC, Hard to Iterate On.
The paradox of LLMs is that they have made POCs (Proofs of Concept) incredibly easy to create, but at the same time, they have made iterating on the product ...
Recent Posts
Google’s Python Style Guide for Data Scientists
Most Data Scientists have learned to code well on the job. Indeed, higher education generally focuses more on theory (mathematics, statistics, etc.) rather t...
The Role of Language in Reasoning and Conceptualization
Yann LeCun's recent statements about language not being necessary for reasoning are as fascinating as they are divisive. Among other things, he emphasizes th...
Managing a Mainstream Open Source Library is Tough
Managing a mainstream open-source library is a full-time job that requires implementing an efficient workflow to avoid being overwhelmed. This is particu...
Is There a Future for On-Device Open Source LLMs?
One obstacle to using open source LLMs is the cost of hosting them on GPU instances in the cloud. But perhaps there is a future for on-device open source? ...
Why bigger context windows won’t make RAG disappear.
We sometimes hear that RAG (Retrieval Augmented Generation) is destined to disappear because LLMs have an increasingly large context window and can therefore...
Padding strategy matters when Fine tuning LLMs
A poor padding strategy can disrupt an LLM after fine-tuning, and the reasons are quite subtle. This is especially true since there is no pad_token in the Ll...
Navigating the Jungle of LLM Quantization Formats, GGUF, GPTQ, AWQ and EXL2 - Which One To Pick?
Quantizing LLMs reduces calculation precision and thus the required GPU resources, but it can sometimes be a real jungle trying to find your way among all th...
RAG vs Finetuning - Complementary not Opposing
RAG and finetuning are often opposed: Should we fine-tune our LLM or do RAG? In reality, the approaches do not oppose each other. On the contrary, they compl...
Assessing the Criticality of AI Failures
What is the criticality of a bad prediction from your algorithm? In my opinion this is one of the first questions to ask before considering a generative AI o...
The Non-Determinism of OpenAI and Anthropic Models
OpenAI and Anthropic's models are not deterministic, even at temperature zero and with a fixed seed, and this can be a real problem. How is it that some...
Math Skills vs. Pragmatism in Data Science
Math skills are often considered fundamental to becoming a data scientist. This is true, but we often forget to mention that in practice, machine learning is...
The Decline of TensorFlow, The Rise of PyTorch
TensorFlow was once essential, but I see fewer and fewer reasons to use it today. PyTorch seems to be establishing itself as the go-to deep learning framewor...
From Classic ML to LLMs: Similar Development Trajectories?
It's interesting to see how certain patterns in Machine Learning seem to recur, including with LLMs: Classic ML: </p> Step 1: The nature of the mo...
4 Simple Tips for Improving RAG Performance
Here are four simple tips for improving the performance of your RAG (Retrieval Augmented Generation): </p> Understand the RAG: The RAG combines a "R...