Welcome back to the next addition and thank you for the amazing feedback for last weeks post! This last week I finished the book The Alignment Problem by Bran Christian, an incredible read that has gained even more significance with the increased relevance of Reinforcement Learning. If you are into non-fiction books this one is well written.
Let’s get right to it this weeks edition.
News
This week was a lot slower than last weeks but there were still some great new things that came out:
Mistral has released its API for building AI agents. Providing a bunch of built-in connectors and support for MCP tools.
Anthropic open-sources its circuit tracing tools. These tools help to generate attribution graphs to help with interpretability research on how specific answers are formed in an LLM.
During Google I/O Veo 3, a video generation model, was made avaiable and its videos have poped up all over social media. Now some TikTokers are taking real videos and claiming for them to be AI in order to drive clicks.
Articles
Here are again three articles/papers that have helped to shape my thinking over the last week:
An incredibly thoughtful and deep piece on AI Feature Design. If you are not just using AI, but are thinking of building products that are being enhanced by it, than you have to read this.
Speaking of building AI products, there is few things more important than Security, so thinking and learning about how to protect against new and unique attack vectors is crucial. This paper dives into lessons learned around Gemini.
The paper introduces Chain-of-Model Learning (CoM), a framework that enables dynamic, multi-scale, and causal modeling in Transformers via chain-based representation. Now while I don’t have a direct application for this technique, I believe it is a good idea to keep taps on what is to come.
Deep Dive
Okay, I hope you still have your API-key handy, because it is time to get back into things.
First I want to address a quesiton that I got: “Why aren’t you using the Google Gemini package?”. This question refers to the Google GenAI SDK. The reason for not using it is pretty simple, I believe that using the API directly via HTTP requests enables us to understand some of the implementation details better. This isn’t because I don’t like that package, but rather to stick with the fundamentals.
You can find a demonstration of this deep in this YouTube video.
With that, let us dive into todays topics and that is types of messages. Last time you might remember that in our example request we send the following to the model:
{
"contents": [
{
"parts": [
{
"text": "Explain how AI works"
}
]
}
]
}Now that might have felt very similiar to just typing into a chat box for an LLM, but with the API we can do more. Instead of just sending the normal text that a user might type, we can also provide system instructions. The system instructions or often refered to as a system prompt influences how the LLM response to the user request. This is part of a larger topic called Prompt Engineer which we will touch more on in the future. For now lets focus on the system prompt.
Let’s add a system prompt to our example call and compare how that changes the result:
{
"system_instructions": {
"parts": [
"text": "You are a pre-school teacher that specialies in breaking down complex topics into easy to digest answer that are appropriate for five year olds."
]
},
"contents": [
{
"parts": [
{
"text": "Explain how AI works"
}
]
}
]
}Without the system prompt I got the following response:
AI learns from data to make predictions or decisions.With the system prompt I got:
Okay, imagine you have a super smart puppy! You teach it tricks by showing it lots and lots of examples. AI is kind of like that puppy. We show the computer lots of pictures, words, or sounds, and it learns the patterns and then can do new things all by itself! So, AI is like a puppy that learns from examples!As you can see these answers very widly. The good thing about using system prompts is that you can set a certain answering behavior and pattern for the LLM with that. But they aren’t gurantees that the model will always respond the way you want.
With the changed input template from above it is fairly easy to extend our code from last week in order to also use a system prompt. The updated code is again available in both Python and SAS. If you get a 400 error code while running this, it is most likely because you are not using a valid API key, so please make sure to check that.