How AI tools like Groq work
Understanding the engine under the hood — without needing a PhD.
What is a Large Language Model?
A Large Language Model (LLM) is an AI system trained on enormous amounts of text. During training, it learns patterns in language — how words relate to each other, how sentences are structured, how questions are typically answered. When you send it a message, it predicts the most statistically likely next tokens (chunks of text) one at a time, which is how it generates a response.
An LLM doesn't understand meaning the way humans do — it predicts what text should come next based on everything it learned during training. But the results are good enough to be remarkably useful.
What makes Groq different
Most LLMs run on GPUs (Graphics Processing Units), which are powerful but designed for parallel tasks like gaming and image rendering. Groq built a custom chip called an LPU (Language Processing Unit) specifically designed to run language models. The result is inference that's dramatically faster than GPU-based alternatives. This is why MG Labs uses Groq — it makes the AI feel instant, which is critical for a good user experience.
How API calls actually work
When you build an AI app, you don't run the model on your own computer. You make an API call — a structured request to a server that runs the model. You send a JSON object with your messages and configuration, and you get back a JSON object with the AI's response. That's the entire technical interface. Everything else is just code that wraps these calls.
What an API call looks like
// This is what happens behind the scenes when you use MG AI
const response = await fetch("https://api.groq.com/openai/v1/chat/completions", {
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json"
},
body: JSON.stringify({
model: "llama-3.3-70b-versatile",
messages: [
{ role: "system", content: "You are a helpful assistant." },
{ role: "user", content: "What is an API?" }
]
})
});
const data = await response.json();
const aiReply = data.choices[0].message.content;The two types of messages
Every API call includes two types of messages. System messages define the AI's behavior — its personality, rules, and context. User messages are the actual conversation turns. Together they form a 'prompt' — the complete input the AI uses to generate its response. Understanding this distinction is foundational to everything you'll build.
System messages control behavior
- Sets the AI's role and personality
- Defines rules and constraints
- Gives the AI context about what it's for
- Example: "You are a customer support bot for a coffee shop. Be friendly and concise."
Open MG AI and have a normal conversation. Then imagine you're setting the system prompt to: 'You are a strict business advisor. Always respond with 3 bullet points. Never use casual language.' Notice how the same model would behave completely differently. That difference — between system prompt and user message — is where AI product design happens.