AI Periodic Table Explained: Mapping LLMs, RAG & AI Agent Frameworks

Does the world of AI feel a bit like this to you? A thousand terms all flying around. Everyone's talking about agents and RAG and embeddings and guardrails, and you're just kind of supposed to know how it all fits together.

Well, how about we put a structure to all this chaos? What if we could organize AI like chemistry organizes the elements into families and periods and predictable reactions? Well, welcome to the AI periodic table.

Now, a quick disclaimer. There is no official AI periodic table like there is in chemistry. This is my take on what the structure could look like, but once you understand it, you can basically decode any AI architecture, any product demo, any event, the pitch.

You'll see which elements they're using, how they connect, and then maybe even what might be missing. So let's fill in this table starting at the top. So, you know what a prompt is, right?

Well, that's my first element, Pr. Now a prompt contains instructions that you give to an AI, like write me an email or summarize this document or explain quantum physics. And where this sits actually matters.

So this element sits in a particular row, row number 1. And row number 1, this represents primitives. Now you can't really break prompting down any further than this.

You could say that it's atomic. And then, this also sits in a column as well, which we're going to call groups or families. So this is G1.

And G1 is reactive, because prompts are reactive. You change one word in your prompt. Well you're going to get a completely different output.

So that's the, uh, the made up element of Pr for prompting. But prompting isn't alone in this group. It has a family of elements that get more reactive as you go down.

And it's not alone in this row either. So we're going to map out five families across the top. And then we've got four rows down the side.

And then you see these empty spaces here. Well, they're there for a reason and we'll get to that. But let's fill in the rest of the primitives, row number 1.

So what's the next element? Well the next element is Em. That's for embeddings.

So if you've heard of vector databases or semantic search, you've probably bumped into embeddings because they are numerical representations of meaning. So you take some text, like the cat sat on the mat, and you can turn that text into a list of numbers that capture its meaning, and similar meanings get similar numbers. It's still in row 1.

It's a primitive, but this is in group two now, and we're going to call group two the retrieval family, because now we are actually retrieving stuff. Embeddings are how AI systems search and remember. Okay now, we're going to skip ahead to the end here to the next element.

That's the element of Lg, the large language model, the LLM, you know, like ChatGPT and Claude and IBM Granite, you know, these. And this goes all the way over here into this family, which we're calling G5. And G5 is the models family.

These are well, kind of the noble gases, these stable foundational capabilities that everything else reacts around. And notice that row 1 here, it only has three elements prompts and embeddings and LLMs. That's it.

And that's because in some ways, everything else in AI is built from combining these primitives. All right. Let's move on to row number 2 now.

And the first element here is the element of Fc. That's function calling. So this is when your LLM calls a tool before giving an answer.

So perhaps it invokes an API. If you ask a model what's the weather. Well the model is going to call a weather API to give you some real data.

So it's still in the reactive family group one it's making things happen and taking action, but it's not a primitive anymore. So this is row 2. And row 2 is all about compositions.

You need a model and structured output and tool integration that makes a composition, and watch how the reactive family continues to evolve. So down here, we've got the element Ag. That's agents.

Now these use think-act-observe loops. So the AI plans it, takes an action maybe using function calls, and it observes the result. And it keeps going until it reaches a goal.

And that is row 3, which is all about deployment, putting these things actually into production. So look at the progression we've got here from prompt to function to agent. Or we could say from control to action to autonomy.

That's the reactive family. All right, let's fill in the rest of row 2 here. So the next element that is the element of Vx, vector databases.

These are data stores optimized for semantic search. You can store millions of embeddings and then query them to find the most relevant ones. This is group two now the retrieval family.

Because vector databases are storage for semantic search. You compose embeddings, that's why it's row 2, into a vector store. Okay.

What's next? Well, the element of Rg, that's RAG, retrieval augmented generation, one of my favorites. done a bunch of videos on this.

So you have a question. The system retrieves relevant context from your documents using embeddings and vector databases. Then, it augments the prompt with that context.

Then, the LLM can generate an answer based on what it retrieved. So where does that go? Which family?

It's going to go under the family of G3, which is the orchestration family? Because RAG orchestrates multiple elements together, embeddings and vector stores and models. And notice that there is no primitive here in row 1.

Because you can't really orchestrate one thing. Orchestration only emerges when you're combining multiple pieces. Okay, next in row 2, we have the element of Gr.

That's guardrails. So we're talking runtime, safety filters, maybe schema validation. Basically just making sure that the AI doesn't say something that it shouldn't do or just kind of output pure garbage.

And that sits under the group four, which is all about validation. And then rounding out row 2, the last element is Mm. That's for multi-modal models.

So these are LLMs that can kind of process images. And they can process audio as well as being able to read text. So it's still column five, the models family.

And that is row 2 complete. We've got five compositions that honestly, basically power most AI systems today. These guys.

All right, let's finish row 3. So we had Ag for agent. The next element is Ft for fine tuning.

So you take a base model, and then, you adapt it. You train it on your specific data, on your domain, on your use case, like fine tuning on medical papers or on your company's codebase. Now it's under the retrieval family because fine tuning is adaptation.

It's baking memory directly into the model's weights. So look at that column. Now we've got embeddings, which encode meaning.

We've got vector databases, which store for search. And then we've got fine tuning, which stores in parameters. So three time scales of memory.

Okay. Next up is Fw. That stands for framework.

So we're talking about things like a LangChain. These are the platforms that tie everything together to build and deploy AI systems. So very much under the orchestration family.

Next is Rt. That's red teaming. This is adversarial testing where we're basically trying to break the AI.

So jailbreaks and prompt injection and data exfiltration. It's under the validation family. And then under models, let's also throw in the element of Sm.

That's for small models. Distilled specialized models. They're fast, they're cheap.

They maybe run on your phone. All right, one last row. And first up is the element of Ma.

That is multi-agent, multi-agent systems. So this is not one AI. It's multiple AIs that are all working together.

They're debating and collaborating and specializing. So maybe one agent does the research, one does the writing, one critiques all of it, and they coordinate to solve complex problems. So this is a new row.

It's row number 4. And I'm going to call row number 4 emerging. Now this is not science fiction.

It's stuff that's happening now but it is still rapidly evolving. This is the reactive family kind of taken to the extreme right. So what else can we add to this row?

Well, I would argue we could put Sy here for synthetic data. This is using AI to generate training data for AI, which yeah, sounds kind of weird, but it works. if you can't get enough real examples, you can generate synthetic ones now.

This is not new, but it is emerging because as we hit the limits of available data for AI to train on, more and more is being done with synthetic data. All right. Notice there's a gap here.

I think there's really no clear emerging orchestration paradigm yet, at least to my eyes. What goes beyond frameworks here? Well, I should be curious to to learn what you think would fill in that gap at some point.

Anyway, moving on, the next element is In. The interpretability. So this is about understanding why a model does what it does, kind of peaking inside of the black box and finding the neurons responsible for specific behaviors.

And it's in the validation family as this is frontier safety work. And then rounding things out, we have the element of Th. This is for thinking models, models that don't answer immediately.

They spend time reasoning. There's basically a chain of thought built directly into the architecture. Test time compute scaling.

These are the smartest models today. Those are the ones that tend to be thinking models. So there it is, the AI periodic table.

At least my attempt at it. But periodic tables aren't just for memorization. They're for predicting reactions.

And let me show you how these elements combine. So I'm going to show you two reactions. But any AI system could be modeled here.

So first let's build a chatbot that knows your company's documentation. It's one of the most common patterns in production AI. So it starts with the element Em, embedding.

So you take your documents and you turn them into vectors. And then, you store them. Where do you store them?

In the element of Vx, into vector databases. Now when a user asks a question, we use RAG. That's the element Rg.

And that is going to query the vector database and retrieve the relevant chunks. And those chunks augment the prompt here the Pr element. And that gets sent to the large language model, the Lg to generate an answer that's grounded in your data.

So we've got these five elements here, all combining. And actually smart companies are going to probably add in one more element as well. And that is the element Gr of guardrails.

Because this kind of wraps the whole thing in safety filters and makes sure that the model isn't going to be leaking sensitive information. So that that's production RAG. All right.

One of the reaction I wanted to show you, which is the agentic loop. So you give the AI a goal. Let's say the goal is book me a flight to Tokyo next month under $800.

So the element we're going to use for that is Ag. Now the Ag agent is going to take that goal, and it's going to break it down. So first, the agent needs to search flights, then check the calendar, then compare prices and then book.

So to do that it uses Fc, function calling. Function calling calls external tools. So the flight APIs, the calendar APIs and the payment systems.

And then, the agent observes the results and decides the next action. So this is basically a loop that goes back and forth here. Think.

Act. Observe. Think.

Act. Observe. So that's the start of the reaction.

But you might also build and deploy this using a framework. And that framework kind of gives you all of the plumbing you need for this loop. So we've got here the Ag element that's looping around Fc.

And it's deployed into Fw into the framework. That's the agentic reaction. And there are so many different reactions we could show here.

AI image generators, for example, which use the prompt as the text description, which is then sent to a multi-modal model to output an image or code assistants, which are fine tuned on code, which is then used to build a prompt with the context and is then processed by the large language model, the Lg to generate completions. They all basically fit somewhere in here. So here's your challenge.

Next time somebody pitches you with a fancy new AI feature, or a new AI product, or even an AI startup idea, try mapping it to this table. What elements are they using? What reactions are they running?

Are they missing a safety element? Are they over-engineering the orchestration? Are they using a thinking model when perhaps a small model would just do the job just as well?

It's, it's all right here, a way to categorize and link all of these AI terms we talked about at the beginning. So what do you think? Will we be seeing a version of this plastered on school classroom walls like the real periodic table at some point?

Let me know in the comments.