Nate's Newsletter · Tech & AI
TIER 4 Sat, 26 Jul 2025 13:01:10 +0000
Watch now (13 mins) | These are the eye-glazing concepts that I find most people need to unlock to 'get' AI: Tokenization, jagged intelligence, and prompt sizing--finally explained so clearly you'll be able to teach them! ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ | | ---|---|--- | | | Forwarded this email? Subscribe here for more --- Thanks for supporting Nate's Newsletter! This post is free and clear for all subscribers, just like the sky (but not in Seattle--it's always cloudy here). * * * --- --- | | Watch now --- # Confused by AI? Nail These 3 Concepts First ### These are the eye-glazing concepts that I find most people need to unlock to 'get' AI: Tokenization, jagged intelligence, and prompt sizing--finally explained so clearly you'll be able to teach them! | | Nate --- | Jul 26 --- | --- --- | | | --- | | --- | | --- | | --- | | READ IN APP --- _For this week 's free post, I wanted to focus on three of the most misunderstood concepts I see when I talk about AI: tokenization, jagged intelligence, and the idea that big prompts are always better._ _I picked these three because I find that when I go from eyes-glazed to "oh!" with these three, people often unlock a lot of the rest of AI on their own. Think about it: if you understand how AI processes data, where it's good at handling that data, and how to communicate intent--well that's the big picture isn't it? _ _A lot of the rest of the AI concepts can fall into place once you get those bit puzzle pieces down. It 's like when I'd do 2000 word puzzles with the family on vacation as a kid and we'd always start with the corner pieces. These are the corner puzzle pieces of AI._ Subscribers get all these newsletters! Upgrade to paid * * * # Three AI Concepts That Will Change How You Work (And Why They're So Confusing) Let's get right to it! What are these three concepts, and why do they matter? ## 1\. What Is Tokenizable Data? I talk about tokenizable distributions, and I can see people's eyes glaze over. What does tokenize mean? Very simply, ask yourself if a piece of data could appear in a document. If it could, that's a good sign it's probably tokenizable. If you can't imagine it fitting in a document, it's probably not tokenizable. (Nvidia has a great token explanation here.) When you ask if AI can do something, people usually think about the task as a whole. But I always start with the data and tokens first. Can I even fit it in? Can the tokens go into the system? Then we get to subsequent questions: Is there too much data? Is it too big for the context window? Is the task too multifaceted for a single prompt? Or is it too complex for AI to handle with nuance? AI often polishes off the nuance in a task. Understanding whether AI can do something starts with the token--just a little chunk that passes into the transformer. It's a piece of a word, about four characters. As an example of something that doesn't easily tokenize: spreadsheets. You need special techniques for spreadsheets. Very roughly, AI is still 10x farther behind on spreadsheets than Word docs. Is it getting better? Absolutely. But it's not where Word doc processing is. You can hand a very large Word document to AI and get a sense of what's in there. Try the same with a large spreadsheet--because you value accuracy in numbers--and you won't get nearly as lucky unless you have a specialized tool. That's why tools like DataRails exist. Tokenized data exists in tiers: **Tier A:** Easily tokenized data. Anything in your Wiki is super easily tokenized. **Tier B:** Spreadsheet-scale data. It fits in a spreadsheet. Not super easy to tokenize, but you can probably massage it and get it in there. **Tier C:** Data lake data. Available for search, potentially through agentic search. But it's too big--hundreds of thousands and millions of rows of time series data that you have to preserve structural relationships for before you can develop insights via LLM. Traditional LLM transformer architectures don't handle that well. You can take small pieces and learn something through tokenization. But when people talk about hooking up LLMs to large data sources, what they're really saying is they've figured out how to search the data lake to retrieve useful information that they can build into insights. There are preparatory steps to do that well, but that's out of scope of this article (you might check out the hybrid search section in my RAG guide for that). Think of tokenizable data as stuff you can fit in a Word doc first. Second, maybe it can go in Excel. Third, anything so big and complex that it needs a data warehouse or data lake will be much harder. What's interesting: the easier it is to tokenize, the more you can shape your destiny with AI and that content. It's quite difficult to pivot and architect AI solutions over data lakes. Organizations wrestle with this constantly, and this makes moving forward on AI harder for larger data problems (but of course you get leverage if you can do it successfully). Successful AI pivots often start with something simpler--company policies in one neat Word doc, or three or four--you can easily get that into LLMs and immediately control your destiny. Learn to think in terms of tokenizable data. Think whether it fits in a Word doc, maybe whether you can sketch it on a napkin, since a napkin test is a handy test for context window size. If you can draw the complexity on a napkin, AI can be very helpful. If you can't fit the complexity onto the napkin, it may be too complex for nuanced AI perspective. Keep in mind you can fit everything from the Amazon Flywheel to the concept of transformer architecture in AI on a napkin, so you have more mileage than you think there. ## Jagged Intelligence Moving from tokenization: jagged intelligence. Again, people's eyes glaze over. Jagged intelligence simply means we have AIs that are in some ways as smart as Einstein and in other ways worse than the worst intern you've ever met. I was a pretty bad intern myself. The problem: AI is not a continuous intelligence surface. It has really large gaps driven by known issues, particularly around memory. If AI can't remember something, it can't learn as it goes. Yes, LLM teams are working on this, but it's a hard problem without much progress yet. For now, it's very difficult to get AI to consistently do certain simple things that require memory. For example, if you talk to AI about your role and ask it to write an excellent article, proposal, or email, you have to re-explain what you want again and again. If you make any mistake re-explaining, your faithful AI will make a mistake. That's jagged intelligence. It's good enough to write those outputs really well but not good enough to remember how to do it or avoid being extremely sensitive to briefing mistakes. Think of it as Shakespeare obsessed with following instructions. If you make any mistake in the instructions, Shakespeare makes mistakes. This is why prompting matters so much--you're trying to get the LLM to do what it does best rather than getting stuck where it doesn't do well. Organizations come up with all sorts of tricks for this (saving prompts, figuring out RAG architectures for memory management, etc.) but ultimately they're all trying to work around the fundamental limitation that you have to re-explain stuff to the AI a LOT. Other low points in jagged intelligence: math. LLMs will call other tools to do math. There are specialized models--Gemini has one, OpenAI apparently has one that does math Olympiad problems. But when it comes to "is 9.9 or 9.11 bigger," LLMs can still struggle. If you're trying to understand mathematical modeling of concepts or weigh business levers, you get some insight from AI, but I find it tends to cluster around existing distributions of strategic advice--think McKinsey deck-level abstraction. It doesn't tend to be deeply insightful unless you're extraordinarily good at giving it strategic intent and excellent context. Then it can reason across your specific information. But even in those cases where I prompt very well, I typically find I'm wading through a lot of "not quite" to get to the gem of "ah--that's fantastic." Again, jagged intelligence. This highlights another tricky thing about jagged intelligence: it can be made less jagged if you prompt better. Better communication of intent can erase some gaps. You'll still feel the gaps. I still feel it because AI is really good at outlining but often not as good at capturing tone the way I want. I'm very picky about tone. I feel it when asking AI to think about strategy--AI is good as a sounding board but doesn't feel as refined as needed. The more you cultivate high taste, the more you cultivate saying "it can be better and I know how," the more sensitive you'll be to jagged intelligence. My challenge: where is your taste bar? If you can't sense jagged intelligence, have you insisted on a high enough bar with AI? I bet you know something better than AI, and you can start insisting on a high bar there. Start cultivating your sense of taste, and your awareness of jagged intelligence will follow. ## When to Use Big Prompts Versus Casual Chats Third concept: when do you apply big prompts versus casual chats? I get this question because people look at my Substack and think "Nate always does big prompts." NOPE. I write big prompts it's true. But here's why: the planning and thoughtfulness that go into an excellent prompt pay off when I have an important task. When I need to iterate and discover as I go, it pays more often to start with a sharp one or two liner. You might think that means I value the big prompt work more. No again. It's more nuanced. If it's important _and_ needs to be anchored around lots of context you provide, a big prompt makes sense. If it's casual and/or iterative, longer conversations with shorter prompts to start make sense. With iterative tasks, you're discovering meaning with AI as you go. So start shorter with just a bit of sharp intent, like a little compass to lead the way. Larger prompts are for anchoring around a specific topic. Sure, you'll have multi-turn conversations, but within that box you've set with strategic intent at the top and a big prompt. If you're trying to iterate, riff, brainstorm, and think through things, it's often much more useful to start with a very short prompt and leave the model room to expand. It's deceptive to think meaningful work gets done only with big prompts, because I can get very meaningful work done with short prompts that I kick back and forth rapidly if I need to discover meaning iteratively. My encouragement: _is this work something I already know enough about that I want to focus on production?_ Probably a bigger prompt. _Is this work something I don't even know the shape of and need to discover?_ Probably a shorter prompt. Both can be valuable. So don't think "Nate writes big prompts, so I have to as well." My ask is that you realize why prompts work the way they do and get intentional about knowing when to use larger, formal prompts versus casual ones. ## Your Weekly Practice Putting this together, you'll get farther with AI this week if you can do a couple of exercises that help you think through these challenges. 1. Find something you can tokenize this week--something you haven't tokenized yet. I scribble stuff on notepads constantly. Terrible handwriting. I find that with the right model--o3 is better at this--I can visually process that data, get it into text, and tokenize it. That's tokenization for me. You can find one for you. 2. Look for something that feels jagged with AI and be intentional about cultivating the strength--the peak, the good part, the Shakespeare part--rather than the part that isn't so intelligent. Especially pick something where you can tell what's good, so you are practicing that taste bar. 3. Keep an eye on how often you feel like the prompt fit the project. If it fits most of the time, that's fantastic. If it doesn't, it's an invitation to figure out how you understand the relationship between your intent, your work, and the AI. These three pieces, if you get them, will help you enormously in understanding how to use and augment AI. These are the corner pieces of the puzzle. If you get these three down, you're going to be able to follow along with a lot of much more advanced AI concepts easily, because you're already thinking about data, how AI understands intent, and where AI is good (or not good) at jobs. Good luck out there this week! I make this Substack thanks to readers like you! Learn about all my Substack tiers here Upgrade to paid | | ---|---|--- _You're currently a free subscriber to Nate's Newsletter. For complete strategic insights that'll accelerate your career, treat yourself to an upgrade._ Upgrade to paid --- | | | Like --- | | Comment --- | | Restack --- (C) 2025 Nate 548 Market Street PMB 72296, San Francisco, CA 94104 Unsubscribe