Scott's Mixtape · Economics & Policy
TIER 4 Thu, 19 Feb 2026 13:47:18 +0000
Maintaining Attention, Reduced Congestion and Human Verification Is The new Skill ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ | | ---|---|--- | | | Forwarded this email? Subscribe here for more --- # Claude Code 21: Attention, Human Verification and Congestion, or Some Problems From Too Much Better Work ### Maintaining Attention, Reduced Congestion and Human Verification Is The new Skill | | scott cunningham --- | Feb 19 --- | --- --- | | | --- | | --- | | --- | | --- | | READ IN APP --- _This is part of my ongoing Claude Code series, which are substack posts discussing what I 'm learning about Claude Code as it relates to quantitative social scientists whose work lives inside of folders and directories on their local machines._ _My claim continues to be that at the moment, there is a surplus of writings about Claude Code by engineers for engineers, and a paucity of writings about Claude Code by social scientists for social scientists. So I am just documenting what I am noticing, sometimes doing video walkthroughs, sometimes writing essays, and this one is more essays about dealing with the need to find verification systems now that productivity is legitimately enhanced with Claude Code. All Claude Code posts remain free when they come out, unlike other posts are randomly paywalled. Everything goes behind a paywall after a few days, though. If you find this series valuable, I encourage you to support it at $5/month or $50/year!_ Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Upgrade to paid * * * As with so many of Claude Code posts, they are fairly stream of consciousness. And this one is no different. The material in this substack is more or less an idea I'm working out which is outlined in this deck which is that even with maintaining the same amount of human time on research, I think there are so many new problems with using Claude Code for research that we could very well be in a very hard place where we have to spend far more time on non-research related activities trying to resolve these new problems that we are not used to encountering. In this deck, I have been working out the problems I am creating for myself with so much new, higher quality research output, where I am inadvertently creating _too many activities_. And in the process of seeing productivity gains, but always with diminishing marginal returns, these new costs are exploding around me in ways I am not expecting. I call them throughout "stock pollutants", almost like litter, and I am trying to figure out which of them are just tolerable, and which ones are absolutely not tolerable. Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Upgrade to paid * * * **Hyper Systematic Organization Interacting with Strange Attention** So back to the problem. Over the past few weeks, Claude Code had helped me generate a massive amount of material for this project that I had revived after months of sitting on it and procrastinating on the revision, trying to pretend I did not remember the deadline was approaching. So I had used Claude Code to break down specifically what the referees and editor wanted done. The revised analyses, the robustness checks, new specifications, figures, etc. I used Claude Code to break down precisely what they were asking me to do, organize those into aichecklist, like a map of tasks, so that I would not inadvertently skip anything. And then we had one by one started doing them. And I could feel my productivity exploding because it was -- there were some things, many things, that I was completing instantly, and there were things that I also felt like I was getting done that was harder to explain. But it had to do with how twisted up I get because of my ADHD "primarily inattentive" stuff. How I can't quite remember where I am in a process, or how I get fascinated with the most trivial details, going deeper and deeper, until I basically nowhere near where I started. And to me this is not just how I am, but if I'm honest, it's how I want to be too. I love that hyper fixation, that flow state, when it happens. I am profoundly curious, to a fault, and in ways that I think annoy coauthors, but so often they lead me to the improvements. It's just that they also spew off a lot of pollution too, and so in my research projects, my coauthors often have to tolerate a lot of it in the hopes that on average we are getting somewhere that will improve things. And my best coauthor experiences don't mind it, see the point of it, and are willing to ride it with me. So then in that sense the research with Claude Code has that same feature. It's just that my productivity is ramped up by 5x, and since those problems are still there, it means that the externalities are also generated at 5x. And I'm not really sure that they are in fact linear in the work. I think sometimes that the externalities may even nonlinear in the productivity, and since my speed of work is now faster, and in a new environment without the guardrails I had spent years perfecting after graduate school to keep me focused and on track with minimal errors and maximized output, the costs associated with the progress might very well be convex, rising faster than the gains. So let me then share a little of what I'm thinking. I think that my style of interacting with the research via my "rhetoric of decks" philosophy, where I keep constant notes in a journal of an evolving scroll of "beautiful decks", largely adding to them, could be creating some challenges. I can't quite put my finger on it, and haven't yet, but I think the decks are necessary _for me_ , and yet they're also the source of stock pollutants growing fast in the research process, making finding what I need like finding a needle in a haystack as I can't seem to at the end, when it's time to finish this up, remember where things are. Some of this is because as organized as Claude Code is, every single idea I give, he generates the code and stores it, but I've noticed he may not always put it in the place I want it. And not only that, he will often generate new code, rather than add to the current file I want, which I think may create these small random perturbations in the pipeline where things are branching off. This happens most of all in old projects being revived I've noticed as the old projects have legacy styles of organization that are not necessarily what I am doing now when I start. Because when I start now, I tend to have a much simpler starting point that looks like this. | | ---|---|--- That is generated above using my `/newproject` skill. It generates that directory structure for all new projects. But for old projects, like I said, I cannot and do not do that out of concern that I will overwrite things, which is a real worry I have with Claude Code -- the inadvertent deleting of data is something I explicitly tell Claude Code _not to do_ in my static` Claude.md markdown`. Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Upgrade to paid Which means though that my current R&Rs where I'm brining Claude Code for AI assistance are messier than intended. When I revive old projects, and try to bring it into the discipline of Claude Code, it looks much crazier because it has this Frankenstein style hodge podge of the old and the new, and I find I am not willing to just flip to the new style and instead tend to grandfather in the old, which this project was since it was an R&R, and which therefore may or may not have contributed to the difficult-to-put-my-finger-on struggle I was having keeping track of just what was going on * * * **Isoquants, Attention, Misplaced Attention** Recall that I gave this talk to the Boston Fed back in mid-December, which now seems like I was trying to bring recently discovered fire to them in light of the rapid explosion in Claude Code awareness through the social sciences, but which at the time I was kind of worried I was going to sound like a manic and somewhat over-reacting seminar speaker filled with prophetic hopes and doomsday predictions. Well, **both can be true**. Anyway, here was my basic framework if you didn't read earlier posts about this (which frankly are posts I used to write as far back as 2023). | | ---|---|--- Recall that my core conviction is that the isoquants from production functions for doing "creative cognitive work" have flatted from being quasi-concave pre-AI -- wherein it was impossible to do any creative cognitive work without using nontrivial amounts of human time -- to having flattened, and for many tasks, actually linear which as economists know means that if I'm right, then machine time and human time are perfect substitutes, even for cognitively creative tasks. Well if they are in fact perfect substitutes, then rational actors will use the cheaper of the two _at the margin_. We pay monthly prices for Claude Code at anywhere between $20 to $200. We don't pay for tokens on a case basis, but we do incur opportunity cost of human time on a case basis (proxied by the value we place on our next best alternative). And so as such there is temptation when using AI for research, almost like we are wearing heavy weights around our legs, for AI to pull us towards using less time on research. I don't mean, though, doing less research note. I mean _less time_. Less _human time_ on research, and if _human time_ is a direct input in attention, as you cannot pay attention to things that you are not literally focusing your time on, we can end up _learning less_ and _doing more_ at the same time. | | ---|---|--- This is really at the core of many of the problems, apart from ethics (though this indeed gets into ethics too as to what degree are you the expert on things you are driving?), of using AI for social scientific research. Reduced time leading to reduced attention, leading to less human capital, despite the completion of actual cognitively creative tasks is where human researchers become more or less optional in the process of doing research. So I outline three possibilities, only two of which are good for human researchers if our goal is to maintain a connection with the knowledge we are responsible for creating. And one of them -- the first one -- is the one I personally hold to which is that I maintain my time use committed to the research so that I maintain my curiosity and learning, because my curiosity is my power, and if I live a life where I drift away from my love of learning, discovery and engagement with my curiosity, I might as well go find a job somewhere else. I simply refuse to live an inferior life where I am not engaged in the activities I love, which is a connection to learning in all the ways that makes my heart sing. This is partly what differentiates me from being purely someone who cares about policy for its own sake -- I am a hedonist. I care about my passions and curiosity for its own sake, and everything else gets swept along with it. I just try to aim that short-term needs with long-term goals so that I am helped in ways that touch on other values I have, like helping people. | | ---|---|--- And so this picture is more or less what I see as my own aspirational goal. Maintain time use on research topics using AI so that the productivity gains happen. This is represented to me as the best outcome because the output gains are the largest on a per unit basis. It is the _same time_ , H*, it just will inevitably be _different time_. But the fact of the matter is that there is a pull, like a gravity force, that draws the researcher down and away from H*. And one of them is arguably welfare enhancing from the perspective of increased knowledge for oneself, and the other is not. The one on the left represents gained knowledge with reduced time use, and the on the right represents extreme automation where time use fell too much such that the human became really nothing more than what I just sometimes call the "button pusher", where research becomes factory work. Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Upgrade to paid And so what I was experiencing in the R&R was that specific manifestation of ways in which using Claude Code to assist me in the research process was mixing me simultaneously among all three of these states. It was creating some kind of internal coordination problem that I couldn't quite put my finger on, but I wanted to now just describe what I think is happening. * * * **The Problem of Too Much Better Work** So part of the problems I think I am having is that as I am going so fast, increasing my work by 5-10x, and using "beautiful decks" to maintain my connection to the progress, like a running diary, I am somehow creating too many decks, with out of order progress. This happens in particular for the truly complex projects, too. Where there may be 5 ways of doing something and where _ex ante_ there is no clear reason to favor one over the other, and so I do all 5 and then have to decide how they can be reconciled, if they should be reconciled, and how to go about positioning these exhibits. Do they go in the manuscript? If so where? If so how will they be displayed? Five tables? Five figures? Five panels? One panel? So I may try all options for aesthetic purposes, but I may too iterate sequentially as I do it, realizing that the right way to do it is to do XYZ, not realizing that that insight came to me after some earlier step of ABC. The problem in using decks this way to maintain my connection with the work is small, subtle details. For one, Claude Code may almost randomly hard code the code output into the decks unless I say otherwise. And if I am not using `/skill` commands for repeated work, and if those /skill commands have not been absolutely perfected to avoid hard coding into decks -- something so specific it may be missed -- you may not realize that randomly throughout the deck are non-replicable work. See, if the work is hard coded into the deck, even though the output ./tex exists, then you may very well have TWO copies of the same thing -- you may have the old copy that is using at-that-time output, and you may have a new copy in .tex generated from `estout` or `outreg2`. So this has been a challenge for me to resolve. How do I maintain a new diary of progress, maintaining my attention, but now dealing with the stock pollutants, let's call it, of stuff surrounding me? If in fact the production of excess waste is convex in time use, then perhaps I have two things happening at once -- I have increased productivity, but diminishing marginal returns as the one law of economics, even moreso than demand sloping downward itself (but which is in fact responsible for demand sloping downward when it does), is the law of diminishing marginal returns to human time. And I have convex cost functions such that each additional use of time increases at an exponential rate rising marginal costs along some dimension that I may not directly understand, but which through repeated interactions in this new environment I absolutely keep encountering. | | ---|---|--- * * * **Maintaining Attention, Reduced Congestion and Human Verification Is The new Skill** I saw Andrew Karpathy say recently, I'll have to dig up the quote, that the new skill is in _human verification_. It's not in 'vibe coding' in this age of Claude Code as there's basically no barriers to entry to telling Claude Code "do this and that". There is no skill whatsoever in dictating "do this complicated partial identification thing I've always wanted done". That takes no skill, and because Claude Code is basically a genius, compliant, and stubborn like an obedient dog to do anything and everything you ask of it, it will do it. The real skill going forward is not therefore in the _doing_. We will all be sitting with jet packs on our back, and that when we decide to lift off, we will lift off -- just not slowly. If we are not careful, we will rip through our surroundings at light speed, and while it's true we will get somewhere faster, we will shatter windows and houses in our way too. I am focused now on just the excessive litter I am creating in my decks with my new workflow that I cannot quite get a bead on. And I've latched onto my "rhetoric of decks" because I am using it a lot to help me keep track of work over time. But I am therefore dealing with idiosyncratic problems too from that being an imperfect solution. So to Kaparthy's point, the new skill is not in the doing. Rather it is in one area he identifies, and two more that I am focused on. 1. **Human verification**. We are responsible for everything. We must therefore find a way to insert 100% accurate verification systems into the research process. There can be no mistakes. And frankly, given the difficulty of identifying mistakes, I suspect in an almost Beckerian like way, the stigma and punishments aimed at even the smallest AI-related errors going forward are probably going to be draconian. Just like in a footnote in "Crime and Punishment", Becker's classic 1968 JPE on the economics of crime, where he notes that Vietnamese rice speculators had their hands cut off for their crimes due to the low chances of detection, I suspect we will see even more of that going forward. Science is many things, but scientific _communities_ tend to regulate their own through sanctions and rewards. 2. **High Level of Attention**. So we must be **vigilant** and even obsessed with **zero error philosophies** now more than ever. And it is really unclear on those time use curves I drew just where that is, and where we need to be involved and where we do not need to be involved, and how we can automate even the verification, and which parts cannot be automated at all. All I know is that the final product must be something we all understand just as much as we ever did, which best I can tell requires a high level of attention. I think this absolutely means for most of us keeping human time use on the research project as high as humanly possible and resist, or even refuse, automation of research. Not so much because we are in principle committed to human work, but because I don't think we are even close to a world where robots have the comparative advantage in automating scientific discoveries. I doubt the isoquants are straight lines -- yet. 3. **Congestion**. But maintaining the same level of time use without addressing these convex costs coming from the stock pollutants associated with the same kind of time use is I think going to be its own problem to be solved. It is related, obviously, to the other two, but I think it is still helpful to separate it out. Which brings me back to all my "beautiful decks". I am not saying that the fault is in my deck philosophy -- of using decks to keep me attached. Some of what I outlined, after all, is absolutely fixable through new workflows where I _always_ use exported .tex files no matter what. But I still think I see the problem a bit more clearly from these overflowing decks because at some point, for any typical research project, I _will_ end up with too many slides, and no matter how "beautiful" those slides are, I will end up with congestion, and I will have a hard time pin pointing exactly where that congestion is occurring. | | ---|---|--- So I'll end there. Some of the ongoing video walk throughs I think will be less about me _doing_ as it is about me dealing with the problems of my doing. I will obviously be doing things. I have a cool new video series that I want to announce but am waiting a bit longer to do so. But I think what you will see is me stumbling around, in real time, trying to document the nature of these rising marginal costs, and then making stabs at trying to shift them down. But that's it for today. Have a great day! Let's hope we can keep going without any accidents! Scott's Mixtape Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Upgrade to paid You're currently a free subscriber to Scott's Mixtape Substack. For the full experience, upgrade your subscription. Upgrade to paid --- | | | Like --- | | Comment --- | | Restack --- (C) 2026 scott cunningham 910 North 17th Street, Waco, Texas 76707 Unsubscribe