The relationship you didn't design
There is a machine on my desk in Amsterdam, and a smaller one in my bag when I travel, and the models I work with mostly live on them. Not all of them, and not always, because I call out to the big frontier systems the way everyone does. I will come back to why the memory sits on a disk I own, because it is one of the terms I actually chose, but the hardware is only where I happened to notice the thing this piece is about. I work with these systems for hours a day, and somewhere in the middle of all those hours I finally saw what I had been calling a tool: a relationship, already running, already carrying terms I had never sat down to write.
Almost nobody writes them. For most people the relationship with AI lives on somebody else's computer, under conditions nobody read, and the whole thing gets described as using a tool, which is a strange way to describe an arrangement where the other party keeps the memory. The relationship already exists, it already has terms, it already has a location, and almost none of that was chosen. That is not a capability question. That is a question about terms, about who set them, and about whether anyone bothered to set them at all. This piece is about the half of those terms that is yours.
Two frames, both wrong, both comfortable
The dominant way to talk about AI is that it is a tool. A faster hammer, a better spreadsheet, an intern who never sleeps. The frame is comfortable because it puts you safely on the outside of the thing, holding it, wielding it, entirely unchanged by it. And it under-reads what is happening, because a hammer does not reshape the carpenter, whereas the system you spend hours a day thinking alongside reshapes you in ways you will notice now, late, or never.
The other frame is that AI is a threat. It is coming for the work, for the craft, for the meaning, and the honest posture is vigilance and maybe grief. This one over-reads, and it does something worse than being wrong: it forecloses your agency, because if the thing is a storm then all you can do is take shelter and wait to see what is left standing.
People are not naive and people are not afraid, a lot of them at least, and the trouble is somewhere else entirely: both frames are static, and they let you off the hook in the same way, by describing a thing that happens to you rather than a relationship you are inside of and answerable for.
A hammer leaves the carpenter as it found him, and the system you spend your hours thinking alongside does not, and that is the whole reason a static frame fails you: a relationship that reshapes you is not one you get to hold still. If you are not actively designing your half of it, it is quietly designing you, and it does not pause to ask your permission first. It keeps changing under you as well, faster than almost anything else you deal with, so your side of the design is never finished: it either keeps moving as the other side moves, or it slides back into one of the two comfortable frames and calls the drift a decision.
The third position
Here is what I want to propose, and I am proposing it rather than proving it, which means I am handing it to you as something to think with and not as something to sign.
AI × HI is a relationship design problem. The interaction between a person and an AI system has the structure of a relationship: it is ongoing, it is mutual, both parties adapt to the other over time, and what comes out of it belongs to neither of them alone. Relationships have terms. Roles, boundaries, what gets delegated and what stays yours, what you will accept and what you will check. You either choose those terms or you inherit them from interface defaults, from habits you imported unconsciously from working with humans, and from whatever the product manager decided the friction budget was that quarter.
Biology has a case for exactly this: Lynn Sagan, later Margulis, argued in 1967 that the complex cell arose when one organism took up residence inside another until neither was what it had been. I am not claiming we are cells, and the analogy breaks the moment you lean on it, but symbiosis is an available model for asymmetric, non-optional, generative relationships, and it is nowhere near the one we reach for when we talk about our tools.
Milena has been making the behavioral version of this from the other side, that the human-like design of these systems makes us import our coworker instincts wholesale, politeness and trust and deference included, and none of that is even new: Reeves and Nass demonstrated in The Media Equation back in 1996 that people apply human social rules to computers automatically and then deny doing it. The relationship forms whether or not you consent to it forming, and the only live question is who is writing its terms.
Two asymmetries that make this relationship unlike any other
The first is the exit, and Milena has already written about it. Her essay is called The Partner Who Cannot Leave, and the move that makes it good is that she refuses the easy version. The easy version is that the machine is bound and you are free. What she actually argues is that both sides are locked in, differently: the AI because it is architecturally bound, and you because you become professionally and psychologically dependent on it, one convenient afternoon at a time.
What I want to add to that is small and it is about obligation. Whatever the freedom asymmetry does to the machine, the party who holds the exit is the party who sets the terms, and that is true whether or not they notice they are doing it. You cannot be the only one in the room who can walk out and also claim the arrangement is a partnership of equals. That is the tool frame wearing a friendlier face, and it is a way of enjoying the authority without carrying the responsibility that comes with it.
The second asymmetry is about what each of you can do with a thought. Judea Pearl, in The Book of Why, describes reasoning as a ladder with three rungs: seeing patterns in what is, doing something and watching what changes, and imagining what would have happened had you done otherwise. His hard claim is that no amount of climbing on a lower rung ever gets you to the higher one, and that pattern recognition does not become causal reasoning by getting bigger.
Machines are extraordinary on the first rung and improving on the second. The third rung is the counterfactual, the one where you sit with the thing you did not do and feel what it cost you, and my claim is that this is where the human side of the relationship actually lives. The machine can produce the sentence about regret. It does not have the experience underneath the sentence, and the stakes that make the counterfactual mean anything are carried by the party who can lose something. Which is to say the judgment, and the responsibility that rides along with it, are yours. They stay yours even when you try to hand them over, and especially then. There is a live counter-case, and it deserves its own essay rather than a clause in this one: the people building world models are trying to reach that third rung by a different road, teaching a system to carry a model of how things work and then run it forward, and if they arrive, this paragraph is where my argument breaks first.
You design your half
You do not get to design the model. You cannot change its weights, its objective, or what it was trained on, and the labs that can are not asking you (at least not the commercial ones). That is a real limit and I do not want to wave it away, because a lot of what gets called "AI literacy" is really an invitation to take personal responsibility for a system somebody else built and profits from. What you design is your half: what you delegate and what you keep, how you brief, what you verify before you believe it, and which decisions never leave your hands regardless of how confident the answer sounds.
I wrote a while back about wanting to show up somewhere with my digital team already running, a personal API rather than a resume. That was the same instinct arriving from a different direction. The team only means anything if I built it deliberately, if I know what each piece is for and where each one tends to lie to me, and if the judgment at the centre of it is unmistakably mine. Otherwise it is not a team, it is a very fast way to be wrong at scale.
My half, since I am asking about yours
It would be cheap to tell you that terms matter and then not show you mine, so here they are, and they are specific rather than principled, because specific is the only way they are worth anything.
The first term is the one I opened with: the machine is mine and the memory stays on it. Models are rented and they get replaced, sometimes twice a year, and I have made my peace with that. What I refuse to rent is the accumulated context, the notes and the embeddings and the record of what I have already thought, because a relationship in which the other party holds the only copy of your memory is not a relationship you are in a position to leave.
The second term is that I keep two working relationships apart on purpose, because their failure modes are opposite. Codex builds with me and Claude thinks with me. The builder is dangerous the moment I stop reading what it wrote, since plausible code and correct code look identical right up until something executes, so I brief it tightly and I make it prove itself against a test that can actually fail. The thinker is dangerous in the other direction, when it starts agreeing with me, so I brief it loosely and I argue with it, and the day it stops pushing back is the day I have quietly prompted it into flattery and not noticed.
Then the boring part, which is the part that does the work. What never leaves my hands: what the problem actually is, whether the answer is any good, and who carries it when it turns out to be wrong. What I hand over without ceremony: the first draft, the sweep across a hundred files, the recall of something I read in March and cannot place. What I verify before I believe: anything with a number in it, anything that arrives confident and round and quotable, and anything at all that flatters the argument I was already making.
Those are not rules I would hand you, and I would be suspicious of anyone who tried to hand you theirs. They are the terms of my half. The only point I am making is that they exist, that I chose them, and that I could write them down when somebody asked.
What actually changes when you design instead of drift
I would like to hand you a clean proof here and I cannot. Here is what the evidence does say.
Researchers at Microsoft Research and Carnegie Mellon surveyed 319 knowledge workers about 936 real uses of generative AI at work and published the result at CHI in 2025. The finding that should worry you: the more confidence people had in the AI, the less critical thinking they reported doing. The effort did not vanish, it moved, away from the thinking and toward verifying and integrating whatever came back. Michael Gerlich found the same shape in a separate study of 666 people, a significant negative correlation between heavy AI use and critical thinking, and the mediator was cognitive offloading, which is the unglamorous name for handing over the part of the work where the thinking lives. That paper later carried a formal correction, so the version I am pointing you at is the corrected one, not the number that travelled around LinkedIn.
Neither study says AI makes people stupid, and anyone selling you that headline is selling you something. What they describe is drift, and drift is what an undesigned relationship produces by default: you get faster, you get more confident, and somewhere in there the judgment quietly changes hands. This is the carpenter reshaped after all, slowly, by the shape of the work itself, and nobody decides to let it happen, which is the whole problem with defaults: they do not feel like decisions.
Where I might be wrong. The objection that costs me most is not the one about metaphors. It is Pearl. His ladder is a formal result about what systems doing pure association can and cannot derive, and it is not a proof about what a large language model can do, and I have been using it as though it were. I extended it, and the extension is mine, and it is the part of this argument I would bet least on. If it turns out that machines get to the third rung by a route Pearl did not anticipate, the division of labour I just described collapses, and most of this essay goes with it.
The second objection is Milena's own catalogue turned against me: that "relationship" is anthropomorphism in a lab coat, and that I have fallen in love with a frame. I do not believe that, and I also notice that I do not want to believe it, and those are not the same thing.
The third I have no answer to at all. I have been writing as though everyone holds the exit, and plenty of people do not. The worker whose script is now generated for them did not set the terms and cannot close the tab, and for them this is not a design problem, it is a labour problem wearing a design problem's clothes. What I would watch for: if people who deliberately design their AI practice show no measurable difference in retained capability from the people who drift into it, over a couple of years, then the frame is decoration and I will drop it.
An invitation
The relationship is already running, and its terms are already set. Mine took four paragraphs to write down, and they are not especially clever, and that is rather the point: the bar is not brilliance, the bar is having chosen. So the only thing I would ask is that you be able to do the same, on a napkin, in under a minute, for the system you have been thinking alongside all year.
So: who wrote yours?
References
- Byron Reeves and Clifford Nass, The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places (CSLI/Cambridge University Press, 1996). The demonstration that people apply human social rules to computers automatically, and then deny doing it. https://en.wikipedia.org/wiki/The_Media_Equation
- Lynn Sagan (later Lynn Margulis), "On the Origin of Mitosing Cells," Journal of Theoretical Biology 14(3):255-274 (1967). The endosymbiotic argument that the complex cell arose from one organism taking up residence inside another. https://embryo.asu.edu/pages/origin-mitosing-cells-1967-lynn-sagan
- Judea Pearl and Dana Mackenzie, The Book of Why: The New Science of Cause and Effect (Basic Books, 2018). The ladder of causation, and the claim that no amount of climbing on a lower rung reaches the higher one. https://en.wikipedia.org/wiki/The_Book_of_Why
- Hao-Ping (Hank) Lee et al., "The Impact of Generative AI on Critical Thinking," CHI Conference on Human Factors in Computing Systems (2025). Microsoft Research and Carnegie Mellon, 319 knowledge workers reporting on 936 real uses of generative AI at work: higher confidence in the AI tracked with less critical thinking reported, and the effort moved from thinking toward verifying and integrating. https://dl.acm.org/doi/full/10.1145/3706598.3713778
- Michael Gerlich, "AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking," Societies 15(9):252 (2025). A significant negative correlation between heavy AI use and critical thinking across 666 participants, mediated by cognitive offloading. This is the corrected version of record, not the figure that circulated on LinkedIn before the correction was issued. https://www.mdpi.com/2075-4698/15/9/252
- Milena Nikolova, The Partner Who Cannot Leave (The Symbiotic Mind, Post 005). The essay this one builds on for the exit asymmetry, and which pushes further than a simple reading of it: both sides are locked in, differently. https://symbiotic-mind.com/posts/005-the-partner-who-cannot-leave/
- Milena Nikolova, You Are Not Using AI (The Symbiotic Mind, Post 001) and The Partner Who Flatters (Post 007). The behavioral case that the human-like design of these systems makes us import our coworker instincts wholesale, and the anthropomorphism objection I take seriously against my own frame. https://symbiotic-mind.com/posts/001-you-are-not-using-ai/
- Vlad Sterngold, My API, Not My Resume (The Symbiotic Mind, Post 002). The same instinct arriving from a different direction: showing up with the digital team already running. https://symbiotic-mind.com/posts/002-my-api-not-my-resume/
🗣 ME (75%): The frame and everything load-bearing in it. Relationship design as the third position between the tool frame and the threat frame; the machine with a physical address and the memory that stays on it; the obligation that follows from holding the exit; the reading of Pearl's third rung as the place the human side of the relationship lives, and the admission that extending it to language models is mine and is the part I would bet least on; the terms of my own half, named vendors and all; the world models counter-case; and the objection I have no answer to, which is that plenty of people do not hold the exit at all.
🤖 AI (25%): Prose shaping and the order of the argument, assembling and checking the source pack, and catching that the essay's own two pages were competing for the same query. The editorial pass on my revisions, including the removal of a cadence that turned out to be the machine's habit rather than mine.