The Pattern Without the Person: Privacy in AI Coaching

A coach I talked with while building Grove told me about Facebook. She started using it in college, back when it was four schools and a handful of students who had no idea what they were building a habit around. They posted pictures. It felt small and new. Twenty years later she looks back and sees something she never agreed to in any moment she could point to: a company that knows a great deal about her, assembled for free, from a thousand small posts she made before she understood what the thing was.

Then she named the fear underneath it. She is afraid the same thing is happening again with AI, and this time the data is not her vacation photos. It is her clients.

That fear is correct, and most coaching-plus-AI setups earn it.

What pasting into ChatGPT actually does

Here is the default move, the one the field has quietly normalized. A coach has a session coming up, opens a chat assistant, and pastes in their notes to get help thinking it through. “Sarah has been missing her morning walks, she mentioned her mother’s surgery, here is where we are stuck.” It works. The suggestions are useful.

It also just sent Sarah’s name, Sarah’s mother’s surgery, and Sarah’s struggle to a server the coach does not own, attached to each other, in plain language. Sarah never met that company. Sarah cannot see what was sent. And the coach, who would never read a client’s file aloud in a coffee shop, has done something with a longer half-life: handed a named person’s private life to a system that keeps it.

Board-certified coaches work under a confidentiality ethic. The textbook says client information is held in trust. The Tuesday-afternoon reality is a copy-paste into a chat box that quietly breaks that trust, because the easy tool and the ethical tool are not the same tool, and nobody set out to do harm. They set out to prep for a session.

Nothing goes to the AI unless the client says so

Grove’s first answer to that is not a clever algorithm. It is permission. Grove does not send a client’s data to the AI until that client has explicitly opted in, and until their coach’s practice has too. Both switches start off. If either one is missing, or a client turns theirs back off later, the request is blocked before it is ever built. The model does not get a careful version. It gets nothing, because nothing is sent.

This is the part that is easy to admire and genuinely hard to do by hand. Consent is a real obligation in coaching, and it is also exactly the kind of thing that is simple to mean and simple to forget on a full week. A coach prepping at 9pm is not going to stop and write a data-use disclosure before pasting into a chat box, and almost no one does. Grove makes the asking a built-in step instead of a good intention, so “did this client agree to this” has an answer that does not ride on anyone remembering to have the conversation.

Even then, the model never sees a name

Consent decides whether a request runs at all. What is inside it when it does was the other half of my bar. I did not want to build coaching software with AI in it unless I could clear both, so even after a client says yes, the model never receives a name.

Not “we promise we delete it later.” Not “it is encrypted in transit.” The model does not receive the name in the first place.

Before any request leaves Grove for the AI, it passes through a step that strips out the identifying details. Client and coach first names come out and are replaced with neutral placeholders. Email addresses, phone numbers, and the internal account IDs that could tie a record back to a real person are removed too. What the model receives reads less like “Sarah skipped her Tuesday walk after her mother’s surgery” and more like a person, unnamed, with that same pattern of weeks and habits and setbacks.

The placeholders are not the AI’s invention, and undoing them is not the AI’s job. After the model answers, Grove puts the real names back mechanically, by direct substitution, in our own code. The model that wrote the words never knew whose words they were. If a name appears in what you read back, it was reattached after the fact, by Grove, not generated by a system that had ever seen it.

Be honest about what the AI does see

A privacy claim is only worth something if it is precise about its own edges. Stripping the identifiers is not the same as sending nothing, and it would be a different kind of dishonest to let you think the model works in the dark.

The AI does see the substance of the coaching, with the identity stripped off. The shape of someone’s check-ins, the habit that keeps slipping, the reflection they wrote, the measurements they logged, the rhythm of a streak that broke. That has to be true for the help to be real. A tool that saw nothing could tell you nothing. The point is not that the model works blind. The point is that it works on the pattern, not the person, and it cannot tie that pattern to a name, an inbox, or a phone, inside the request or across requests.

So the comparison is not “Grove uses AI” versus “this coach is careful.” A careful coach can leave the client’s name out of the paste, and the conscientious ones do. Two things are harder to strip by hand. The first is yourself. You are logged in, so the provider knows which coach is asking, and a record builds under your account across every session you run, nameless clients and all. The second is consistency, because the name has to come out perfectly every time, on the rushed Tuesday as reliably as the slow one, and people are not perfect at that. Grove removes the client’s name, the coach’s name, and the IDs that would tie one request to another before anything leaves, by default, so the protection does not depend on anyone remembering. The client’s identity does not make the trip. Neither does yours.

The wedge underneath this

There is a smaller idea hiding inside the privacy one, and it is the part I care about most. The reason this is even possible is that Grove treats the AI as a tool inside a system a person controls, not as the place the relationship lives. The coach stays the one who knows Sarah. The software carries the draining parts, the remembering and the drafting and the pattern-spotting, and it carries them on data it has deliberately blinded to identity. The judgment, the name, the relationship, the decision about what to actually say, those stay with the human who has the duty of care.

The coach who told me about Facebook was not anti-AI. She wanted the help. What she did not want was to look back in twenty years and find she had given a system access to her clients that none of them understood they were granting.

That is the right thing to want. The work was making sure she would never have to choose between the leverage and the promise she made to keep what her clients tell her.