Why Health Data Isn't Enough: Healthcare's Missing Intelligence Layer

Why Health Data Isn't Enough: Healthcare's Missing Intelligence Layer | Dr. Robert Wachter & Mike Haney

In a recent episode of A Whole New Level, Levels editorial director Mike Haney sits down with Dr. Robert Wachter, professor and chair of the Department of Medicine at the University of California, San Francisco. Wachter coined the term "hospitalist," has been ranked among the most influential physician-executives in the country, and is the author of the New York Times bestseller The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine's Computer Age and, most recently, A Giant Leap: How AI Is Transforming Healthcare and What That Means for Our Future.

The conversation traces how American medicine went from paper to digital in a single decade, why that transition disappointed nearly everyone, and how that imperfect digital scaffolding now underpins the AI revolution arriving in hospitals and clinics — as well as what has to happen before a world of continuous health data from wearables, sensors, and smart devices can actually improve care.

"This is going to be the greatest experiment in the history of medicine. It could go off the rails in a hundred different ways." — Dr. Robert Wachter

Why a physician-leader moonlights as a journalist

Mike Haney: Well, Dr. Robert Wachter, thanks for joining us today.

Bob Wachter: It's a great pleasure, Mike. Thanks so much for having me.

Mike Haney: So maybe as a way of bio, you've written a bunch of books, as we were just talking about before we started, and I think the first one was in the early nineties. And then most recently a couple of books which we're going to talk about a lot today: A Giant Leap, which just came out this year about AI and healthcare, and then The Digital Doctor, which I think was 2015, all about sort of the EHR revolution. I'm curious, because your day job has only gotten busier, I would suspect, since that first book in the nineties — you now run UCSF Medical, which is one of the biggest academic hospitals and medical centers in the country. So what motivates you to moonlight as a journalist, and when did the technology side of healthcare start to interest you?

Bob Wachter: Well, first of all, thanks for having me on. What motivates me is about every ten years I seem to find an issue that I just cannot let go of. I'm still a practicing physician, although most of my job is leading a very large department at an academic medical center. But they tend to be issues that are pretty close to the ground, that involve real doctors and real patients and nurses, and how we diagnose and how we treat and how people think about medicine and health and health care, rather than kind of cosmic issues about insurance policy. I don't get all that jazzed about those.

And it seems like about every decade an issue comes along that I just think, wow, I need to learn more about this. The explanations I'm hearing for things that I don't understand make partial sense to me, but always feel like they're coming from a variety of different perspectives, and there needs to be somebody who can kind of disentangle the whole thing.

And my first two trade books, lay-oriented books, I kind of just wrote them based on what I knew. When I decided to write The Digital Doctor and sort of do a deep dive into medicine going from paper to digital, my wife, who's a journalist, said, "You have to do this journalistically," and I said, "What does that mean?" She said, "It means you have to go talk to people." And I said, "I hate people." "I know that, but—" So it turns out not to be true. I actually love the process of being an amateur journalist. I went out for each book and interviewed over a hundred people. And you know from your work, I mean, that's how you learn a variety of perspectives, and everybody knows some part of the story, but not the whole story. And I kind of pull it all together and try to articulate in a way that's useful.

I have in some ways an advantage that it used to give me imposter syndrome, which is I don't really know very much about the technology itself. I think people that are tech experts tend to get just lost in the details of the tech. And I'm what happens when a political science major becomes an academic physician. So I think about the big picture, the people, the politics, the money. And I think not really being a techie person per se liberates me from getting too deeply in the weeds.

In terms of your other question, how do I sort of make it all work? First of all, I'm a very good delegator. I'm surrounded by amazing people in my day job. There's some synergy between kind of what I'm studying and what we're trying to do, for example, at UCSF where I work. And I'm a very efficient writer. I can write at, you know, on nights and weekends. I'm an adherent to the old Hemingway line of "write drunk, edit sober," so I can write for ten hours, get something down, come back the next day and say, this isn't very good, but I think I know how to make it better.

And so, you know, this is an issue. And finally, the tech issue — I've been very interested in just how to make the healthcare system better, because it's not very good and it needs to be better. And so if you were interested in that, and I've been studying medical mistakes for a long time, you had to be thinking, like, if we could just digitize the damn thing, it would make it better and safer and more convenient and less expensive. And The Digital Doctor happened because we finally did, and only after the federal government bribed us with $30 billion of incentive payments. And I was flabbergasted by how badly it went and how everybody was unhappy with their electronic health record. And I kind of decided I wanted to learn more about that, and that's what led to that book.

This book was sort of obvious — when the new AI happened on November 30th, 2022, with GPT, it was clear that this was going to be a revolution. My concern about writing a book about it was, is it going to be out of date five minutes after I'm done? And luckily my wife, my publisher, a few other smart people said, if that happens, if that's the book you've written, you've written the wrong book. So you really need to try to helicopter up ten thousand feet and really say what are the big picture issues that emerge when this new generative AI hits the healthcare. And so I tried to do that, and I'm glad most people seem to think I got it pretty right.

The $30 billion push: how American medicine finally went digital

Mike Haney: Well, we're gonna start with the EHR book. So I'm gonna ask you to do the thing you probably don't do in a lot of your current interviews, which is go back to the last book you wrote ten years ago. I do think, you know, the thing that was pretty revelatory for me in reading that book was, A, just learning all the kinds of ins and outs and crazy stories of how we ended up where we are, but also realizing how much that is the scaffolding for the AI and healthcare experiment we're running today, and then what we're going to talk about later, this sort of — what this series is built around — this theory we're positing that there's going to be a continuous data revolution coming in the next 10 years as well. And I'll say the thing that really struck me initially in the EHR story was how recent and how rapid it was. So I think the stat was something like in 2008 there was something like one in five hospitals were using an EHR.

Bob Wachter: And I think, if you go to one in ten — actually, one in ten in 2008. And by 2017, one in ten were not. So over a decade, we went from a paper industry to a digital industry.

Mike Haney: Yeah, so that pace of change, but also the recency. I mean, if you would ask me what year was one in ten hospitals using an EHR, I would have said 1995. I would not have said 2008 — that is a year after the iPhone came out, just to help people sort of place that. And so in that time, 90 percent of — a lot of medicine was still happening sort of on paper. What I'm curious about is, okay, so that happened over 10 years. We're now 10 years out from when you wrote that book. And the thing I kept wondering reading the book is, where are we now? What has happened? And so I want to walk through a couple aspects of that, which I think will kind of get us into the AI story.

And the first I'm curious about is the government and regulatory side. Because as you mentioned, the government threw $30 billion into this — really seemed to be the impetus for that rapid change. It wasn't coming necessarily from industry. It was coming primarily from the government driving that. But it was also the source of so many of the challenges that came up, through HITECH, through meaningful use. So I'm wondering if you can just talk a little bit about where we are now in terms of the government's involvement in health IT, if we want to take that sort of broad lens. We're going to talk about AI and government's role in that, or lack of role in that, in a little bit. But just in terms of how the government is interacting and mandating how healthcare is using digitization now, where are we?

Bob Wachter: Yeah, I mean, in some ways the main government regulation that governs our use of health information technology — I guess the two main ones are the role of the FDA in regulating digital tools, where, you know, the FDA is not perfect by any means, but knows how to regulate a new drug, and sort of knows how to regulate a new device, a new X-ray machine or a new pacemaker, et cetera. But — we'll talk about this, I'm sure — has a really hard time figuring out how to regulate AI, which is a very different animal than anything it's had to regulate before.

The second sort of main rule that we operate under is HIPAA, which relates to the privacy of data. And HIPAA, of course, was written and put into effect, I think, in the mid-90s, so really before anything was digital. And it's one thing to be talking about what you're gonna do with a piece of paper that has patient data on it. It's another, obviously, to talk about what happens when you have data flowing everywhere, whether it's in my electronic health record at work or coming out of your watch or your ring. I think in some ways it's very clear that our thinking about data privacy has to be modernized.

I mean, the story of the electronic health record — and the reason I thought it was worthy of a book to tell this story — is you're absolutely right, every other industry that I know of digitized a decade or two before medicine. Despite the fact that medicine is now twenty percent of our GDP, is an industry that fully depends on data, whether it's data about your health or data from the medical literature. And it was a massive market failure that that just didn't happen on its own.

The federal government, I think, to its credit, recognized that it wasn't happening on its own — that in 2008, in the average American hospital, if I saw you, if I was your doctor, I would be scribbling down my observations about you on a piece of paper. Your laboratory results would be on computer. I would probably have scribbled a prescription that periodically hurt people when somebody couldn't read my handwriting. So kind of crazy that that didn't happen on its own when the travel industry and the retail industry and the financial service industry had all computerized a decade or two earlier.

The feds — sort of an amazing story — in 2008, it was in the state, the Great Recession, that the federal government decided it needed to throw $700 billion at the American economy to try to get out of the recession. And it was only almost the happenstance that there happened to be some health policy advisors in the White House who saw that as an opportunity to throw $30 billion to get us to digitize. We were sort of ready to do it, but if you were the average hospital, you said, this is going to cost $100 million, it's going to be massively disruptive. It's gonna screw up not just the way we kind of collect data, but it's gonna change every process about the way we've organized, the way we do work, and that's just too hard. When the incentives came out, they didn't fully pay that hundred million or five hundred million dollars, but maybe they paid ten million or twenty million, and there was now a threat that if you didn't do it in the next couple of years, we were gonna ding you on your payments for Medicare. So it got us all to go very quickly. The meaningful use regulations were a whole bunch of standards that these computer systems had to meet. Some of them were perfectly sensible, some of them were bureaucratic overkill.

And my own take on the overall era is a lot of people were unhappy about the electronic health record, unhappy about all the trivia that we had to record, about sort of the bureaucratic documentation requirements. But to me it was absolutely necessary to get us digitized. Our mistake in thinking about it was that we thought it was the be-all and end-all, that the key maneuver was if we could just have our data in digital form, that will make healthcare better and safer and more convenient and drive a consumer revolution of people shopping for healthcare, lower costs. Virtually almost none of that happened.

And I think what I've come to recognize is that just digitizing the data, and to some extent connecting some of the parts, although imperfectly, was foundational to the revolution we need. It did not lead to the revolution we need. That's not that shocking if you think back about it, because the foundation of the internet was really important, but it took several years before DoorDash emerged, Airbnb emerged, Waymo emerged, et cetera. You know, the companies built on the capacity of the Internet didn't emerge when the Internet emerged, but they couldn't have emerged if the Internet wasn't a thing. Same thing is true in medicine — that we needed digitization, we needed our data to be in digital form, in order to change the fundamentals about how you get your health care, and now, increasingly, how patients can take control of some aspects of their health care that they used to depend on the system. But it didn't do it automatically. And I think that's where I got it wrong and where most of us got it wrong. It was just sort of the first step.

And now where we are today is, you know, we now have these new tools in the form of generative AI that do a bunch of things that prior AI and prior digital tools could not do. I'd say most importantly is read language. You know, data that were computable until four years ago were numbers. You could compute the correct— the connection between a patient's blood counts and whether they ultimately got cancer, for example. But anything that required that you understand the conversation that I might have with a patient, or what I document in my note, or the medical literature — we didn't have the capacity to do that. We do now.

And I'm— you know, we'll talk about sort of the government's role now, but, as you well know, the government has taken a very hands-off approach to regulation of the new AI. Personally, I think that's not a bad thing, and I'm not anti-regulation in general, but I think this thing is moving so fast. I think in healthcare systems like mine, there's enough guardrails that already exist. We're not going to implement an AI tool that we think is going to be dangerous or that we think is going to hurt people. We don't do it on moral grounds, but we don't do it as well because we know we'll get sued if something bad happens or if we leak data. So there's, I think, some built-in protections in the system that mean aggressive government regulation of this thing, I think, has more downsides than upsides. But we'll have to see, because at some point these new tools will harm some people, and of course there'll be an uproar. And I think we're already beginning to see a little bit of that in the mental health sphere, where you start seeing some examples of the AI, you know, saying some bad things and kids hurting themselves or killing themselves. And of course you get a massive uproar. And it's an appropriate response where people say we need to regulate this.

Mike Haney: How much is HIPAA still a sort of impediment to what we're trying to do? Has HIPAA modernized to the degree that it needs to?

Bob Wachter: It has not really modernized at all. The rules, as far as I know, are pretty much the same as they've been for about thirty years. I think people sometimes pin things on HIPAA that really are not HIPAA's fault. The data from my Apple Watch into my Epic electronic — you know, into my chart. And the answer is, you can. Or that healthcare systems say, you know, we can't have an agreement with a company that could take our data, or take your— you know, my patient's data, and do magical analyses of it. And it turns out you can. You need a business agreement between the two organizations.

But I think it makes everybody pretty risk-averse about, you know, hoarding data. And I do think there's a fair amount of data hoarding where people use HIPAA as the excuse — you know, "I cannot send your data to this company" — whereas you as a patient might want it sent, because that company might give you some insights about your health, or maybe guide you to a place where you could get cheaper care. And it may be that a healthcare organization doesn't really want you to do that, because they want you to get your MRI from them, not from Joe's MRI down the street. And they'll blame HIPAA. They'll say, you know, we're not gonna send your data around because we're worried if there's a data leak or breach, the fines are enormous. And sometimes when they say HIPAA, what they really mean is, it's not to our business advantage to do that. And I think that's the thing we have to disentangle.

AI scribes and the productivity paradox

Mike Haney: I also want to talk about how the doctor experience has changed, maybe, since that book was written. I mean, I think one of the things the book really does well is — it's easy as an outsider to look at the digitization of health information, when you say, oh, there's a whole book about EHR, and think, well, how difficult can that be? As you said, other industries did it. What's really clear throughout the book is, like, oh, this is orders of magnitude more difficult because of the depth of process, the density of process that surrounds everything that happens in a medical system. You know, the fact that you guys had to like flip a switch, and one day it was paper and one day it was digital, was like giving me the shakes. So I thought, oh my God, to live through that had to be crazy. And that sort of rapidity, again, of that change seemed to be what drove a lot of the discontent early on.

You had a quote that I really liked, that I think was from David Blumenthal, that said the implementation of health IT is not a technical project, it's a social change project. And I wondered how that social change project is going today versus how it was going in 2015. Are doctors still grumbling about EHR? Have you worked out these processes that surround digitization in a way that you just hadn't in 2015?

"Just plunking in a new technology into a complex workflow does not achieve what you think it will achieve." — Dr. Robert Wachter

Bob Wachter: Well, I think in some ways the most fascinating thing, or one of the most fascinating things, about the modern era of AI is many of the early use cases are designed to fix the problems that the last digital revolution created. And so the first kind of ubiquitous tool that we're all — almost everybody — is using is what's called an AI scribe, or sometimes called ambient intelligence. Basically what it is is, if you come in to see me as your doctor, you may have noticed over the last 15 years, since we have an electronic health record, that my head is down in my computer and I'm typing away. And the patient gets the sense the doctor is paying more attention to the computer than to me. And the doctors are completely unhappy about this — the fact that they've become an expensive data entry clerk.

That was a problem created by the electronic health record, because the electronic health record could now make the doctor do stuff. When I was scribbling on pieces of paper, nobody could make me do anything. If anyone wanted to audit whether I was providing high-quality care, or I was writing my note in a way that created the best bill for my hospital, they'd have to pull this paper chart and review 300 pages. Whereas now, of course, that can be judged in real time, and the EHR can be created in a way that makes me do a whole bunch of stuff designed to optimize certain things. And one of the things that really pisses doctors off is those things often aren't really clinical care. They are designed to optimize the bill that we then send to Aetna — which, I mean, we're part of the problem, because that bill partly pays our salary. But what it did then is create a documentation requirement that was massive and did not exist before, where I have to— when the doctor's looking down at their computer, what they're doing is they're checking a whole bunch of boxes. They're putting in certain terms that are buzzwords that create a better bill.

And so one of the early most successful use cases for AI is — most of us are using AI scribes now, and an AI scribe is: you come in to see me today, and I'll say to you, is it okay if I record our conversation just for the purpose of documentation? If you say yes, I'll put my phone down, it will record our conversation and then create that note. Still optimized, again, for billing and malpractice prevention and all the things, but at least prevents me from having to look down at the computer. I can now look at you. You may think that's not that big a deal — like, we've had voice-to-text translation for 20 years — but it turns out that wasn't good enough. A transcript of our conversation is worthless. Actually, it needs to do a whole bunch of things that are very special and highly formatted a certain way.

So, but I think, you know, the broader point is one of the main sort of themes of the first book was what's called the productivity paradox of IT. And the productivity paradox is something that's been seen forever, which is a new technology comes into an industry — and I'm talking here about general purpose technologies, things that really transform the whole way the work is done — and there's great hope and great hype that it's going to be fantastic and improve quality and improve productivity. And the thing comes in, and lo and behold, nothing happens. And everybody's left shaking their head. One Nobel Prize-winning economist said in 1986 — he was not talking about medicine, because we didn't have electronic health records, we wouldn't have them for a generation — but if you went to the factory floor, or you went to the trading floor of a Wall Street, of Goldman Sachs, he said, you can see the computer age everywhere except in the productivity statistics. Meaning there are computers everywhere, but it's not yielding what we hope.

What the lesson of that mantra is, is that just plunking in a new technology into a complex workflow does not achieve what you think it will achieve. The ones that eventually do yield benefit — and the good ones do — often take a decade. And the decade is partly the technology getting better, but much, much more importantly, the changing of the nature of the work, the workflow, in some ways the governance. The leadership may need to die off to have new people come in to say, why are we doing it this way? And the answer is, oh, because we always did it that way and we put a computer in. So that's sort of a fundamental problem in healthcare, and I think in organizations in general.

This one is easier — and when I say this one, I mean AI meets healthcare, as opposed to the electronic health record — for a bunch of reasons. One is there are a lot of individual problems that can be solved with sort of point solutions. Meaning that documentation problem: okay, I'm gonna buy this thing called an AI scribe. It's not that hard to use. I don't have to go to a 10-hour training course. I kind of turn it on, and it's obvious what it does, and it immediately solves a pain point for me. And actually the patients like it too, because patients see that their doctor's paying attention.

Part of what I need to do before I even see you as a patient is I need to review your old medical record. One out of five patients has an old record longer than Moby Dick. So if you've forgotten your great literature course, that's 600 pages. And the idea that I'm going to be able to review 600 pages in the two minutes I have to do it is a joke. It's impossible. I now can press a button and it will do that in 30 seconds. Not perfect, but it's better than I am. And so these are the sort of — and again, takes no learning curve on the part of the organization.

Now, what is going to be much harder is weaving all these tools in together into sort of a holistic, whole, sort of integrated thing, so it's not a complete mess of 50 different tools that we're using. But the overall act of moving from paper as the way we do our work to an electronic health record — which really transformed every bit of work and needed one big monolithic integrated solution, as opposed to what we're all doing now, which is sort of picking out different things that are annoying and don't work and trying to solve them — I think what we're doing now is just easier. The tools are pretty good, pretty intuitive. Where things will get dicey is trying to weave it together into something that really works as a holistic thing, because there's a lot of what we do that's interconnected.

And then, as we get to the topic, your topic du jour, we're talking about a whole new set of problems, which is like what happens when we're not just talking about taking the data that I gathered from you when you came in to see me in clinic — which is a single blood pressure reading, a single measure of your glucose, a single measure of your oxygen — but instead we're trying to deal with data flows that are coming continuously, from your phone or your watch or your ring or cameras in your house or your Alexa or who knows. I mean, we don't have the foggiest idea how to begin doing that, but that's going to take a whole new way of thinking about data massively overwhelming the system. The data we already have overwhelms the system, and we haven't even begun to think about how to deal with that kind of data flow.

The patient side: piecemeal data and a doctor in your pocket

Mike Haney: Before we get to that — and we will — I wanna just finally touch on the sort of patient experience and how that's changed. You know, one of the things that hit me reading the book, which of course is obvious in hindsight, is that my ability to access my medical information was only possible once this got digitized. Like, that was a really fundamental change in the sort of experience of healthcare. It went from being this absolute black box, where I went to a hospital, I went to the doctor, you wrote down a bunch of things, and then I didn't really know what that was, to I can log into MyChart and I can see my labs and I can see your notes and I can see all this stuff.

At the same time, I was just on MyChart this week. It still looks like an interface from 2015. You know, it does not look like a modern tool. We still do not have any kind of universal health portal. I was recently — I got some vaccines, and I was trying to see when I had got my last tetanus vaccine, and I've changed, you know, health systems, insurers a few times over the last ten years. I had no idea where to find that information. And I started trying to create one. I thought, all right, Apple Health, I'll just make that my center. And I was trying to dump in CVS vaccination records and One Medical vaccination. And there was no way to do it. They would not. Everybody's tried this, and that still doesn't exist. So how has the patient experience of sort of interacting with their data — where has it gotten better, and why is it still, in these kind of ways I'm describing, so bad?

"You have access to a tool in your pocket that is as smart in medicine as I am. Actually smarter. It knows more medicine than your doctor does." — Dr. Robert Wachter

Bob Wachter: Yeah, as you describe that, it strikes me, Mike, that there are sort of two revolutions on the patient side. And one is, as you say, a very imperfect revolution, which is access to your data. And you can now get your data from UCSF, but that's not the same as your data from Walgreens or CVS, and might not be the same as your data from another doctor that you saw in a different state. And the feds have tried to create some rules and standards to make it easier to weave that stuff together, and it's been massively hard. Part of it is, you know, in some ways HIPAA and the privacy rules.

You know, you can imagine a world where all of your data were organized around your social security number, for example, or some unique patient number. And it doesn't matter where it's coming from, it automatically flows into this one portal that you have access to. That would be a spectacular thing. But the obstacles to that — there's actually a federal law against a unique — a single unique patient identifier. And so, and none of the entities — you know, my health system, your health system, Walgreens, CVS, et cetera — don't have any great incentive to spend money or political capital on making that happen. So, I mean, you want it to happen, I want it to happen, but it's actually going to require stronger government intervention and probably some government funding to make it happen. I'm a little skeptical that it'll happen, because I don't see the government sort of stepping up to the plate there.

So I think you have a fundamental problem with the democratization of care, which is you're going to have some access to your data, but it's going to be quite piecemeal and therefore not give a complete holistic view of your entire health. Now, that said, I think for most people, if they're being followed largely in one health system, the stuff that's in MyChart or in your electronic health records is probably a reasonably complete view of you — maybe not perfect, but reasonably complete view of you.

But the second revolution, which I think is a very much more recent revolution, is until a few years ago, you had access to all of that data. So the asymmetry between the data I had as a doctor or a health system and the data you had was really profound, and was really breached by now things like MyChart. But you still had a massive asymmetry between what I know and what you know, because I went to four years of medical school and three years of residency and two years of fellowship and I've been practicing for 40 years. And you are, quote, a novice — when it comes to, you know, you know a lot about you and the things that are important to you, but really don't understand medicine. That asymmetry has now been breached by virtue of AI. So the tools that you have in your pocket, if you're using whether GPT or Gemini or a more healthcare-specific tool, makes you at least potentially as smart as I am when it comes to medicine.

Now you might say, all right, what the hell do we need doctors for anymore? It turns out that me using the same tool that you're using — and I'd say you generically; obviously, as a science and healthcare journalist, you know a lot — but for the average patient, when I use a tool, I will get better answers than you get. And the reason is I know what to put in. Often when I'm seeing a patient, I might have 200 pieces of data at my disposal — everything from you came in 'cause your throat is sore, but you're on these ten medicines, you have this family history, you just went hiking in the mountains and drank fresh water from a lake, et cetera, et cetera, et cetera. You had an appendectomy when you were fourteen. The act of taking that and creating a prompt that I might put into AI — this is a 62-year-old man with a history of diabetes who comes in with a fever, a white count, a creatinine of 1.7, and an ALT of twelve. I mean, that's Latin to you, I assume, but completely natural language for us as physicians. And that is what you need to put into the— into the AI to get the right answer.

So there still is some asymmetry, but it's weird asymmetry, because by all appearances the asymmetry is gone, because you have access to a tool in your pocket that is as smart in medicine as I am. Actually smarter. It knows more medicine than your doctor does. So we're kind of in a weird time where you now have access to most of the information, but maybe a little bit piecemeal — but that's actually true for me too. But now you have knowledge tools that really take away this hierarchy of I know stuff and you don't know it. But it still has a little bit of hierarchy in a more subtle way, because there is some subtlety about what you put in to get the right answer that I know and you don't know. So it's sort of a bizarre time. And then, layered on top of this, we're gonna have these new data flows that we're gonna have to figure out where that goes and how to make sense of it.

Pablo's overdose: what happens when humans trust the computer too much

Mike Haney: Maybe going back to the sort of doctor experience, or how AI — maybe as a bridge to how AI is now being used within your system — you have an anecdote in Digital Doctor, a really vivid anecdote, that kind of illustrates why digitization of information isn't necessarily a panacea for all mistakes and in fact can cause mistakes. I'm wondering if you can tell the story of Pablo's overdose, and then talk about how that might be different — how would that situation play out today with the sort of AI tools that are available? And if the answer is not much different, how it might play out in, say, three years or five years?

Bob Wachter: Yeah, I don't know if it would be different today. It's a good question. In some ways the spine of that book ten years ago, The Digital Doctor, was a case that we had at my hospital where we gave a kid — I think 16-year-old kid — a forty-fold overdose of an antibiotic. And it was because of our electronic health record, because it turns out not to be all that hard to toggle between the — it gets a little complicated, but — the dose that you want to give of X number of milligrams, and what we do in kids, which is we say you should always put in the dose per the kid's weight. It doesn't matter for an adult whether you weigh 200 pounds or 160 pounds, the doses are the same. But for a kid — it might be you're talking about a three-pound preemie versus a 130-pound adolescent — the doses better be very different. So the computers are sort of programmed to put in the doses per the kid's weight. In this case, through a series of sort of almost comical mishaps, the computer was set for the dose per the kid's weight and the doctor didn't realize it. So the doctor put in what she thought was the whole dose, and the computer then multiplied that by the kid's weight, which was 40 kilograms, and therefore it was a 40-fold overdose of a common antibiotic.

Then things really got crazy. And by that I mean, this is a dose that is absurd. This is a dose that any doctor or nurse would look at and say, are you kidding me? Forty Septra tablets? That's crazy. The equivalent would be if you were driving the highway and saw a sign that said the speed limit is 2,500 miles per hour. That is a 40-fold overdose. It's patently absurd. You would know it's absurd. And yet people had such trust in their computer that they saw this dose and they said, huh, it must be right. There must be something I don't understand. Because the computer has all these checks built in.

It went to the pharmacy — but of course, in the pharmacy now there is a computer check. But the computer check is not that this is not an absurd dose. It's: is the dose that the robot has pulled out of the bottles — is it what the doctor wrote for? So if the doctor wrote for a forty-fold overdose, the computer's job is to make sure that it pulls up the, quote, right dose, which is the absurd dose.

And the final step, which was sort of the most amazing thing, was a young nurse, who happened to be floating on a floor that she usually wasn't on for a bunch of happenstance reasons, sees this dose, says, this is kind of weird. And yet then she barcodes the pill, which is this modern computer machine that we have to be sure she's giving the right dose. And she barcodes pill number one, and the computer says, well, that's one out of forty. You need to do forty doses. So she has to barcode forty pill things in order to give it to the kid. Basically, what had happened — she had turned her brain off. She no longer trusted her judgment. She trusted the judgment of the computer more than her. Gives this kid this massive overdose, he has a seizure, spends a week in the ICU, and just dumb luck that he doesn't die.

And so, you know, I think in some ways the computer has made care safer. It's gotten rid of doctors' handwriting on a prescription. Prescriptions now go directly, electronically, to Walgreens or CVS. That is ultimately a safer system. But I think that case pointed out that these systems can also cause their own problems — problems that did not exist when we were on paper. And part of the problem is that the humans will turn their brains off, quite naturally will tend to give the computer undue amounts of trust and sort of no longer trust their gut.

And so what does that mean for the current era? I think the current era is that on steroids. As AI gets better and better, begins taking over certain of our decisions or making suggestions to us — whatever it's doing, whether it's writing my note for me or reviewing your 600-page chart or suggesting diagnoses or suggesting the right treatment for you — the final choice is made by the doctor, or I'm supposed to review the note that it just drafted for me. This is probably true in your writing as well. There are a number of flaws in that model, and one of them is de-skilling, which is, over time, as I become more and more dependent on the computer, I'm less good at the thing than I was in the beginning.

But probably just as importantly, maybe more importantly — if you think about the tasks that humans really stink at, I would put remaining eternally vigilant when I've come to trust a technological tool pretty high on the list. It's like, if it was right the last 20 times, are you gonna be— are you asleep at the switch on time number 21? The answer is yes. If you're human, if you're busy, of course you are.

And so we just have to think — I think the fundamental issue as we move into a new era of AI is: if the AI was right half the time, it would be worthless. If it was right 100% of the time, it would be great — I'm worried about what we're all going to do for a living, but that's a different issue. But I think the moment we find ourselves in is, the AIs that we are implementing in healthcare, whether it's suggesting a diagnosis or drafting my note, are now correct often enough to be useful — really useful — and wrong often enough that they do need a human final arbiter to check its work. And that is a system that sounds better than it is. We're gonna have to really think carefully, like, what is that dyad, what is this kind of co-worker or wingman, how does that actually work? How do we be sure the humans still have agency, keep their brains engaged, still have judgment? This is going to be true for patients too, as they begin to use these tools. I think we've only just begun thinking about that. We focus so much on what the tools themselves can do and haven't really focused sufficiently on how does the whole thing work together. You know, 'cause it's not going to be right all the time, and you're going to want your clinician to stay engaged and keep their brain on.

"If you think about the tasks that humans really stink at, I would put remaining eternally vigilant when I've come to trust a technological tool pretty high on the list." — Dr. Robert Wachter

Mike Haney: I think they call that automation bias — I forget what the specific bias is — but where we tend to trust the machine, and as you say, we see that all over with AI now. The couple of other things I thought about — and again, maybe this was because I read the first book first, and the — or the second book first — and the EHR book afterwards, so I was thinking about these AI systems when I was listening to that anecdote. Was, one, you could imagine a world in which there was AI integrated into all of those kind of dumb computer processes, right? The barcode scanning, the robotic pharmacy filling. That would be intelligent enough to flag that kind of a dosage and just say, wait, no, this is crazy. Somebody step in and double-check this.

The other place I thought that I could imagine this being different — you know, part of the story of that nurse was she didn't want to bother anybody, right? She's on a shift that's not her own. She doesn't want to wake anybody up, it's late at night. And I thought, well, could she — if in today's era, she could pull out GPT and say, I'm about to give this kid 38 pills or 40 pills. Does that sound right? And GPT would surely go, that's crazy, don't do that, go ask somebody. Do you think that — are either of those kind of instances, either AI systems within the kind of what I call dumb kind of computer processes, there today? And even at sort of the nursing level — you talk a lot about the sort of doctor use of AI today — at the nursing level, are they pulling out GPT or OpenEvidence and integrating it into their work?

Bob Wachter: The answer on the latter is yes. I think we're all using now AI tools to sort of mitigate our uncertainty. I use OpenEvidence, which is sort of GPT built for clinicians. The way I frame it in my new book is I use it as my curbside consult. Meaning — lots of patients; I'm a generalist. I'm something called a hospitalist, so I take care of sick people in hospitals, but I'm a general internist. And so many times on rounds in the day, I have a question that, you know, I don't need a full-on consult, I don't need a cardiologist to come to see the patient, but I'm 90% sure of the right thing to do, but not a hundred percent sure. And in the old days, I kind of would have hoped that I ran into my colleague in the cafeteria. And if I did, I'd say, Joe, can I run a quick case by you? But now that's what I use AI for. And I think the nurses are beginning to use it for that purpose. So the answer is yes, it's nice to have an advanced knowledge tool.

A forty-fold overdose of this antibiotic does not require fancy GPT. I mean, that is like Textbook 101 from 1970 says don't do that. So the problem here was less sort of an information deficit. And the computers, by the way, did fire alerts periodically saying this seems like a weird dose. The problem — and AI, it'll be interesting to see if it gets this better, if it makes this better — the problem is, if the computer fires every time it sees something that either it knows is wrong or it doesn't quite match the normal template, you will have an overwhelming number of alerts, because there's a lot of stuff that happens in medicine that is a little bit outside of the norm. And the challenge, of course, is alert fatigue.

A nurse researcher at my hospital several years ago did a study where she asked, in our intensive care units, how often does the monitor next to the patient's bed — that's monitoring their blood pressure, their oxygen level, their EKG — how often does that thing fire an alert? And the answer was, across our 70 or 80 ICU beds, it fired 2.5 million alerts per month. One every eight minutes. To the point that — she told me the story — was she was standing by the bedside of a nurse in the ICU, and an alert is firing every five or six minutes, and the ICU nurse seemed completely chill about each of the alerts. And finally the researcher said to the nurse, like, what would make you worried? These alerts are going off, you seem not to be bothered. What would make you worried that your patient was in trouble? And the nurse thought for a second and she said, silence. She said, if there were no alerts, I'd be really worried something was wrong.

So the challenge is, can we— And so we as doctors and nurses, particularly if we have any experience, have learned to just ignore a whole bunch of alerts that in our experience are always false alarms, or 98% false alarm. The problem is, every now and then there's one that's real. So the question — and I don't think the answer is obvious — is, is embedding sort of newfangled AI into our system so much better than what we've had that it will be better at calibrating when the alert is real, and, you know, giving you a real live super-duper alert when it really is necessary, and does not fire when it's gonna be a false alarm? I think we can get there, but it's not like we turn on some fancy GPT button and that automatically happens. It's a really complex problem of calibration, because there's a lot of stuff that we do where, if you say, does this violate the sort of standard textbook way you do this thing, the answer is yes — and we do it all the time, because we know empirically that it works.

I mean, one anecdote that I put in the last book was I spent a day at Boeing, and spent it with the cockpit engineers. And they told me about the way they deal with alerts in the cockpit. And they were just flabbergasted when I told them that 2.5 million story. They were just— they couldn't believe it. Because they know that every unnecessary alert increases the probability that the pilot's gonna do the wrong thing and kill everybody on the plane. And so there may be lots of stuff that's going on that goes to the ground engineers, so that they see how the performance of the plane is going, but they would never ever dream of mainlining that information to the pilot, because it would be a distraction.

And the story that stuck with me forever, as I said — and then, when we do give the pilots alerts, there's a gradation of seriousness. Whereas our alerts often are generic — doesn't matter whether it's, you know, the patient shouldn't take this medicine with grapefruit juice, or the patient's about to die, the alerts often look the same. AI should get that better. But the story they told me was, if the plane is stalling and everyone's gonna die if they don't do the right thing, a red signal comes up on the computer — it's the only time we ever use red in the cockpit — the steering wheel starts to shake, a robot voice comes out, says, "Stalling, stalling." All designed to get everybody's attention. And I said— and they said, then there's a level down, where the color is yellow, there's no steering wheel shaking. And I said, what's an example of that? And they said, well, let's say one of the engines is on fire. I said, that's not the highest level alert? They said, no, it will extinguish itself, but we think the pilot should know. It's like, are you kidding me?

So there's sort of a much more sophisticated way of thinking about this in other industries. In healthcare, there's so much kind of chaos that I think we have to figure out how to calibrate, how to deal with the false alert problem. I think AI can help us with that, but it's going to take a lot of work, a lot of thinking, a lot of engineering.

Can AI act more doctorish? Differential diagnosis and judgment

Mike Haney: Yeah, the comparison between your alert system and all the stuff in alert fatigue was, frankly, kind of terrifying — to realize that so many of those alerts were being ignored. And then the clarity of the Boeing example was really illustrative. But it raised that question for me about judgment, right? About the role that AI can play not just in sorting information or searching information, right, which it's really good at, but in actually making calls about things. So one could be that, right? You could turn an AI system loose on the alert ecosystem and have it judge which time should you think should be red versus yellow. One could imagine that.

The other place I thought about this was — you talk about the concept of a differential diagnosis, and this maybe takes us back a little bit to how consumers use AI as well. And, you know, it occurred to me that the way that the models basically work today, when you're talking to them about health — maybe GPT Health will be different — but they don't think in terms of differential diagnosis, right? And you can explain what that is. But they basically give you what they think is the most obvious answer, unless you really press them, as you probably would as a doctor, to say, don't just tell me what you think these symptoms are. Give me the 20 most— give me the 20 things they could be, and then give me a probability ranking of what you think.

Bob Wachter: That actually is a good thing. So this is like, if you do a Yelp search, and you search, and then you say, give me by rating and then give me by distance, for example. Those are the two sorts that I want to see in terms of thinking about the list of potential diagnoses.

Mike Haney: And how good are the AI tools today at that level of judgment?

"Patients have no real ability to know what to put in and how to interpret the results. It doesn't make them stupid, it makes them normal." — Dr. Robert Wachter

Bob Wachter: Pretty good, but only if you prompt them to do that. And that, again, is something that I will naturally do, because that's the way my brain works, and a patient might not naturally do it. And one of the things that's been interesting as we sort of look through the literature about research on AI and healthcare is, you know, the tools that we are all using — this tool called OpenEvidence. It's a company that didn't exist a few years ago. It really is GPT built specifically for clinicians, and mines not the entire literature — and therefore The Onion and Reddit — but is really focusing what it's mining to get its answers on the medical literature, on respected journals, and on guidelines that come out of respected societies. And in the space of two years, it's become almost sort of ubiquitous in medicine. The last month, it had close to 30 million searches.

But when I use a tool like that, again, I know what to put in. And when I look at its output, I know — you know, I have a sense, like, that first diagnosis it gave me, oh yeah, that's good, but I thought of that. Number two — I hadn't even thought of that. That's good. That's really helpful. Number three — no, that's absurd. I'm going to ignore it. Patients have no real ability to know what to put in and how to interpret the results. It doesn't make them stupid, it makes them normal. They're normal humans. They've not gone, what you know, from novice to expert, which is what you hope your physician has gone through.

Does that mean that it's futile? Not at all. I think the question is, can tools be built that act more doctorish than the current tools? Meaning, you say, I woke up this morning with a sore throat. I would never give you what I think your diagnosis is. I now have about 10 questions I need you to answer — do you have a fever? Are your lymph nodes swollen? Are you on any medicines that suppress your immune system? Et cetera, et cetera. I can't even begin to answer what's going on and create that differential diagnosis — meaning, the likeliest thing is this, but I'm also worried about this, this, this, and here's the tests I'm gonna do to sort that out. Can't even begin to do that until I have the answers to those questions. I may also need to do a physical exam, which obviously the AI can't yet do.

But where this gets interesting on the consumer side — it was a wonderful study that was in— out of Oxford last year or early this year, where they created prompts that had obvious answers in the medical world. Here's a good example: patient woke up this morning and had the worst headache of her life. You tell that to any doctor worth his or her salt, they will say to you, that is a bleeding around the brain, called a subarachnoid hemorrhage, until proven otherwise. We learned that on the second day of med school. And the patient needs to go to the ER, like, stat. Do not pass Go. Right away. When they put those prompts into GPT, that's exactly what GPT said: this is a subarachnoid hemorrhage, go to the ER. What they then did was they gave the prompt to the patient and said, now interact with GPT. And the patient took "this is the worst headache of my life" and put into GPT: I woke up this morning and I had a really bad headache. And GPT said, oh, I'm sorry to hear that. You should take some Tylenol and rest. Because the patient had no idea that "the worst headache of your life" is actually a term of art that we use to differentiate bad headache from awful headache.

So, you know, I'm quite hopeful that these tools will be more and more helpful. Part of it is, as GPT for Health or Claude for Health now do, part of it is they now have the capacity for you to put in your past record, and for them to know about your background, which sometimes is quite useful — because a headache in one person might have a very different meaning from a headache in a person who was on immunosuppressive medicines or had a history of cancer. So part of it is knowing enough about you to give you a more customized answer. But I think in some ways more importantly is, is it acting — does it have the judgment? Does it know the right questions to ask that a good doctor would have? I don't see any reason the tools can't develop that, but the generic tools aren't quite there yet.

The continuous data revolution: promise, hype, and heavy lifting

Mike Haney: I think that's, excuse me, a good bridge into the kind of continuous data world that we want to talk about before we end here. So the thesis behind this — excuse me — the thesis behind this series that we're doing is that we're trying to imagine a world in which healthcare data is much less episodic and much more continuous. And so you can start to see the little glimmers of this in the Apple Watches, the Whoops, the consumerization of continuous glucose monitors. And even in the kind of breadth of things we can measure — so Abbott just announced a continuous ketone monitor to go along with the glucose. There's a whole bunch of labs working on various sensing modalities, and we're going to talk to them as part of the series, of different ways to do this kind of measurement.

But then the question is, what happens with all that information once it sort of gets into the system? And maybe where I want to start there is, how much is AI today — well, I'll frame it this way. I talked about sort of the sensors and the tools. I think that's a kind of push model of how this world comes to be, and I think five years ago we would have said that's what's gonna drive it, is that the Abbotts, the Dexcoms, the new startups of the world are going to want to make these sensors. Now I see a real push-pull part of this, rather, which is the foundational models companies want to get into healthcare, because it's an enormous market and they are ever more data-hungry, and so they're gonna start to drive, I think, some of the development of these tools that can bring them more data all the time. And I'm curious, where do you think the AIs today are held back by the amount of data or the diversity or distribution of data? I was thinking about the Mayo Clinic Platform — and you can describe what that is — which seemed to be a bit of an effort in this direction, to grab a more diverse array of data to inform its tools and how it's thinking.

"Up until very recently, the only unit of both measurement and intervention was the doctor's office visit. And that's got to change." — Dr. Robert Wachter

Bob Wachter: Yeah, well, first of all, I agree with your premise. I think over the course of the next decade or two, it seems inconceivable to me that we will not be leveraging this more continuous measurement system to have better insights about a patient's health, what they need, and better ability then to have an effector arm that, in the case of an insulin pump, sort of changes the medicine you're getting in real time, but in other cases may just deliver to you coaching to say, you know, if you change your diet this way, or change your exercise regimen or sleep regimen, that things will be better. There's a lot of steps between that vision and where we are today.

I'd say this is an area that's hyped a lot without a ton of data demonstrating that that kind of measurement really leads to improvements in patients' outcomes. And when you think about the patients that need this the most — the kinds of patients that, you know, I might see in my office, have five chronic diseases and are on ten medicines and are really at high risk of a health problem — that's very different than the patients that I think are most likely to be using these tools, who are often relatively healthy young people who are kind of into a wellness mindset. Which is all perfectly nice, but I think it's yet to be proven that taking all this data and putting it in some fancy AI machine and spitting out an answer is demonstrably going to improve their health. I can see it— I can't see a reason why it wouldn't, as we come to understand this data better and understand the connection between, you know, your heart rate is doing this, or your urine analysis from your wired toilet tells me this, and we know that that is associated with this bad outcome ten years from now, and we know that if you change your diet this way, your exercise this way, that that will decrease the probability of that bad outcome. That seems logical that that will happen, but there are a lot of steps between now and that. And I do worry a little bit about sort of overhyping this, because I don't think we've made those connections yet.

That said, how can it possibly be that the best way for me to manage your high blood pressure is I measure it once in my office every six months, and at that point I adjust your medicines, and I'll see you again in six months to see whether you got better — when you now have the capacity on your wrist or on your ring to have your blood pressure measured in real time, to potentially even have your blood pressure adjusted with medication changes or lifestyle changes in real time, to get you real live-type coaching, like, it doesn't seem like you took your medicine today, or, you know, I think we agreed that your regimen should be, you know, ten thousand steps, and it looks like you only had two thousand today. I mean, that could be annoying, but I think the conceptual change here is, up until very recently, the only unit of both measurement and intervention was the doctor's office visit. And that's got to change. And I think that's a very exciting era — that's going to change it.

Now, where does AI fit in? First of all, in taking all those data and trying to make sense of it, which I think is partly a research question as opposed to an individual care question. But once we understand the associations between those data and better health, then, yes, trying to figure out ways that the stuff is measured, you get insights into what's going on, you have recommendations that are evidence-based about what you should do, you have coaching that helps you do it better. I mean, that all, I think, has to happen.

Where are the challenges here? To the extent that what we need to know in order to make sense of your data is not just what's coming off your watch, but what's in your electronic health record, those two data sets need to get connected. To the extent that what is being measured may at some point mean you actually need to see a doctor — like, your blood pressure now is way too high or way too low, or you're now in atrial fibrillation — what's the connection between this ecosystem of ambient real-time measurement and actually connecting you to a health care system? Because what you have is scary.

What can't happen is all of that data that you're now collecting in real time gets mainlined directly to your doctor. Your doctor's already overwhelmed by data. The idea that they're now going to get continuous data feeds on the eighteen hundred patients they're following — they'll quit this afternoon. So somehow AI needs to be monitoring all this, getting you the information as a patient that you need, connecting to the healthcare system when the algorithm says you're actually not doing well and in trouble and need to see a doctor. Know what has to happen: do you need to go to the ER, or do you need an appointment with your doctor sometime in the next two weeks? If you need to go make an appointment, does it connect to the scheduling system and get you an appointment, and then get the doctor the information that that is why you're being scheduled, because this thing is being measured?

So even just saying that is exhausting. I mean, the idea of connecting all of those pieces together — I think that's like a decade-long — and not clear who's gonna do that. Is Dexcom gonna do that? Is GPT gonna do that? Is Epic, the maker of the electronic health record, gonna reach out from the EHR and do that? Is Apple gonna do that? Each of those companies — or is my health system going to do that? Each of those entities has some part of the puzzle here, but in order to make this whole thing work, it's all got to get woven together into something that's like the world's best air traffic control system. That's gonna take a decade or two to get right, if we ever do. It's really — it's a pretty heavy lift.

Mike Haney: Yeah, that's what brought me back to that sort of universal health portal question, right? If I can't even get all my vaccines into one place today, the idea that my Whoop data and my CGM data and my whatever new sensor data that I have is gonna all go together and then do this intelligent dance with my health system does sort of both seem like a very plausible future, as you say, but also seems really hard to imagine how we get from here to there.

Bob Wachter: Yeah, but it's just important to almost disentangle those two things. One is, can you get all your vaccine data and the ten medicines you're on in one place? Those are in some ways relatively static pieces of data that, once they're in there, okay, they're in there and they're gonna get leveraged into sort of my health system. That's fine. There's sort of a second volume problem, which is, all right, if we do that — but we're no longer talking about ten discrete pieces of data that maybe one of them changes every four months, or I get a vaccine once a year, it's got to update my flu vaccine. But now it's a data stream — and twenty different data streams — being developed constantly, any change in which might be trivial, or maybe should influence the way I think about how many steps I take tomorrow, but might mean also a life-threatening emergency that therefore needs to connect back to the healthcare system. I mean, that's a problem — that's a mind-blowing set of problems that you don't fully solve just by getting all your data into one place.

Mike Haney: Yeah, and that's the spot where I find myself wondering if how difficult it is for me to wrap my head around that is just me being limited in how I'm thinking about the capabilities of the AI tools or the laws of scale when it comes to compute, right? Because they're already doing things that five years ago we would have said were impossible. They're all obsessed with hyperscaling, and until the community start really, you know, holding out the pitchforks in the data centers, they're going to continue to hyperscale. And how much of it is that lack of imagination on my part, and how much of it is really structural, both sort of in terms of the infrastructure — is it just too much data to sort of do anything with intelligently? And how much is it structural in the sense of we would be asking the AI to do something that it can't yet do — that the kind of connections, the kind of analysis it can do, it just can't do?

And that's what made me wonder about the sort of training data side of it, right? Like, it makes sense that OpenEvidence can look at all the papers that are out there in the world and then give you a pretty smart differential diagnosis. If I'm asking it to look at a complex array of values for me, and then tell me something about it, given the heterogeneity of humans — is that where we're gonna bump into a thing where, like, well, it's only trained on this little bit of data that OpenAI could somehow steal from the internet, and it was all in, you know, 40-year-old white guys in San Francisco, and now we're asking it to look at a diverse array of people? How much do you see the — do you bump into any places now where AI seems to be limited by its lack of medical training data? Or do you see that as a potential hurdle coming down the road?

Bob Wachter: Yeah, even just as you describe all this, you know, my head is exploding on trying to figure out how all this works. I think it would be hard, as you look at the last four years, to say that the lack of training data is going to be the obstacle, or the lack of compute is going to be the obstacle. I think the obstacle— or, you know, I think the integration of all your data into a single data pool that gives the AI a complete picture of your health — that also seems surmountable. It seems like that's not completely infeasible, that we could get there.

I think where the real challenge here — the one that strikes me as being maybe the hardest one of all — is essentially medical research. It's the connection, once, assuming some mythical state of a decade from now: your data all are in one place. We've managed to connect all the dots. The limitation on collecting and dealing with all this much data is not— we've overcome that. We're connected to the healthcare system in a meaningful way, so that you're able to self-care where you should, and where you shouldn't, and you really need to see a doctor or get a test or whatever, that is connected in a way that the algorithms understand, and those wires have been laid.

Where I think the real challenge is — and this is where I think things get a little bit in danger of being overhyped — is most of the interpretation of, you know, the meaning of your heart rate variability, or your temperature variability, or your sleep patterns, or whatever, or your urine that's being analyzed by the smart toilet, or your stool that's being analyzed by the smart toilet — most of those sort of connections between that piece of data and what your health outcomes are, and what you should be doing to improve your chances of living better and longer, I think have not been worked out. I mean, the hypesters would tell you they have, but they just haven't. We don't know what these things mean, and even when we kinda know what they mean, we don't really know what the interventions are.

Now, you might then say, this is the field, what we call real-world evidence — that let's get over this old fuddy-duddy idea that we need to do some double-blind controlled trial of 10,000 people where, you know, where they ate more cheese, and 10,000 people that ate more tofu, to see which ones did better. You know, that's the old way of thinking. We'll just analyze a hundred million people and look at the patterns that emerge that tell us what the right things to do. It may be. That's a really hard problem from a kind of medical research standpoint, because of the heterogeneity of people, because the confounders of someone who's living a healthier lifestyle in a lot of ways — and then trying to take one piece of their heart rate variation, or their oxygen saturation, or their sleep pattern, and say that is the thing that, if you could just fix that, they would live longer and better. There's a lot of steps between A and B there.

And so I think that's the — you know, the things that we're talking about — compute, connectivity, connectivity to health system, connectivity of all your data pools into one place — those strike me as sort of human problems and maybe technological problems. But actually truly understanding what are the interventions that flow from that, that actually are meaningful, make a difference in your health and healthcare — you know, I think we'll get there. I mean, I think today I would like to know what your blood pressure is all the time, rather than just when you're in my office every four months. But for a lot of the other measurements that you're taking all the time — and, for some patients, obsessing over all the time — you go to your doctor, and the doctor says, I don't really care about that, because I have no idea what its meaning is. And there are hypesters out there who will tell you you need to know that piece of data because it's massively consequential as to whether you live to 100. I think we're gonna have to prove that that's true. I don't think we're there yet.

What's actually worth measuring continuously

Mike Haney: Yeah, I think one of the things we've seen — you know, we've sort of played in the continuous glucose monitor space, so people without a diabetes diagnosis paying attention to their blood sugar, and a whole bunch of benefits to that. But one of the things we've definitely observed is that it risks pathologizing normal variability, right? That people look at a thirty-point spike when they eat blueberries, and they go, oh my God, I can't eat blueberries anymore because they're not good for me. I'm curious — you mentioned blood pressure. Are there any other areas, especially as a kind of generalist, that strike you now as, boy, this would be better if I had longitudinal data rather than episodic data?

Bob Wachter: Well, yeah, I'm glad you brought up diabetes, because I think, in some ways, people are extrapolating from diabetes. You know, in an area, you know, that is a measure that we used to insist that patients — I mean, I remember when I was in my clinic, patients would come in with this graphical spreadsheet where they'd put dots down on all of the— And they're checking it by sticking their finger four or six times a day. And we now know that tighter control is better than looser control. And so the ability — I mean, if you told my thirty-year-old self that we'll have a place where we're able to measure that variable constantly, understand its implications, give that data to the patient and or their clinician, and adjust their insulin in real time through a closed-loop system, I would say that's a dream. I mean, that's magical.

But I think there's a danger of extrapolating that — this one metabolic parameter that changes all the time, and we can influence in real time, and we know that there is real live health impact to what happens with that piece of data — it's a little risky to extrapolate that to your heart rate variation or your sleep patterns or your urine, et cetera, et cetera. So what else can we— what else are— what are other variables like that? I'd say sort of your glucose level is the one that's sort of most obvious to me. Your blood pressure is a little bit like that — but there, I don't need a measurement every second. I could use— maybe once a week would be great. Certainly for the patient with a known pathology, like atrial fibrillation, I would like to know how often they're in atrial fibrillation, which might ultimately influence their risk of a stroke and might influence my decision about how to try to control it. But for someone without known pathology, God, I'd have to do a lot of head-scratching to know what are the variables that I really want to see continuously versus episodically.

Now, it may turn out that, you know, the cholesterol I check on you every six months or every year — if I knew what its pattern was, and it goes up and down, I might see that, you know, actually, your cholesterol looks good every time you come and see the doctor, because you go on good behavior and change your diet, or you actually do take your Lipitor, the week before you come in. But if I knew what it was two months ago — you'd stopped taking your medicine, and it was awful. That might be useful. But I'd say it's important to say the glucose-insulin example, at least in today's understanding of medicine, is by far the exception rather than the rule. And for a lot of the other things, I think it's a little bit of hand-waving to say we really know that more continuous measurement of this thing is super salient.

And I'm thinking in medical terms. I mean, it may be continuous measurement of your mood is really important. And if we can determine that from your tone of voice, or the number of words that you speak per second, or whatever it is, that that is super useful — you know, there's stuff that I can imagine being useful. But I think before we just sort of say it is, because we can measure, you have to weigh the downsides of pathologizing everything and making people crazy over the upsides of measuring it, intervening on it, coaching on it, et cetera. And, you know, I think right now that's a pretty short list.

The greatest experiment in the history of medicine

Mike Haney: Maybe as a place to end — you know, at the end of the first book, I've heard you describe it as twenty-five grumpy chapters and one optimistic one. I have to say, the book did not strike me as that grumpy. I think in both books there's actually a real tone of optimism, particularly in the AI book, and a real tone of balance. And I wondered if your journalist wife sort of helped you with that — I feel like you did a very good job of threading the needle of kind of, you know, objectivity, and it didn't feel like a sort of angry screed around, you know, EHRs. I'm wondering where you sit right now in terms of that optimism as you look ten years ahead. Do you think — given where the gov— all the things we've talked about: where the government is in terms of its intervention or lack of intervention in here, where the relationship between the medical systems and the health tech companies is, where patient demand or understanding is right now — what's your sort of rosy picture for where this could be in ten years?

"This is going to be the greatest experiment in the history of medicine. It could go off the rails in a hundred different ways." — Dr. Robert Wachter

Bob Wachter: Yeah, I mean, I'd say my last book — it wasn't super grumpy, but it was written largely out of frustration about, like, how badly the digitization went, when in some ways it's measured against the yardstick of how optimistic I was about it. It was that disconnect that I saw in a lot of my colleagues, and that I saw, that led me to write. And I came to believe — I think it ends on a note of optimism, because I said, I now see where this is gonna go, but we need better tools, and just digitizing the record was not the answer. It was the foundation. So in some ways it anticipated the moment we're in now.

The moment we're in now is one where I have a fair amount of optimism, in that these tools are magical and can do things that we've never been able to do before. But also, the healthcare system is in such desperate need of help, and has a number of problems, largely related to the amount of data — and it's only going to get either better or worse, depending on your point of view, when all of these new data sources flood into us — that now we have tools that can address that. To me, that's extraordinarily exciting. It opens up massive possibilities for good.

But the way I frame this is: this is going to be the greatest experiment in the history of medicine. It could go off the rails in a hundred different ways. The democratization of care is fantastic, I think — allowing people to get the things that they need, have much more agency, be more empowered, probably do things less expensively and more conveniently. It also is the opportunity for immense amounts of mischief, because you're no longer seeing a doctor who took the Hippocratic Oath, but you may be getting your care from a company that's there to make profits off you, and the opportunity for mis- and disinformation has never been greater.

Within the healthcare system, you know, these tools can be used for good, but they also can allow us to turn our brains off and lead to levels of de-skilling, that we're giving the tools undue amounts of agency and undue amounts of trust. And I'd say the trust issue is particularly challenging with generative AI, because it seems so human-ish. That, you know, we tend to trust people more than we trust companies, but these tools, when you talk to them, feel like you're talking to a colleague — and even a sycophantic colleague who'll tell you how wonderful you are.

And then, on top of it, you have the general mistrust of these tools, of AI, which is growing, for reasons I completely understand. The backlash that's likely in every profession, including nursing and medicine, if people feel like their jobs are being threatened. The overall risk that we're all worried about, of bioterrorism, of — you know, if we hit 15% unemployment, I think there'll be a revolution. All of that kind of stuff means that, as wonderful as these tools might be, the implementation of them in healthcare systems has to deal with the fact that there will be some pushback by the incumbents. Has to do with the fact that, even if the tools are really good — I use the Waymo example a lot in the book. Waymo is demonstrably safer than riding in a car that you're driving or Uber's driving, and yet six months ago, as you probably know, Waymo ran over a cat in San Francisco. It became international headlines. So we're going to hold these tools to a very high standard, and at some point they're going to kill people, and that will become a cause célèbre, and everybody's going to talk about how, you know, see, we can't trust them — which will lead to a backlash. So there's a lot of tricky kind of socio-political, ethical steps that probably, to me, are more complex than the technology steps.

But when I take a step back overall: you have a healthcare system that is not delivering what people need. Quality, safety, convenience, the burnout among clinicians, the cost of care. And these are problems that I think, if you net them out, it's likely that AI will make substantially better compared to the status quo. But part of my optimism is I think the status quo is pretty terrible, and I can't see how we get to a much better place without embracing these tools and trying to do it thoughtfully and sensibly and ethically.

Mike Haney: Well, I look forward to the book in ten years reflecting on how that revolution went.

Bob Wachter: Yeah, I would look forward to writing it.

Mike Haney: All right. Well, Dr. Bob Wachter, thanks so much for joining us today. Appreciate it.

Bob Wachter: Thank you, Mike. It was a joy.