Roy Bahat: ’Learning’ and ’doing’ should no longer be separate

Roy Bahhat, head of Bloomer Beta. Very happy to have you on our still burgeoning but but I think you know relatively in the maturing mode show aiEDU Studios. Roy, you and I met a long time ago before most people were talking about AI.

You were talking about the future of work. I don’t know if you would have think thought of yourself as an AI person. You certainly were one of the the thought leaders on the conversation about the future of work which was about more than just and I think is about more than just AI.

But how do you describe yourself now? How you know is is it any different from from when we would have first met back in 2018 2019? I mean I we’ve been investing in AI as a firm since 2014.

And so for sure AI would have been then my number one investment area. And I don’t know, am I an AI person? We’re all still figuring it out.

So like the most expert people I know in AI are the ones with the least confidence about what is happening right now. And so together we’re all trying to shape this understanding alive as it’s happening. So I think in that way it’s a lot the same.

You know, I teach at Berkeley. And so I have some limited amount of understanding of the educational setup and a parent of teenage kids. And so that’s how I relate to all this.

What do you teach at Berkeley? I teach in the business school. I teach teach two courses.

One on the business of media and the other on leading and organized or organizing workforce because I spend a bunch of time with organized labor. So, you know, I’m a VC at Bloomberg Beta who believes that AI is the most important technology trend that’s affecting our world and workers organizing is the most important human trend that’s affecting at least our economic world, our working world. And so, those are the areas where I spend time.

Excellent. Yeah, I would like to dive into that to that. Putting putting on your hat as I mean there’s so much that I you know, I was thinking about what to talk with you about.

I want to start with actually just sort of like the most recent stuff that I’ve been obsessed with and I’m curious if you’ve had a chance to to think about it. So first Bureau of Labor Statistics they published I think it was in February but they do they still publish statistics? Well this was this was in February.

So this is that that question will have to be answered sort of in real time. I don’t know if this is like an official update to their their stats. I’ll be honest.

I’m not extremely deep into BLS’s sort of the arcane details of how BLS works, but they they basically published this update to some of their predict their projections to incorporate AI. And the thing that that jumped out at me was they’re predicting I think like 18 to 19% growth in computer science jobs over the next 10 years which stands in stark contrast to what I have heard from folks who are really on the front lines of this technology and their expectations about it’s the impact it’s going to have on on employment. Hot take I mean I don’t know if you if someone all employment predictions are stupid.

Yeah. I mean the BLS as last I recall took current trends and just extrapolated. They have been wrong many times but that’s because prediction is really hard and you know one of the reasons why it’s so hard is first of all you have two effects.

You have an income and substitution effect for the economists. It’s basically like you make it easier to be a software developer and make software developers more productive. There are reasons to think you might want less, fewer software developers because each one can do more.

And there reasons to think you might want more because you want more software developers compared to marketers or something like that. And so both of those effects happen at the same time. But the other thing that happens is work constantly gets redefined.

I mean what a teacher does today compared to what a teacher did 30 years ago. Many of the things are the same and some of the things are different. And so the faster we we do these things, these changes, the more we reabel like what is a software engineer today versus 10 years ago.

It’s a different kind of a job by the way. It’s a different job now from what it was a month ago. And so meaning the AI tools have been improving that fast.

And so I don’t know how to make predictions about it. And I’m not even sure why they matter. Like let’s say I told you all that we were going to have 10% more software engineers and not 10% less or something like that.

How much would it affect things? I’m just not sure. I mean I Well, let’s I think it would affect things.

I mean certainly if you’re I mean if you’re a junior in high school Okay. And you’re trying what job should I prepare myself for? Well, and also, should I take on $50,000 of debt per year to go to a tier 2 computer science program?

Totally. And I think the answer to that is as hard to answer now as it was to answer 5 years ago, which is honestly I just don’t know in a lot of cases. And one of the things that I believe is happening and this is a place where AI can be very helpful in a learning environment is, you know, the old model, the model that I went to school under is first you learn and then you do.

You prepare yourself and then you do the thing. The tools have become so good that there’s no reason why learning and doing should be so divorced from each other. So I look at schools like I think we’ve talked about this high school for entrepreneurship in Fresno, the Patinho school.

Those students graduate with businesses that is wild. When I look at a tool like Replet, we were investors in Replet. Replet started out as a learning tool.

We are investors in Replet. Started out as a learning tool for classrooms to learn how to code. Now I just type in plain English what I want the software to do and it does it it makes it it and you know I have to learn how to do that.

It’s not automatic. I have to learn like the way you learn riding a bicycle. But what I so so my own kids ask me what job should I go for?

And I’m like I honestly don’t know. But I think the parents you this should this should give a solist to parents who feel like they don’t know the answer to the question because none of us knows. Roy Bahhat doesn’t know that.

The only answers that I think are true are either true by luck or they are And so what I tell my kids is learn how to make your own money. Learn how to continuously improve your own skills because you know I just spoke to a group of HR professionals yesterday and this learning and doing thing had lots has lots of in the workplace analoges which is it used to be you got to the workplace and you expected your employer to teach you how to do your job. You went to training sessions and of course all that is still true to some extent but it is the people who will themselves learn how to use the skills and be I call this in the terms of use of technology being the CIO of your own life the chief information officer of your own life if you are constant if you’re the person constantly doing that I mean somebody on that panel described perplexity which is a company that we in tech know well it’s a search engine that I use as an alternative to Google because it looks things up and finds the articles on the internet using AI and then writes a summary.

It’s like what I want Google to do. And somebody on stage called it that pea company. And I almost laughed because it’s like the level of knowledge that those of us have who are paying attention to trying to improve the skills and the level of knowledge others have is just really different.

And so I think my they couldn’t think of the name. What’s that? They couldn’t think of the name.

Yeah. It wasn’t even couldn’t think of the name. I mean it it was more that it they it struck them as some new fangled tool that had just come out when it’s been part of my core workflow for two years.

Like I wouldn’t forget the name of Gmail. And so you know to me the the the advice to students is going to be less about which job title and occupation do you choose and more about skills that will be robust across different occupations. And so is it worth learning computer science?

I definitely think it’s worth learning. Is it worth taking $50,000 of debt to do it? That I can’t speak to because that gets to like your family’s financial circumstances.

And so I’m not going to say yes, it’s for sure worth it. You should do that because I just don’t know. I think this might be actually an area where you and I you know have sort of like slightly differing points of view.

I can learn from I my my instinct is if you look at some of the research of of how generative AI is used right now in knowledge work what concerns me is that you’re seeing like a few things like there’s a deterioration of skills among people who are overly reliant or comfortable or confident in the tools and so it’s almost like it’s ironically like the more confident you are in AI it’s actually you’re actually less and less effective because what you need is someone who is sufficiently skeptical and like really paying attention. It’s like the if I don’t know if you have a Tesla, but I have my friends who have Teslas will will tell you that like this all drive I’m a venture capitalist in San Francisco. Of course, I have a Tesla.

I’ll sticker that says I bought this before Elon went crazy, but Good. So, we we Can you tell it? I mean, have you do you use a self-driving mode?

Do you still I don’t I do you find yourself like are you hold two hands on the wheel? No, of course not. Like all of us, I do the thing where I try to pay as little attention as needed before the system times me out.

And I even have special sunglasses in my car so it can’t tell if I’m paying attention cuz there’s a camera, right, that will actually sort of like look to see. It’s creepy, too. And this is despite you knowing that there are plenty of stories of people who overly relied on self-driving mode and got into accidents.

Yeah, for sure. Isn’t isn’t this going to be something that companies care about? If you’re an accounting firm, if you’re a law firm, of course, the idea that well, you don’t I mean, because I I think what I’m pushing back on, and I’m not to I don’t want to straw man what your your point of view, but what I push back on, you know, at the summit that you spoke at, I was there as well, and you’ll hear people say stuff like, well, all people need to know is just they need to just learn how to use AI, and that can replace sort of like the the the knowledge and skills that they’re currently learning.

And I I I never said that. You didn’t say that. And so, I would never say that.

So, so, so help me, help me thread this, Neil. Yeah. So, this is how I thread that is if you rely on the AI’s outputs and take it at face value, it’s the same as if you hired somebody to work for you.

I think of AI as like a really good intern. A lot of people have said this. If if I rely on the intern’s facts, I mean, I remember I wrote a post about learning to code in like 2012 or something like that.

Asa Hutchinson when he ran for governor like excerpted that piece and put it off as his own policy and somebody Buzzfeed or somebody wrote a story about it. Why did that happen? Because the intern put it there and passed it off as their own work.

The governor, the gubernatorial candidate didn’t do that. In fact, he had the class to call me and apologize. And I didn’t care.

I was just happy that the work got out there. But I cared a lot that he was classy enough to own his work. So I don’t think you can own I don’t think you can rely on the AI’s outputs and just treat it as finished.

But I do think that the people who are more reliant on AI to get the work done to take parts of the workflow over are going to perform much better. So that’s the trick. And like here what I don’t want to have happen is every like you know it’s really sad that everybody’s learning how to bake bread because they just don’t know how to make flour anymore.

And it’s really important that human beings know how to mill the flour themselves. It’s like no, somebody has to do that and I need to know as the buyer of the flower that they’ve done it in a way that’s safe and blah blah blah. But what we are learning right now is intuition over when can I trust the thing and when can’t I when do I have to double check I mean somebody just did a profile on me that a fundraiser did where they used OpenAI’s deep research to do like a dossier on me and it was amazing because it had insights in there.

Was like, "Oh my god, if somebody tried to fund raise for me using that, it would totally work." Like, it understood my psychology. But the very first bullet said I was the son of Holocaust survivors who went to NYU, and I’m not the son of Holocaust survivors, and I did not go to NYU. I am the grandson of people whose siblings died in the Holocaust.

And so, it’s kind of truthy in a certain way, but that’s so I think the way to square the circle you’re describing is we need to learn the skill of when to trust the outputs and when not to. And blind faith is basically always stupid. And and what you just when you describe for example like a school leaning more into providing students with this opportunity to to do entrepreneurship like as as I can attest you know building a company requires a heck of a lot of expertise and like the rapid acquisition of expertise in things that you don’t know.

So to me that’s an that’s that’s what you’re describing is actually different formulations of how students are building and capturing that expertise and how they are representing their their mastery and knowledge and how we’re assessing them. It isn’t necessarily that but but I think this is important right you’re you’re not saying that they just need to get really good at the it’s like necessary but not sufficient. I think being a user of the AI yeah words like replace I am really hesitant to use because replacement first of all it’s lazy intellectually because you sort of imagine the job as it is and you the only thing you imagine is the replacement so like instead of me doing the dishes there’s a robot doing the dishes you know something like that instead what I think ends up happening is that the activities shift around as we redefine how we do it I mean replacement to me makes me think you know that story about how the early ads on television were just like radio ads where the announcer just sat there and read the ad.

That’s replacement. Replacement is stupid. But what will happen is redefinition of everything.

What what experiences will I need and won’t I need? And it’s going to keep changing. Like it’s not the other model that’s broken is the model of learn then do because you know we’ve all talked about lifelong learning, but learn then do assumes you get trained at the beginning of your career, then you do your whole career and yeah, you improve as you go by learning on the job.

But I actually think it’s going to be much more continuous, which maybe is obvious. You don’t make predictions. So maybe you can just sort of give me feedback on this.

An idea I have is that there there’s this assumption right now that AI is going to be really good at replacing entry- level jobs. So maybe start with that assumption. Does do you buy that?

That like if you look at sort of the jobs that that a company has, it depends really just the entry-le jobs. I’m not I’m not convinced by that. No, I mean I think what is true the the big surprise with generative AI so tools like chatbt and Claude and replet etc. And you know there’s lots of other kinds of AI like we invest in AI that makes aircraft flight routes more direct.

You know that doesn’t have anything to do with generative AI or very little anyway. They may use some generative AI somewhere. There’s the the big surprise is everybody used to be worried about all the low wage work being automated and now I know a lot of software developers who are worried and lawyers who are worried and etc etc. and so I think that’s a major shift.

I think that the other thing is it’s going to vary a lot depending on the nature of the work. So like some care work is going to be really hard to automate. Like you know I think we did focus groups you know we’ve been working on these things for a long time and in 2016 I think we did focus groups on on people who in addition to their job cared for an aging relative cuz the multigenerational thing is going to be much more of a thing.

Like by the way, the one prediction I’ll make about work is people are going to be older. Yeah. Which is huge and so obvious that we don’t even think about it.

Like every like how are the kids going to learn AI? You’re working on education. It’s essential.

But I’m actually much more worried about what 55 year olds are going to do than I am about what 15y olds are going to do because there’s going to be more 55y olds at work than 15 year olds or 18 or 21 year olds. Interesting. Yeah.

And and the proportion of the workforce that is older is going to grow. And so I don’t know. I guess I’m not sure if it’s giving you like a satisfying answer because I think a lot of this is just unknown, but it’s going to vary by occupation.

The thing that people will want is anything where the person demanding it wants it to come from a human. And the focus group example is we talked to a guy in the focus group. I was behind glass or whatever and he was saying how he cares for his like aging aunt or something like that.

And the focus group person said like, "You’re a successful person. You’re like a he was an executive at an insurance company or something like that." And she’s like, "You can afford to have somebody care for your aunt. Why don’t you do that?" He’s like, "Cuz my aunt doesn’t want somebody.

She wants me." And that you can never take away. By definition, you can never take it away. But, you know, could call center jobs be less plentiful?

Yeah, of course they could. Might the people who do those jobs need to do something else? Yeah.

I mean a 100% of the work is going to be automated because 100% of the work always gets automated. Like if you look at what my look at you and me if you ask my great-grandparents about work and you said this is what we’re doing right now. You and I are both at work right now.

They’d be like that’s not work. Like my grandfather lost his eye at work. And so we can my point is just that we continually redefine.

And the really meaningful question is what produces a life that somebody wants to live, values, experience, and enough money that they can include it as part of that that they can feed their family and live in a safe and stable way. Those are very open questions. We can talk more about the different kinds of AI and how it might affect things, but those are the big questions to me.

Yeah, this is I mean this is I think what I mean people folks in the education space struggle because they hear this you know setting aside actually talking about artificial general intelligence but you know you hear folks who are really at the front lines of this saying things like you know it’s actually possible that the vast majority of work is displaced and it just for folks in education their heads kind of explode because it’s not really like it’s easy for us to think about okay how do we sort of shift education to orient students towards a a slightly different set of career pathways but then when you have someone come and say well actually there may not be pathways for a lot of folks and we need to sort of think about how we the role of education in there and it’s how productive it is I think that’s a busted model a mental model in the following way we will always have pathways because we will always redefine what we do as work whatever we do in the US you know like again what you and I saying no work you’re saying what we think of as work will change that’s right that’s exactly Exactly what I’m saying now. But I do think it’s an open question about how will they be able to earn enough to live and how will they enjoy their lives and that’s where things like government policy you know one of the reasons I believe in a much higher social floor and as a proud AFT member you know my union has advocated for a higher social floor you know in many many ways just more gener because I think it’ll stabilize many of those transitions because just because there might be more jobs in total doesn’t necess necessarily help you know Alex Notran who is the other Alex who doesn’t have a you know a good job that he loves like running an AI education nonprofit. One of the heristics that I use is if someone’s because like sometimes people want much more actionable.

They’re like is my job at risk and as you say it’s hard to say. No no no actually I think that’s not hard to say. The answer to that is yes.

The answer okay the anybody who asks you is my job at risk the answer is yes. The question there are questions around which aspects of your job right you know we invest in a company called work helix that does workforce analysis for big companies where they basically determine here are the proportion here are the jobs that are more and less vulnerable right now so there’s a question about degree of vulnerability there’s a question about timing but everybody like the notion of I’m going to pick a career it’s going to be safe for a long time and I’m going to be fine there might be some exceptions but in general I think that’s gone away and look we invest in a company called campus that is national highquality community college and people who do learning in a new environment like that I actually think are much more likely to be successful than people who think the old ways still apply like the old ways if they worked 10 years ago which I don’t think they did they for sure don’t work anymore one of the one of the heristics because I because yeah sometimes I I think people are still not necessarily happy with well all jobs are at risk so one of the heristics I use is, you know, open up your calendar and like look at how much time you spend interacting with people and how much time you spend sitting in front of your computer and and and what I say is it doesn’t really matter what you’re doing at your computer, whether that’s writing or researching or coding or analyzing. That that is probably not a good sign if too much of your day is solo sort of like like you know creation of stuff or or writing stuff.

In a previous conversation though, you sort of like challenged me a bit because you said, "Well, right now we assume that like things like empathy and communication are sort of the bastion of of human work, but I don’t want I want to paraphrase, but you were like, I’m not so sure about that." I defin No, I I for sure believe that if we tell ourselves bedtime stories, like humans are inherently empathetic and machines are not. I mean, there’s already studies suggesting the AI has better bedside manner than the typical doctor. Yeah.

So, I think we are there’s a great book, The Man Who Lied to His Laptop, about how people anthropomorphize machines, written by the guy who, if folks are old enough to remember Clippy in Microsoft Word. He figured out why people hate Clippy so much. He was a Stanford professor, Cliff Nass.

And I think that we shouldn’t tell ourselves bedtime stories and instead, you know, it’s about figuring out where is the enduring value and assuming it’s just going to keep evolving. And I I just don’t see people are looking for the safer place, the higher ground. And I just think that’s not a great way to think about it because it’s all going to get flooded at some point and you just need to learn to swim.

Did you did you do you watch the Y Combinator podcast about vibe coding? But you’ve seen you’ve heard the the the top lines, right? About like a quarter of their cohort report that 95% of their code base is written by AI.

None of this is in conflict with any of the any of the things that you’ve shared earlier. Have you have you heard similar things from from Bloomberg betas? I hope it’s true.

I mean, look, I first of all, we had an emergency meeting of our portfolio last month because so many people were freaked out by how much faster they could go using AI tools to code. They wanted to make sure they were all learning from each other. These are expert startup founders.

The second thing I’d say is this isn’t new in some ways. Like the paradox is it’s new and the tools are new and it’s moving fast, but the principles are not that new. And in a way, I think that’s an analog to education.

It’s like people learning how to think for themselves, how to assess what’s right and what’s not. Moral, I mean, there’s all kinds of stuff where the principle is still going to apply. It’s just a question of how.

And so, the reason I say it’s not new. In 2012, maybe 2013, I was very interested in this question of how people could learn to code because I assumed everybody would learn to code at some point because I just saw how useful it was becoming, how much easier it was becoming. And a friend of mine introduced me to this guy who was the professor at Stanford who taught the intro computer science class just so I could learn from him.

And he ended up teaching that class I think for almost 30 years. And he said this is now more than 10 years ago. He said nobody programs anymore.

I was like what do you mean? He’s like well because the tools have gotten so good that the abstractions they use are not real programming. And to me both I think he was right by a certain definition but it’s a little bit again like just cuz I bake no don’t know how to mill the flour doesn’t mean I can’t bake the bread and there’s skill in baking the bread and by the way there’s skill in owning the bakery and buying from the person who bakes the bread or hiring somebody who bakes the bread or whatever and so where people move up and down the layers of abstraction in order to do something valuable that to me is a major question.

So I mean, look, I’m I’m completely aligned with this. I the the the baking analogy I think there’s I’m I’m always in search of analogies and that’s not just because I like them. I think one of the biggest challenges we have again from the perspective of education is translating what is at least initially a relatively opaque or obscure or arcane topic AI to a lay audience.

It’s really hard and I think people sometimes sort of they either like revert too quickly to sort of like the shortcuts. So prompt engineering for me is like you know there’s I think there’s a lot of folks who have spent a relatively small amount of time with probably just chat GBT and they’re sort of going around and sort of banding about this idea that well everybody is just going to be a prompt engineer and that’s going to be the job of the future. And that just seems like lazy thinking to me.

Lazy thinking, but it’s also I mean I I mean people are new to therefore everybody’s going to be a prompt engineer. That’s sort of like in the first two weeks of co was like look it works to work remotely. Everybody’s going to work remotely.

It’s the future. But I think part but part of it Yeah, I think that’s true. But part of it is like you’ve been in the space way longer than me.

I’ve been in the space way longer than most people who are now talking about AI. If you weren’t one of like the nerds, there was, you know, prior to November 2022, AI was literally just science fiction. And so you don’t necessarily have a like barometer of like what the velocity is.

And and they’re also not on Reddit. They’re not necessarily they’re not on Discord. They’re not seeing they don’t know what a reasoning model is.

They don’t know that the like the way I think that you don’t need to. I mean again I don’t need to know how to mill the flower to read like here I’ll give you a practical suggestion. Ethan Mllik Ethan Malik is a pen professor as you know who does practical tips about how to use AI.

Everybody should subscribe to his newsletter who is at all curious about this and then you’ll see how fast it’s going. So that was what I was going to ask you is like how do we so subscribing so it’s just sort of digesting information from people who are taking the role of being those translators. Yeah.

It’s a it’s what you know Malcolm Gladwell would call a maven, right? And and obviously this is the work that we do as well. Not that I have anything to myself.

No. And that’s why I think it’s very valuable. But I I still wonder about you know aiEDU Ethan Malik Roy Bahhat there are not enough of us going out and doing the work of informing the general population given what you have shared earlier which is this is a literal like the scope of this challenge is everybody like we are not we are not talking about you know how do we make sure that x% of students can improve their literacy because they’re not graduating you know being able to read and right or you know how do we deal with like you know lagging math scores we’re talking about even if you are a top performing student who is going to become a doctor it is actually I’ve actually talked to my brother’s a doctor and his boss has been an early user of AI and he was telling me like honestly doctors are actually worse like at least near-term like they’re probably going to be impacted sooner than the nurses which kind of blew my mind and it’s just like your your point about knowledge work the the the conversation about AI previously was well basically the poor people are going to be impacted. What what are we going to do with those those poor people?

It’s just too bad. And and that was those were the conversations happening at like the World Economic Forum and you know and then and now it’s like slightly different. It’s like oh shoot this is this is software engineers.

This is lawyers, accountants. But looking back to analogies I mean when you look back at the internet or you know some of these prior technology revolutions what worked really well to keep people from falling behind? Is there any is there any sort of analog?

Besides obviously like newsletters and it’s a great question. I mean I think the bad news is it was really bad in past transitions. How so?

I mean we had two world wars after the industrial revolution. That part seemed to suck. Okay.

Okay. So you’re going back right. Okay.

And you know like you know manufacturing transition in the US hollowed out entire places you know like pick your thing so the risks are real and big and that’s why we need to be prepared for a societal response and you know my best mental model for that is my best mental model for that is hurricane and disaster response like at some point there may be an AI hurricane that some version of FEMA economic FEMA needs to respond to the way that we responded to co honestly economically speaking and I’m not saying we did all that right but we swarmed for sure then the second thing I’d say the good news is it’s easier to learn this stuff before if you wanted to transition careers in the past you might have to move cities you might not have the tools available to you but like you know for the very technical people or more technical people one of the great AI teachers who was one of the legendary teachers at Stanford and built the first version of the Tesla full self-driving software. Andre Carpathy is on YouTube. Yep.

That’s wild. Yeah. And it’s and yet we seem to be gravitating away from long form content.

It’s it’s it’s weird that how YouTube is going super long form. I don’t think we’re gravitating away from long form content. I think it’s doing what everything else is doing, which is it’s just bifurcating.

It’s bifrocating. It’s either very short, but if you look at like the top 10 podcasts, I mean, Joe Rogan can talk. Dwares Patel can talk.

And so it is I I think it’s bifurcating is what’s happening. Do you do you have any sense of I haven’t looked at the numbers in terms of like who’s actually watching the long form versus the reals. It’s it’s not necessarily like older people are watching long form.

I actually don’t know. I my sense is that it is bifurcating for everybody and that age is more about channel. It’s more like young people are on TikTok and old people are on Facebook.

But that’s not a deeply researched thing. That’s just kind of a a sense CI like every student being their own CIO. Is this is this within reach for a underresourced public school?

Like setting aside like a private school that you know has one of the other cool things is almost everything has some version of the tools that is free. And so I don’t know if there were ever something that was accessible, you know, unlike accessing the internet where you needed all this equipment you didn’t have yet and laptops and internet connections. You know, again, I don’t know the educational context.

I definitely don’t know the K-12 context very well. But at least in principle, most of these tools are free or have a free version that’s powerful enough to learn a lot from it. Yeah.

I mean, but can you missing something? No, you’re not missing something. Unpack this.

Like I’m I’m cognizant that maybe not even everybody knows what a chief information officer does. Yeah, good question. So what does a chief information officer do?

They decide what technology tools a workforce should use. Okay, they try new things. They figure out should I build something myself?

Should I buy something? And look with tools like Replet, you know, somebody who doesn’t know anything technical can make what the founder calls personal software. They can make their own software.

And you know, but it’s that process of not waiting for somebody to tell you what tool to use, but going out and researching your own that I think is very very valuable and becoming more valuable. And and this is interesting because a lot of school leaders right now are obsessed with what is the one tool that we get every student to use. And what what I’m hearing from you is that may actually be counterproductive to the ultimate goal, which is equipping students with the ability to actually navigate different tools and make decisions about which ones are appropriate.

Great observation. Yes. Although I also think, you know, pick any chatbot and make sure everybody has access to it cuz you know, I can ask Claude, hey, I want to use a tool that communicates securely in the following way.

And shockingly, Claude is generally pretty good at that. How do you use I mean what do you have an AI LLM of of preference for your own work? No, I use them against each other.

So like in my writing process, I frequently will will basically go out and do a bunch of research, get LLMs to make an outline. I mean, I can talk you through my whole writing process if it’s interesting on how I use AI. I would love if you’re if you’re willing, I would love to hear.

Yeah. I mean, I can even send it to you in writing. I mean, basically the short version is I have the LLM’s I I first of all, when I’m doing research, I’ll often just talk.

So, I’ll record audio files of like, oh, I’m thinking about this, but what about this argument? I’m not sure. Maybe the other one, blah blah blah blah blah.

And then what happens is I will take that and I’ll feed it into an LLM and I’ll say, "Please go research this other stuff that I’m curious about. Find me a fact on this, blah blah blah." And then all that leads to an outline. I’m not getting it to draft for me, but I’m getting it to make a rich, detailed outline.

And then and then I’m often times doing the same thing in another LLM. So I have two outlines produced by the same prompt. And then I’ll feed each one the other one’s outline.

I call it dueling LLMs and say, "This is the one that came from another LLM. Please improve it and incorporate it with yours." And then they’ll kind of converge into something great. And then I’ll look at it and I’ll sometimes give a little feedback and then I’ll cut and paste the outline into like Google Docs or something like that and I’ll just type until I’m done writing it and I can go pretty fast, much faster than I used to be able to before.

I just wrote a blog post on on what questions a person should ask before joining a startup that was based on that. And so yeah you know it is it’s a process that has been working for me and that I’ve iterated on like and I expect that you know literally just today my team was talking about some piece of writing we had to do I was like okay could we try using the AI in this way and so we’re constantly probing the limits of what it can do. Yeah I I I also I mean I got into I got my scholarship to Ohio State because I got like I won a little bit of money from like a writing competition.

So I got a partial scholarship. I’ve always been a writer. I I actually resonate with teachers when they when they stress about students using AI tools.

It hearkens me back to one of the first English teachers that we worked with. We we we did this training on how, you know, this is like the very first days of ChatGPT. And I caught up with her six months later, asked her like, "How is it going that are you using AI in the classroom?" She was like, "Oh my gosh, it was so engaging.

My students loved it." She she had all these really fun activities that she created with AI. Is the most engaged that she’d ever seen her students and and then I was like and we actually had a New York Times reporter that wanted to talk to a teacher that was using chat GBT and I said oh like well how are you using it today and she’s like oh I don’t use it anymore why don’t why did you stop using it she said well you know I teach freshman English and we got to the portion of my class where students were learning how to create an outline and I realized how part part of the way through that se section that they they were not learning how to create an outline they were learning how to ask and prompt engineer for an outline. And she’s like, "Those are those those are actually different skills." And it strikes me that you’re like what you described sounds great, but I think that my my suspicion is part of the reason why you’re so effective with LLM is because you’ve always been a really strong writer.

And that you are able to So, I really appreciate you saying that. I don’t know. I think you’re a good writer.

Well, thank you. But, but you also don’t know what I’ve written versus what the LLM has written. I’ve read your stuff prior to Chad GPT.

Yeah, fine. Fair. Thank you.

But the thing I would just say about that is I’m not suggesting that people should stop learning how to write. What I’m suggesting is that it’s a different process now. And that the same way, by the way, like you could have said the same thing about spellch check.

And I remember I’m old enough to be like penmanship is important to learn. Is penmanship important to learn? I I don’t think so.

I do think that writing is thinking. Sure. I I agree with that.

And so my point is not that writing is like penmanship. My point is that of course it’s important to learn. And the question is how and in what context and that we should get away from well if they’re using the LLM they’re not learning to write.

It’s like no they’re learning to write in the same way that when I use spellch checkck I’m still learning to write. It’s just different and I need to stay pay attention to how do I edit, how do I how can I compose a paragraph on my own and we just don’t know what all those lines are going to be. I agree with that.

I my my in my instinct is that the the whole the whole thing about like writing is thinking and the whole thing about like why is it why does it feel wrong when a student is just getting to the outline so quickly and I think that is that there’s actually a lot that you gain from sitting in front of a blank p piece of paper and like struggling to figure out how to get started like you have these thoughts in your head and this like sort of productive struggle that comes from writing, especially early on, you know, like once you do it enough, you kind of build the like the muscle memory to figure out how to get started and, you know, like for me it was always like I would spend a lot of time with the first paragraph and once I had that, the rest kind of flowed. But the process was not just it’s not it’s it’s I I think that I actually built and learned a lot from the the the the challenge that that I faced in writing. And I and I and I wonder if something being so easy yeah and and maybe it’s just different ways of creating productive struggle right I think it’s different ways so I think productive struggle is for sure essential but I think the different ways is the question because I sort of hear like when I went to school I had to walk up you know walk uphill both ways kind of thing and okay struggle is great it’s just got to be necessary struggle and unnecessary struggle can be fine as an exercise.

Like look, I go to the gym. I li I don’t really, but I should go to the gym and lift weights and that’s unnecessary struggle, but it’s necessary for my own health and learning, you know? Yeah.

And so, but I but I think I’m not trying to dismiss the idea of productive struggle. I’m trying to dismiss the idea that which struggles are productive is fixed because I think it’s malleable across time and we should find new productive struggles. That’s my kind of general take.

Yeah. And this is and this is where I I find that the issue with teachers being worried about students cheating is is so fascinating to me because on the one hand I I don’t think it’s correct to just think of a student using AI equals cheating. Of course not.

At Berkeley by the way in my class I require the students to use AI. The only rule is they have to tell me how they used it. But okay.

Okay. So, but but but the nuance is if the student is more effective at using AI than the teacher and certainly if the teacher doesn’t have any comfort or experience with what AI is capable of, I do think that it’s possible for the students to get around the teacher’s ability to create that productive struggle because like if if the teacher doesn’t know what the AI is capable of, you know, they’ll do things like what my favorite is when they say, "Oh, I figured out the Trojan horse strategy." And this is one of the things that was shared on our Facebook where you put like, you know, in white text some like, you know, Trojan horse to sort of fool the prompt into like including the word Frankenstein somewhere in the in the homework assignment. And and to me, this is like this is this is what this represents is teachers are not they’re they’re spending too much time trying to figure out how to actually like enforce and like stop students that they’re actually they’re not investing the time into figuring the tools out for themselves, which I think would get them to a place where they figure out, oh yeah, I mean that is a you kids and your rock and roll kind of moment.

Do you have any So what other strategies? I mean so you ask the students to use the AI and show you how they use it. I have no other strategies.

How do you how can you validate I mean like are your I assume your homework assignments are not they’re not doing multiple choice tests is my I don’t have a cheating problem in general because the students can’t even disclose their grades they’re paying a lot of money to be there it’s business school so my view is right it’s not if they cheat it’s fine but I’m not there to enforce anti-cheing against them and there’s no right answers to the question they ask like the final assignment is a personal reflection essay on the class. Like I don’t even know what it would mean to cheat on that. Like did you not share the thing that you and so it’s just different for me and easier.

But I do empathize with the fact that in a different context if you’re a math teacher, if you’re an English teacher assigning an essay on Sound and the Fury, you know, cheating could be a big deal. So but I but but I hear you on the desire for people to figure out what they’re doing. Being something that is sorry figure out the tools being something that trades off with blocking others from using the tools.

So that seems like a real issue to me. Okay. I this this this self-reflection.

This is one of my favorite ones. Been something else that I hear a lot is teachers say, "Well, self-reflection, something that that that that’s personalized to the student." and the challenge is I think you can actually get a pretty damn good self-reflection with a single sentence prompt. And and you could do a little bit more prompting.

It would take you probably less than 5 to 10 minutes to get to something that’s probably high enough quality that totally maybe not you, but by the way, if I had to do a self-reflection, I’d be like, "Hey, AI, look at my emails that I’ve sent relating to this class." I’d upload the emails directly into ChatGPT or Claude or something like that and I’d say based on this and also my Slack messages which I’d cut and paste in what are my reflections and then I’d edit that and then you’d edit it and but I guess the challenge is what like how and and so the point is taking that if you’re an MBA and you’re cheating it’s like well why are you spending the money like but also but also like look if it’s easier or harder I’m not going to the printing press and laying out the print type on the machine anymore. More either. I’m not doing mimographs, you know, like there’s a lot of conveniences that have come along the way and I think we have to embrace constructive convenience and still find a way to learn the things we need to learn.

So your point about productive struggle I think is so valid but the mere fact it is a lot easier does not disqualify it in my perspective.