The band is back together! Paul and Mike return from separate events last week and are ready to talk the latest in AI. And there’s a lot! Catch up on the latest from Google, the responsible side of LLMs, and hear more about labor and productivity in the age of AI.
Listen or watch below—and see below for show notes and the transcript.
This episode is brought to you by MAICON, our 4th annual Marketing AI Conference. Taking place July 26-28, 2023 in Cleveland, OH.
00:02:39 — Google AI Updates
00:16:13 — What’s next for Claude
00:23:55 — More productivity and labor talks
00:34:02 — Rapid fire discussions
Another week of big news from Google
Google just announced major AI updates, including an AI makeover of search. The updates were announced at Google’s I/O developers conference and some of the more important updates were discussed on the podcast.
A new next-generation large language model called PaLM 2, “excels at advanced reasoning tasks, including code and math, classification and question answering, translation and multilingual proficiency better than our previous state-of-the-art LLMs.” Next, an AI makeover of search through Google’s “Search Generative Experience” will deliver conversational results to search queries. This will become available to users who sign up for Google’s Search Labs sandbox.
Additional improvements include new AI writing tools for Gmail, the removal of the waitlist for Bard, and the ability to create full documents, generate slides, and fill in spreadsheets across tools like Docs, Slides, and Sheets.
What’s next for Claude
Anthropic, a major AI player and creator of the AI assistant “Claude,” just published research that could have a big impact on AI safety. In the research, the company outlines an approach they’re using “Constitutional AI,” or the act of giving a large language model “explicit values determined by a constitution, rather than values determined implicitly via large-scale human feedback.”
This concept is designed to address the limitations of large-scale human feedback, which traditionally determines the values and principles of AI behavior. It aims to enhance the transparency, safety, and usefulness of AI models while reducing the need for human intervention.
The constitution of an AI model consists of a set of principles that guide its outputs, and in Claude’s case, encourages the model to avoid toxic or discriminatory outputs, refrain from assisting in illegal or unethical activities, and aim to be helpful, honest, and harmless. Anthropic emphasizes that this living document is subject to revisions and improvements based on further research and feedback.
More on the economy and knowledge workers
In a recent Brookings Institution article titled, Machines of Mind: The Case for an AI-powered Productivity, the authors explore the potential impact of AI, specifically large language models (LLMs), on the economy and knowledge workers.
The authors predict LLMs will have a massive impact on knowledge work in the near future. They say: “We expect millions of knowledge workers, ranging from doctors and lawyers to managers and salespeople to experience similar ground-breaking shifts in their productivity within a few years, if not sooner.”
The productivity gains from AI will be realized directly through output created per hour worked (i.e. increased efficiency), and indirectly through accelerated innovation that drives future productivity growth. The authors say they broadly agree with a recent Goldman Sachs estimate that AI could raise global GDP by a whopping 7%. But there’s more to it, so be sure to tune in.
Listen to this week’s episode on your favorite podcast player and be sure to explore the links below for more thoughts and perspectives on these important topics.
Disclaimer: This transcription was written by AI, thanks to Descript, and has not been edited for content.
[00:00:00] Paul Roetzer: imagine you're able to create a synthetic version of yourself, like a digital version that kind of looks like you, but it's, you know, has voice split like you and you could actually sell a subscription service to that. That's the kind of. Business opportunities we're heading into.
[00:00:13] Paul Roetzer: Welcome to the Marketing AI Show, the podcast that helps your business grow smarter by making artificial intelligence approachable and actionable. You'll hear from top authors, entrepreneurs, researchers, and executives as they share case studies, strategies, and technologies that have the power to transform your business and your career.
[00:00:33] Paul Roetzer: My name is Paul Roetzer. I'm the founder of Marketing AI Institute, and I'm your host.
[00:00:44] Paul Roetzer: Welcome to episode 47 of the Marketing AI Show. I'm your host, Paul Roetzer, back with my co-host Mike Kaput from his trip to Punta Cana. What's up, Mike? How was Punta Cana? We haven't even had a chance to talk since you got
[00:00:57] Mike Kaput: back. It was awesome. I mean, obviously we ran into a few little internet issues, but you know, the beach and the sunshine made up for it.
[00:01:05] Paul Roetzer: Well, yeah, it was, it was weird doing it solo. I think I got a little slap happy at the end, like talking to myself for 50 minutes. But, yeah, it's good to have you back. It's good to be back. I'm not winging this on my own. All right. Today's episode is jam packed. So we are going to, we're going to get through this.
[00:01:22] Paul Roetzer: We have a, we have a deadline. We have 55 minutes max. We are going to move fast. There's a lot of rapid fire items. I feel like we could spend the whole show just talking about the Google IO conference, but we won't. So this show is brought you by the Marketing AI Conference, which is again, is returning to Cleveland, July 26th.
[00:01:40] Paul Roetzer: The 28th. We are, trending. Pretty high on, on the ticket sales. Again, our, our goal was originally 400. We're, we're blowing past that. So, yeah, I mean it's looking like, you know, maybe 600 plus, for this event. So we would love to have you there. The lineup looks incredible. We're still adding to it.
[00:02:00] Paul Roetzer: Still going to announce some keynotes over the next 30 days or so. But check that out. It is MAICON. Just M A I C O N dot A I. We would love to see you there. Again, it's July 26th to the 28th, so if you are new to the show, three main topics. Mike and I go through Zoom chat all week, back and forth, and it's usually like 30, 40, 50 items in there.
[00:02:24] Paul Roetzer: Mike then spends his Sunday night curating those. And, turns it into the main three topics, and then everything else gets dropped into rapid fire. Like I said, today is a jam packed episode, so we will get right into it with our first topic.
[00:02:39] Mike Kaput: All right, so a huge topic. This week Google just announced some major AI updates, and this includes an AI makeover of search that everybody is talking about.
[00:02:50] Mike Kaput: So Google had its IO developer's conference and announced. A slew of updates related to AI across different products and developments that it has made on its own models. So some of the most important updates, and there are quite a few of them, so definitely check out the show notes to go through the whole list.
[00:03:09] Mike Kaput: But the ones that jumped out as really important include they announced a new next generation, large language model called Palm Two. They say that Palm two excels at advanced reasoning tasks, including code and math classification and question answering translation and multilingual proficiency, and it's better than our previous state-of-the-art large language models.
[00:03:33] Mike Kaput: Google also announced an AI makeover of search through the, what they're calling their search generative experience. So this will essentially deliver conversational results like a ChatGPT, right in Google's core search engine. And this is going to be available to users who sign up to Google's search labs Sandbox.
[00:03:54] Mike Kaput: Again, that link is in the show notes. Some other really notable updates. There are new AI writing tools for Gmail where it will write emails for you. Google has removed the wait list for Bard. It's kind of experimental, conversational AI engine, and you will now have the ability to use AI to generate full documents, slides, and fill in spreadsheets automatically across tools like docs, slides, and sheets.
[00:04:21] Mike Kaput: So, This was such a huge announcement, Paul. I mean, there's been a ton of speculation that Google has fallen behind lately in the AI race with companies like Microsoft OpenAI, et cetera, but these announcements certainly seem quite the opposite of that. Not only is Google back in the game, but they appear to be in quite a strong position here.
[00:04:42] Mike Kaput: Can you unpack the significance of the latest round of these announcements from Google?
[00:04:47] Paul Roetzer: Yeah. You know, I've thought a lot about this and we've talked about this on the show, quite a number of times that, listen, there, there probably isn't a more advanced organization in the world from a perspective of AI than Google.
[00:05:00] Paul Roetzer: And that. Is pretty widely known, I would say. I mean obviously there's major players like Meta and it is still not talked about enough with what they're doing in Yann LeCun, that team, Microsoft has been doing this stuff for 20 years. Amazon's been a major player by DO and video. Like there, there's just, there's a lot of major players and obviously OpenAI gets a ton of press.
[00:05:23] Paul Roetzer: What what sort of jumped out to me on this event was, There seems to now be a willingness on Google's part to do things they weren't willing to do before. So they have obviously a massive, lead in, in ad the ad, digital ads and search in particular, and there's been no motivation on their part to disrupt that.
[00:05:52] Paul Roetzer: So then when Microsoft and open eyes show up and start kind of rattling the cage, Google has to sort of look themselves in the mirror and say, are we willing to now do things we wouldn't have done before, like cannibalize their own business? And what we've been waiting on is would they be willing to do that?
[00:06:08] Paul Roetzer: And I think that's been the big question in everyone's mind, certainly in investor's minds. And so I started thinking last week, and I think I might've got asked about this on a different podcast, and, and so I just, I made a quick list of like the things that. HA Google has going in its favor that this, these announcements sort of start leading into, and that I'd be a little worried about if I was the competition.
[00:06:32] Paul Roetzer: So first they have the people, they have over 2000 AI ML employees. Now they have lost a lot of top talent. People like Aiden Gomez went on to f found cohere, and I think it was, was it Known Brown that went on to find character.ai? Like some of the people who were core to the innovation in the last like five, six years in, in generative AI and in language models in Google have left to do their own thing.
[00:06:58] Paul Roetzer: And in part it seems like some of those top people left because they couldn't commercialize their own technology. So certainly as a entrepreneurial spirit, people are going to leave and go build their own thing. That's going to happen all the time, but you just don't hear about that at like Apple. Like you, you don't hear about high profile people leaving Apple as much.
[00:07:15] Paul Roetzer: So it seemed like it was a little bit more of an issue here. But because they, again, they weren't really willing to commercialize what they had. They have these two major research labs, which are now a single AI research lab, and they're working on all this stuff. And so imagine you're one of the top AI researchers in the world and you're inventing the transformer architecture and you're doing all these things and it's like they never see the light of day.
[00:07:37] Paul Roetzer: I would imagine that becomes a bit demoralizing or challenging. And so to work in a company where the things you build can actually, you know, see the. Consumer end, is probably a little bit more interesting. I would guess. Again, I'm not a researcher, I'm just sort of theorizing here. And so I think if they're willing to now actually start putting these innovations out into the market, maybe you slow down that talent leak and you can actually really recruit, the best people in the world.
[00:08:05] Paul Roetzer: So they have people, they have 20 plus years of AI research. OpenAI was created in 2000, what? 16. So, I mean, it's newer. They have the data centers and the compute power, which almost no one has. OpenAI relies on Microsoft for, it's why they sold a third of the company for 10 billion. They had to have access to compute power.
[00:08:27] Paul Roetzer: So Google has that, they have their own chips that were purpose-built for AI for machine learning called TPUs. TENS are processing units. So the w the industry was powered by GPUs, which are made by Nvidia. Google built their own TPUs for more advanced and kind of customized versions of ai. So they have their own chips.
[00:08:46] Paul Roetzer: They have Google DeepMind and Demi Saabas, who we've talked about on the show before. So they have. Again, a decade plus of advanced research into some of the frontiers of where AI is going. They have 115 billion in cash according to their most recent earnings call. They have 93% market share of the search market, and they have over 6 million Google Workspace business customers, which is more than 50% of the office suite market.
[00:09:15] Paul Roetzer: So if you stack all this up and they actually have AI as advanced or better than OpenAI, OpenAI doesn't have any of that stuff. Microsoft doesn't have any of that stuff, so I just look at and say if, if, if the end game here is actually artificial general intelligence, which, which I believe it to be both from having conversations from people at Google who are working on AI and from everything we hear in the industry is the belief that this is where we're trying to go.
[00:09:41] Paul Roetzer: It's with Demi Savvis and DeepMind were, you know what? He created it for him and Shane Leg and their, their co-founders is to achieve agi I, well, AGI I is. Trillions in market cap. So if you can build a machine that's super intelligent or superhuman at multiple capabilities, at many tasks, the value you can unlock is, is massive.
[00:10:03] Paul Roetzer: And so if they think that there is a viable path to AGI and that they're looking at potentially trillions in market cap, then they may be willing to do things to their ad business and their search business that were not. Thinking about, or that the average analyst would look at them and say, oh, there's no way they'd cannibalize their ad business.
[00:10:25] Paul Roetzer: Well, maybe they would if, if the alternative is a trillion dollar market. So, I don't know. I just, I think that, they, they may be willing to do some things here that most people don't think they're going to be willing to do. And I think that, you know, it's kind of just like that. You hope the bear and the bear is like, ready to go now and.
[00:10:49] Paul Roetzer: I don't know. Like I'm very bullish on Google. I think that the biggest roadblock honestly may be regulatory like that if they do start heading down this path. You know, I think part of their concerns was that they were going, if they release things that don't work as well or that do harm, that they were worried that they, regulators were going to come after them.
[00:11:11] Paul Roetzer: And I don't know if maybe they think that because everybody's doing it now, there's more cover for them and they're not going to get, you know, I isolated in that. But I do see the regulatory being, being a potential issue. But overall, I don't know. A couple of quick thoughts. So that's like kinda my macro level thoughts.
[00:11:29] Paul Roetzer: My other thought is looking at the search massively disruptive. I've talked with a number of people in the last week, you know, more expert on, on search than I am, and it seems pretty universal if no one has a clue what this is going to do. But it sure seems like it's going to be a massive disruption to SEO and content publishing for brands and.
[00:11:47] Paul Roetzer: Because the big question basically becomes, well if I get the answer right there, what do I need to go to the publisher for? If I over time don't need to go to the publisher as much, then what is the motivation of the publisher to publish anything? So I, did I give you a quick example? I ran, I use Bard.
[00:12:02] Paul Roetzer: So the Bard, the Palm two, you mentioned the new language model is powering Bard now. And anybody can go get, it's just what? bard.google.com. So you said there's no wait list, nothing. You can turn it on now I will say you can turn it on for your personal Gmail. But if you're a Google Workspace business, your administrator actually has to go in and turn on, there's an early access for apps.
[00:12:24] Paul Roetzer: You have to go in and turn that on and then give some data permissions. So if you have a corporate account that's using Google Workspace, or if you're the administrator in one of those accounts, you have to go in and make it an adjustment. I'll put in the show notes a link for how to do that. But anyway, so once you go in and turn it on.
[00:12:41] Paul Roetzer: You can actually start to experiment with Bard. And so I did one because it is connected to the internet, unlike OpenAI ChatGPT Outta the Box. So I did one that asked it about, our conference. So I said like, you know, tell me about the marketing AI conference. And it did, it, you know, had the right dates, right location, right everything.
[00:13:00] Paul Roetzer: I did ask it one. Where, and it was pretty much everything I needed and I could still go to the event site, but it didn't list it. And then I asked at one where I wasn't sh I was like, okay, let me see if it actually cite things. And so I asked that, what is the law of uneven AI distribution? Now if anybody listens to the show, they've heard us talk about this.
[00:13:18] Paul Roetzer: I wrote it in probably March. So it's, it's fresh, it's original. Like I knew there was nowhere else on the internet. That would've talked about it yet. So I put it in and it did like a 400 word summary of the law of uneven distribution that I wouldn't have needed to go anywhere else. Like it was everything I needed, but it did actually have citations at the bottom that went back to our blog.
[00:13:38] Paul Roetzer: So, I don't know. I mean, I think that it's going to change dramatically. Bard was interesting. No, not nearly as creative as GPT-4. So I haven't pushed this thing through the limits, like, you know, the, the Twitter thread people who, you know, jumped on this thing and ran 50 use cases through it.
[00:13:56] Paul Roetzer: But I ran like five or six use cases through Bard. Yeah, it's nowhere near as creative. I gave it the same out of office prompt that I've talked about. I gave GPT-4 and it was like really dry. And I said, can you make this more humorous? And it gave me another output. I was like, that's not funny.
[00:14:13] Paul Roetzer: Like, try, can you, can you make this funny? And it's like, I'm just a language model. Funny is subjective. I'm like, no, it's, it's actually not very creative or funny. So, yeah. And, but then I gave it like, you know, my, my family's going on a trip to Remedy Italy. Create a recommended destination and it did with a day-to-day itinerary.
[00:14:32] Paul Roetzer: Like it was, it was really good. I asked it to summarize, mentions of AI during the most recent HubSpot earnings reports. And it gave me a summary from, I dunno, it was like February. And I was like, isn't there a more recent one? And it came back and said, oh yes, there's a more recent one from May. And then it gave it to me and.
[00:14:50] Paul Roetzer: And then I kind of, I actually asked it like, what is, this one was fascinating. It was like, what is the, what advantage does Google have over Microsoft and OpenAI in, you know, the race for building generative AI technologies? And it gave a really good analysis that was actually kind of on point.
[00:15:10] Paul Roetzer: And then I kind of kept drill on that. So, I don't know. I guess overall massive impact on search. Once the stuff gets infused into Google Workspace, so it's in sheets, then docs and slides, it's going to be really interesting. I think it's going to be not as impressive as the demos. I think it's going to be a little bit, of a learning curve, but I do think turning this stuff loose on the world when knowledge workers have no real training for it, and everybody in the organization all of a sudden has these tools, it's going to be game changing and we're not ready for it.
[00:15:46] Paul Roetzer: Yes, we're, we're getting close to ready, but as a society we're not ready for.
[00:15:51] Mike Kaput: Wow. Yeah, that's, that's an awesome analysis. It's really helpful to kind of cut through all the information and noise out there about this, but
[00:15:57] Paul Roetzer: there was like 25 products analysis. Yes. So again, Mike and I are just hitting the basics, what we see as the high level.
[00:16:04] Paul Roetzer: But yeah, I mean go, go read this stuff. Go look at some of the deeper analysis of some of the specific tools, because it was a lot to process.
[00:16:13] Mike Kaput: So next up we have an interesting announcement from Anthropic, which is a major AI player and creator of the AI assistant model cla. They just published, kind of an approach to AI safety that could be a really interesting and significant, approach moving forward to get safer, better AI outputs.
[00:16:34] Mike Kaput: So, In a blog post, they outlined an approach that they're using called Constitutional ai. What this means is they give their large language model explicit values determined by a constitution rather than values determined implicitly via large scale human feedback. So basically, you know, this is designed to address.
[00:16:55] Mike Kaput: The historical limitation in large language models where it kind of learns what's quote unquote right or wrong by what humans determine. And that traditionally determines what we might call the values or principles of how the model behaves. So with this approach, The constitution of an AI model consists of a set of principles that guide its output.
[00:17:18] Mike Kaput: So in Claude's case, Anthropic has encouraged the model to avoid toxic or discriminatory outputs, refrained from assisting in illegal or unethical activities, and aimed to be helpful, honest, and harmless. And so Anthropic says that this constitution is a living document and they actually. Drew it up using various sources, including the UN Declaration for Human Rights, some trust and safety best practices, and some principles proposed by other AI research labs.
[00:17:49] Mike Kaput: They also mentioned that it includes principles to reflect non-Western perspectives and values identified through trial and error. So they say, look, this is basically a v1. It can be revised and improved based on further research and feedback. So you're essentially giving a model. A set of guidelines that the company infuses into it from the start as to how it should, should, and should not behave versus allowing that behavior to be dictated by human feedback.
[00:18:16] Mike Kaput: So first off, I guess Paul, like, can you outline how big a problem is this, that they're trying to solve? It seems like we often get harmful, toxic, or negative results produced by some large language models pretty
[00:18:29] Paul Roetzer: often. Yeah. I mean, it's certainly one of the largest challenges that. Generative AI is dealing with is how, how to do this, and we've talked about this on the show before about OpenAI, you know, When GPT-3, GPT-4 come out, there's pushback from one, you know, again, we'll thing about political sides in the United States, because this's we have to live with all the time.
[00:18:51] Paul Roetzer: So you have one side saying, oh, it's too woke, it's, you know, this and that. And then the other side, you know, has their own issues with it. And so then OpenAI basically says, listen, eventually you're just going to able to tune this thing however you want and you're going to be able to have your version of it and you can kind of adjust, you know how it.
[00:19:07] Paul Roetzer: Responds, but we're going to stop putting the guardrails in ourselves as much. They're going to put their own in for universally accepted things, but when it comes to anything related to politics or what, what some may consider facts, it's kind of like going to let you have your own run it. How, how it outputs basically.
[00:19:27] Paul Roetzer: And so it's a really messy problem to try and control what it it says and how it positions people in places and countries and economic beliefs and religious beliefs, like all of these really messy issues that a lot of times as humans, we just avoid talking to each other about because we don't want to like cross some line, well, these things can't do that.
[00:19:48] Paul Roetzer: You're going to ask 'em these questions, either because you're trying to figure these things out, or you're just trying to challenge the system, but you're going to push it. To give you answers. And as these things get better and better and more accurate, people are going to come to rely on them as fact and truth.
[00:20:03] Paul Roetzer: And so what is truth? What is fact becomes really, really important. And again, as a society, we have a really hard time deciding what truth and fact is right now. And so how do you. Tune a model to give you truth when we can't agree on what it's, and so that's the problem they're having. And so this is a really novel approach and it's, you know, something I've been kind of watching for the last few months is how they're doing this.
[00:20:27] Paul Roetzer: Now, as a reminder, Anthropic isn't some small player here. They've raised 1.3 billion, their Series C deck that was talked about in Crunchbase. What was this one? This was just April 6th, and I think we talked about this on the show. So the pitch deck for their Series C fundraising discloses that they're looking for as much as 5 billion over the next two years to take on OpenAI and others, and that they're, building what they're calling a frontier model called Claude Nextt, which is 10 times more capable than today's most powerful ai, but that will require a billion dollars in spending over the next 18 months to do it.
[00:21:05] Paul Roetzer: So, and they're founded by former OpenAI researchers. So th they're a major, major player in this space and I just think it's really worth noting how they're doing this. There's a research paper on kind of the details of how this works, and they just published, which we'll put in the show notes, kind of how the Constitution AI works, because they've been kind of guarded about what exactly it's trained on and how it works.
[00:21:28] Paul Roetzer: So, yeah, I think it's a really novel approach and worth paying attention to. And, well, the White House meeting, we talked about an episode or two ago, the CEO of, Anthropic was one of the people at that table with Microsoft, Google, and OpenAI. So again, they're a major company worth paying attention to in this space.
[00:21:45] Paul Roetzer: Yeah. So
[00:21:46] Mike Kaput: do you anticipate. Other companies kind of developing their own constitutions for language models. I mean, how would that, how would this impact how companies use the models or the outputs they're getting from them?
[00:22:00] Paul Roetzer: I could definitely see that. I mean, I think what's going to happen is, you know, again, we touched on open versus closed models, and are you just going to go get an, you know, out of the box, model from OpenAI or somebody, are you going to actually like, Train these things on your own.
[00:22:15] Paul Roetzer: And in most cases, corporations are going to, infuse not only their brand guidelines and their own knowledge base, but I could imagine, you know, where you could have your own constitutional AI approach to it where you're training it on your morals and beliefs and the things you want it to be able to do and say so.
[00:22:32] Paul Roetzer: Yeah, I think this is again, just kind of a developing area and something to pay attention to. And then the other thing I think we touched on is, they recently put out an update where you could, re-prompt this thing with a hundred thousand tokens. So again, tokens are kind of how these work, but that's equivalent of 75,000 words roughly.
[00:22:54] Paul Roetzer: So, you know, basically a thousand tokens is 750 words and just keep scaling up. So what that means is like when you're using these models, you can actually put 75,000 words, into your prompt. So you could take, like our book, the marketing Artificial intelligence book is 60,000 words. We could actually give it.
[00:23:16] Paul Roetzer: The manuscript and then like have a conversation with it about that book. Like you can take all of that in. That is, that is a massive leap. And I think just kind of a, again, an idea of where this is going with being able to, you know, do. Some pretty incredible stuff where you can actually extract, knowledge from all of the content.
[00:23:38] Paul Roetzer: You have all, all the show content, all the written content. It's just incredible stuff.
[00:23:44] Mike Kaput: Yeah, that's a super exciting use case and kind of marketing and business getting a much faster time to knowledge with the books and the resources you engage with. So, Our third main topic today is about a recent Brookings Institution article, and it is titled Machines of Mind, A Case for an AI Powered Productivity.
[00:24:06] Mike Kaput: The authors. Of this explore the potential impact of ai, especially large language models on the economy and on knowledge workers. And notably, one of the authors of this article is Eric Olson, who wrote a really formative book we read in the early days called The Second Machine Age, and it's about AI's impact on labor and the workforce.
[00:24:29] Mike Kaput: So the in the articles insights, They pull out a few really interesting takeaways that add some kind of nuance and context, I think, to the ai. Impact on employment conversation. So first off, they predict that large language models will literally impact millions of knowledge workers in the next few years, ranging from doctors and lawyers to managers and salespeople.
[00:24:54] Mike Kaput: And these groups will all experience similar groundbreaking shifts in their productivity within a few years, if not sooner. They also say that the productivity gains from AI will be realized directly through output, created per hour worked. In other words, increased efficiency and also indirectly through accelerated innovation.
[00:25:16] Mike Kaput: That drives future productivity growth. And the authors of this paper, this article actually broadly agree with a recent Goldman Sachs estimate that AI could raise global G D P by a whopping 7%, which is a crazy number when it comes to G D P. They also note that the rate of change from generative AI will be unlike any technological advancement we've seen before, which makes it very hard to forecast as they admit, you know what happens next.
[00:25:45] Mike Kaput: What I found interesting about this, Paul, was that there's a really vigorous debate over whether or not AI will take away jobs or create more of them, and this certainly seems to be leaning towards AI being a long-term net positive for employment and productivity. Was that kind of what you took away from it?
[00:26:02] Paul Roetzer: That was certainly how they tried to spin it. I actually looked at this and thought they tried really hard to convince themselves that it wasn't going to be massively disruptive in the next couple years. But all their data seems to imply the opposite. that it's going to be really painful.
[00:26:21] Paul Roetzer: So, yeah, I mean, like, I shared this on LinkedIn and I put kind of the synopsis you just went through that they. Address the fact that it's going to have a massive impact on knowledge work. They talk about, you know, the 49% of the workforce could eventually have half or more of their job tasks performed by ai.
[00:26:39] Paul Roetzer: There's going to be massive productivity gains through direct output, you know, created per hour and indirectly through innovation. But then it becomes very apparent that they actually have no idea how to predict this, like how to model this. Yeah. And so that was the part that really struck me was that they, they were sort of, it felt like they were kind of grasping at straws for how to project out the impact this was going to have, because economists really liked to look to the past for answers.
[00:27:09] Paul Roetzer: And so you look at past general purpose technologies like electricity and you say, okay, what happened when these general purpose technologies came into the, came into the world? And so they talk about diffusion of this stuff. So kind of the law of uneven AI distribution, like we talked about a little bit.
[00:27:25] Paul Roetzer: But the diffusion of this stuff can take decades in some cases till it has the impact. And by that time, you can have a net positive in job creation. But they very specifically say like, they, this is going to happen way faster than any of that. So as an economist, if you're looking at past models to try and predict the impact on knowledge work, there aren't really models that parallel what what is about to happen.
[00:27:47] Paul Roetzer: And just to give you a sense of how I feel like they were trying to put a positive light on this and trying to kind of like get, get, get some recent, reasonable projections. But I almost felt like every time they did it, They talk themselves back and be like, okay, we might not be ready for this. So let me just give you an example.
[00:28:04] Paul Roetzer: Like, and this is a little bit kind of technical from a math perspective, but anyway, it shows you how they're just kind of making up numbers right now. So they talk about, okay, so they go through the Goldman Sachs, as you mentioned, the 7% g P, which is a staggering number. I mean, this is a massive number.
[00:28:21] Paul Roetzer: GDP is like, what, 22 trillion or something like that. I don't even know what it is, but it's massive. So seven percent's huge. And then based on their analysis, they kind of agree with them. So then they talk about the first channel is increased efficiency of output production by making cognitive workers more productive, more efficient, the level of output increases, which improves the economy.
[00:28:42] Paul Roetzer: Economic theory tells us, so again, we're looking at economic theory here. We're looking backwards. Tells us that in competitive markets, the effect of a productivity boost in a given sector on aggregate productivity and output is equal to the size of productivity boost multiplied by the size of the sector.
[00:28:58] Paul Roetzer: So basically, the bigger the sector, the bigger the boost, the bigger the, you know, productivity game. So then it goes on though and talks about the second and ultimately more important channel is the acceleration of innovation and thus future productivity growth. So this is the one where it starts getting real, like this is the real important part cause this is the part that determines the impact, but it becomes very apparent that they're not sure how to model this.
[00:29:19] Paul Roetzer: So, okay. Cognitive workers not only produce current output, but also invent new things, engage in discoveries, and generate the technological progress that boosts future activity. This includes r and d, what scientists do, and perhaps more importantly, the process of rolling out new innovations into production activities through the economy, throughout the economy.
[00:29:40] Paul Roetzer: What managers do, so this is the part where it starts, like they're just putting, throwing some numbers in there. If cognitive workers are more efficient, they will accelerate technological progress and thereby boost the rate of productivity growth in perpetuity, for example. If productivity growth was 2% and the cognitive labor that underpins productivity growth is 20% more productive, this would raise the growth rate of productivity by 20% to 2.4% in a given year.
[00:30:10] Paul Roetzer: Such a change is barely noticeable if it is usually swamped by cyclical fluctuations. So basically like you wouldn't even notice a 20% gain overall macro level, but productivity growth compounds. After a decade, the described tiny increase in productivity growth would leave the economy 5% larger and the growth would compound further every year thereafter.
[00:30:32] Paul Roetzer: What's more, if the acceleration applied to the growth rate of the, of the growth rate applied to the growth rate of the growth rate? For instance, if one of the applications of AI was to improve AI itself, then of course growth would accelerate even more over time. So basically they're just like, we have no idea, but every report we're currently looking at, whether it's Goldman Sachs or some of the others ones they cite seems to imply we are heading down a path we've never been down before in terms of productivity gains.
[00:31:04] Paul Roetzer: Now that can be great because the US economy in particular is in massive debt. We're in the process right now of trying not to default on our debt. And so the way out of that is to increase productivity. So that seems like a positive thing. But the question just becomes what impact does it have on the people doing the work, and how important are those people to future models?
[00:31:25] Paul Roetzer: And so again, it just goes back to a few episodes ago and we talked about we could be looking at millions of jobs in the next 18 to 24 months that are impacted. Well, certainly impacted. That's unde debatable. How many are lost? Is the real question and my. Thesis was like, I think there's a greater than 50% probability that we lose millions of jobs in the next two years.
[00:31:47] Paul Roetzer: I think it may gain over time and you can go back and kind of listen to that episode and we'll put the show notes, the link to it. But, you know, I think that this article just hit for me because these are really well known economists that are not sure what to make of this is kind of how I took it, and I did feel like they were trying to put a positive, like optimistic spin on it.
[00:32:12] Paul Roetzer: But I read the article twice. Like their, their data doesn't seem to support the optimistic perspective, is kind of how I took it. Interesting. Yeah. And I want it to be more positive, but that's just, I don't know.
[00:32:25] Mike Kaput: Yeah, it is interesting you get, like you mentioned, these quotes throughout the article that shows this tension because after some very positive stat or outcome, they'll say things like, quote, instead of the lowest paid workers bearing the brunt of the disruption, now many of the highest paying occupations will be affected.
[00:32:45] Mike Kaput: These workers may find the disruption to be quite unexpected. So then you get statements like that and you're like, oh wait, this could be a little rockier than they're. Describing,
[00:32:55] Paul Roetzer: and you can just read, I mean, even honestly, like if you just go in and read the italicized part at the top, they do a pretty good, like 75 to a hundred word synopsis of the whole thing.
[00:33:05] Paul Roetzer: And then, like I said, on LinkedIn, I kind of called out the three main things with a few quotes. But I, again, my, my overall thing is here is don't, this isn't a doomsayer kind of thing. Like nobody knows, the economists don't know. And so it goes back to what we've always said is like the best thing you can do right now.
[00:33:22] Paul Roetzer: Is, embrace this stuff, figure it out, figure out what it means to you. Find ways to start applying it every day. Bring value to your organization, help others figure this stuff out, and you're, you'll be fine like that. But if you don't do that, I can't. I don't know what's going to happen. Like that's all we keep saying is like just.
[00:33:40] Paul Roetzer: Take the next steps, learn the stuff, figure it out. Find ways to improve your own career with it and like just stay focused on that. Don't let all this other stuff kind of bother you, but you have to know this other stuff's going on at a macro level and that a lot of really, really smart people are trying to figure this out, and no one seems to have the answers yet.
[00:33:59] Mike Kaput: The really good takeaway. We have a ton of rapid fire. I'm just going to sit back and let you, I'm going, I'm going to, I'm going to move fast here and each of these probably could be a main topic on its own. First up, a 23 year old influencer named Karen Marjorie with 1.8 million Snapchat followers introduced Karen ai, a voice-based chatbot.
[00:34:23] Mike Kaput: Of her that is built as a virtual girlfriend. So she used AI to create a clone of her voice and her personality or persona from thousands of hours of her own recordings. And so this actually debuted on an app called Telegram earlier in May. And it enables individual chats with users and they pay $1 per minute to engage in intimate conversations.
[00:34:50] Mike Kaput: Despite being in beta testing for just a week, Karen AI has already generated over $70,000 in revenue for this influencer. This just seemed like a weird and wild story of what's now possible with this technology. What did
[00:35:05] Paul Roetzer: you think?
[00:35:10] Paul Roetzer: I thought, I have an 11 year old daughter and I just don't know where, where the future is going. I had a lot of thoughts when I saw this, but I'm probably not the place to get into this. It's kind of like the 900 numbers of like the 1980s. I think I saw a meme about that. Like, it's just like the dial, dial a minute and, you know, pay by the minute kind of thing.
[00:35:31] Paul Roetzer: I'm going to get out of the, How, why people are paying this realm and like go into the, kind of the business side of this. I guess imagine you're able to do this as a strategist or as a celebrity, or you know, as a business leader. So if your purpose for creating it is actually to assist people and to help them.
[00:35:57] Paul Roetzer: And you can train it on your knowledge base. So again, like think about that example we said about just assume every blog post you've ever written, every book you've written, every podcast you've done, every piece of knowledge you've ever created or your company has ever created, imagine that a language model is trained on that.
[00:36:14] Paul Roetzer: And then imagine you're able to create a synthetic version of yourself, like a digital version that kind of looks like you, but it's, you know, has voice split like you and you could actually sell a subscription service to that. That's the kind of. Business opportunities we're heading into. And so again, if you're a celebrity, you know, think about these, what's that?
[00:36:31] Paul Roetzer: I can't remember that company where you could like pay money and a celebrity would show up on a Zoom call or something. Yeah, cameo. Yeah. Yeah. Like imagine you could do that, except you could be in a hundred places a day. So a cameo AI version. There you go. Cameo. If you don't have that in your roadmap, like you're welcome.
[00:36:47] Paul Roetzer: But that's what we're talking about here. I saw something last night. Was that Justine Bateman, I think. Was she, I forget what she was on. It was like a 1980s actress. I think. But she was talking about this writer's Guild strike. Mm. And how they're already doing this. Like they're creating synthetic versions of actors.
[00:37:10] Paul Roetzer: And so you'll be able to basically license away your, yourself, your digital being. And so you could be recording like four movies at once. Like Wow. Just basically. And so I just, I think this is just a really interesting example when you extrapolate out of the, You know, people paying a dollar an hour for intimate conversations.
[00:37:29] Paul Roetzer: And we talk about the business side. There's, there's a ton of use cases and implications for business. This is just maybe like a early foray into it. Wow.
[00:37:41] Mike Kaput: Next step. We got some big news from OpenAI. They are now rolling out their web browsing and ChatGPT plugins to all ChatGPT plus users over the next week.
[00:37:53] Mike Kaput: So basically, if you have ChatGPT plus very soon, if you happen to already, you will be able to use the browser plugin that allows you to access the internet and there will also be 70 plus. Third party plugins that they've already announced that have been available to certain users will now be accessible by everybody.
[00:38:11] Mike Kaput: This seems like a big announcement, especially on the heels of Google and Microsoft with their big announcements. Yeah,
[00:38:18] Paul Roetzer: so quick notes here. So I've had the browsing plugin for a few weeks. it, it does, it's interesting. It works pretty well. I've got the code interpreter plugin and that's it. I don't have the rest of 'em.
[00:38:27] Paul Roetzer: So I have seen some people online who have gotten access to these. So you can look online if you want to learn more about it, but I was a pretty early wait list person. I don't have 'em yet. Could be turned on at any time. The thing I will say though, that I have noticed is, a lot of people who have tested 'em are like, yeah, it's really interesting.
[00:38:45] Paul Roetzer: They don't work. Like a lot of 'em just do not work, so don't assume like this is going to get turned out in the world. I mean, it's just changed and nothing's ever going to be the same again. There's a good chance you're going to go in there and play with 'em, like these kind of suck. Like, I don't want to use this, or I don't want to connect my data to this one.
[00:39:00] Paul Roetzer: And that's expected. I mean, it's an emerging ecosystem. There's going to be junk in there, there's going to be some that you find that are super useful. So if you got time to play around with them and explore, awesome. But it, it's not, we talked before, like the plug-ins are a huge deal. Don't expect life-changing, things to happen from this first run of them once they're turned on in your portal.
[00:39:25] Paul Roetzer: So next one. Oh, and you do have to real quick, you do have to go into your settings and turn on beta user. So if you d if you don't. Have it turned on already. Go into your settings in the bottom left, in the in, in the Chet interface and then click on, I think it's beta, and then make sure that you're toggled on.
[00:39:42] Paul Roetzer: So that'll at least give you once they do, roll 'em out, the chance to get 'em.
[00:39:45] Mike Kaput: Gotcha. That's an important note because I can imagine people like myself just sitting here excited. Waiting. Waiting. Right. So next up, another interesting kind of company brand use case for ai. Wendy's, the fast food chain is actually starting to test an AI powered chatbot next month in June, and they will begin to talk to customers and take drive-through orders with ai.
[00:40:08] Mike Kaput: So, This is a system powered by Google Cloud's AI software. So they're building on top of like Palm two and the, some of the technologies that we just discussed. And basically they will allow a system to understand speech, answer questions and fill your order. And I think a fun fact as well, I think the first one is in Columbus, Ohio, so relatively close to us here in Cleveland.
[00:40:31] Mike Kaput: Yeah. What did you think of this development,
[00:40:33] Paul Roetzer: Paul? Yeah, it's I think we're just going to see tons of these. I mean, I think behind the scenes right now, every major organization, enterprise is having these kind of conversations about ways to infuse this and I think throughout 2023, we're just going to see tons of interesting use cases.
[00:40:49] Paul Roetzer: So again, kind of a sign of things to come. Yeah.
[00:40:52] Mike Kaput: And definitely probably adoption, being driven by fast food, especially being, having chronic labor shortages. Yes, it sounds like
[00:41:00] Paul Roetzer: these days. So, yeah, and I think that's an important point, Mike is. We talk about the loss of knowledge work jobs, potentially.
[00:41:06] Paul Roetzer: But there are a lot of industries that are having massive labor shortages. So in some cases, AI replacing, or significantly automating jobs is, is going to be a really good thing, especially for small business owners who are struggling, you know, to find workers. And it's everywhere still. So, yeah, I mean, again, like you're always looking for the positives here, and these are ones where it could.
[00:41:33] Paul Roetzer: Be a positive.
[00:41:36] Mike Kaput: So another big AI announcement from Meta, they announced that they are. They have built and are open sourcing something called image Bind, which is the first AI model capable of binding information from six modalities. Now, what that means is that the model can learn, you know, not just from text image and video and audio.
[00:41:56] Mike Kaput: But also from sensors that record depth, so like 3D thermal sensors, so infrared radiation and inertial measurement units. So basically motion and position. So what this means is we now have an AI model from meta that will be usable by anyone from an open source perspective. That gives equips machines with a holistic understanding of how of connecting objects in a photo, understanding how they will sound, their 3D shape, how warm or cold they are, and how they move, which opens up an exponential amount more of possibilities for building on top of such a model.
[00:42:37] Paul Roetzer: going to be bonkers. Like, I, again, I mean, I think so many people are new to AI, are just blown away by the language generation capability. And it's really hard to imagine a world in, you know, 1, 2, 3 years where all of these models are multimodal, where they all learn from an output. Images, videos sound like.
[00:43:03] Paul Roetzer: it's going to get so crazy. And a paper like this, like we've said before, like these research papers and these releases from companies like Meta, if you dig into them and you start connecting the dots of where it could go, it gets really, really crazy. And so I just, I think we're going to see in 2023, we're probably going to hear a lot about multimodality.
[00:43:25] Paul Roetzer: We're going to see, you know, GPT-4 or whatever version of GPT-4, G P T it is that's going to have. Image capabilities and eventually video capabilities. And it's probably going to happen in, in pieces. It won't be like you just flip a switch and it's all multimodal. But that's certainly where Google's going.
[00:43:44] Paul Roetzer: Google probably has a, a headstart there. If you think about all the data Google has, they have YouTube, they have Google search with all those images, like again, advantages. Google has training data. I didn't list in my original, but. I got more of it than anybody. And yeah, this, I don't know. This is fascinating stuff.
[00:44:02] Paul Roetzer: I haven't read this paper yet, but. Meadow believes Yann LeCun believes that like a worldview is very important to, to general intelligence that the AI has to be able to observe like a toddler would. So the way, if you think about a toddler, how they learn, it's not all coded into them. They actually just observe the world around them and they, they learn things that way.
[00:44:22] Paul Roetzer: And so that's, I think part of Jan La Koon's belief is that language models on their own aren't enough. We, we actually have to be able to kind of. Observe the world. And so this sort of innovation is actually a precursor to the ability for these AI agents to observe and learn from things happening around them, which I think they believe is a, a better path to agi.
[00:44:45] Paul Roetzer: I. Wow.
[00:44:48] Mike Kaput: So next step. A leading generative AI platform. Our friends@writerwriter.com just announced some big new features, and you can read all about them in the show notes, but some big ones are there's a knowledge graph now in this platform that connects all your important data sources, so you can actually fact check and ask questions in real time.
[00:45:08] Mike Kaput: There is a self-hosted large language model so you can update, configure, operate, and update your own model without relying on third party services. There's a cool service in the platform now where you can upload anything including PDF stocks, PowerPoints, et cetera, and actually, Query the content, kind of like we discussed a bit with Anthropic and actually generate content based on those documents.
[00:45:33] Mike Kaput: There's a writer Mac app now, and then there are commands in the writer web app that actually make it much easier to research, create, and edit without having to go through a bunch of different tabs. So, you know, Paul, we've kept tabs on and have a good relationship with writer, over the last.
[00:45:48] Mike Kaput: Several years. And they're kind of, you know, an enterprise grade platform built for teams. So what did you think of seeing these feature updates?
[00:45:56] Paul Roetzer: Yeah, it was, they're super cool. I mean, they do a lot of smart stuff. They're, it's good people. Habib was one of our speakers at our AI for Writer Summit.
[00:46:03] Paul Roetzer: Shell should be at Mayon as well. So, yeah, I mean, just, just continually doing cool stuff. The PDF one is interesting. I think I've seen that a lot lately. Like that's one of the use cases I've been waiting for is, yeah, like these research papers that we talk about, that we read, some of these things are really dense.
[00:46:20] Paul Roetzer: They're like 40, 50 pages. And so the ability to take a research paper, take a segment of that research paper and just feed it to it and say, here, like, what is it? Summarize this for me. Pull out the key data points. So I think just some of these more common use cases are going to start to be made really simple within these platforms.
[00:46:38] Paul Roetzer: And so you know, writer, again, if you're an enterprise looking for, to figure out your strategy for large language models and generative AI is certainly one of the companies to keep an eye on and have a conversation with. That's
[00:46:49] Mike Kaput: awesome. So another big update is we've talked quite a bit about the European Union's Artificial Intelligence Act.
[00:46:56] Mike Kaput: So this is pending standards for ai, in the eu. And they actually just agreed on some text for the act. And there's, you know, we'll link to the full updates here because they're quite extensive, but a couple big ones. And you know, someone on Twitter put it really well. Think of this as broadly GDPR for artificial intelligence.
[00:47:17] Mike Kaput: Now. As part of these regulations, two that really jumped out at me, the builders of foundational models are going to have to disclose details about how the models were trained. So, you know, the OpenAI, the Googles, the Anthropics of the world will actually have to be, will be forced to show how they're training their models to the EU in order to be in compliance with this act.
[00:47:42] Mike Kaput: Also foundation models will have to clarify the copyright status of the data they use to train the model, which is a huge issue right now in Europe and across the globe. Those seem alone, like two pretty big updates. Paul.
[00:47:58] Paul Roetzer: Yeah, I know this isn't law yet, but you know, the compromise, this is part of the barriers.
[00:48:02] Paul Roetzer: Those are very interesting, notable items. I would be fascinated to see how that lays out because as we've talked about, like. GPT-4 in particular, like we, we have no idea what was trained on, like, you can guess. And I mean, there's been some research projects trying to figure it out. But part of the reason some people think they didn't disclose it is because they, they stole stuff.
[00:48:24] Paul Roetzer: They, they took copyrighted stuff that they maybe didn't have the rights to. And that's in the training set. So, yeah, I don't know. I mean, it'd be interesting, but, you know, there's a reasonable assumption that they're all training on similar data that they maybe shouldn't be training on. So at the end of the day, I don't know.
[00:48:44] Paul Roetzer: I just, I don't know that like it's going to have a major impact, but it'll certainly make for some interesting headlines when all this finally comes out.
[00:48:51] Mike Kaput: So another big announcement with partnerships in ai, Jasper, you know, a generative AI platform, the one of the leading ones we're very familiar with.
[00:49:01] Mike Kaput: They actually just announced a new partnership with Google Cloud that will actually enhance Jasper's AI engine and make it more accessible to millions by being listed in the Google Cloud marketplace. This seems like a pretty big deal. We've seen multiple. Big AI companies, third party startups, you know, partnering with some of the bigger platforms.
[00:49:23] Mike Kaput: What did you take away from this?
[00:49:25] Paul Roetzer: Yeah, I, the night before Google IO conference, so that was last Tuesday, they announced a series of partnerships. Jasper's one of 'em box was another. I mean, if anything, it shows that these application layer companies aren't relying on single cloud providers. So, you know, they're building on a bunch of.
[00:49:45] Paul Roetzer: APIs. So they're pulling in, you know, GPT-4, they're pulling in, you know, Palm two. Like basically if you are a pla an application layer, you, you can't just build on one language model. You're going to have to have a collection of them. And it seems like what they're all doing is basically saying, Hey, you as the end user, the VP of marketing, the C M O, whatever, don't worry about picking the right language model.
[00:50:10] Paul Roetzer: We have five of them, or whatever it is. Based on your use case, our system will determine which model to pull from. And so I think that that's part of their play is to take some of the confusion out of the marketplace and be the one that does all the infrastructure so that the end user doesn't have to worry about going right to OpenAI and picking them and then realizing, oh, that was a bad choice.
[00:50:34] Paul Roetzer: So if you're able to sort of have a diverse set of models to access. I think that's what it seems like the play is, so yeah, it's just Google's making their play and getting in the space. All
[00:50:48] Mike Kaput: right. Last but not least, we have another
[00:50:52] Paul Roetzer: old friends at ibm. Our
[00:50:53] Mike Kaput: old friends at B m are back in the game here at their annual Think Conference b m announced B M Watson X, which is a new platform that delivers tools to build AI models and give access to pre-trained models.
[00:51:09] Mike Kaput: In the generative AI space. So basically help customers better and easier use artificial intelligence kind of in their operations Now, you know, b m Watson was one of the kind of foundational AI systems we wrote about in the book. You know, they were on Jeopardy. I B M has been somewhat quiet in the last few years, it seems like.
[00:51:27] Mike Kaput: When in the AI race. What did you think about them being back in
[00:51:31] Paul Roetzer: the mix? Yeah. I mean, it's good to see. I think they, they made a massive bet for years on Watson in healthcare, and I don't know that it played out the way it was supposed to. But yeah, I mean, our origin is from Watson. Like I've told the story before, but you know, Watson, winning on Jeopardy in 2011 is actually what piqued my curiosity about what AI was, which led me to spend years trying to understand it, which led us to.
[00:51:58] Paul Roetzer: Create the Marketing AI Institute eventually and help other people understand it. So, you know, I have a special place in my heart for IBM and Watson and, I, you know, I I think it'd be really interesting if IBM showed back up. I mean, if thousands of AI patents, they've, yeah, they've been at this again for 20 plus years.
[00:52:14] Paul Roetzer: We don't really talk about B because they don't have, you know, it's not a massive play in, in the marketing space that we look at and. They don't have a bunch of tools that are readily accessible, but maybe they're going to move in that direction. It'd be really interesting if IBM showed up and wanted a seat at the table.
[00:52:29] Paul Roetzer: So I don't know, it could be fun. So I'm kind of cheering for 'em, but I haven't like dove into Watson X yet and seen what it's all about. But it's cool to see that they're part of the conversation again.
[00:52:40] Mike Kaput: That's awesome. Yeah. Don't count anyone out in the AI
[00:52:42] Paul Roetzer: race. Yeah. They, they're never like, when I'm list, like literally I'm listening all these people.
[00:52:46] Paul Roetzer: I never list IBM anymore, so maybe I'll have to start throwing IBM in the mix again.
[00:52:52] Mike Kaput: Awesome. Well, Paul, thank you as always for the time and insight to kind of unpack what's going on in ai. As I'm sure listeners can tell, there's way too much going on all the time, so it's, thank you for clarifying it for us and demystifying
[00:53:07] Paul Roetzer: it.
[00:53:07] Paul Roetzer: Yeah, good to have you back. And oh, by the way, happy belated Mother's Day to all the moms out there, our, our three, Tracy, Tamara and Kathy at the institute. You know, we just, and my, you know, our own moms, but, just a couple days past that, but I wanted to make sure we recognized all the amazing moms out there, especially those working moms who I don't know how you do it.
[00:53:28] Paul Roetzer: So, thank you to everyone. Thanks to all our listeners, and we'll be back next week.
[00:53:36] Paul Roetzer:
[00:53:36] Paul Roetzer: Thanks for listening to the Marketing AI Show. If you like what you heard, you can subscribe on your favorite podcast app, and if you're ready to continue your learning, head over to www.marketingaiinstitute.com. Be sure to subscribe to our weekly newsletter, check out our free monthly webinars, and explore dozens of online courses and professional certifications.
[00:53:58] Paul Roetzer: Until next time, stay curious and explore AI.