Voice AI, Speech-to-Text, Sentiment Analysis, Intent Recognition – Damien Murphy (Deepgram) #0to1AI

okay so we are here at Cloud flare we just participated in a hackaton uh it was pretty amazing a lot of people a lot of interesting conversations and we met here Damian Murphy from deepgram uh the company that provides uh uh SDK for voice uh for like maybe you will tell us more a little bit about that in a moment and one of the sponsors and one of the sponsors so Daman helped us a lot during our struggle with building the app uh but before we start we wanted to ask you uh how did you start with Ai and programming uh what was the the path uh that you yeah so brought you here yeah I started uh programming probably back on a BBC computer oh wow so my father uh had a little side hustle where he'd like you know work on computers so you know started out on a on a terminal basically okay uh and then from there over the years just kind of learned more and more you know went to university did a lot of different things and um yeah immigrated here about 10 years ago from from Ireland and uh yeah I was working with sap at the time okay so large multinational um and yeah just really wanted to get into the startup space MH uh so you know left sap joined another startup in the mobile cicd space and after that you know chat GPD came out and it's okay got to dive in got to got to got to change I either adapt or die I think when it comes to AI so definitely wanted to hop onto that wave so was deep CR the first uh your yeah first first a company okay interesting and so maybe you could tell a few words about what deepgram is yeah absolutely so so deepgram is a speech to text and text to speech AI company so we build our own foundational models and we run our own infrastructure in our own data centers uh so we're vertically integrated right we basically you know we we don't give away our revenue and we we keep it to ourselves um and we reinvest it in research and because of that we've actually been around since 2015 you know long before chat GPT came out um but you know ever since chaty BD came out a lot of people realized they had a you know a treasure Trove of information stored in audio that they didn't really have a way to leverage right um and now you can actually unlock that audio data turn it into text and then take that text and turn it into insights and you know intents and and sentiment and all these different kind of value things when you think about the the voice and like transcribing or changing from text to speech or speech to text it seems very easy but you at Deep gram you're doing much more than that right for example we discovered uh just today that you are doing sentiment analysis what else is there in in in in deep grum API yes so we we do sentiment analysis intent recognition uh entity detection topic detection uh summarization um and like with all of that you can actually build up a very clear picture about what a phone call is about right and you know one of our key use cases is analyzing phone call data right and so you know when you make a phone call and says you know this may be recorded for uh training and and quality purposes that allows them to actually run analysis and say okay you know which of my 10 million calls this year do I need to actually look at right um you know what what are the good ones why were they good what are the bad ones why were they bad right um nobody can listen to all of those calls um but AI now can actually get that information um and what we're seeing big like growth in right now is you know AI voice spots right so you know you can think of traffic patterns into call centers that come in waves right so imagine you're a you know large uh service provider and you have an outage you're going to get a sudden massive Spike that you just can't answer the phone for um so the ab ability to scale up with AI and you know take that uh sudden demand it's really useful it gives you Competitive Edge right in a way yeah definitely and it and it and it does good right because you know you can have 911 issues right like imagine there's a natural disaster suddenly all the phone lines are like nobody can get through you can't collect anybody's information and it may be too late by the time they do get through to actually uh save their life right so being able to have an AI That's always on always able to answer that phone can can be the difference between life and death and you doing something very interesting uh because when we were preparing for the hackaton we were just scheming for the docks and then we discovered you told us about that for example you can uh create with deep gram uh this like a system or let's say process where you can in interrupt right and you can really you can detect when a person stops talking and uh could you elaborate on that how it how it works and yeah yeah so the the inference engine uh for real time it's actually processing the audio extremely fast right so you're you're taking slices of the audio right and you may may take like five slices of a single word uh and that allows us to bring the latency down of detecting words right so now you're getting very fast words and but we also want to increase the accuracy so while a person is speaking um you're increasing accuracy the more you learn right so like you mightn't have very good accuracy on a single word but as you get a sentence right now you can actually really solidify that and so over time what we're doing is we're we're basically getting you the best transcript and then locking that away and saying okay that's finalized you know next bit right and then as soon as you stop speaking uh we'll detect that silence and we'll say okay we're not going to receive any additional context so let's take what we have right now finalize that and then that gives you that super low latency high accuracy transfer yeah and this this has been tremendously useful for us today yeah because we were trying to do it in the browser API and it's a struggle there's a a lot of like uh let's say mathematics involved and deep is doing that that for us right and at the same time I was impressed how fast it is the late because before we tried the like rest apis when you sent pre-recorded uh tracks and here we could use the web c connection send it real time and it was really really fast so I was wondering because uh if we talk about the future of AI with deepgram you can for example imagine those interfaces where like a fluid interfaces where you can talk the way you talk to a human right because deepgram allows for that uh because of the low latency and generally of all those features you mentioned how do you see the future I mean of a John do you think it's like a good thing or bad thing or we will see I mean I I think AI is going to bring a lot of good um but you know at the same time people can use that for bad right so you know one of the big things that we do is responsible Ai and making sure that you know when we do release say voice cloning um that we put in place you know um water marking we put in place you know terms and conditions that don't allow it to be used for nefarious uses um and really just making sure that you know the system we have in place to clone a voice prevents people from cloning other people's voices right and you can do that with various things like you know in order for you to clone a voice you're going to have to read out this uh number and this Alpha numerics right and we can then verify whether you know you actually said that right so it's very hard to get somebody else's voices to say you know 659 ABF right and and that gives you then that voice kind of proof that this person you know was present when the voice was actually being cloned um but you know getting back to the low latency AI stuff um it's it's actually really powerful in a lot of different ways right so so you can you can offload a massive amount of um kind of monotonous work right um if you think of like outbound sales calling right you have people that basically are making phone calls repeat the same thing no answer right okay make a phone call get a voicemail make a phone call somebody answers oh amazing somebody answers would you like to bur right so how much time do you think people want to spend doing that right very little um so there there's companies we work with that basically automate that process now they don't speak to the customer on the other end but what they do is they're able to figure out when somebody answers using Ai and then that person that actually wants to talk to somebody right you know because that's their job now they get to talk to somebody right they don't you don't have to go through that pain of you know constant hangups and and and so it really optimizes the fla right yeah yeah and it's a productivity boost right you know makes the work more satisfying for them as well yeah it could be a 70x increase in you know productivity on outbound sales and then with customer success I wanted to ask you uh because you you focus on deep gram and and voice AI let's say uh but do you see any other like interesting or what's outside of that this this little ble in AI what what interesting projects have you seen and uh I think AI agents right I think that's going to be uh the future right because like we have right now a very powerful uh AI system using Transformers and and we'll probably iterate on that and have better ones um but they're very good at a specific thing that you get it to do right so you could have an agent that's like a returns agent another agent that's cancellation agent another agent that's like a you know an upsell agent complaint agent right like you know you can have an agent for nearly every single use case a pipeline of agents and then at the front of that is like you know instead of this menu press one for this press two for this right because essentially that's like a decision treat right you can have that rooting agent like be completely seamless just like hey how can I help you today and they you just speak you just say what your problem is oh my my internet's really terrible you know I need some technical support oh of course I can help you with that right so none of this on hold waiting and moving like you can swap agents out really easily um and and and that's just if it's like inbound calling right now imagine what you can do with autonomous agents um you know when you when you apply them to other use cases um like there is a lot that prevents you from like cold calling people with AI um but you can call your customers right so imagine I'm a large soda manufacturer and I want to call up all of my stores and ask them how many cases of you know uh you know some some sort that you want right um that's a very manual task so you you can you can automate that like you know order taking across your entire customer base and you know it's probably going to be a lot easier right you can get through very much more people much quicker it means they get the product quicker right so you know all the turnaround times really really improve and do you see that as a part of deep gr or rather like a layer on top for other startups to build on top of De so so we're a foundational AI company right we provide you with the models and we provide you with the API and a lot of our customers a lot of our largest customers are resellers right so they sell transcription to their customers right um but they sell other things as well right they they sell business value on top of it right so you know like you look at that use case with the podcast right um imagine if he started selling you know transcription podcast services to his customers right you know so it kind of it allows us to provide you know very highly scalable API but not have to worry about building you know complex business products for every different use case and the interface yeah awesome and I'm I'm wondering if we uh because we're trying to Target different audiences and different levels of like Advanced and beginners uh I'm wondering if you could have an advice to people who are just getting started in programming what do you think should they still like learn what what we've learned when we started or should they to take totally different path do it differently yeah right now I I think there's value in understanding the basics right like you know you can't do complex mathematics unless you can add Divide Subtract right um so I think there's definitely basic steps right for Loops things like that if you understand fundamentally programming that will make you a better user of AI right because now you can understand the limitations of the language that the AI is going to write um but I don't know if it makes sense for people to you know spend hundreds of thousands of dollars going to the best school to learn to program Java or python when it'll probably be relatively obsolete pretty soon like the the programming language has always tried to ER towards natural language right so like for Loops are for because they want it to be natural language right like there's no other reason for it to be for um only because that sounds familiar right um but under the hood it goes down to assembly and then then the binary and you know now you're running like all this kind of stuff so like the ultimate evolution is just natural language and and I think if if you can be very good at controlling an AI to achieve your goals um that can get you probably like 80% of the way and but you you're still going to have to go in there and tweak things because you know like trying to get an AI to modify a single character is kind of a waste of money right you know if if if you know that you want the the text one pixel bigger you know do you really want to prompt an AI every time like you know so much trouble sometimes yeah yeah uh thank you very much and for this inspiring interview we learned a lot and uh yeah we we put the links to deep gr and you shared with us very interesting uh repositories like examples there's a lot of things to do thank you once again thank you byebye byebye

Join 0to1AI 👉 https://www.0to1ai.com The video you provided features an in-depth discussion held at Cloudflare during a hackathon, including a detailed interview with Damien Murphy from Deepgram. Here’s a chapter breakdown with timestamps summarizing the key sections: Chapter 1: Introduction and Background (0:00 - 0:57) Damien Murphy shares his personal background in AI and programming, starting from his early days programming on a BBC computer, his migration to the U.S. from Ireland, and his transition from working at SAP to joining startups focused on mobile CI/CD and AI. Chapter 2: Introduction to Deepgram (0:58 - 2:18) An overview of Deepgram's services, specializing in speech-to-text and text-to-speech technologies. Murphy elaborates on how Deepgram builds its foundational models and runs its own data centers, ensuring vertical integration and reinvestment in research. Chapter 3: Advanced Capabilities of Deepgram's API (2:19 - 3:26) Discussion on the advanced features of Deepgram's API, including sentiment analysis, intent recognition, entity and topic detection, and summarization. These features help analyze large volumes of audio data for actionable insights. Chapter 4: Real-Time Processing and AI Utilization (3:27 - 5:39) The conversation shifts to the importance of real-time processing in AI applications, particularly in handling large spikes in call volume during events like outages or emergencies. Murphy explains how AI can scale to meet demand, potentially saving lives. Chapter 5: Interactive AI and Future Perspectives (5:40 - 7:33) Murphy speculates on the future of AI, emphasizing responsible AI practices and the potential of voice cloning technology. He discusses the need for security measures to prevent misuse, such as voice watermarks and stringent user verification processes. Chapter 6: Enhancing Work Efficiency with AI (7:34 - 9:56) The discussion wraps up with the benefits of AI in automating monotonous tasks like outbound sales calls, enhancing productivity, and improving job satisfaction. Murphy highlights how AI can streamline customer service and other business processes. Links: https://deepgram.com/ https://www.linkedin.com/in/damienm1/ Follow us: https://www.linkedin.com/in/zaiste/ https://www.linkedin.com/in/mmiszczyszyn/ https://www.0to1ai.com #VoiceAI #Hackathon #AIInnovation #TechTalks #deepgram