Building an AI App in TypeScript & Bun with Groq SDK using Function Calling

hi in this video I will show you how to build an AI app in JavaScript typescript specifically using Gro there are other LM providers but I really like groc it's not only fast Innovative but generally pretty cool and the goal for this series so this will be a series hopefully the goal for this series is to build a voice agent with low latency something that feels like human and the Grog will be a center the center of that uh application and then I'll be using other uh interesting Technologies and products like deep gr cesia and some maybe other so in this first video we will try to uh see how we can integrate with Gro how we can use Gro SDK and I will also explain uh how to use tools so how to help AI uh to fulfill certain tasks for example search the web or ask some Services external services for some data to integrate uh in the response so let's start uh so first of all I will create a repository uh directory rather and I'll be using ban uh so let's initialize that and let's open Okay so let's try it is it working it works okay so to make it work you need the API key you can generate it here I've generated that so you need to place that into n file so let me just uh copy a file I already have and uh let's build something so first of all this code is it could be improved a little bit so instead of like do doing the require we will do import Gro Gro uh SDK I cannot type uh but we haven't installed that so it should be good and now we can do Gro and we don't have to Define anything here it already knows that it should look for Gro API key so we can leave it leave it empty like that and we don't have to use the main uh function we can just start writing top level in this file so I can just do something very simple like result Gro uh chat completions create uh yeah uh this should be good except the model would be using Lama free and here tell me something [Applause] about something like that so let's see if it works um let's run it of course I need to await uh it works so I just need to extract the actual so it's Choice zero message content and uh yeah it provides a response we can maybe specify be concise and here I don't know choices [Applause] maybe something like that let's see if it works yeah it works so now the next step would be to use tools or maybe before that uh let me show you something pretty cool we can do streaming right so instead of like waiting for the whole response to arrive we can start um writing it so maybe I should keep it result and then we can use for a weight we have chunks and uh in that case the result is the actual stream and here we can do choices and in this this time Delta uh but instead of write using console lock we should use uh right because we want to we don't want new lines and uh here it's going to be empty like n let's see yeah it works and it's streaming the each chunk uh arrives after another um let me commment that out and yeah let's return to this uh previous but not Delta message [Applause] rather okay so now let's define uh tools so we will Define uh tool so type function and function name search for searching the web like so and here we need a description search to for information it's good and then [Applause] uh and required so now we are saying that we have this function called search and that LM can use that to get some information but we need to write this function and I was playing with those apis for search and they are not optimal because you have to register and so we'll be cheating a little bit here so I'm just defining a function which checks if this is NVIDIA or Gro and return some text and this is like the idea to simulate or to like imagine that LM can now invoke this function to get some data uh so now since we defin the tools we have this new um property called tool calls so we can see if LM used uh this function and if it used uh the function um it will ask us to invoke us invoke this function so it will provide what needs to be searched for so we haven't defined anything here um search the web for Gro and then search for NVIDIA and let's [Applause] see so I'm just displaying the the tool calls and because I asked to search for two things we should have two elements in this uh array yeah I think it's [Applause] okay uh sometimes it's not it's not consistent so yeah Gro Nvidia so once we have that we can now uh we need to invoke this uh so for each tool call uh we will be um invoking our function with the call uh function arguments uh but I think we need to parse [Applause] it and query because we have this yeah as a query function arguments yes and yeah for now let's just display the result okay so it returns uh Nvidia maybe the web let's be more explicit yeah we have two responses so now we can use those responses we can chain them uh we can append them to the messages here so I will just extract that messages uh but I would prefer this instead of that [Music] [Applause] and we are passing this here we have an error because we need to um chart completion uh message BMS I think so this is not ideal I think this type should be coming from the top level uh but yeah it works so that's our query our message uh here the function that imitates an external call uh we are invoking it using providing that we have tools functions that LM can use and we are invoking so when we get a uh information that LM needs to invoke some function we are invoking that for for them for it and uh now we can append so first of all we should append this information uh so result choices message and once we have the result we should also need to append [Applause] it so it's important the role will be um tool and the content will be result and we also need to uh Define the ID because we have two uh calls so we need to associate each uh call with the proper result and finally as we appended those things to the messages list invoke uh result to maybe await yes something like that and console log so this should work yeah so I think it grabs um yeah it integrates it's not deterministic but it integrates the information about Gro and about Nvidia and then responds to our uh query so what what is inference in the context of our LMS and then search the web using this function so that's pretty much it for this video and in the next one we will try to improve that so I will be uh showing you how to use versal AI SDK uh to make it a little bit more uh streamlined hopefully that was useful thanks for watching and see you in the next one

Welcome to the first video in our series on building a voice agent AI app using TypeScript and the Groq SDK! In this video, we dive into the basics of integrating with Groq, a powerful and innovative language model provider. We’ll also be using Bun, the modern JavaScript runtime, to streamline our development process. What You’ll Learn: • Setting Up the Project: We’ll start by creating a project directory and initializing it using Bun. • Using Groq SDK in TypeScript: We’ll cover importing and utilizing the Groq SDK to create chat completions. • Basic Functionality: See how to set up simple AI interactions and refine the code for better performance. • Streaming Responses: Implement streaming responses for real-time feedback. • Defining Tools: Create functions that the AI can call to perform tasks like web searches. • Handling Tool Calls: Learn how to handle and respond to tool calls made by the AI. Stay tuned for the next videos in this series, where we’ll explore more advanced features and integrations, including using virtual AI SDKs to streamline our app further. Don’t forget to like, comment, and subscribe to stay updated with the latest tutorials and updates! Chapters: 0:00 Introduction 0:45 Project Setup with Bun 3:30 Groq SDK Integration in TypeScript 5:00 Basic AI Interaction 7:15 Streaming Responses 9:00 Defining and Handling Tools 12:30 Final Integration and Testing 15:00 Conclusion and Next Steps Useful Links: • X: https://x.com/zaiste • GitHub: https://github.com/zaiste • LinkedIn: https://www.linkedin.com/in/zaiste/ Thanks for watching, and see you in the next video!