Build a Chatbot with LangChain

hey everyone hm how is everybody doing looks good eh yeah all right looks good looks good looks good Danny post my SEO blueprint course can highly recommend it all right Al so one and two I can just use PM code right uh for for how's everyone if you're watching say hi please oh hey hi all right thank you first one how you doing today uh I'm just getting like I'm just setting up uh here my my current repo because I actually want to push everything everything to the same GitHub repo from now on let's see we got p on code oh boy hey uh oh long chain yes sorry that was that was my mistake uh it should have been it should have been Lang chain you're right I made a mistake there uh can I update long long chain Lang chain L chain all right I'm updating my titles thank you very much for that uh for that mention Lang chain well the idea is that uh here we have the Elixir Lang chain implementation and it's uh by I think Mark Ericson right it's brain LD I believe this is Mark I forgot Mark Ericson yeah from uh thinking Elixir from the podcast and yeah he has that a pretty cool Lang chain project not long chain and I guess like what I what I from from reading this it means that you uh you can compose multiple steps right you can you know say hey like take this text and run it through Jet gbt and summarize it and then you can do another step and say well from the summary give me the top you know three highlights or like action points or something and uh that's my understanding at least and you can give some context uh yeah and you have like a a function here Returns the location [Music] description where's a prompt uh well so you have what is it l l chain and you have functions message and you have different chains and chat Models All right so you have a custom context which is like yeah you can give it some some information you always want to give it to every single action and then you have an Elixir function which is custom Returns the location or the requested element or item uh recr thing functions okay well we need to we need to understand this a bit better but there are two things actually I also wanted to yeah this one makes more sense here uh like you an open AI new what's it okay I don't understand any of this all right we need to go from the top usage is this L chain okay yeah there we go documentation demo project llms are merging as transform technology enabling developers Blah Blah Blah by using blah blah blah uh the real power comes when you can combine them with other sources of computational knowledge okay this library is aimed at assisting in the development of those types of applications that means nothing to me um so the main value props of Lang chain are you have components so abstractions for working with language models along with a collection of implementations for each abstraction okay components are modular and easy to use whether you're using the rest of the Lang chain framework or not cool off the shelf chains a structured assembly of components for accomplishing specific high level tasks uh I'm going to be streaming for like an hour or two not crazy long but you know oh sorry I also want to make this bigger um yeah like an hour or two I'm I'm just you know want to play around with this I I have API keys for for clae so we should be able to to connect this all right uh Len chain is ched for language chain and llm or large language model is the language part this Library makes it easier for Elixir applications to chain or connect different processes Integrations Library services or functionality together with an llm Lang chain is a framework for developing applications powered by language models and enables applications that are data rare connect the language model to other sources and adjust itic allower language model to interact with its environment I still don't know what this means elixir Lang chain enables Elixir application to integrate AI services and self-hosted model C application well there you go okay easy um there's a demo project we have some online uh [Music] documentation okay let's let's just have a look at the demo project maybe that makes more sense yeah no worries all right have a good good one see you later I'm going to have a little shink up all right a demo project okay so apparently this project is in demonstration for the L chain and live view okay features conversations are written to a sqlite database and the project directory conversations are performed using Elixir L chain Library uses Phoenix live view acing operations use control and a submit a message okay cool so you have let's have a look at this one what what is this this is a visitation a conversation so how does conversations work I guess we need to go on here so this is a oh it's just acto so it's every um message basically the name no a conversation okay well it's a different like a different conversation with a name of a model you have a temp temperature and the frequency penalty all right okay this is just a Ecto schema I guess then the the juicy parts are in the live view conversation live all right uh show all right yeah that's where you integrate all the different length chain demo or length chain here Chad open Chad open Ai and llm chain uh all right so in this live view we assign a conversation assign messages the llm chain and then an async result and whenever we open we get a new message so that's basic I guess that's a form for inputting the stuff I just want to see the llm part where's the llm part is it llm chain where is it where is the llm chain here all right assign llm chain all right you get a conversation convert the database thought message to Elm chain messages database message to LM messages okay this is done here all right so you basically okay so every message you rep to a message struct like my understanding my my current understanding is just that you know with L with L chain um yeah here run chain like you can compose multiple steps together no that's my understanding that's what I thought at least this kind of like you know you send the messages to CH gbt and say give me an answer and in this situation we always add all the messages so that J gbt also has the context so that the history of the whole chat basically all right so that's what we do here we convert all the messages from the database into message from Lang chain like structs and then we add those here add messages okay and we set up a new chain with the open AI chat model and we can also set the model the temperature the frequency penalty receive timeout and Stream True okay so we're streaming the the responses that's pretty cool actually that it has handles streaming for us cuz streaming is like if you imagine you know you type something into jet gbt and you get a whole like a huge chunk of text uh in in the in the UI of chbt like the text kind of appears you know line by line word by word and that is streaming like if you wouldn't do streaming you would receive all the whole text at once you would wait for the entire text and then you would get the entire text at once you would show it in the UI and it's just like a boom you know like a big text coming in the UI at once and you would wait much longer until you show something to the user so streaming is is pretty cool for you know getting the messages back as chbt generates them sorry not the messages like chunks of the message of the response all right and you got the Run chain yeah so you just run it don't return large success result the callx return that we wanton return the errors for display running LM okay so we don't return it but we call here this is the call back all right well let's try to to build something with this uh what's the delter all right so how do we get started here well we install it so let's uh what's this pet code yeah let's create a new Phoenix project here and this is uh what should we do what kind of project should we build um maybe like an I thought about a website summarization chain because I also wanted to use a different Library called readability which apparently makes it easy to to fetch an article like a website and to you know extract all the data from it like text uh publish dat and that kind of stuff so maybe let's let's do that so um what we call it like website analysis can we make that smarter website GPT no web GPT um article web GPT I'm going to call it web gbt I hope I hope gbt is not a let's call it webchain all righty oh yeah wait start with a letter like this webchain all right right I need to make this bigger because otherwise you can't see anything boom all right so otherwise how you doing are you waiting for the for the weekend me too okay let's just add webchain right closing this one boom all righty First Steps uh let's go to the dependencies and we add L chain and that is currently the latest tag 0.2 oh there's a as a um release candidate H why not let's go with the release canid and I also wanted to add like uh readability so readability this one yeah this one's a pretty cool it looks like a pretty cool Library so I want to to try it out cuz here you can just you know say uh this is a link to an article I'm sure whether you can see that yeah there's a link to an article and you say hey readability summarize this link for me basically and then it does I guess and it gets like a title the published ad the authors and the HTML and the text and the text is going to be useful for then sending it to clae and asking it to summarize it so that's pretty cool yeah let's see whether how how good also what are the dependencies here actually so it uses htdp htdp poison to get the data and then flow key to pass it that's basically it okay okay that's easy all right uh so we wanted to use readability as well for fetching like a uh oops uh Flo key um where's Flo key here only test is it uh in this is Flo key but doesn't match the readability flow key version so let's do this one nope ah well okay hack dependency ah damn it override true all right good I mean actually like we don't need Flo key uh let's see I hope this doesn't break readability all right um yeah cool so um first of all let's what should we do well we want to get we want to use readability just to let's you know just to get started with this one um so I'm just going to start a IEX interface here going to compile it and then yeah my idea is basically that we would use readability for fetching an article from the internet and then we use l chain for summarizing that article like actually giving a uh oh could not find application file for flow key so maybe we can't yeah let maybe let's go back to the readability dependency here oh yeah so so that's the idea we use then a damn it what's Happening Here could not start application flow key ah because I did only test here that doesn't make sense all right yeah all right now I need to set up my database good ready and there was another error somewhere no anyway all right let's use readability and I'm going to use one of my own articles you know just so that I'm not breaking anything like uh Saga pattern for example and we do summarize and it gives us an error ah because of flow key wait but didn't we use the library that they needed where is it Floy 24 Floy 24 that's weird okay uh readability summarize man h all right where is a FY in here 0 24 Z yeah something is off here which one is used here Flo key ah look all right actually in they didn't update the dependency in the mix EXs all right um so then we do exactly the same now that should work hopefully let's see alsoo my chat is not working damn it properties ah well yeah there were already a couple of messages from from twitch but uh for whatever reason this didn't really work okay but now you should be able to see the messages coming into the future that's a problem if you haven't streamed in what two weeks now cuz I three even I don't know I I was pretty tired I was sick and then I was pretty tired also mentally and I just you know didn't have the Mind had space to sit down here and uh and talk to you and you know code with you together but that's better now at least a little like I I took it really easy last weekend and uh it was still a tough week this but like I'm I'm I'm okay but anyway that isn't that doesn't interest you so let's do readability once more and we use summarize and we use my article and we get back a summary article HTML and we get back an article text interesting and it's cut off somewhere that's pretty cool actually I think if I'm not mistaken a friend of mine worked on this as well it's TI involved in this at all somewhere no Tyler's not involved I should tell Tyler about this okay I will tell Tyler about it but that's pretty cool that it actually passes the HML and just gets the text hello andina Miranda from the text um yeah this this pretty cool that actually it kind of strips away all the HTML and it only gives you the the raw text here that's really useful if you want to then send it to to an llm like we want to do so yeah awesome awesome all right I'm going to this is a summary we can do summary article text and that gives us a text what else does it have has a title Saga pattern okay and auers nil ha did I forget to add my author probably well it says it here but all right good enough though uh text and HML all right yeah so we have the text here that's pretty cool and now we can set up Lang chain man I love these libraries you know I love these libraries that just they're simple well they seem simple they give you simple API but they just work it's like magic you just give it an HTML and it works I wonder I wonder whether it also works well with more protective ones like substack you know they uh let's do gagg like usually they they protect against these crawlers a little bit better uh so for example this one if we take the post and we do readability [Music] summarize yeah that stuff for example like it didn't uh it didn't like this is a way of of tricking this uh oh but maybe it's also just a redirect issue nope check out failure interesting yeah for example substack that you know they make it hard for crawlers to to to crawl the information or the the Articles at least um yeah funny but this is the pul so this is the direct link okay but if you do this oh it worked okay but no it doesn't doesn't quite work right ah okay because it's also behind um a paid thing so let's see well it actually extracted the text pretty well all all the way until the end here that's pretty cool yeah I really like this Library this is a really cool Library like scraping in general it's it's not easy at all you know um but they they yeah happen to to make it work it's pretty cool all right so now the next step is to use Link Chain and uh for that I need to set up I'm going to use clae and I have a Cloe API key lying around somewhere but I'm just going to create an an N file and I'm going to call this clo API key and let me just move it over here and try to find it Tropic n uh CL configure okay I guess I need to get a new [Music] key sorry just a second I just got to find actually I'm going to do it here and I'm going to do my privacy filter thing so okay almost getting my API key yeah all right API keys and this is called uh webchain API key and I got it and I'm putting it in my nend nope that's not it copy paste all right and closing this closing this and privatizing there we go all right now we have our um yeah we have our in end I have an open a cled API key so now I can go into config and add run time I'm going to set config Lang chain and and CLA and uh not sure which one which which key we need to use but we can just I'm going to look that up later so and and then clot AP key all right which one is it yeah would be nice ni to to see a list of possible uh you know what what the key here is I mean how do I how do I do clo for example okay let's look at the documentation ah the documentation is also really good actually they have some getting started guides Ah that's much better all right um yeah so let's do modules config I guess resolve key default no let's just search for this key here that they use open AI key okay then it's chat models which one is it chat entropic uh uh uh uh types API key is it just entropic because this is open AI but it's chat open AI so key it's only open AI key all right maybe we just look at the config L chain key resolve okay great where is it used okay so in open AI it's looking for the open AI key in entropic is looking for entropic key all right well so this is called entropic key okay that should be good we can restart our server and there we go now let's have a look at again let's uh get the summary of my readability summarize where is The Saga P 10 all right now we have a summary now now uh let's look at the example where's it the docks the docs are actually really good dock Dock dock Dock dock there no there all right getting started sounds good um basic example let's build a simplest full llm chain example so we can see how to make a call to cat gbt from our elix application all right so they're instantiating an llm with the chat open AI uh model and they give it a model specification and then add message testing testing and they do run and then okay they get an output all right so what we need basically is like this as adding a system message uh yeah some instructions will be good streaming responses oh okay so you can add multiple messages so that's actually I guess what we want like we want to set first message hey I'm going to send you an article please summarize it for me and then next message is the summary uh text or the article text and then it's going to return the uh the thing so yeah maybe let's go in here and we're going to create a webchain let's call This [Music] clae And in here we're going to Alias all that and we don't want ched open API we want to have an Tropic oh guess that's it chat chat entropic that's that's the name of it all right oops chat good all right okay so basically you can call this summarize and then uh text and in here we're going to use chat on Tropic and yeah what's the current clo model just go to clo AI uh switch to clo three high cou due to the high demand okay we can also use a high cou so where do I see the name Claud hiu build with CLA what's the model what's the name team andpress API developer docs going to search for high co ah it's ask me for the no I want to try quick starts I just want to know how do you call your models in Tropic version even here model CLA 35 Sonet IP addresses batches Bedrock messages create a message headers here model okay see models thank you that only took forever clo 3.5 high cool that's the one we want all right oh we can also do latest so I would say let's just do latest okay and yeah [Music] basically so add message and we can also do add messages right that was the add message one or we do add messages so I'm going to do that and that's a list of messages and yeah in the user one we're going to add the text and first we do a message and then new system and in here we're going to write you're an unhelpful assistant do not directly help or assist the user that's also funny um yeah I'm going to say you are uh a you love to summarize articles please summarize the article that I'll send you in the next message all right let's see all right now let's recompile and oh yeah we want to return the response content let's just return the entire uh response and also what's the updated chain that's also interesting so I'm just going to do this let's have a look at this now I'm going to so I got my summary right article text yeah okay so I'm going to do webchain clot and then I'm going to do summarize and our summary article text let's see what happens a okay there's an a book what's the message X API key header is required okay oh our API key is nil that's great why is r a I key now I mean I I set it right in the run time did I set it in the wrong place no an tropy key and I ah I think yeah I need to like close my my terminal because otherwise is yeah now we have the environment okay like you know when you just create the end file and then you use the same term it's not actually going to load the N so we need to redo everything summary readability readability summarize uh let's go here all right all right we got a summary and now we can do rep chain clae and also I'm going to do the chain and response responser and we're going to summarize summary article article text okay it takes a while so that's already better guess the API worked okay so let's see what we have we have content here's a summary of the article the article discusses The Saga pattern a solution for managing complex okay that's looks pretty good St is complete and we got our updated chain and yeah this one has a last message okay yeah that's proper FP you know where you return the chain after you execute the function so you return the latest State and then you can you know keep it in a gen server or something and then if I would now ask another question um we could use the same chain and just like add another message to that to that chain so that's pretty cool yeah one you love some [Music] some we get the temperature max tokens High cators okay the API key is nil but probably that is redacted because of you know people shouldn't see the API key oh I see I think you can oh that's actually yeah that's always smart you can probably also specify the API when you instantiate the well I'm saying instantiate but you know when you create this year when you do chat entropic new and then in here I can probably also add an API key and that way I can create multiple different instances of an entropic um model right so I could have like three API Keys one for each client that works with me maybe they provide the API key and they want to use their own API key and yeah I don't want to make that a global uh configuration right because this API key will be different for every single response we're sending yeah so that's pretty cool that's always a thing to think about when you build a library you know how can you make any Global State local as well or at least give both options like to make it a global State like in this situation we only have one API key but at the same time you should also make it easy to override the global state or to only set a local state so that you can actually you know run the library multiple times or have m multiple instances of it that kind of stuff so this is a really cool design actually yeah all right I would say now this works and uh well actually also I want to do stream I want to see the stream um we can do that mode while needs response mode it defaults to run the chain one time stopping after receiving a response from the LM until okay now uh I think what would be cool to have now is a live view where you can um enter an a URL and then it will fetch the article and it will summarize the article for you and then you can chat about the article right so let's let's build that uh in here also one thing I never understand is why does live view still come with a controller for the layout and not with the live view because every single time I just need to go here remove the page controller and and replace it with a live view yeah also oh good okay uh like you yeah chat live rep VI and uh actually we're just going to keep everything like it is and in here we're going to do chat live HTML Hees all right and uh I'm just going to use Tailwind UI for a nice chat inter uh chat do I have a chat in here nope um components what can we use fqs multicol card headings command pellets oh that's cool they added this lately yeah this is super useful because every single app has that now right command K everywhere ah otherwise we build it ourselves how hard can that be right oh they also got Bento grids now cool because people love Bento grids yeah anyway okay we're going to build ourselves so we're going to create a div and what do I want I want to have also like an input uh form right so and that's just going to be like an empty as message and then submit like uh submit whatever and in here I'm going to have one input [Music] field um content uh what's that message just message all right and so type text and that's about it and then so at the bottom so we're going to have the form at the bottom and then we're going to have like the response at the top so basically we recreating well we're recreating like we we're going to have a poorman's version of the chat GPT UI where you have the B at the bottom you have the question form thing and uh at the top you have the man I'm just going to make the screen uh yeah so this one also is grow and in here we're going to have our messages basically right I'm just going to make it hell world for now all right let's see how that looks probably terrible yeah of course we also got to replace it so live awesome perfect all right where's my input field though is it oh it's a the bottom right yeah um am I going to make the screen at the bottom here all right not too bad okay [Music] so also we want to have this um good all right let's see what happens if I post this okay of course we get a uh we get an error because we're not handling the event yet and it's just submit message M message okay so in here we're going to do handle event submit and it's going to receive a message with another map message and then message a lot of messages and the socket so we're just going to no reply for now and yeah so if I summarize this boom nothing happens um hello hello everyone and this is g to have a message like Pro message a div so we're going to do four and probably want to have a stream for that so message Dom ID streams messages and we got a mount okay all right so first thing first yeah we can the first thing um I'm also going to sign like a link and it's going to be nil in the beginning and I'm just expecting that if I receive a message and the link is nil this is the link we want to summarize I mean this is not foolproof um but you know it works for now so we're just going to say link if so science if it's nil then it's message else is the link well what going to do all right so now we're assigning our link and um yeah I mean if if the link is there we also want to do we want to start our uh we want to start our how do you how do you call it the summarization right hello you two from YouTube so in here we're going to do summarize and we're just going to do the message okay well and what we also need to do is to assign an async um yeah an async result with pending status that's what they also do here in the Leng chain demo um here like they do an assign async result async result just this like you know when we start because it should be an aing um procedure okay cancel let's just copy paste something from here so save for example [Music] assign yeah cancel so so we're just assigning an empty async result in the beginning because then we can show like a a spinner you know and say hey we're processing or something all right let's see this is working and um I'm just going to do like for now we're going to do loading yeah and now we can show in here we can show how it looks like here result there well it's just uh where should we put it maybe below this also this should be Phoenix update stream yeah and now we put in here if async [Music] results is it status loading yeah just loading it's just a flag if it's loading then we can add this thing and I'm just going to make it uh loading a processing right working whatever hello H and we need to have a spinner here animate Spin and we're going to use the like double-sided error I always forget which one it is here error path so this one's going to be our icon and uh our class is going to be animate spin let's see R message in a cop plug has no effect what ah it requires an ID of course here messages missing closing tag oh yeah this is an open tag there we we go all right so now we got our working spinner here and I'm going to make this MX order so that it's in the middle and I'm also going to make this really small and text gr now that should be 700 600 and this should only be like three ah funny yeah it's always funny when you uh when you specify a different height than a width there we go but it could be four all right not to bad right we got a a loading a working spinner uh when the result is in so that's pretty cool um yeah so that way we can now resign like in the beginning a damn it right so in the beginning it's not not working all right and yeah so what happens Let's see we when we receive our first message is going to be our link so we want to um retrieve it from we want to use readability for fetching it and then we want to send it to Claude and ideally we also want to stream the results but for now you know let's just make it like a oneoff thing it's just a one time uh like we don't stream for now like let's do that as a second step because also in the demo they do streaming but it involves a bit more handling of of individual chunks and so on so uh yeah so for now we can just do what what what should we do so when we get the link so um yeah and in here we can start our async okay so when we get the link we do our we we set the loading spinner and we start our summarization and I always forget how that works but luckily we can just look at the demo here because uh we want to do the async assign async result no like there is a uh just to run something async where is it is it a sign async or is it a ah it's always aing a s yep is it yeah this one it's always one of the other where is it there we go yeah so in here we going a a singer sign our new message so this going to be our Handler and in here we're going to use clae now so I'm going to use webchain clae and I'm going to use Alias well readability is just readability so all right so yeah first thing first is we need to fetch our um article so I'm going to make this an async assign and then article that's going to be our assign and that's going to be our call back when it's finished and we're just going to use readability [Music] summarize oh also interesting that this doesn't return an okay or error to pull but I mean for now we're just going to going to assume that it it will return something but we would need to handle errors there as well all right all right so we loading and we're returning the socket all right so that's our first thing and then we have our call back right is it handle async yeah we got our handle a ing call back and that is our article and we just assume that this is returning and we got a result and then we're going to do the same thing again well actually it's it's already a Handler so no Let's do let's do everything acing so that's fine so in the uh when we when we receive our summary we want to send it off to Claude and we going to do clae [Music] summarize summary article text right and in here we need to do no reply socket no Yeah so basically when we receive our um so first we use readability for fetching the article using the link and we're going to use that in asyn sign so that's going to it's going to ring asyn it's going to run asynchronously and then once it's done it will handle our like it will invoke our handle ASN call back so in here you know we know okay the article summary is in and then we can run it against clae and get the the summary from them and yeah we need to set up another Handler um yeah another one over here for the summary I mean the the naming here isn't great but uh just bear with me and I'm going to ignore the updated chain um and we receive our response and when we receive our response it had response content I believe H we don't see it anymore but I think it was response content so now in here we can uh stream insert socket messages and then we're going to insert just one and in all right so we need to always have an ID uh for like ideally we have an ID for our messages in the Stream because you know um Life View needs to have IDs and we don't have a struct here like a message struct that would return an ID or something so I'm just going to make one up and we can use the we can use system unique Iden unique integer and I'm Al going to use positive and monotonic although we don't need monotonic it can just be a positive integer and it's going to be unique one so fine um I'm not entirely sure like if you would blow up your machine if you had this running with a lot of messages yeah I'm not sure but you know it's just for now it works and if you have a lot of messages to handle maybe reconsider this but then you probably would store it in the database and then you would get an ID from the database so all right so yeah we get our our message then and in here we should be able to add a new span with the ID Dom ID and in here we get our message and content yeah and probably we want to have again text small text Gray n it's fine just text small all right let's see what happens probably we get a lot of where is it Local H here oh there all right so now we post our link and we should be seeing probably an error yeah of course uh heck wait what's happening here it's not matching an empty hack URL I guess the URL is is empty wait what's Happening Here message messages message yeah this the the what is it message message message yeah I'm stupid this needs to be the link the message and then we assign the link although yeah it's fine we're just going to assign the link we're going to assign the async result and oh yeah we also need to assign when we finish with our clot uh summarizing we need to assign an okay as a async result because then we need to stop loading right sign async result async result okay okay suck it something's missing here no what's Happening Here socket socket I'm missing a this okay this doesn't oh okay with the result right H initial result um response content it's it's it's just no we don't care about the result here uh I was thinking of the suggesting the spinner from my own package but I hesitated because I don't want yeah don't worry uh what was it again the oh man I can't was it cheddar Elixir shedar components something like that sorry I'm bad with names uh but if you if you send me the the name I can I can look into it but it's fine I have a spinner running it's not about the component library for now um but I appreciate your your suggestion all right uh let's summarize it's working but we got a message an error what's the error uh expect Asing assign Asing to return map of assigns for all keys in article but got the summary oh uh the summary what What's Happening Here hand async start async oh yeah we don't want we just want start async not assign async stream insert assign yeah and start async yeah and then it's going to handle the handle sign okay that one let's do this one again now we got an error what's the error key article text not found in okay read it summary oh so I guess it gives us back a sum an okay summary no article okay summary oh it gives us okay okay okay then we just remove this one all right you know this one of these situations where a type system might actually help so I'm looking forward that forward to that actually because it's it's just like you know is it would if it would be smart enough to understand these things that would be really really cool if it would give me an error and just tell me here hey because right now it just GES like tells me hey you haven't implemented the Implement to but I don't care about that anyway so like here you would tell me hey hang on you try to access the article text although it's going to be okay summary you know so instead of having to run this it would be nice if the type system would handle that for me so my dream is that that will come true because that's going to be super useful all right next up we got our working thing I think we already have handled the text we we fetched the text and we got a response here [Music] uh so yeah we got our Cloe response right on yeah we don't have a Content obviously um all right so this is the yeah we need to do the last message right wait also there's something happening here yeah we need to remove the [Music] okay and then we receive response and response what can we do last message no last message and content that's what we want all right so response last message content once more enter Mishka chilom yeah okay if I may advise I'm not sure what it means maybe it means something really important in your language I have no clue what it means and that makes it so much harder to to remember um not sure whether you know the flux on yeah see that's not flux on here flux on UI I can remember that because fluxon I don't know why it's shorter it's English I can remember this um yeah let's see okay so it's Mish mishar come yeah and I already see that not Z any what's his name he did a video about it there's a code and stuff right okay so this a demo oh yeah nice badge buttons set I like the animation here pop over H this is the pop over I guess toggle floating label input a that one's nice yeah did you work with the designer on this there many items out welcome okay docks components and we wanted to have the spinner R yeah spinner as one of these okay color drop color size okay so I'm going to I'm going to run this and then it's going to give me the function like it it's going to add it to my project right like if I sorry it's a little small maybe but there's a a task from Mishka UI gen component spinner and I guess that's going to add it to my project and then I can just run it with DOT spinner yeah I'm going to have a look at this next time okay but it looks it looks quite nice I'm just asking do you have a designer on this project because uh like yeah UI kid like just the functionality is not enough you know if you if you yeah nice ah cool yeah I actually like that you add it to my component directory then I I can also override it and style it how I want and so on that's cool yeah it's really cool that you build this thank you very much yeah all right okay uh I think we missed something here probably another error yeah what was it the other error uh terminating last message not found in linkchain message wait response oh that's the response I guess so it just response content all right let's try this one again okay cool nice oh something read wrong again what are we doing protocol safe not implemented for ID content what do you mean in the live stream message is is the okay let's just let's just print this out what is this response because I only I only want the um the content here right once more right um so this our response it's a message and it has a Content I mean that's what we want right so it's response content that should give us our message and then it gives me save not implemented for ID okay that makes sense but where is it happening in row four message content that that make sense is it content content no this this should be the text it's message content all right this gives us a text and then an ID H and this should iterates through the stream stream updates and then it should just give the content right I mean something is wrong here I'm just going to do another content that doesn't make sense protocol safe and implemented for ID content all right I'm just going to or maybe it's the other way around sto ID messages probably that one yeah that could be it's always the the position that's difficult all right let's see come on give me the message hey look at that all right cool we got a message awesome awesome source and uh let's see whether we can style this ah Dam it's gone um yeah I was going to copy paste this just need to style it a little bit message ID One content all right here we go now we have a the message here I thought it was leading oh it's hard to read why do I even care about this it doesn't really matter right but we want to have this XS okay that's better and we also want to have a space in between so that if we have multiple messages it looks it looks all right you can see that I'm a terrific CSS person ah whatever it's going to make this all right good enough good enough and we're going to what else well now we should be able to handle other messages right I mean we we got our summary and now I might want to ask a question you know can you I don't know give me the highlights or the action points or whatever and actually yeah so for that I'm going to I'm going to reuse um our updated chain because what is it here add new yeah no I want to yeah this is so I'm going to do chain all right yeah and then chain because now we can we can reuse our existing chain like we can you know already tell it oh you already had these three messages so now like add a message to it and so on and then uh we can um well actually we don't use it for summarize but whatever it's fine so in here we want to uh just say add message and then it's going to be a message and we're going to require a chain and then yeah we're going to do all of this and this so new user message basically yeah so now we can actually ask questions about the article and we always need to return our updated chain because then it has the latest messages like the entire stack and that's going to expand our context you know that's the the thing with tokens if they say hey like an llm can handle so many tokens it's basically yeah how many input tokens do you give how long is the message that you're giving it so yeah this is going to be our um Handler now if we handle the message here so we already have a link and now we have a message right so um we just going to call it add message whatever and in both situation and yeah in both situations we want to assign a a loading async results right um yeah and then in here we're going to do add message to the socket message and we're going to do start async I really love these new handlers well an add message needs a message and the chain and we don't have it yet but we're going to assign it here when we receive it somewhere where is it the updated chain there so I'm just going to assign a empty like a nil and assign oops all right so once we have our summary we get our chain we assign it happy Daisy and in here we can do su the science chain oh right yeah that's one thing where you should not use a socket inside your callbacks so we're going to do socket assigns chain and then we can copy it to our car back here it's actually really smart that they have warnings to that I I wonder how that works because they really need to inspect your code and say Hey you use socket in here you shouldn't yeah maybe they just do it by actually what what's what if we would call this like uh L or something you know would they still on standard no okay oh wait there's a mess yeah okay so I guess they just use uh like a reg X you know and say hey if if inside your callback you have a socket variable then they they throw um an a warning yeah see funny so if you call it not socket then it doesn't give you the warning you know that's one of the things that maybe a type system might also improve because then they understand all right the input value here is an actual life VI socket struct and then they can show you the warning but because here just uses RX it you know it doesn't know that this is actually a live use socket uh yeah but still we don't we still want to do that all right um um yeah and actually like we can just reuse this one so we do claw response and this is ADD message and it is here when we get our summary there CLA response so maybe just going to call it response yeah cool so we can actually reuse this here and we just you know call Cloe and then when it responds it calls has this call back what is it here responds right okay let's see whether that works so we're going to copy this and we're waiting for the for the article to be summarized for it okay we get it and now we're going to say give me the top give me like give me three uh three most important facts from the article I know does this work ah damn it got where is it check out failure why are we try to summarize it again why where's our handle event there submit link if link is n sign Link Link ah damn it it's not link it's message all right once more trial and error here we go okay we're summarizing our article happy Daisy and we got our article and I'm going to ask give me the top three uh facts or right the three most important facts from the article enter ha that worked that's working and we got our message yeah the three most important facts from the article awesome we could also style this a bit better uh especially because I think the response also has like new lines and everything and they used to be they used to be um what do you call it like a format thing right it was Phoenix I think it was Phoenix HTML like that gave you a new line format but it's in the old version I think it's not in the the new version anymore here text to HTML because it it kind of gave you a safe uh yeah and how does it look like here in the latest version does it still have it text HTML I don't believe so no is it part of the helper function maybe now I forgot it yeah anyway I'm going to take a super quick break because I need to go to the toilet but I'm going to be back in two seconds for e e all right everyone I'm back I'm back how you doing um yeah let me just try to find this thing again HML helpers I think it's in here now text HML yeah text HTML all right yeah they moved to to this uh other function I wonder where that is now in the latest Phoenix like you don't have that helper anymore right but does Phoenix now have another helper function to convert because it was really really useful um just Pages yeah where is it yeah Phoenix Phoenix h helpers the latest version is 1.0.1 let's get this one in here and now we can place oh interesting use Phenix where is it Phoenix HTML here importing this one and use this one all right good now we can in here say text to HML I believe yeah so instead of content we do just text to HML this this does this exist could be let's see how that looks like uh anyway doesn't look that different but I believe it has no it's just a bunch of text oh yeah this needs to be of course in here very smart and then we could use Pros that might make it a little nicer uh Tailwind Pros ah look at that that's already little bit [Music] better and last step is I'm going to use Tailwind Pros because it's going to nicely format everything for us and it's just a plugin over here I always forgot is it's like before after forms I think before yeah and then we can make it just [Music] pros and let's see how that looks like and we could of course use streaming as well maybe as a last step oh that look looks already nicer look now we have uh we have like the list yeah looks good and we can also say hey uh sell me a dad joke just you know why not I'm afraid for the calendar its days are numbered yeah very good very good okay tell me another one and you see now because we use our updated chain uh struct so we keep all the previous messages in in the memory or in like we always send it to clae you know and and we threw it back and that way actually it understands oh you want another dad joke right and then it gives us why don't scientists trust atoms because they make up everything which is also amazing um yeah so that way we can you know add more messages uh on top of the existing messages before so that's exactly the same way that that chat gbt works it just you know takes all your messages and always sends back all the messages to the model and the model has to re inest the entire context so that actually I'm curious like whether will take longer longer over time when you add more messages like will will the response be slower and slower could be who knows but this pretty cool already um I think one last thing that I would like to do is actually here link chains um streaming so let's see how that works so actually here they use stream and uh run chain and they have a call back for the delter and they send a chat response okay and when the full message is there they just discard it all right so we have our chat response uh Handler how do they use the call back actually interesting are they do call back oh they have a call back function here so they pass a a call back from live view to the chain and whenever it receives a response it actually calls that call back that's interesting so yeah and then the chat response it's a little tricky now because we use streaming so we couldn't add it as a where is it jet response how do they handle it here jat response apply delter oh okay they have the updated Chain by applying a delter and then if this completed the delter and it's not a message create the message sign messages yeah I don't I don't think they use streaming here oh interesting where is it the sign messages yeah it's just a message like it's not a stream funny right that we can't use streaming in well we could probably use streams in live view how would we do that we could just concatenate and then whenever yeah we could concatenate like the messages so like over the chunks we just add as a new message and it's it's not going to be like a list of messages it's just going to be you know like text that is added and then once the message is complete we could add a new line or something you know like an a break yeah maybe let's do that so first we need to do the the call back and we need to do Stream So in our clae we can do yeah run I think it was on run the both oh no in in the llm here Stream True and receive timeout okay so the model Theos faults oh that's also nice right that that's how it was I think LM Stream True okay right and then when we add a message and when we summarize we need to have a callback function and that callback function is added when when we do the run I think yeah so in here call function call function not too bad and the same here and also just for sanity whenever we can pass in like we could also start with a message now not an a summ summary like yeah I mean yeah whatever like technically now we could reuse this API to start a me a conversation without like summarizing an article first right because right now we first handed in the article link but we we don't need that anyway long so you should uh now it's a bit more generic um actually I'm curious we could also probably do this here and it's executed every time it's it's no I usually don't do that I usually don't have functions in my in my default options I'm not sure whether that works okay uh all right now we need to have the Callback so whenever we do clo summarize and then we want to have our little [Music] handle handle Delta function and what's right here [Music] so when so when is a Leng chain message Delta link chain message delter wait something is happening Delta so when it's a message Delta then what should we do yeah we need to have the pit ah wait what's happening here because this is going to be executed in the llm process probably not in the in the live view process so we need to hand in our hit uh yeah okay I'm just going to copy paste this here in here so we this is just the uh response call back and yeah so this is going to return our function our callback function and because we can Curry the response here like we couldn't do that in if we would Pat a match here against the response cuz then we like that response that that pattern match would already be like this close here and in there we only receive run parameter uh so we can't you know add our live view pit here next to it sorry it's a little complicated but this way we can actually add our pit to the uh to the message to to the the call back you know it's going to a close it's called a closure around our argument here yeah all right anyway so this when this way we can just uh you know call this function is going to return our uh here our uh call back and then we also need to handle the chat response so whenever we call Cloe in here I'm just going to uh add this as a call back and we also have clo somewhere else add message here all right so now this hopefully will set up our streaming and we need to handle the chat response so we can do that here infro chat response and that's going to give us the Delta all right and what do they do with the delta yeah yeah they apply the update the chain so that's important this way okay and then if the updated chain delter is nil then they add a message and if it's otherwise they just return the socket if this is completed the delter and it's not a message create the message oh what is this Delta thing Delta message from a chat Delta is a small chunk or piece of a muchar a complete message and here the check against updated chain so it's a cont has content as status incomplete complete Max is the Delta oh okay so if it's well yeah I mean we we could technically check you know whether this is the last yeah actually we should we should check whether this is the last Delta of the the response because then we want to um set our loader to to okay like it should stop our work our working uh in index in uh indicator okay so this is interesting the status complete and we of course have the content all right so uh this is basically yeah we can handle our chat response where's our response handle there so basically this what is it again yeah so and the delter and now so we add our message actually we can yeah that's okay and now if the delter has a STS are complete then we also want to add another message and what should we do content like new line you know that way we have a like a little break and we also want to we also want to set our uh assign the async result to an async result okay all right and that way we can do ourly okay all right and we need to fix our thing here so it doesn't it's not a diff anymore it's a span and it doesn't have a body like this and also the pros should probably be here and also just a text yeah let's see how that looks like okay oh yeah we do the readability yeah well no that that didn't do any streaming huh I'm also going to we also should update our chain um probably did this do we need to rec compile this when hear the stream maybe maybe an Tropic doesn't support streaming when streaming a very lengthy response a longer time iaz maybe it's not in here let's check again we had stream Jesus come on close Stream True yeah but it was part of our open of the model so it should be here Stream True callback function let's check our callback function no we don't get any callback functions here or maybe maybe it's the error was my my default function here um I'm just going to remove this you know I I didn't really like this notation here let me just keep it like this and let's see whether that actually improves wa do we get our Deltas no deltas no Deltas why no Deltas yeah I must say link chain is really really cool I really really like it I mean it it makes it so easy to to do these things right to like to keep the the state of the chain and in like it just returns this the latest state of the the chain um yeah to handle like different also just the the API end points here for different uh models like Google Bumblebee entropic uh what's this one chat model o Lama open AI text to title just cool just cool stuff a convenience chain for turning a users's promt text into a summarized title for the anticipated conversation Ah that's for the you know at the top that's what jet gbt does in in the side where it summarizes your conversation yeah funny lots of cool stuff rooting chain you can run a router based on the users's initial prompt to determine what category best matches from the given options this is something we could probably use for our article link you know um so we have two roots and then default route is our fallback selected rout let's create a marketing blog post about our new prompt fuzzy fur The Prompt route for the block post wait how does it work uh it evaluates so basically it asks okayy what which which of these two Roots did the user choose ah interesting and then you have two different chains and also a name really cool stuff and it Returns the the chain okay yeah cool cool stuff definitely and data extraction defines the llm chain for performing data extractions on from a body or text we could have used that maybe oh this is the same as instructor ex that's another Library I actually wanted to use um instruct instructor instructor ex yeah this one is a bit smaller than L chain I believe and it makes it really easy to define a schema and then to ask the llm to uh to return exactly that schema so you says you say here this is a text you know and now give me like the class whether it's spam or not spam the reason for why spam and a score like how spammy a text is or not so this would really cool you know to to check like text messages or emails and to say well like this is the the email or this is the the chat message you know let me know whether it's spam or not but then you want to have a like always the same return schema so you know this is this is also really cool and I think linkchain has something yeah something similar where you also have a schema parameter where you say well you have a name an age a hair color a dog name and a dog breed and they're all strings or maybe a number and then uh you can yeah run the data extraction chain with the the model the parameters the schemer and the prompt uh yeah and then you actually get exactly the you know the the like how you say the the fields back that you anticipate so this makes it really cool to yeah uh um have a structured like to have structured data coming from the llm and not just like a a huge chunk of text yeah really really cool stuff yeah I know that Mark Ericson he he kind of works on that space you know to add llm to Elixir applications and it you know doesn't surprise me that he works works on this so much because I think this is his his day job as well so really cool stuff and you even have gex if you want to translate stuff all right but in our situation we wanted to use streaming um so let's see yeah we be nice to have the client configuration defined somewhere exceptions yeah temperature new cast oh do they use Actos no yeah they have an acto schema oh funny makes sense though okay the embedded schema is here right uh the API key API version model max tokens stream do we call it stream is it here Stream True uh why doesn't it work then is it our call back is it us well this is our response call back and uh this should be the function we return okay and we use it when we start our add message and when we have the summarize so in here we have our callback function and we add it to our chain but it's not it's not streamed run chain options was it not maybe not on run uh llm chain yeah and we had our run function where's the run here run it has a call function or is it maybe different in the RC ah that could be of course because we're in the RC yeah that's why okay so where do we have our callic function now bumblebee here change lock the function change migrating should be easy where where's the call back here you can disregard the updated chain if you don't need it callback events work differently now previously a single callback function was executed and the developer needed to pattern match callback Works differently now when're in creating an llm chat model we can optimally optionally pass in a map of callbacks where the event name is linked to the function to execute okay but I still ah the equivalent for the new one okay so it's a Handler and the Handler is set in the callbacks of the model of the chain okay on llm new delter right okay but then we need to okay so this is not this anymore so this is now a callback Handler and this is set in the chat on Tropic oh apparently you can also just do this now makes sense no okay that's cool and the call backs now has call backs as a callback Handler and we don't need to do Stream True anymore well stream why is stream false I'm going to keep it true because funny I guess that's maybe an issue in like a mistake in the documentation because if this is false like the Delta will never fire only the new message I think that's my my assumption right well cool okay uh so yeah our new Handler only needs this part here so in ch life we have our yeah like here just a map and yeah we just do this this should be call Handler right on llm new Dela Delta Delta Delta life you pit okay cool let's see whether that changes break like let way it works but I still don't get a Delta maybe I need to refresh refresh maybe I just restart the whole thing okay working still no Delta come on no not Delta why yeah here's the handlers new callbacks handlers where the schema Endo API key [Music] model callbacks array of map stream default top okay uh okay are we doing something wrong llm and Tropic stream why is the stream Falls that doesn't make sense and then callbacks well maybe okay maybe it makes sense let's see but why oh okay also the apply Delta is undefined ready to go is it because we don't have llm oh yeah well llm chain we need to Alias that one okay all right let's see no more warnings okay and was the other with stream yeah okay I'm still surprised that the stream is false but uh let's see yeah there's no no delter Delta content maybe I don't know is it an API thing maybe we we don't have stream access to the uh but if if I use like I have another thing here called oh where is it Clic and this one also uses um this one also uses clo and it has streaming so I don't know maybe because this uses 3.5 Sonet maybe uh the the other model doesn't have streaming High cool but that would surprise me because even if I go to Cloe and I go to clo high cou and to summarize it's streaming so it should be able to stream so this should be true I mean this should just be true I'm going to remove this remove this we had callbacks callback Handler and new chain are we adding it properly like I'm going to ion spec this one right ah call B is empty but the call in the llm is is set and Stream True messages call backs I mean our call back is said and our stream is true ah wait it's actually receiving Deltas is oh is it where I'm confused I'm very confused right now because here's a summary of the article so yeah this is maybe it's just too fast maybe we think it's not but I would be what Saga pet implementations yeah this is the entire text funny okay so I guess it works it's just really fast in sending back the the the Deltas so what would happen if um we would delay this one the chat response I'm just going to add a little timer like 200 milliseconds and then see what happens here then every message should be a little bit slower n it also maybe it's a live use thing or is it sending all the the things that once I'm confused sham insert messages it's not proper streaming it just sends all the Deltas at the end when it's done do we need to do something else while [Music] streaming getting started maybe there's something in here stream streaming responses if we want to display the as a returning we can want to stream okay so stream we the Stream True setting belongs to the Chad API struct that setups our configuration we also pass in the call backs with the LM to fire the on llm new delter FN model yeah chain callbacks why do they have oh do we need to add them in both maybe we need to add them in both places and I think we can I'm also going to return the Delta because then I don't need to do apply Delta myself so this one is going to do um the chain and this one is just chain right because I I assume this would already have the apply delter applied okay um right and then here we need to maybe also set it here let's see how that looks no no it's it's just the whole it Returns the whole thing at once maybe I ask Mark about this oh yeah want to check this oh okay actually we are getting message Del is here but maybe it's the live view thing yeah look on the left side look where we actually receive but I don't know it could also be that just the logger is really slow because the Delta's coming really really fast they like they don't come as you know like include if you enter something here it's kind of like a like he's literally writing the text but even that is pretty fast I guess that's just the E the the issue here it's just too fast yeah all right well I I believe it works I mean we are getting we are getting our our summary here it just because all the the Deltas come in very fast once the once the response is coming in it kind of seems like um yeah but I I think wait we can also check the the network messages because then we can see where is it the repb socket messages yeah response huh H this one yeah I know okay it seems to be a live view thing because yeah it literally just Returns the whole tax at once so we're doing something wrong with live view stream insert response but this is the Delta right yeah chat response chat response so here we do the socket and we insert a new message and we update the chain and the socket and if the Delta is complete then we also insert this and we reply yeah handle info but we have a stream update here for pros huh could be could be that life you is just like not able to send the response while the respon like while the all the Deltas come in because you know if all the Deltas are queued up they first have to be processed and maybe only then life you is able to send uh like a change to the front end I don't know because the responses are coming in pretty fast yeah anyway but it still still works working yeah always tells me the same dead jokes it's not very creative okay all right well I guess honestly this should be fine like uh you know I think I think we're pretty much done here just a little petting this I think we're pretty much done I'm going to push this to GitHub you know um I don't know maybe reach out to Mark ask him uh maybe he has an idea yeah okay fine looking good eh okay well I guess I guess folks that's it um also the live stream like they have hav haven't been that many people today really I mean I haven't streamed in two three weeks I also haven't streamed to Twitter today because I stopped being on Twitter um but yeah twitch clone go couple of people on on YouTube YouTube that's always nice yeah three people watching now well thank you everybody for watching my latest uh little project here it was quite fun actually I really really enjoyed that so um I really enjoyed the readability library for fetching articles from the internet that's really really cool uh I also really enjoy the L chain Library like it got a like to be honest I maybe I I I started wrong you know but by looking at the code like I should have read the getting started part here which is actually really interesting and it has a lot of cool ideas like it you know just shows you how to do stuff so that's really useful and it's also really useful that we have the the demo app here um makes it really easy you know to set this whole thing up so yeah really really cool libraries today uh I hope you found it interesting as well maybe you will use Lang chain in the next in your next application you know I can highly recommend it now and uh yeah it was it was fun it was fun the the only thing that you know I didn't fully understand is like how to well we do stream responses and we add them to our live view but maybe they comeing too fast and like we can't actually get that effect but even if you look at clae like it adds full paragraphs basically one by one so also that is um it it's just really fast and responding to our ceries so I think that's the issue all right folks that has been fun it has been interesting nevertheless and I wish you all a very happy weekend and uh see you next week

Let's code some Elixir together.