Webhook Testing Without the Headache: A Developer's Sanity-Saving Tutorial

[Music] hi and welcome to programing Percy have web hooks ever given you a headache they have for me at least when we're going to test them in this tutorial I hope I will help you solve these headaches by showing you how I tackle testing web hooks we will learn how to test web hooks effectively and we will also learn how we can receive web hooks directly into our local host uh so we can test the application while we are developing all without needing to host our application on a public domain so we will also cover how we can take those web Hooks and turn them into marcks using wire Mark so why will we look at both well usually I use nro to receive the web hooks into my local host and then while I have inspected the payloads and learned how they look I turned them into mocks using wire mock so I can have a completely offline environment I can test how much I want without needing to trigger the web hook on the third party software or anything I can have a completely offline works whenever I want solution Also let's not forget that some Services actually done bill you for using the web hooks so we're just going to avoid it all together while developing so in this tutorial we will learn how we can take web hooks from third party Services into your local host and we will also learn how we can mock web hooks for local and offline development so let's begin really easy let's begin by exploring what web hooks are so basically web hooks web hooks is you have your service here and we have a third party service here in the form of GitHub a web hook could be us telling hey GitHub you know what whenever whenever something happens let me know on and let's say HTTP myip.com so let's imagine that we could tell GitHub that we want a notification whenever a pipeline fails so we would give this URL leading to a API of our to GitHub and then whenever GitHub figures that something has happened and it should let us know it will use that URL and get back to our server so really that's everything and that's how it works you tell GitHub that this is our URL whenever something happens get back to us on that URL and they will send us a web hook on that URL it's kind of like giving your friend your phone number and the friend will call you whenever they have news think of it like that the web hook instead of a number is a URL and instead of a friend it's GitHub but we all know GitHub is the best friend we have so I mean right so as you can imagine and as we discussed a little bit in intro this can be problematic because if I'm developing my application this won't work because I won't have an application hosted on a URL or we can set that up of course but it's a little bit much what we will do instead is we will give GitHub a madeup URL which we will receive from a service called enrock and let's say enro tells us you are now the temporary owner of ABC 123 and and then we can tunnel that traffic from ABC through enrock and when enrock receives traffic on ABC it can tunnel that traffic into our Local Host so this is the setup that we're going for we and you might wonder how this works there's a lot of documentation on Ang Ro's website which I recommend you to look into but basically we will start up a enr service on our computer which will be acting as a agent so it can route the traffic to us using that agent I know all of this can sound like a lot of work but don't worry it's actually pretty easy so before we begin I really recommend you to visit Ang rock.com and set up an account it is free and it's really easy to set up an account I use my uh Google account you can use your GitHub account and where and whatever so it's really really easy to sign up when you sign up you will see a dashboard and inside that dashboard there will be a authenticating a authentication token do not share your authentication token with anyone let's visit enr.com and ooh I'm a ghost so I'm going to go ahead and log in and once you are logged in you will see a dashboard like this visit setup and installation and select your operating system and it will show you how to install I'm using Linux so I will copy the commands that they have given me and basically we're going to use we get I'm going to see if I can zoom in and I'm going to use vget to get a tar tar file which is their agent so we have download it their agent and I'm going to unar it and we can see Ang Rock uh also when we unar it we are placing it in SL user SL lo/ bin so the angro agent should be available for us so let's go ahead and type angro and you should see a little help section with information which is great now before you begin working you will actually need to add something called a authentication token so that enro can actually reach your computer using this token and you should never share that token never share that token with anyone because then they can access your tunnels so what we do is angro config add off token and then your authentication token should have been provided or the command you need to write is provided in the enrock dashboard so type in your token here and once you go ahead and do that so once we have that we can go ahead and type angro we're going to host a HTTP API since web hooks are using HTTP and we're going to host it on a application that is available on my Local Host 8080 currently I have no application on that Port but we can start the agent to see what's going on so go ahead and run that you should see a bunch of information showing up one of those those informations is the generated uh URL that we can use for the web hook so this is great whenever this URL receives traffic it will get tunneled into my local host and 8080 and this is really everything we have to do to start using web hooks on our Local Host it is that easy to test this we kind of need to have something triggering web hooks to us and in this tutorial for Simplicity I'm going to use GitHub so you can go ahead and visit GitHub I have created a new repository for the purpose of really testing this so what you can do is go into any GitHub repository that you own and then go into settings inside settings here in the drop down you will see a web hook button so select that and you can see a button up here that we can add web hooks now you don't have to use GitHub to follow along this tutorial you can use whatever service you like that has G that has web hooks it's almost always exactly the same to set up so in here the payload URL is going to be the URL that was generated for us by the enrock agent so go back to enrock select the URL that's printed in the terminal copy the URL go back to the payload URL and copy paste it and we can use Json as the content type and the secret the secret can be used to validate the web hooks so you won't really know when you receive traffic on your API you won't really know where the traffic comes from you can use the host Etc but as a little bit of extra protection we can pay uh we can set a secret here which will be used to generate a hmac hash which we then in our application can use to look up if it's really from the correct source so you can use I'm just going to use test my app we won't be doing the validation in this tutorial but it's uh very easy in the GitHub docs how you do it there's even some examples so once that's done let's just go down to the bottom and and add the web hook and we can see a little bit of status if we visit it you can go here and you can see recent deliveries and we can see right now it has tried sending a ping but it failed what really happened is it tried sending a ping but we don't have an application running on my Local Host so there's no response so it's going to fail so let's move on and create a simple that really accepts the web hook and let's run it on my local computer and see how this just works out so I'm going to open up a new terminal and I'm going to create a new application and this application will be listening on Port 880 and it will accept a a simple HTTP call on that port and just print the payload because that's what we want to do for now we will receive the web hook we will print it and we're done so let's go ahead and do go mod in it programming pury SL webhook testing and I'm going to create and I'm going to create a main. go so inside our file we're going to create a super simple super simple HTTP server so let's create a main function in our main function we will simply start a HTTP listen and serve uh on Port 80 so let's go ahead and do slog doino listening on Port 880 let's do a code action import slog then down below we're going to do HTTP listen and serve and we're going to listen on port 8080 and leave the second parameter nil so basically so basically serve app on 880 880 and important to use the same port as enro agents is configured to use if we're not using the same port the tunnel won't work and let's just make if error not equals nil because I don't want that angry yellow text to jel at me and let's just Panic if something goes wrong oh my bad and if error isn't nil let's just Panic right we're listening on Port 880 but we're not really we don't have a Handler so let's add a super simple handle function and we're going to listen under Ro root path and then we're going to add our HTTP Handler our HTTP Handler is going to be like really simple so let's just do a defer body. close we're going to close the body and we're going to read the body so whenever we receive a request we will simply read the payload let's read the payload and then if we don't have an error let's just Panic if we have and then we will simply print the payload as a string just so we can review it let's do payload and I'm going to do string data and we need to import IO so let's import that so this is a really simple go program it will listen on Port 880 it will set up a HTTP Handler that will just print the payload that we receive what we can do with this is we can close that file and we can do go around main.go and um yeah let's just run it so we're listening on Port 880 now we need to re-trigger the web Hook from the third party application and this is also kind of a reason why we will be looking into wire Mark later because we don't want to do this we just want to programmatically have everything run when we're testing but for now let's visit GitHub and let's go inside this uh recent deliveries and let's have it resend the Ping so we can actually open up here and we can do redeliver and yes yes we want to redeliver to the same URL and you can see it's actually a check mark this time saying it was a success let's go back and oh my God we can see a lot of data so this is great our web Hook is actually working it's going from GitHub into the local host on my computer through the enrock tunnel so that's actually pretty sweet so we can see and review the payloads and basically this this allows us to test our web hooks really easy and wow my green screen turns me into a ghost so that that looks pretty cool so this is great we have angro set up we can receive payloads but what if GitHub goes down then we can no longer test our application we're screwed webook doesn't actually charge us any money but some services do so that's not nice and we had to actually visit GitHub and fire the web hook so that's kind of that's not how we want to do it and gladly we can solve this using Wick and if you're unfamiliar with Myck a big shout out to them it's a fantastic tool to mock data from rest grbc graphql web hooks basically anything you need to mock you can us usually do that using wire Mark and wire Mark is actually quite easy to use so let's turn down turn off our service now to use wire Mar we need to have these routing conf configurations that tells wi Mark what to what to respond when it receives traffic on a URL and the data it should expect and the data it should return so you can really set up a fake rest API using wi Mark in our case we want my wire Mark to return a web hook and let's take a look at how we can do that so before we begin we need to create a new folder and let's call it wire Mock and Wick has this directory called mappings and mappings is where we specify the routing rules or the mock rules that wi mock should respond to so let's go ahead and create a new file inside the mappings called Web hook. Json wirar accepts Json as a configuration file and it's pretty easy to configure so let's go ahead and configure that file it's going to be a Json file and we are going to begin what wire Mark is expecting so we're telling wire mark it will expect uh it will expect requests on slash send web hook let's say that we and we also have to specify the method so let's go ahead and do method get so Yark is now going to expect requests on slend web hook and it's going to expect get requests and then we have to tell wire Mark that it should respond with something in our case we will just send a HTTP status 200 back to whoever sends the request so if we send a request to/ web hook we will receive a 200 HTTP back but that's not a web hook right exactly so we need wire Mark to after after it has returned a response we need wire Mark to go back and say hey you know what I'm going to fire a web hook and that can be done by serve event listeners serve event listeners is an array and this array can takeing stuff that it should do after it has sent the response so let's just name this a web hook so we're going to send a web Hook Once we're done and we need to set some parameters and the parameters is basically how how the HTTP request that will be sent as a web hook should be defined so let's say that we want to send a post message as a web hook and we're going to specify the URL in our case we don't need to use the enrock uh URL anymore but we can use Local Host because we're hosting the application ourself so let's go ahead and specify some headers and our headers will be content type and we are sending application SL Jon and uh let's see I need to wrap this my bad like that and we need to specify the body and the body is a yon request this is a little ugly what I usually do is that I take the payload that we received through angro and printed and just copy paste it right into here and that way we can mark the data that the real service is sending and just have a fake event go out we won't do that now because I will just show you guys how it works so so inside this body we actually need to escape uh the the double quote otherwise it will break the Json so make sure you add the escapes before in front of the double quotes basically this is how it looks this is the wire Mark configuration that we were going to use we're receiving request on send web hook we're responding a 200 and then then we're firing an event from the server and it's going to be a HTTP request to our API that's expecting the web hook that's really what we need to do that's all we need to do so let's go ahead and save now the easiest way to run wire Mark is to use Docker always Docker but we need to mount the folder we have the wik folder and the wire mark folder holds our mappings and we need that inside the docker container so we're going to mount that so let's go ahead and do Docker run and I'm going to do interactive I'm going to remove it once it's done I'm going to name the docker I'm going to do Network equals host and we're going to the reason why we're running it on this on the host network is because our API is running on the host Network and we're not using compos or anything so the easiest way is just to mount it now in the current directory slre Mark we want to host that to slome slem which is the default location inside the wirek docker um where the mappings should go so let's just do I have ver version wire markk 3.3.1 we're going to do verbos and we need to change the port used by default wire Mark uses 880 but we're already using that so let's go and put it inside 881 do you want to correct the no I don't and let's see error loading the file unrecognized field okay I have a typo my bad let's go ahead and open it up [Music] parameters there's the the typo it should be parameters save the file go back and restart the docker file and now it's running so we have this nice little splash screen saying that it's running and we can see it's running on version 8081 small note web hooks is an extension for wire Mark but it's enabled by default in the latest versions so you shouldn't have to activate it if you need to activate it make sure to update your wirek version okay this is great we're running wik so open a new terminal and let's go ahead and run the goang application on 880 I'm going to split my terminal and I'm going to send a curl request to wiar wiar is running on Port 8081 slend web hooks so let's curl that that and you can see that the goang application is receiving the web hook so that's great so what I recommend you to do is set up enrock to receive real web hooks from the service that you are developing against once you have received the payloads and you know what they look like I do recommend you to set up copy paste the payloads set them up in wire Mark as you saw the it's not a lot of configuration to get wire mark up and running if you have a developer environment with darker compost or anything it's easy to just spin up wire Mark in that darker compost and always have it running and just you can and then you can Mark and have your applications look like they're running the real thing so in this tutorial we have created a really simple way of accepting web hooks from services to our locally running application using enrock which tunnels the traffic from a generated domain and as I said before I usually Begin by setting up Ang Rock receive the real traffic inspect the payloads get to know the payloads and write your code for them and then go ahead and take the payloads into wire Mar so you can always have this kind of safe uh offline environment to test and uh work against so you don't also if you do use wi Mark you don't need to have the enrock tunnel up while developing which is also like I don't like the idea of having to have a tunnel routing traffic to my computer up when I'm developing against a web hook service it's just much easier do that once marck it and be done with it now this this was quite a short tutorial I do hope you enjoyed it and I do hope you learned anything from it and I really enjoy feedback so I mean shoot your feedback towards me let me know what you felt feel free to reach out with to me with any ideas a lot of you guys have been reaching out asking about all kinds of tutorials and I'm going to be working on them as we go so Don't Be Afraid just reach out with anything thank you for watching hopefully good luck with your web [Music] hooks

Effortless Local Testing - Learn how to test Webhooks on your local environment In this tutorial, we'll dive into the art of testing webhooks effectively. Discover how to receive third-party webhooks directly on your local machine using Ngrok, all without needing a public domain. Get ready to streamline your webhook testing process like never before! You can also read my article on https://programmingpercy.tech/blog/webhook-testing-without-the-headache/ if you prefer reading. 00:00 Intro 00:01 What Are WebHooks? 00:03 Using Ngrok 00:08 Setup Github Webhook 00:11 An Api To Respond on Webhooks 00:16 Mocking WebHooks with WireMock 00:26 Conclusion Ngrok - https://ngrok.com/ WireMock - https://wiremock.org/docs/webhooks-and-callbacks/ GitHub Webhooks - https://docs.github.com/en/webhooks/using-webhooks/creating-webhooks Webhook validation - https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries