On Writing Software Well #2: Using callbacks to manage auxiliary complexity

welcome to episode 2 of on writing software wealth I am David Hana Mary Hansen and this is a follow-up to yesterday's pilot episode that I did on a couple of small changes in Basecamp dealing around comments you can look for that episode in the playlist if you want to go back and check that out but the response to that episode was great I wasn't really sure whether anyone would have been interested in watching 10 15 minute videos on diving into sort of in some cases my new code considerations but it seems like there was so I'm gonna keep going for a while at lone at least as long as I still have the motivation to do it and right now I do so today we are going to dive into something a little more meaty than what I showed yesterday we basically talk for what 10 minutes about changing one line of code yesterday so let me show you through something a little bit more substantial this is an episode about callbacks about tracing one callback wrap it hole all the way down and seeing what that does to the code I'm a big fan of callbacks I think callbacks allow you to take incidental complexity and considerations and move them sort of off to the side since that most of the code can just pretend to be in a simple default path and then when you're up for it you can look into the more specific details of something that happens as a side-effect side effects really is just another one of those words that in some circles has gotten also a bit of a bad rep right like functional programming and so on it's all about not having these side-effects well I think side-effects are wonderful for a whole host of uses and I love having side effects that as I said sort of go off and adorn or progressively enhance a main flow that is then not considered or not alluded with that complexity so that's what we're gonna look at today we're gonna try to trace down one specific feature at base camp all the way down through all the callbacks that happen so I hope you join show so the future we're going to look at today is the mentions feature base camp 3 basically you can app mention someone who's already on a base camp project and that person will get a notification notification can either be an in-app browser-based notification it can be a native push notification it could be a email notification we have a host of different sorts of notifications that we send out but that's not really so important what I want to focus on today is sort of this callback flow how we find those mentions and send something out so what we're looking at here is the messages controller messages is sort of one of the main forms of communication to Basecamp and this is to create method and the create method uses this recording bucket setup that I'm going to dive into more in a future episode it's a really interesting pattern I think actually it's the predominant and most powerful pattern that we have in Basecamp 3 but it's kind of a long episode I want to think about a little bit of how to present that and talk about it so we're gonna save that for later for now just think about this as though we're creating a new message it belongs to some parent recording a specific message board it has a status which can be trapped or it can be active there's a number of default subscribers or additional subscribers it's banana that wants to know about this and messages can also have categories none of these things really matter for what we're trying to figure out here but I just want to show you that in none of this code when you read this controller is there any talk about this mention right all we do here is we record the new message which let's take a look at that the new message down here it's basically just a subject and the content right these are the things that we allow to be created as part of the new message now as I said this doesn't mention anything about mentions it doesn't say anything about how these things are recorded all those considerations about how we do that exists somewhere else and what we try to create is one of these it's or several of these actually in fact it's a mention an dimension is something to belong to specific recording one of these recordings the one we're looking in this case is a is a message and there's the person who mentioned someone and then there's the person who got mentioned and I kind of love to come up with these specific words to admission me and mention air to describe these things these are I don't know if there even is a mention me a real word I don't know but it doesn't really matter I think the wonderful joy of programming is that we get to invent words sometimes we get to invent concept and as long as we just package them up and they're consistent and coherent we have the freedom to do so wonderful right in any case a mention is created as as I said belonging to recording someone mentions someone is mentioned and it's the mention II who gets the notification so that's really what we're trying to do and mention has a graphical representation in Basecamp you see the little avatar and you see the name but what really matters is that the mention II gets a notification right so here's the class that basically encapsulates that mmm not really terribly interesting there's not a lot going on what you can see though is that there's a delivery going on so there's a callback that once this mention has been created we're automatically going to send it out to the mentioning we're unless the mention er is the same as dimension II which is sort of one of those little wrinkles here we don't do it in that case but but otherwise we do but how do we create these mentions that's really what's I think the interesting point and mentions are a concern I love concerns - a concern of recording recording maybe I should just mention is this broad encapsulation of all the different types of content we can have in Basecamp so a message could be a recording a chat line can be a recording a to-do item could be a recording and that's how we share a lot of these generic concerns between these different types of content that all acted the same way and all of these pieces of content whether to chat line whether it's a whether it's a to-do description they can all have mentions and this is a way we get to apply this logic to to all of them in any case what I want to focus on here is basically this callback train right so we're mixing this into the recording which means that every single time we save a recording which as you remember is what we did here we'll do a bucket record it'll save the new message creative recording for it and this recording has its concerned for mentions that's going to eavesdrop on the content and it's the eeap's dropping part to really find it interesting and fascinating to dive into first of all I love just the domain language around that eavesdropping is just such a perfectly delicious term to use for something like this that you have this system sitting on the side listening for new recordings listening for mentions in those recordings and then acting upon those mentions so the first thing to notice here is a little bit of a trick that I've tried to extract into rails several times but been unable to really find a good way into it which is the notion that to track dirty attributes whether an attribute has been saved or changed in the last update we have these neat sort of automatic methods for it for example changed from drafted which is a method that checks basically whether an enum that we have for status changed to the drafted status right we're given this by the enum definition and to use these change set methods you have to still be within the original transaction once the transaction is closed we clear out the dirty changed set and you can't really querying zin the same way anymore so by using this after save callback we basically take a look at these change sets and then we just make a little note and that little note happens as an instant variable here called eavesdropping to remember whether we need to eavesdrop because the thing with eavesdropping is that it has to act on a saved object right like this has to exist in database which means it has to be after the transaction is closed which means it has to be in an after commit ho right and as I just said once the commit is has happened and we've wiped all these things out we can't worry about these things anymore anyway no really none born just think that we're setting up the decision on whether G eavesdrop or not in advance this is really kind of actually enough to my station we could have Beatrice dropped every single time but scanning the content if the content didn't change doesn't really make a lot of sense right there could be some other change there could be some for example a comment could touch this message recording when it gets added and that triggers a new save we don't want to eavesdrop in that case because the message itself didn't change the content didn't change so that would just be wasteful in fact that's how we launched we launched with that wasteful version and then I forget how it popped out but somewhere along the lines we just realized oh we're doing a lot of work that we don't need to do and it's happening all the time so let's not do that so that's where remember chief jobs comes from and that's basically just looking at what are the conditions upon which we would want to eavesdrop well if there's an active or an archived recordable recordable is the concept for messages that to do's and whatever if they have changed if the message in this case has changed and it's active or or archived we want to eavesdrop we also wants to eavesdrop if we had a draft that became active you can make a draft message in Basecamp but that's supposed to be secret right so if you app mentioned someone in that draft we don't want to alert these or notify these mention E's once the draft is safe because we want to wait until the draft is published to do so so that's what draft become active is all about and then we basically we have a check here about whether we should be eavesdropping at all there's another few ways of sort of getting out of this this is first of all the the eavesdropping instant variable we set up here in remember to eavesdrop but there's also this notion of suppressed which I will come back to which is a really key concept for dealing with heavy callback scenarios because the thing about callbacks is you want to use them on the default path you want it to happen most of the time but there are scenarios where you don't want them to happen and I think perhaps that's what have bitten people in the past that they didn't really consider how they could opt out of these callbacks well we have a concept for this called suppressed that we will look at at the end anyway we're eavesdropping so if star 4 mentions and this is another interesting point here I love you saying callbacks together with jobs that you basically have this body of work that you need to perform in this case you have to scan a piece of content for mentions in order to create these mention objects that didn't get delivered and so on so forth but that work does not have to happen at the same time that we save the message right like if we go back here we do the record of the new message it's not a terrible amount of work that happens here it's just a few savings to the database but if we start scanning the content every single time well we're we're sort of just holding this up right we're holding up this line that until all the work here has been completed well we can't respond we can't respond to the user and send them on their way now if we take some of that work that is not required to happen at the same time as the recording of the message and shove it up on basically a different queue and have it happen in a job then we're doing less work that's blocking which mentions this great pace for that because the whole reason we're doing this mention is as I said was to notify the mention ease well what they're notified the second something is posted or a second and a half later doesn't really matter but it makes a big difference in terms of the workload that we have to perform within sort of the scope of a single request so we're setting up an eavesdropping job really that's what this mentions concern is mostly about if you see what is the work here actually being performed it doesn't actually do the real work what it does is it checks should we be doing the work obviously it sets up the association to dimensions once the work is done and then it takes off a job to do them at work that job - to do the work is this eavesdropping job which really is a pretty thin shell - we don't actually do the work in the eavesdropping job itself we have a dedicated class for this calls the eavesdropper which would just seed with the variables that we get from from queuing this job and we create these mentions so the real work itself happens down in eavesdropper so this is already interesting right like we start here in the messages control we record a new message no mentions of these eavesdropping things going on then we have to mention its concern that says oh let me just check this new recording that was come in do we need to do the eavesdropping do we not need to do the eavesdropping if we do let's in queue a job - to actually run the code so this is one of those things where you're like there's a fair amount of indirection here but it provides a very clear path of reading what's going on in the messages controller most of the time you're not really concerned about this auxilary complexity of the mentions themself right anyway let's have a look at the eavesdropper itself the eavesdropper is a class that takes a recording and a mention er the mention er is basically just a person currently doing work I think we're getting they mention it nope we're getting dementia nerve all the way back from here from current person which is another one of these interesting points here current is this new concept in rails 5 - or did we already have I won I forget concept to basically make Global's pretty and Global's is another example here of a concept that needs to be treated with some respect right like Global's is not something you should just litter all over your application but the notion of a current person or a current account is something you need to keep - and over and over again that passing this stuff around all over the place it isn't actually that helpful in this case we do need to pass it because we're passing we're issuing a job which won't be in the same request as the current person has been defined in so we need to pass it along I'll do another episode I think on current and idea of Global's and how they can be helpful because I think it's a bit of a misunderstood and overly feared consideration in a lot of forms of web programming just like callbacks Act yes we talk about writing one thing at a time so eavesdropper here where the actual mentions are created is it's basically this great mentions method that's called straight from the job see here we're calling we initiate instantiating the eavesdropper than calling create mentions which relies on a method called mention ease which is where the real scanning is happening and here you can see we're just that the rabbit hole keeps going right like does the eavesdropper that's going through and creating these mentions actually also need to have the work to actually scan the content itself I don't think it does so we have another dedicated method just for that just for extracting out the mention ease from from a recording there's been a consideration here we have different kinds of recording some recordings just have plain text and some recordings have rich text and there's two different ways of extract these mentions out doesn't really matter that much what we what we want to do here is we want to find these mentions in either forms of content and in the rich text these contents are is already encapsulated because when you do these add mentions we have some JavaScript that creates a little bit of HTML and you extract it out from that when you're extracting good call sound call signs just straight out of plain text content which is doing basically a rec X print the scan right anyway doesn't really matter when you're reading to understand what the eavesdropper does all you basically have to trust is the fact that have a scanner it can take a recording and that scanner can in extract some some mention ease and then once we have the mention ease we have the recording dimension air and the mention ease so now we have all the three components that we need to create mentions with so this is what's happening in create mention we have a recording with dimensions and we find and initialize by mentioned by mention ease we only want one mention per Minchin ii set up everything and save it now as I said we've created one of these mentions now when that happens is the work is still not done right now we have the domain model kind of set up we've scanned the recording we have a bunch of mentions we need to deliver these two right that's the whole purpose that's why we're doing this so there's another after commit deliver hook here at that triggers on both create an update that then runs the mention delivery and I mentioned delivery takes two forms we have a notification which is yet to be extracted framework within Basecamp three that I've been calling action notifier it's a framework for dealing with notifying people through web notifications in the browser or native notifications on both Android and iOS anyway this is another encapsulation that and of course we have male missions and this is just a standard action mailer that sends out an email notification to two people in that all right that's sort of a lot of weaving back and forth here I hope that makes some sort of sense I want to come back to the point we mentioned at the beginning which was the notion about suppression I have a class here called the project copier which is a way of copying things from an earlier version of Basecamp into the current version of Basecamp and as it's doing this copying stuff it's creating a bunch of new recordings that may have mentions but we don't want to actually notify people we don't want to create these mentions on on all this copied content so what we do as we're doing these copies of uploads and doctor whatever we suppress a bunch of things that happen we basically suppress a bunch of callbacks and one of the callbacks we want to suppress is the idea of eavesdropping right so that's what this line does if the eavesdropper just heals a block that within that block anything that would have called the eavesdropper to trigger will just get suppressed and essentially ignored that's all coming from a concept of what do we have that yes in the eavesdropper we have suppressible which basically just sense it sets up this variable to say what the something should be suppressed or not and then we can skip doing the work if if we don't need to if we don't need to do it so that's one example of how a single feature mentions works from the perspective of callbacks and hooks and I hope it's clear I don't know I mean an easier way to make it clear would be to take basically all this logic that we have the eavesdropping the scanning whatever then jam it into what the controller a service object some other form of trend or transaction script I don't think any of these things get clearer if you do that the fact is mentions is an auxilary concern it's not the main purpose of what we're doing here when we're setting up new recordings so it deserves to be on the side it does not deserve center stage but of course it needs to be there and we need to be able to dig into it when we want to which is where these concerns are just wonderful we don't have to burden the recording class itself with all this logic and all this setup to it we can treat this whole aspect as one cohesive unit even though it's mixed into a class even though recording gets a bunch of new private methods it doesn't really matter when we're reading through the code and trying to follow the flow of it this is a much easier way to read through things so that's it I hope you enjoyed it and I will talk to you in the next on writing software well

Callbacks are a powerful technique for making that which isn't essential to a primary flow live on the side. It means we can reason about just that one aspect when we please, but otherwise treat the main flow as though it didn't exist. It pairs very well together with concerns. In this video I explore how we used both techniques to create a mentions system in Basecamp 3.