Ritz, The Missing Clojure Tooling Hugo Duncan

so my name's Hugo Duncan I'm going to be talking to you about Ritz which is IDE tooling for closure I work with pallet most of the time and and spent a lot of time doing doing debugging so large part of autumn I'm gonna be talking about its debugging my Twitter handle there's Hugo Duncan so a bit of history to start off with debuggers an enclosure got really started with Georg jihad and he came up with this the realization that you could use macros to access the current variables within a within a frame when you're in your sin in your code and then evaluate arbitrary expressions using that information so that led to the introduction of the ampersand end argument in in macros then I ended up putting that debug repple stuff into a swank closure which was the the project that Phil Hagar boat maintained for a long time and and then sometime after that George George had decided to make something more powerful so based on the JVM facilities for debugging he started to put together a rebel toolkit called CDT that allow you to break and lines and fence and inspect the stack etc and about the same time I started RIT's with the aim of putting the slime debugger making the slime debugger work in closure so since then things have changed a little bit we now have n ripple in rebel was Chaz's inspiration for a project to allow the user a closure of JVM across the remote connection it's now tools to unravel the official and the official closure repositories and n raffle itself is made up of a couple of things there's a transport so it allows you it opens a socket for you to communicate we got your closure JVM on and it provides a protocol to talk to it so being code I think being code was picked so that it wasn't limited to just Lisp clients so they be easy to build clients for for a closure ide outside of you without using lists also in n ripple is the the concept of a middleware so n ripple communicates based on messages so it transfer six messages from the clients and sends it to the closure JVM and each message is passed through a stack of middleware so you can think of this a bit like ring middleware in in your web stack so when you start that the the n ripple server you specify a list of middleware and it puts those into into it into the path of each messages and each one each each middleware gets a chance to inspect that the messages that get sent finally it provides a session management and by that I mean it provides a consistent environment for your dynamic VARs and so because it's message based each time you send a message if you update a var in one message you'd expect to see the same values next time you evaluate a message so that's what the session session management really does so n ripple n report or L then is a client for n ripple a separate project don't go hassling your chairs with stuff that doesn't work in and ripple VL n report that he has a project led by Tim King it's an Emacs client and it's a essentially a replacement for us for slime but ridden from the ground up specifically for closure as I said it had this has this middleware concept and it communicates by messages so this is just an example of a message in a middle way so in on the client side we have an OP a map that specifies an OP for for loading a file that message is sent to the to the server where it's passed through the middleware stack and the load file middleware says I I recognize that up I'm going to handle it it loads the file you've requested sends a status message back to the client and the client can can display whatever to say that the file has been loaded the interesting is really the thing about this is that it has support for not just closure but closure scripts so you can either out some of the differences between the backends in your in your client and your client doesn't have to worry about that the differences between a closure closure script for example middle-way also lets you have alternative implementations for things so if you don't like the the completion that comes by default in n ripple and then ripple VL you can switch your own one in so it gives you some customizability so in in n raffled VL there's an N raffle events buffer that you can actually go and see these messages being being sent backwards and forwards which is great for debugging in riffle though lol I'm not sure if it's useful for many other things but if you're interested in how it works you can you can see the details so Ritz as I said Ritz started off as a fork of swank closure it's now been refactored several times a very different codebase much more modular split up into different projects although the original focus was just to provide debugging for the slime it now includes all the implementation essentially of swank'd closures all the features that were in SWANCC closure can be provided through ritz and it's been refactored to support an repple so there's a there's a core there's a core ripple utils library and built on top of that there's a debugger library with that without any user interface involved there then on top of that you have the user interface libraries one for swank one for n ripple and there's an N ripple middleware library which is usable outside of the the RIT's the RIT's world usable with the standard and ripple servers so the the repple utils library provides all the facilities you'd want for inside a ripple so evaluation completion Javadoc namespace dependency graphs it's all there as a library it's very low dependency it just has the one dependency which is important because this code is going to be running inside your user user process so you don't to have conflicts between dependencies or to your user your own code and repple code the only dependency is and there is Dino path which is used for dynamic path put their class path for manipulation so you could use this this library essentially to implement a ripple in in any environment that you like I know there's at least one person trying to writing a web-based a ripple that using this so on top of that library then we have n ripple middleware so these are middle ways that you can use in any and any n ripple server Javadoc apropos there's a couple of completion ones there is the simple completion which is just like tag on things at the end and then there's fuzzy completion which will match things like I think of a good example if you just type it anyone suggested a two-component function name ok reach drink so I I - s you to type R - s hit complete and it was complete to read string so fuzzy completion so those are usable within Ritz and within the standard n reference server there's a couple more components within the Ritz framework Ritz umbrella as n ripple codec which allows you to have a middleware that will look up the history of a function so you select a function in your client hit a key and it will bring up the the history of that function by looking it up in codec it doesn't actually load the codec database that's a separate nation a separate problem and there's a menu rifle Hornet queue which runs n rifle over a message queue so Hornick user and in a nice Java based message cube that you can actually run embedded so if you if you have a city s the situation where you want to ripple but you can't use sockets then holic user an alternative their spring just to the the debugger so they the Ritz debugger is built on top of middleware like all the other functionality and it uses the the JVM facilities for debugging which is the Jaypee da as and there's JP da libraries and the way that this works is you need two processors so there's the on the right there's the user process which which runs and in the middle we have the debugger process which controls the user process so it talks to the user process using a protocol that comes with the JPD a toolkit which is JD WI I think I've got it right now JDI and it uses the JDI library all the JDI classes within provided by the JVM platform to do that and then your client here in ripple VL talks to the debugger process rather than the user process and any operation that is sent by the client to the debugger that it doesn't recognize it just passes on to the user process the user process handles handles it as normal and then returns it back to the debugger process which falls it back to the client but the the the debugger process recognizes all the debug functionality that's not going to be covered yeah covering in a minute so both the user process and the the debugger process both run and ripple servers the N ripple server in the in the user process isn't that actually listening on the port it's using the the JTA facility JPD a facilities where as the the N ripple server in the debugger process is listening on a port so it's just likely it appears like any other n ripple server because it's implemented in middleware you can use this with any client you like essentially that provides the that knows about the middleware so you could use this to influence a debugger in in a browser so it's just a another example of how the debugger middleware works so there's a when you type all decks break an exception to turn on breaking on an exception it sends a message to the debugger process there isn't a reply at that point it's doing whatever will be equivalent to a long polling in HTTP it's just giving a message idea it can reply on so in the second message we're going to eval some code which is going to throw an exception and at that point it replies to the first message with details of the exception which get then displayed in the client the client can then select a frame so that in this case we're going to look at the the source code for for a particular frame yes they ask for the free air for the source code allocation for that frame goes and looks it up takes the four replies with the file and the line number and the user and the client can then display that again and there's another example over there's a frame evaluation so we can evaluate for a particular frame happens by messages as well and finally the final component then is an ripple or I can't even spell and ripple grits and slime rips which are the Emacs packages which provide extensions to the base slime functionality in the base and ripple functionality for n rifle this this provides a a version of the the slime debugger but foreign Ripple so I'm gonna go very quickly through the install process because you can you can look it up really later the first thing it's all the packages are available in marmalade I think that was mentioned in the talk before this one you can set up marmalade and you have to do a package initialize and that gets all the Emax packages that you have to be available i just the old x package install n raffle and n report ritz which is the extensions to n ripple to run the the to n ripple servers there's a line plugin called line ritz and you add that it's made that globally available you add that to your user profile in the line ian and finally to run a ripple and rippled VL provides n ripple jack in and ripple ritz provides that row for its jacking so to use these functionalities you navigate to a buffer a closure buffer within your project and then enter these commands and it will start a session an n ripple server in the context of your of that project and connect to it and finally the it's actually use middleware the way you select which middle way you want to use is again using lining and you can do that in your your global profiles or you can do it in project specific profiles okay now to the probably more interesting bit a little demo so I've got a namespace here I'm going to start a I would thanks Andrew for its jacket in okay and that's going to start a ripple server within the context of this project and at this point it's actually starting up to JVM processes so it starts takes a little longer than the standard Henry will start up and I'm going to begin by showing a little example of our exceptions are handled in n ripple al and we can switch on the exception handling there's my prompt so in standard n ripple by default you just get a an exception message which is kind of nice if you don't want to see this the stack traces sometimes it's useful to do that though so that's all customizable within within n River ripples or yell using the standard Emacs customisation facilities here I'll just just set it and then a click and enwrap will shows you a stack trace if you have the closure exceptions library loaded it'll actually give you a nicer stacktrace than that so now I'm going to turn on the exception handling that's built into the debugger and there are some reasons why this isn't on by default which I'll go into in a while basically turn it on and this time we get a somewhat bigger stack trace so again you see the the at the top you see the the exception message and the exception type and below we see the the customary stack trace and then we have these these restarts and restarts were the idea of restarts this comes from Common Lisp the condition system in Common Lisp and in in that system basically you can write code that controls what happens when an exception or a condition is raised and that code then runs without the stack being unwound in the context of when there when an exception is raised and we can do something similar using the Java debug tool kit except that we we just provide some some praetor's predetermined restarts so yeah you can't actually pro add add to this list dynamically but the same mechanism works quite well so when we get an exception we can choose to choose to do several things with it if we continue it's going to pass that exception on to the program and you can see it's been caught and that's being wreath row in the game so you can do that a couple of times we can abort which then says okay don't catch any more exceptions just go back to the top level so I'm back at my prompt and then we can choose which exceptions T to C and this is where one of the issues we have the Java toolkit provides facilities for catching court and uncaught exceptions what you think would be very useful to expose but unfortunately doesn't really help us in the closure world because every try finally block creates a catch context essentially so if you if you have any try quatre try finally block in your code all your exceptions are effectively caught and it's a pretty common idiom within within closure to have this so a lot of the macros expand to forms that have try finally in them so it's meaningless to court care to filter on court and on court so we have to provide some other facilities to be able to do that and that's what these these other restarts are about so we can choose if we choose ignore will ignore we won't stop on any exception that has the same type as this exception so if I if I ignore this we're getting a better get back to the raffle and if I plot the expression again it didn't stop so of course it'd be nice to be able to manage this I forgot all my shortcuts and ripple in writs there's a buffer built in they shows you all the all the exception filters so this is the one I just created this is still a bit rough and these look a little little nicer in the UI but they this is the exception I just created you can see there's a whole load of default exception filters because by default it catches every single exception so all the exceptions within the closure implementation and the default default filters try and filter out all the all the common ones that come up from within the implementation so I can I can control the I can control the the filters from this buffer using the bindings that are here so I'm just going to disable this this exception filter with the and now if I go back to the revel it again once against brakes on that exception so we can filter on exception type we can filter on message which is an exact match by default we can filter on catch location and we can throw on throat air we can filter on on throw location so using these filters you can you can control which exceptions you you you break for in the debugger and in the exception filter buffer and this one if you hit s it will save your exception filters so next time you start up the project it will have all these exception filters already loaded and you have you don't have to set them again so so that's how you control the exception filters the the stacktrace is a little richer than the standard one each frame is open herbal so if you just hit enter on the frame you get to see the local variables within that frame it's not just the first frame it's every frame closure or java so and within this frame you can you can evaluate expression so if I hit E I can I can enter expression x 2x and it's going to give me an answer in the at the bottom if you have a more complex expression see if I can find it more complex expression 1 somewhere will do the form if you hit D and enter an expression it will pretty print the result in it in a separate buffer which is useful if you've got a long expression if you want to copy and paste things and you can pull up the source for for a a frame with V so excuse the whitespace highlighting so that's pulled up the the divided source you can see where we're actually here that's a will D up from from the Java let's find a closure one close your eval so that's within the closure implementation screen resolution sucks for displays FF presentations okay so it's going to the middle where each time exactly so when I when I enter on this frame its it's actually going and asking sending a message to the to the back end and it's saying give me all the locals for this frame it's getting the reply and putting it back on the screen yeah I feel free to interrupt if you have questions or you want to see anything in specific please uh yeah pee hand up okay so that's the the snacks stacktrace so it's gonna continue what no I'm gonna but go back to my example code I'm gonna compile this and you can see here it's actually pulled up in compiling to pulled up a class not found exception which is pretty normal because it's trying to find stuff that hasn't been compiled yet so I should really add this to the standard filters that that come up but class not found is a pretty common exception and you probably would like to see it in general so you can you can just ignore this it will carry on compiling so now I've got a little function here and I'm gonna set a breakpoint on this so these control C control X control B I get a little breakpoint and it's shown as a little arrow in the in the fringe there's also a a browser if I can remember them with the shortcut for or it's break points so there's a little buffer that shows you all the breakpoints and if you enter on a breakpoint it'll jump to that break point and you can you can kill break points from that buffer etc so now when I run this expression I'm going to call f1 it's broken at that break point and we can see we have our local variable a which was which is a 1 and again we control the stepping or control the the break points through the restarts so in this case we've got continue which just continues on from the restart got continued clear which will continue and remove the break point we have step which will step into the next instruction actually there should be the next instruction rather than next line step next steps the next line step out steps out of the the function not at the moment map yeah it could I don't know how useful how many people would use that sort of thing yeah okay sounds like this demand for it that was a limitation it should now be fixed I'll show you in a second okay so um so we have stepping I'll just step next and by default it'll show though that the code location actually the next line was an exception so this is the exception that it's true so if I recompile this code fingers crossed he says no demonstration effect it didn't work are you supposed to work okay so I have a more complex example here which I includes that seems to have killed my my session okay there's an another feature which is both X and ripple Ritz threads which will give you a display of all the threads that are running and we can actually see that one here is suspended for some reason and I'm just going to resume that with an rap all writs resume resume all okay I got my way place your Arabic hopefully I can now show you the complex example and that is still not liking it yeah NREL Ritz there's a also in ripple connections control-c old L list your n ripple connections so I'm just gonna get kill this with ctrl K and go back to my example and restarts a memory for Ritz session so one of the the issues in with closure and debuggers is that there's locals clearing so often when you expect inspector a stack trace variables variables show us a nil even when they're not because and that's done so that you don't hold on to references to lazy sequences so you don't end up with lazy sequences fully realized because of the debugger and ripple Ritz break on exception the the breakpoints at the moment depend on break on exception being set so it's a limitation that's going to be removed I can feel seeker okay it's going to compile that so in this case we got the we've got the the local variable and the the locals clearing is actually improved steadily a little bit different new versions of closure so less and less is is nil by default when you show it see if I can find one that's it's gonna give me nil anyway so if you in in the slime interface there's a prefix for the compiled commands that will actually disable locals clearing I haven't got around to porting that to n raffle yet that's so you have to set it manually with alter of our route compiler options and you get a sort disabled locals clearing true and you have to recompile and it shows you that all the local variables reliably so I think that's that's pretty much yes yes yes no I've tried it in Eclipse and IntelliJ and just with the Java try try finally block it has exactly the same behavior no it's not it's not the boy code that's being generated I don't see how you can avoid it without possibly if you put in special byte code for for that situation so you allow the debugger to recognize it but that sounds like a bit of a hacked yeah it's it's pretty frustrating so if we could come up with a solution that would be ideal it should well we can try it you'll see it should just display some dot dot dots I haven't tried it recently so we'll see if we break it I can't type today like oh I didn't set any breakpoints everything sir let's just try that again no you gotta feel a fully realized list there what try it with some more okay hey it didn't like it oh no it didn't like it yeah you should be displaying dot dot I'll fix that because obviously for some some lazy sequences you really need that yes with the the D key yeah you can pretty print so I go back to life I go just pretty print this expression with D and C it should it's gonna be slow because it's coming over the Tourette's okay thanks I should have picked us a smaller example that's going to kill it and start again okay any other questions well unfortunately the way candy Valor a type expression which is like the the hacky way to do it unfortunately the inspector is imported ten rifle yet so the slime inspector which is really useful for that sort of thing isn't they ideally that would go into the main enric rather than just in reports you can set breakpoints in Java so there's some some facility there if you if you have an Interpol server running yes yeah obviously you can't eval code so it's probably a little bit a little bit more limited but if you've got an n ripple server you must have a closure runtime in loaded as well so it might be interesting to explore that a little bit more any other questions so this is essentially just an expansion to me yeah yes isn't it it's not the useful it is jvm specific because it uses the the JVM platform debugger tool kit to read the the middleware operations could be implemented I'll I'll go and all then say they could be implemented but I don't really know if I were to implement yes you should be able to use the same you may say he max interface to close your script if you can get the the middle waiter implemented okay great ok well come see me if you have any more questions thank you very much [Applause]

from infoq