The Taming of the Deftype Baishampayan Ghose

so hi my name is BG and I'm going to talk about the different mechanisms that closure provides for modeling domain data and how to choose one technique for another and once you have done that how do you make your own types interoperate with closures own data structures before that I'm VG and I work for a start-up back in India called help shift or comm and that's me with a dead mouse that I caught in the office a few days back yeah okay so why are you here well closure provides multiple mechanisms of creating your own types the most basic one being just a map which is I think the most widely used mechanism and then on top of that recently closure introduced protocols and records and and and types and so now we have basically three different things to choose from and all of them have different different features and different trade-offs so and the point being is that all these different constructs provide different levels of power and have different types of features so for a newcomer it's kind of like a very complex but a mysterious kind of machine which is quite hard to understand so my goal today is to pretty much demystify on these three constructs and help you in choosing one or the other and once you have done that you will be also be able to customize those types to behave well with approaches own set of abstractions so we'll start up the basics the three things right they kind of create a Trinity the map and the record and and type also known as the Deaf type let's talk about Maps first a what is a map really it's basically a collection of mappings of a domain a bunch of domain values two domain attributes right and maps are also extremely flexible and they are very widely used and they will probably solve 90% of your use cases maps are also extremely flexible so in the sense that you can just use maps to emulate any kind of type and and dispatch on that using multi methods one of the problems with maps being that is that since maps do not create a name type you can't do faster polymorphic dispatch on the type of the map you have to use multi methods which are not as fast as they could be next up are records records pretty much behave like a hash map but they also create a name type and that lets you not only dispatch on the type of the record but also lets you implement different protocols and interfaces so that you can make it interoperate not only with closures own data structures and abstractions but also interoperate with different your existing Java code one of the limitations of I won't really call a limitation but one caveat with records is that records already implement closures own abstractions in a certain way so if you want to customize that behavior and that's not really possible for example since records behave like hash maps and if you want to create a record and want it to behave in a slightly different way as a hash map it's not really a good good a good idea to use my course for those use cases so records basically provide a baked in implementation of closures collection tractions so if you want to customize that behavior you can't use a record but otherwise if you just need a type which behaves like a like a map but also has a name type I think I think a record is a great choice to solve that problem and lastly we have types also known as the DEF type types they set up the lowest level so when you create a new type you don't get anything for free you have to take care of implementing the right protocols and interfaces yourself and make sure that you're doing it the right way types also provide you with the most amount of power but again you have to be very responsible when you wield that power types also lets you create objects with mutable fields which is not possible with records or just plain maps so if you need mutable data structures if you at all need mutable data structures then def type is the only way enclosure right now yeah in summary def types provide you with a name type they have full control if you need high-performance deaf type also will let you do that you achieve that mutable fields are achievable with def types and they also you can also implement to your collection abstractions and make it play well with closures own data structures deaf records are similar but they don't have the mutable fields and the full control part but they also provide you with hash map semantics so different codes by by default behave like like a normal map so performance is also as good as def type because a def record creates a name type and you can do fast polymorphic dispatch on the type of that record def Accords also implement collection abstractions but I put a s there because that's a caveat the caveat being being that deaf records the implement closures own abstractions in a certain way so if you want to customize that behavior it canticles records you will have to resort to using the Deaf type itself and lastly you have the map so if your use case does not fall into any of those other use cases then you are just fine out fine using just a normal map well in short use Maps if you do not know any better maps are the right thing to go ahead with and use them wherever possible so let's say so now that let's assume that we need to implement or use a deaf type so how do we go ahead with that how do we figure out which closure interfaces we need to implement to make it behave in a certain way how do we really create types with mutable fields and stuff well the best option is to look at the closure source code because that's I think the the most efficient way of learning about closures own abstractions and that's how I have learnt about closure zone interfaces and and the different the class hierarchy and then how closure itself uses all these interfaces and abstract classes to implement the collections now if you look at the source code of closure you'll see that there are all these very important interfaces which which will come up especially the eye persistent map I persist in stack I look up see cable all these interfaces they provide certain behavior and to make your own type behave like a map for example you will need to implement if you feel of those interfaces and implement the provide implementations for those methods in the interfaces do you see any something and something or unless like is there any something which is which should not be here yeah that being object it's interesting object is not really a an interface but you still need to implement three important methods in your dev types and those three being there to string the equals and hashcode because if you are using def records then all these three things are implemented for you you get those those things for free but if you are using a def type you will need to take care of implementing those things yourself so so so far the whole process has been about finding out the the useful interfaces that you need to implement and implementing those things on your own types but it's quite tedious right can can we do any better finding or looking at the source code finally or maybe looking at some book or some other documentation finding out other interfaces that you need to implement in a in a non dynamic way is tedious can we reduce the tedium yeah sure scaffolding it's a it's a very interesting function very simple but very interesting and useful function created by crystal grand what that function does is that it uses closure Java's reflection api to find out all the ancestors of of a of an interface or a only or a class and and all the methods and the different elytis of those methods and prints them out in a very nice way so that you can just copy paste that into your editor and go ahead with that so a sample invocation of that I don't know whether it's visible but if I say scaffold closure dot lang assistant map it will immediately print out a list of interfaces that you need to implement to get a behavior to which which which is similar to a real hyper system map so it prints out I persisted map itself and in all the methods and the era T's java.lang iterable associative I personal collection all that but it doesn't do everything though because you will see that couple of methods here are actually duplicated because there are more than one interfaces which specify certain methods so you will do you will need to do the deduplication yourself before you go ahead with that because if you duplicate that then obviously it won't work I put this code up just steal it and use it make it a part of your toolkit it's a very useful function in node about - it's safe so - it specifies 20 different era T's you will need to implement all of them because if you don't do that then and if someone calls your your type with a with a charity which you have not implemented you will get a very weird error called the abstract method error which is not really user friendly so you better implement all those invoke methods and in throw the throw out the proper exception saying that that that arity is not supported instead of throwing the abstract method error immutable types closure is immutable but sometimes you may need to create beautiful data structures for performance or some other reason so closure provides two different ways of creating immutable fields in your own types these things are implemented using just metadata which you attach to a certain field in your record or not record but the type so the the metadata being one being volatile immutable so what that does is that it demarcates that field as a volatile field and that makes it a java volatile field and being volatile it provides certain behavior semantics around that field one being that it guarantees that the reads read or write on that field will be atomic and the the code will be executed in in the program order the JIT or the on the VM won't reorder those instructions for optimization or it will not let you ever see a cached value it will always show you the latest value of that of that field and then secondly we just have the vanilla Java mutable field called an synchronized mutable which is a normal Java field you might need to use that sometime but again as with any kind of mutable data structure need to be careful of all that because if mutation mutation is not controlled then you might end up with something which is completely meaningless yeah next up we have a printing so once we have created our own types how do we print them out because that's that's also a very important thing because a weeb we really want our types to be readable so that I can read that type back in and can serialize it in a proper way so that the reader can read it back in so closure again provides two different things these are just much implemented as multi methods so what you do is that you dispatch let that multi method dispatch on the class or the type of your custom type and you provide the behavior there there it's there itself so the first method is called a print method and that's for human beings so if you call PR STR on that on the on the on that instance of your type this invoke this function will get invoked and we'll show you give you the print representation for that if you look at the implementation the DEF method basically takes the object itself and the write-up and basically just writes any string to that to the writer next one is a print dupe friend OOP is for the reader so closure has this facility where you can print anything any object readable so once you do that it will give you a readable representation and when you call read string on that on that string you'll get your object back another thing to consider here or just know about here is that recently closure introduced this object literal reader syntax when which lets you print out any Java type in any readable way so if you print out your data structure or your own custom type in this way and then then once you run the read string on it it will instantiate that object accurately then we have a factory functions factory functions are often overlooked but I think to provide a good user experience to your users factory functions are indeed important they basically hide implementation details and you basically avoid importing of the from the classes so once you create a new def type if you want to instantiate it you have to call import because require won't work because left eye basically creates a new java type and you need to use import to get it and then only can you instantiate it so but if you provide a factory function around it just requiring that namespace will give you the factory function and just you can use it and then also you can do other things like the dev type constructor does not let you do any kind of adding in the dev type constructors don't let you add any preconditions on the on the arguments right so if you want to have preconditions then the only way to do that is to implement is to provide factory functions for your users so for example this is the first one uses a factory function and the second one it does not have a factory function so as you can see the user experience is not really that good yep so recently I hacked up or something which behaves like it says a custom data structure which behaves like closures own a hash map but has a different semantics so I just wanted to show this to you as a demonstration of how these things work okay just a second okay so this is called the chain map it's inspired by pythons own data structure called the chain map so what it does is that so for example say you you have multiple maps okay and you you give in a certain key you want to first look up that key in the first map first and if it's not available there you look up the second map and so on and so forth a lot of people I mean you can actually use March to merge all these maps together and get just one map but that's probably not so efficient so you might want to keep all these separate maps together I mean separate maps around so that you can find out I'll pick them out later so what Python does is that Python provides this data structure called the chain map which lets you create they call it context so every map is called a context think about server settings for example so you have default settings and then on top of that you have local settings and then you can have custom settings so and then there is this kind of this hierarchical relationship between all these maps so if you provide is a new value for a certain setting in your local setting then and if you try to look that up it will use a local setting first and if it's not available in the local context and it will look up the next one in the chain so forth and so on and so forth so I implemented this chain map data structure using deff type and just wanted to show you how it works so initially idea what I did was that I have created a protocol called the pea chain map protocol and it has three different methods one called parents parents basic returns that a new chain map with all put the first map and then I have I add child agile returns a new chain map with a new map prepended and Maps just returns all the maps enclosed in that chain map in the Deaf type implementation itself firstly I implement the P chain map port of all and then since I want to make it behave like a normal map what I did was I basically use scaffold here so I'll just show you how scaffold works so I want to make it behave like a map so a map implements thigh persistent map protocol so I just run scaffold on it and it will print out all the different interfaces that I need to implement with with the with the methods so yeah it implements I persistent map itself then java.lang iterable then closure lang associative I persist in collection see cable I look up and counted so that's what I did here so here is the C cable implementation so seek C cable yeah so C cable has only one method called seek and to make your type-c cable you need to implement this method so I just implemented that here and I persist in map later associative so on so forth so another thing that I did was I created a factory function here which takes a variable number of maps and creates a new chain map out of it and returns that I also made it readable so if you print out that custom type it will print out properly which is readable by the closure reader and I also implemented hashcode and equals in a way that it behaves normally as it as it should so bring it to bringing us the same value semantics that closure itself uses thus creating one just created a new chain map here and even though it prints out like a complex object okay I can call in the normal closure functions on top of it so I can call seek on it so let me just store it somewhere yeah so I can call seek on it and I defined my own behavior right so what it does is that it removes all the duplicates this gives you the contexts but with their duplicates removed I can just call as you can see from the behavior it basically looks up the key on the first map first and in a second and so on and so forth marg etcetera all these things will also work so since the chain map itself was the first argument it returned a new chain map but if I reverse the order it will do something different it returns a normal map printing it out is also so when I say PR STR on it prints it out in a readable kind of form and I can call read string on it and it will return me the same object so I can also check equality here true right so that that basically lets you serialize your custom types properly I have another example to show you do you have any questions so far any questions okay another example is by map by map is I didn't write it it was written by Christophe grant and the pie map is basically another map implementation but it lets you reverse the lookups as well so you can look up a map by by its value as well so there are implementations in various Java or guava libraries as well so it's a pretty interesting thing so let me show you how it works just compiling it so so just created a bomb up here and they store it somewhere okay so I can look up the keys normally as I would with any other map here so but I can also look up by the value so but before that I have to reverse it or Sania so when I call our map yeah so when I want when I call our map on the by map it basically gives you a new buy map in which the whole key value Association is reversed and that thing is done in in constant time so the key look up again only only reverse map the key lookup is same old lock 32n so yeah so I think I'm done actually so I finished quite early on this one so take change map as well go steal this code and and go forth and and use your create your own data structures yeah thank you if you've any questions [Applause] then you use your regular technique where you store your implementations default implementations in a map and use extend to mix in those implementations right and as far as all implementing all those invoke era T's is concerned that's also a macro way just create a new macro called f-type plus or something like that which will detect if you are using if you are extending iPhone and will patch in the missing in book implementations with a proper implementation which will throw the error nicely yeah yes prodigal methods or all these methods can do not support variable arguments so it depends so are you dispatching on type or is it a complex function on the map if it's a complex function then multi method is the only way to go right but if you are dispatching on type so if you have a key called tag or something like that in your map and if you're dispatching on the just a type then I think a record is a better choice because you will get better performance that way and yeah if in the future you need to implement some other behavior then you can upgrade to a proper def type itself yeah so that's that's by design so when you merge it only takes the first context but and that's that's implement the implementation detail really I I honestly I just copied the behavior from Python I don't really understand the use case of that but yeah they had it and they were quite happy about having that data structure and I built it in an hour so yeah okay thank you yeah sure so if you if you want two syntaxes this is exactly like this guy needs to be exactly like this if you use def record though you'll see that if you print it out without printing on it will use a different map like syntax but you can't use that because it's a it's a it's a built in the compiler itself and I don't think you can implement that but this is this is the one thing that you can use this this thing syntax which is namespace qualified class name and then within a vector all the constructor arguments yeah so what is the best way to figure out exactly what the semantics so and do it gotcha maybe remembering this backwards but if it's supposed to 55 try to represent an empty collection and I return an empty collection I think you need to look at the source I think I mean that's what I did if there were probably probably new books are coming up every day hang on close I write every once in a while there's a new book probably in the near future some book will cover these things in in detail but those are so many implementation itself is changing for some I didn't talk about closure script here throw the script does not have any interfaces all these things are implemented as protocols in enclosure script so the experience is much much better with closure script so and if you look at closer scripts own source intro script is written in closure itself and it was very beautiful that way but unfortunately closure right now is not written enclosure yet so there is a lot of Java ISM and Java stuff that you need to look at to understand the semantics I mean yeah yeah thank you [Applause]

from infoq