ast-grep

Hi, I'm David, your developer on duty. And in this video, we're going to have a look at the search tool as Grap, a CLI tool for code structural search, linting, and rewriting. Now, what's the main problem this tool is trying to solve? Let's say you have a code base JavaScript and you want to search for all console.lo statements because you don't want to ship productive code using console.lo lock statements. Now, what you typically would do is use a tool called RIP Grab or other textbased search tools to search for console.lo. But it could be that you don't find anything. But what might still be the case is that there are still console lock statements, but they are just written a bit differently. So, let's have a look at the source code. Here you can see a console lock statement, but there's a space between the dot and the lock. So that's why your textual search tool did not find it. And if you don't apply constant formatting, there are quite a lot of different variations on how to express the same thing. So this is still console.log. still works, but it's getting harder and harder to find these occurrences using textural search tools like Rep. When code is parsed, it's transformed into an abstract syntax tree or a for short, which is a tree representing the source code structure. This already resolves much of this ambiguity. And when searching for code, we can use the same technique. We can pass the code into the a or to be more precise into a concrete syntax tree or CST in short which is a more detailed representation of the code structure and then search for patterns in this tree. So how does an a look like? I opened here Neo Vim and Neovim uses tree setter to parse source code into an a and the cool thing about neoim is it lets you inspect the a by using the built-in command inspect tree. Now I can move my cursor on the right to any position and the respective source code on the left is highlighted. So in this case the console lock statement is represented as an expression statement called expression function member expression object identifier property property identifier arguments arguments with an identifier in there and this identifier is this x here. So now that we understood a little bit of theory behind it let's see how to use this tool in practice. I can run sg in short for a grab and I can provide a pattern with minus p and here I just provide a string of what I'm trying to find in this case console.lo and you can see I found it even though there have been new lines in it and of course the ambiguity in the source code is also reflected in the ambiguity in the search string. So if I would add a space in here, it would still find the respective console.lo statement. Now let's see what happens if I add parenthesis. It will not match anything. And the reason is in the source code I had console.log of x. And here I'm searching for console lock statements which do not have any argument inside. So if I would provide X in there, I would find it. But of course, this will only find it if the variable I'm trying to log is X. So if I search for let's say Y, it will not find anything. So the question is how can I search for console log invocations which have any arbitrary number of arguments. So for single nodes in this a I can just write a dollar and then something with uppercase letters for example ABC. This will match any node in this abstract syntax tree. Think of a dot in the text space but in the a space. However, this will not match it if I have multiple nodes. So let's just adjust the script again. Let's say I also have y and I print both. Now if I run it again, it's not matched anymore because now the console log statement has two arguments and I'm searching here only for one. And in a similar way, if I remove all arguments altogether, I also will not find it anymore. So to provide an arbitrary number of parameters, I can use triple dollars. And now it will find it. And it will also find it if I add two arguments in there. Now let's just modify our script again. Let's just say we have here this function as an optional parameter and if function is defined I want to call function. And here when I call it I can define my function to be a function which prints hello and I can provide it. Now JavaScript has a shortcut for this namely fn question mark parenthesis to call this function only when it's defined and we want to now search for such occurrences in the source code. So I go back to my friend a grap and I provide a pattern which is something and something called and you can see I can find the occurrence. Also be aware that this A must match this A. It's not like placeholders for anything. So for example, if I would have said FN2, I would not find this occurrence anymore. So let's just undo it again. All right. And now let's just say we want to modify the code to use this optional call syntax. So to replace the source code, I can use the pattern to search for it. And now I can just say I want to replace it with dollar a which is the one I found before. Question mark dot parenthesis. And I will do minus I for interactive. And now for each of those occurrences I can accept it or not. In this case I'm going to accept it. And now if I look again in my script it's been changed. What comes in handy is that you can include a grab also in your text editors. For example, if I open neovim, I have a keybind to open fetf lure which allows me to query the code base for ag grab queries. And here I can write dollar a question mark dot parenthesis to search for optional method calls. And if I run this, I will find the occurrence in a nice overview. The way I've done it is I set a key map to leader FA and then this lure function is called where I retrieve my query from the user input and then I call FCFX that is a function from FCF Lua and I call agre with that query and provide here some additional options. for example what the default action shall be and I want the built-in previewer and again if I run this I can say a question mark dot parenthesis and I find the occurrence and I can directly jump to it and of course you can do similar things also in telescope and other plugins so I recommend to try out this tool there's also this nice website as grabb github io and it has a lot of nice features, searching and rewriting code as you've seen before, but also using it as a llinter. Um, and you can also use it programmatically. So, give it a try. Let me know what you think. Post it in the comments. As always, thanks for watching and stay tuned.

Let's have a look at the CLI tool ast-grep to search and replace source code. Contact: david.devonduty@gmail.com #devtools #neovim