Git support for Lisp improved in 2.55.0

[u/ScottBurson]

In a Git diff, each consecutive subsequence of lines near a difference is called a "hunk". Each hunk has a one-line header that might look something like this:

@@ -316,8 +322,9 @@ int main(int argc, char **argv)

The numbers indicate which lines of each version of the file appear in the hunk. The rest of the line is intended to be the first line of the function, class, or other top-level definition that the hunk is within. Git finds that line using a regexp corresponding to the source language. It's just to give the reader a bit more context; nothing else depends on it — or should depend on it, anyway, since it can be missing or wrong.

The regexps that tell Git how to find the header lines are called "userdiff drivers". A driver for Scheme was added a couple of years ago, but it didn't work for Common Lisp or many other Lisps, as it failed to match (defun lines. I have modified it to be more general, and the relevant changes are in the recent Git 2.55.0 release.

I was unable to persuade the Git maintainers to name the driver "lisp", however, given that one named "scheme" already exists. The argument that Lisp is the family name, and Scheme one dialect within the family, was not sufficient to overcome their resistance to having two closely related languages with separate drivers — understandable, since too lax a policy about adding drivers would surely lead to there being hundreds of them. And of course, we couldn't just rename the "scheme" driver, because people are already using it.

So that's why, starting with Git 2.55.0, the way to get correct hunk headers for code in Common Lisp, or probably almost any other dialect of Lisp, is to have a .gitattributes file containing this line:

*.lisp diff=scheme

The Scheme regexp is still there and will still match all the same constructs, but there's also now a much more general regexp that simply matches any unindented open parenthesis, or (def preceded by one or two spaces. (The latter is to catch defining forms grouped together insde a top-level form like eval-when, but without the false positives that we would get if we didn't require a name beginning with def.)

[u/stassats]

I was unable to persuade the Git maintainers to name the driver "lisp"

Are they short on hash-table space or what? This is incredibly silly. Just in line with the overall UI of Git.

Worse is indeed better.

[u/stassats → u/stassats]

Now that I've tried, it does look better. Especially since I've been working lately with large similar-looking functions.

If only github would do the same.

[u/ScottBurson → u/stassats]

Yeah, I don't know how long it takes GitHub and GitLab to pick up new versions of Git. A few months, I would guess. But eventually, they'll have it.

[u/stassats → u/ScottBurson]

But will they have *.lisp diff=scheme? And they might be using an unrelated differ.

[u/ScottBurson → u/stassats]

I expect they'll use whatever .gitattributes file is in the particular repo. So you can update yours now and get the benefit of it locally, and then at some point GitHub will upgrade their copy of Git and it will start working there too.

I don't think the differ matters — this is internal to Git itself.

[u/stassats → u/ScottBurson]

I used a global .gitattributes. I suppose I can change .gitattributes for all repositories, but that's suboptimal.

[u/alkalisun]

Very cool! I never thought that one could configure git like this to be "smarter" about code diffs.

Thanks for sharing.

[u/ilemming_banned]

If you're still confused what this is about, let me try to explain:

You can trace individual function body history with git, which is very cool. Magit in Emacs adds two commands magit-log-trace-definition, and magit-diff-trace-definition. The former typically gets called from magit-file-dispatch.

What it essentially does, it calls git log with -L arg. Read more in man -P 'less -p "^ + -L"' git-log. So you can basically see how any given individual function changed over time.

The way git knows what "a function" is through the regexp patterns - and for some languages there's a built-in support - https://github.com/git/git/blob/master/userdiff.c

So, if you want to trace history of let's say Elixir functions, you just need add to ~/.gitattributes file:

*.ex   diff=elixir
*.exs  diff=elixir
*.heex diff=elixir

And it would just work, since Elixir's pattern is present in the mentioned C module.

But here's the thing - it's pretty trivial to add any language - because it's just a regexp. Let's say you want to have it to work with Lua.

~/.gitattributes

 *.lua diff=lua

~/.gitconfig

 [core]
     attributesfile = ~/.gitattributes

 [diff "lua"]
     xfuncname = "^[[:blank:]]*((local[[:blank:]]+)?function[[:blank:]]+[^[:blank:](]+|[A-Za-z_][A-Za-z0-9_.:]*[[:blank:]]*=[[:blank:]]*function)"
     wordRegex = "[A-Za-z_][A-Za-z0-9_]*|0[xX][0-9a-fA-F]*|[0-9]+([.][0-9]*)?|[^[:space:]]"

[u/ScottBurson → u/ilemming_banned]

A few years ago I tried to use this approach to add regexps for Common Lisp, but failed. My recollection is a bit fuzzy now, but I think the problem had to do with needing multiple backslashes to escape the parentheses, and for some reason I couldn't figure out how to do it. I know, this doesn't seem like it should have been prohibitively difficult, but somehow it was. (Or maybe that was a red herring, and I had something else wrong that I was overlooking.) Anyway, I figured that if I had trouble, other people would too, so eventually I resolved to get the language built in to Git.

Getting the regexps in Git exactly right wasn't trivial either, but after several rounds of back-and-forth with the Git maintainers, I am confident that we did so.

[u/HoldMyBow]

Thank you for doing this! That'll be a big improvement for the community.