Neovim highlights are extmarks

TL;DR: vim.highlight.range() and vim.api.nvim_buf_add_highlight() are implemented by using extmarks. I did not find this information in the Neovim docs and it took me an hour to make this connection. Proof.

nvim-ts-rainbow is a great Neovim plugin that highlights matching parentheses in matching colors. Nested parentheses are highlighted in a different way, so it looks like a rainbow (or whatever colors you configure) in the end.

It has also a notion of an extended_mode that also highlights JSX tag names. This makes the following code:

<StyledArticleCardContainer className={className} as="article">
  <StyledArticleCardTitle as={titleAs}>{title}</StyledArticleCardTitle>

  <ArticleMeta
    readingTimeMin={readingTimeMin}
    createdDate={createdDate}
    tagNames={tagNames}
  />

  <div>{summary}</div>

  <Link href={getArticlePagePath(slug)} passHref>
    <Button>Read more</Button>
  </Link>
</StyledArticleCardContainer>

be highlighted in the following way:

The bug

There was one particular nvim-ts-rainbow bug that irritated me for some time. The problem was that with extended_mode enabled, the rainbow highlight would include not only the JSX tag names and the angle brackets, but also the props. In other words, instead of the expected highlight, the code looked like this:

Notice that the props of JSX elements are highlighted in the same way tag names are. This makes them easy to spot, but we lose information this way. For example, notice that getArticlePagePath is highlighted in green on the first screenshot, making it stand out more, since it is a function that is called. it does not stand out as much in the second screenshot.

Moreover, JSX self-closing elements (ArticleMeta in this case) is not highlighted by nvim-ts-rainbow at all.

The solution

In case the bug description interested you and you would like to see the fix, it is in this PR.

The solution involved rewriting some treesitter queries to only capture JSX tag names and not the entire tags.

Accessing Neovim highlights programmatically

After fixing that bug I wanted to add some tests to make sure the highlights are correct. This meant accessing the highlights programmatically.

I knew that nvim-ts-rainbow added the highlights using vim.highlight.range. I had no clue how to then access these highlights. I looked through :h vim.highlight.range and :h nvim_buf_add_highlight but I did not find any functions that would return the highlights.

Only after I read the source code of nvim_buf_add_highlight I understood that Neovim highlights are implemented using extmarks, which are well documented (:h extmarks) and easy to access programmatically (:h nvim_buf_get_extmarks()).

Equipped with this information, I was able to implement automated tests for nvim-ts-rainbow highlighting.

Conclusion

If anyone is looking for a way to programmatically access Neovim highlights defined with vim.highlight.range or nvim_buf_add_highlight: highlights are implemented with extmarks. Use nvim_buf_get_extmarks to get extmarks (highlights) defined within a given range.