The linked paper is a much more interesting read: https://arxiv.org/pdf/2608.16157

I think there are many other significant inference improvements which an be built out that are being ignored because the API between client and inference stack would be tricky to nail down.

This is not a criticism of the research but instead the presentation but the repo looks very fishy, it isn't clear that this is from a bunch of researchers from Berkley. It also does not make very clear (on the GitHub) what optimizations or performance they are targeting.

I have read the repo and website. I have no idea what this project does and what exactly its features are.