Well here I am on the train to New York once again, tired in week three of the fall events season that’s seen me run up and down the accela corridor. That means I need to give my mind a break, so let’s write, teach, and illustrate some AI.
In the past week or two there’s been much enthusiasm about JEV, a new AI model tool from Typesafe AI. The excitement about it comes from the massive proclaimed increases in efficiencies of JEV vs the frontier models, and there’s nothing more soothing to me than improved unit economics. The complication of JEV is that it’s not a large language model, the thing we all mostly think of as ai now, it’s an evolution of another member of the family, classifiers.
Classifiers, I would tell you in 2023, were the most interesting/useful tool coming out of machine learning. I worked with them a lot previously, when trying to automate the process of interpreting documents through the prior generation of ai. Rather than predicting text, a classifier takes an input, and it marries it to one of a set of inputs you provide. Back in the day, you’d take your wee baby ML model classifier, hire a team in India (shoutout Gurgaon!), and the team would look at hundreds or thousands of examples, matching them to the labels, which would teach the model how to understand the relationship between the input and the labels, allowing it to gradually accurately recognize them. This practice, supervised learning, was largely superseded by unsupervised learning, but the same reliance on human judgment shows up again in RLHF, where people rank LLM answers, a significant part of the “tuning” process that’s led to LLMs improvements.
The difference is the range of what a LLM can produce vs JEV means it can only be used for these classification problems, rather than for writing back anything you want or doing the other neat things LLMs do (like helping my write the dumb games on my personal site). The positive tradeoff there is that within the classification limits, JEV is super fast, and super efficient relative to the LLMs, and builds on the prior classifiers by adding more useful probability responses with each answer, changing it from only being useful for binary decisions to allowing you to set deliberate protocol for degrees of indeterminacy. How much so? An independent test by Every.to’s head of evals measured about 0.35 seconds per passage versus 8.83 seconds for a frontier model, at roughly 580 times lower cost. All of that and I don’t need a team to train it for any special purpose, it comes ready to go.
That efficiency opens up incredible opportunity to build new things- when you change the unit economics of a question, the whole space can shift. To get that though you really need to understand the mechanics of what it’s doing, which I propose to do in the dumbest way I can think of.
What is a dumpling?
A question that has challenged my friendship with Matt Ryklin for close to a decade now. In the lunch room today there was an active debate over whether a “bacon bit” constitutes bacon, or is so denuded of its essential character so as not to be bacon. This reminded me of the dumpling debate and so I called him up and had him recite his farcical position to the whole room, which was roundly and rightly rejected. Matt is of the overly democratic sort, positing that any cooked pocket containing a filling is a dumpling. Your classic “prescriptivist/descriptivist” argument, just turned slightly on its side.
So to illustrate how Jev works, I present: the dumpling index, located at https://jev.tedsweetsersignalsinc.com/
To use it, type a description of what you think constitutes a dumpling into the text box, or select between mine, Matt, or Nona’s presets and hit “ask Jev”. It will read your definition, classify and match that to nineteen essential attributes of what a dumpling is, and then apply them to 48 foods to tell you how far into linguistic madness you’d descend with Matt.
There’s a million uses for this — the immediate ones I’m excited about are for sentiment analysis, prebid analytics of the kind PurpleLab supports for audience quality scoring, and most excitingly to me, one-shot ETL mapping in milliseconds that mean I never have to write my own exposure data log mappings again.
Have fun, drop me a line, consider PurpleLab for your data needs.
Thanks to Brandon King, MBA for reminding me of the “hot dog, not hot dog” bit from Silicon Valley, a considerably more concise way to explain this, and to Matthew Ryklin for the obstinacy.