As decision model use skyrockets, is customization the next frontier?

Spread the love

In October, more than one in four orders for Vercel’s AI model router went to decision models, not master’s degree holders — an enormous share of work for a type of computer intelligence that had only been invented a month earlier.

Decision models are LLMs that do not produce output codes; Instead, they were set to use their general intelligence to quickly and cheaply decide between pre-selected options, while recording their level of confidence in the choice. They are useful in production environments, where quick decisions can be more important than the logic provided by the slower, more expensive parametric model.

TypeSafe became a $7.5 billion company after launching its Jev model range, which still dominates usage (at least where we can). See metrics From services like OpenRouter, which handled more than 1.5 billion TypeSafe form queries last week).

However, the decision model is easy to make, and dozens have been launched in Jeff’s wake. An Amazon engineer built one over the weekend, and soon every hyperscaler had their own option. So is his company, Microsoft CEO Satya Nadella said on Tuesday Rolling Its decision model, Microsoft-Decision-1.1, is available for use at scale on Azure immediately.

The action occurs at both ends of the ecosystem: SquishThe UK-based AI startup behind a decision model called Kettei Fat announced a $3 million seed round today. Co-founders Ushnish Sengupta and Federica Freddi, who met when they were graduate students at the University of Cambridge, were developing small LLM programs when Jev arrived on the scene, and they realized their architectures could be reused as a powerful model for decision-making.

Data seems to be the key to a successful model in this category — TypeSafe CEO Diogo Almeida told TechCrunch that the hardest part of Jev’s build was precisely calibrating his judgment using statistically adjusted synthetic data. Sqwish also works with synthetic data, but its unique offering is built-in post-training tools that use the client’s own data to improve the model’s judgment.

“I think what makes us not a commodity is the fact that we offer our customers the same power…they can continue to upload their production data, and in the same way that we trained our models, their models can also continue to get better at the tasks they care about,” CEO Ushneesh Sengupta told TechCrunch.

Trying to classify decision models is difficult; Sqwish’s “kettei fat” model has performed well on some benchmarks, but most technology creators say the real proof is in the pudding: It’s generally easier for developers to replace one model with another and see how it works in a specific context.

“The standard is just an invitation to try out the model, so continuous fine-tuning is why they stick with us,” Sengupta said.

The Microsoft-Decision-1.1 model is also the result of the company carefully balancing cost, latency, intelligence, and calibration. Achint Srivastava, the engineering vice president who led the project, agrees that focusing too much on standards is a mistake.

“What we are seeing is that the ‘standard’ models are falling flat with real enterprise use very quickly,” Srivastava wrote in an email. “Our testing underscores how difficult it is to deliver accuracy, reliability, speed and cost efficiency in a single model. Microsoft-Decision-1.1 and Jeff are among the few companies to achieve strong performance across all four dimensions, even when evaluated against ‘pending’ or unseen criteria that reflect real use cases.”

For startups like Sqwish, competition with hyperscalers and frontier labs represents an opportunity. Freddie predicts that future systems will see so-called First Order intelligence, such as decision models, working in concert with Second Order intelligence, like law students at the frontier.

“Even in that future, System One might be worth $10 billion, or $50 billion [opportunity]“OpenAI is doing this IPO with a market cap of $1 trillion and $2 trillion in mind,” she told TechCrunch. “This is all we care about; for them, this is just a side project.”

When you buy through links in our articles, we may earn a small commission. This does not affect our editorial independence.

Source link

By admin