Jonathan Frankle on why frontier fashions outran our potential to specify, consider, and curate, and what which means for design leaders transport agentic methods.
The metaphor lands as a result of it names the embarrassment.
On his return to Invisible Machines, a dialog recorded final summer season, Jonathan Frankle, Chief AI Scientist at Databricks and head of Mosaic Analysis, describes the subject as having constructed one thing like nuclear fusion: an considerable manufacturing unit for intelligence, imperfect however terribly highly effective. What now we have not constructed, at comparable scale, are the energy traces — the specification instruments, analysis self-discipline, software patterns, and information hygiene that flip uncooked functionality into one thing an individual or a corporation can use.
Jonathan is not making a modesty play about fashions. He is making a methods argument. If we froze frontier progress for 5 to ten years, he suggests, we’d nonetheless see an explosion of innovation, not as a result of the fashions received smarter, however as a result of we lastly found out what to do with the ones we have already got.
For design and product leaders, that reframing issues as a result of the trade’s default UI for AI technique is nonetheless a mannequin selector. Jonathan’s counter is older and fewer glamorous: be a scientist.
AI Technique Goes Far Past Mannequin Purchasing
Jonathan is specific that he is not a big-credentials individual, and that there is nobody on the planet who totally understands this subject but. He argues that orgs want to sit down, work out how to measure success, and climb the ladder of methods, testing alongside the method as a result of each use case behaves in another way.
The context-window fable is the clearest instance. One million tokens does not imply you cease managing paperwork. Generally efficiency will get worse as you retrieve extra materials, as a result of distractors multiply and fashions are nonetheless imperfect at relevance. Lengthy context earns its hold in multimodal work (photos and video eat tokens in methods text-only groups underestimate), but it surely is not a common substitute for curation.
Robb Wilson connects this to their earlier dialog with Jonathan: the mixology metaphor nonetheless holds. Rubbish in, rubbish out — whether or not the rubbish goes into the context window or the coaching pipeline. The arduous half was by no means the pour. It was deciding what belongs in the glass.
Jonathan additionally notes an actual shift: at most firms he visits now, there is somebody doing science in a proper method he respects as a peer. That is progress, but it surely does not substitute measurement.
Scientists Who Ship
Jonathan pushes again arduous on hiring filters that deal with hyperscaler résumés as proof of experience. Expertise is broadly distributed; rigor is uncommon. The individuals who matter are scientists who ship, who can maintain computing, experimentation, and product constraints in the identical head. DeepSeek, in his telling, is the proof level: good science with time and care can compete with cash shopping for time. Pedigree is a shortcut, not a assure.
The influence dialog turns private. Naveen Rao’s recommendation — maximize influence on the world — frames why Mosaic joined Databricks. Jonathan contrasts his well-known lottery-ticket dissertation with pre-PhD work on police use of facial recognition: the paper made grad college students excited; the report modified legal guidelines. That is the worth stack beneath the fusion metaphor.
The Specification Hole in AI
The longest technical thread is specification: the way you flip intention into one thing a system may be held in opposition to.
“Construct a benchmark” is a cop-out, Jonathan says. Not as a result of benchmarks are ineffective, however as a result of good ones are vanishingly uncommon and automatic eval tooling nonetheless disappoints, together with work he has taken cracks at himself. The more durable query precedes measurement: what would it not imply for this AI system to clear up your drawback? Is specification thumbs up and down? A paragraph of intent? A library of prompts with out good solutions but? Software program engineering solved unit exams, integration exams, and regression. AI has no equal self-discipline for non-deterministic conduct, and packages, not like fashions, are in some sense self-documenting when nicely written.
From there, Jonathan and Robb go additional: in the event you use a closed API and by no means contact weights, you are nonetheless coaching: your parameters are simply phrases. Immediate engineering is not a separate magisterium from fine-tuning; it is the identical computing sample in a unique encoding. Robb describes boiling prompts into physics-style formulation; Jonathan’s emoji-prompt thought experiment makes the identical level. Tradition shifts round what effort appears to be like like in communication (formal electronic mail vs. three bullets) are signs of the identical underlying truth.
Josh raises the design-generation pressure: constructing agent expertise in pure language whereas nonetheless wanting a visible again finish to delete errors — and McLuhan’s old-media-inside-new-media body. Jonathan admits he could also be getting old out of the chat-only interface; the scientific level survives both method — people want predictable edit surfaces and verifiable conduct, no matter the chrome appears to be like like.
Rewriting How We Management Intelligence
The second half of the episode is the place it’s best to lean in: who owns the reality when solutions are not from your web site?
Jonathan expects a cottage trade round LLM-oriented publishing, parallel to website positioning, unplanned by the platforms, incentive-aligned via capitalism’s messy effectivity. PDFs will persist for hundreds of years; parsers will nonetheless fail; useful information will stay locked in varieties people tolerate and machines hate. Static FAQ pages beat dynamic PDFs for what you need fashions to ingest; tentative information ought to keep out of the crawl path.
Robb presses the curation failure mode: unlocking a doc repository with out hygiene scales each fallacious PDF into organizational reality. Jonathan solutions with a broader digital sample — self-driving fleet bugs, centralized information breaches — localized human error vs. systemic scale.
Josh closes the thread on model management: extra folks will meet manufacturers via LLM feeds than via owned websites. Jonathan’s north star is separating information from reasoning — a devoted reasoner hooked to a curated information base, up to date when model tips change. We are not there but. Till we are, manufacturers have shrinking management over illustration in feeds they do not personal.
Who Owns the Reply?
None of this requires the listener to care which mannequin dropped final Tuesday. It requires them to care about the work between fusion and smoothie: specification, measurement, curation, and the organizational honesty to check slightly than vibe.
Jonathan’s first Invisible Machines go to (Season 2, MosaicML period) is nonetheless the craft dialog — environment friendly coaching, lottery tickets, a whole lot of mini-cupcakes. This episode is the enterprise engineering sequel.
When you are constructing agentic methods or designing the surfaces they run via, the query is not whether or not your context window is massive sufficient. It is whether or not you possibly can describe what you need, show you bought it, and curate what the system is allowed to consider.
Pay attention to the full conversation with Jonathan Frankle on Invisible Machines.
Disclaimer: This article is sourced from external platforms. OverBeta has not independently verified the information. Readers are advised to verify details before relying on them.
