Google Started Generating Calculators Inside the Search Result. In Its Own Test, They Beat the Top-Ranked Page 90% of the Time.

Your dealership website has a payment calculator on it. Almost every franchised store does, usually next to a trade-in estimator and something that answers "how much car can I afford." On August 19, Google started shipping the capability that builds those tools inside the search result instead of sending the shopper to yours.

Google announced a batch of study features and buried the load-bearing sentence in the first one: "Generative UI capabilities have launched globally in English in AI Mode and have started rolling out as well in AI Overviews." Search "ph scale" and get an interactive visual inside an AI Overview. Ask a follow-up and AI Mode builds a custom version for your exact question.

The back-to-school framing is packaging. The system underneath it is general purpose. Google Research describes generative UI as software that "dynamically creates immersive visual experiences and interactive interfaces, such as web pages, games, tools, and applications" in response to any prompt at all. Google has already shown it running a mortgage calculator in AI Mode that lets somebody compare loan terms, and Search Engine Journal reports full availability in AI Overviews is expected in the coming weeks.

That last detail is the one that matters for a car store. AI Mode requires a shopper to choose it. AI Overviews show up on the default results page whether anybody asked or not, and that is the page where somebody types "what would my payment be on a 2026 Tahoe."

The number Google published about its own feature

The research behind this landed on arXiv in February under the title "Generative UI: LLMs are Effective UI Generators", with twelve Google authors. The evaluation section is the part worth your time.

Google put five versions of the same answer in front of human raters: a custom website built by a human expert for that exact prompt, the top Google Search result for the query, a generated interface, and two plain-text formats. Then it collected head-to-head preferences.

Generated interface over the top-ranked search result: 90%.

Read that slowly. On the prompt set Google led with, the page that won the ranking lost to something a model invented on the spot in nine of every ten comparisons.

Now the counter-case, which is sitting in Google's own paper.

Google is grading its own homework. Nobody outside the company ran this test.

The main prompts came from a chatbot arena. They were sampled from LMArena, so they look more like chatbot requests than search queries. On Google's second prompt set, built for information seeking and much closer to real search behavior, generative UI beat the top search result 73.5% to 19.0%. Still lopsided. Meaningfully narrower.

Human-built pages still won. Expert-built custom sites beat generative UI 50.0% to 35.3% on the first set and 55.5% to 30.0% on the second. In ELO terms: expert website 1800.3, generative UI 1736.2, top search result 1352.2.

Those expert pages were cheap. Google contracted independent developers and paid them between $100 and $130 per website, three to five hours of work each, with instructions to prioritize user experience "without any foreign considerations (such as SEO optimizations)." So the human ceiling was never measured. A hundred-dollar page built in an afternoon beat the machine.

It is slow and it still breaks. Google names both limitations: generation "can often take a minute or two," halved by streaming, and JavaScript, CSS and HTML errors occasionally slip into the output.

The trend line is the uncomfortable part. Google calls robust generative UI an emergent capability, and the output error rate on its main test set fell from 60% on Gemini 2.0 Flash-Lite to 29% on Gemini 2.0 Flash to zero on Gemini 2.5 Flash, 2.5 Pro and Gemini 3. Whatever gap you measure today is the widest it will be.

Summaries compete with pages you read. Interfaces compete with pages you use.

Google has been absorbing small utilities for fifteen years. Weather, unit conversion, currency exchange, tip math, sports scores, flight prices. Every time it did, a category of thin utility site lost its reason to exist. Those were static answers dropped at the top of a results page.

AI Overviews extended that pattern to prose. Pew Research Center found that Google users who saw an AI summary clicked a traditional search result on 8% of visits, against 15% for visits without one. They clicked a link inside the summary itself 1% of the time, and they ended the browsing session entirely on 26% of pages with a summary versus 16% without.

Generative UI extends the same pattern to interaction, and that is a different kind of loss for a dealer. Your tool pages were never written to be read. They were built to be used, with a lead form waiting on the other side of the result.

What this means for your dealership

Sort every tool on your site by one question: does this run public math on numbers the shopper types in, or does it need data only my store has?

Public math is now reproducible inside the result. Monthly payment from price, rate, term and down payment. Lease versus finance comparison. Amortization schedule. Sales tax and doc fee estimates. Affordability from monthly income. Fuel cost between two trims. Every one of those is a formula wrapped around generic inputs, which is exactly the category Search Engine Journal identified as easiest for a generated interface to match. Google does not need your page to produce any of it, and a generated interface is built from the query plus web and image search alone.

Your data is not reproducible. The specific VIN sitting on your lot with 14 miles on it. Your advertised price after the exact rebate stack a shopper in your zip qualifies for this month. What your used car manager will actually put on their trade after looking at it. Whether your lender approves that customer at that tier. A model generating a page from a search query cannot see your inventory feed, your incentive stack, or your desking software.

A calculator sitting behind a lead form is now the worst position on the board. Plenty of dealer sites gate the number: enter the vehicle, the down payment and the term, then hand over your name and email to see the payment. That was always a poor trade for the shopper. The competing option is now a generated tool in the results page that simply shows the number and asks for nothing. Gating public math stopped buying leads. It buys a bounce.

Look hard at the tools line item on your vendor invoice. If you are paying monthly for a calculator suite that gets sold to you as an SEO and traffic asset, the traffic case for it is weakening in public, on Google's own blog. The retention case for it, keeping a shopper who is already on your VDP moving toward a deal, is fine and probably improving. Those are different products at different prices, and your invoice should reflect which one you are actually buying.

Measurement is going to be indirect. Google's Search generative AI performance reports launched in June 2026 and report impressions inside AI Overviews and AI Mode, and they are still rolling out to a subset of properties. Combine that with the GA4 problem we covered on August 19, where a meaningful share of AI Overview traffic books as direct, and you should expect to see a trend on tool-page queries without ever proving the cause.

What to do about it

1. Take a baseline this week. Pull Search Console for the queries landing on your calculator, trade-in and finance pages, and save impressions and clicks for the last 90 days. If those queries slide over the next two quarters you want the before-picture in hand rather than a guess.
2. Ungate the math. Show the payment first. Ask for contact information after the shopper has a reason to give it, which is a real vehicle, a real price, or a real trade number.
3. Wire every calculator to live inventory. A payment tool that starts from a VIN on your lot, at your price, with this month's incentives applied, is something Google cannot generate from a search query. A generic loan formula on a branded page is something it can.
4. Turn your trade-in estimator into an actual offer. A range pulled from a third-party book value is public data with your logo on it. A number your store will honor for seven days is not.
5. Stop buying tool pages as a link and traffic play. If the pitch is "this calculator will earn links and rank," the ground under that pitch is moving. If the pitch is "this shortens the path from VDP to appointment," that still holds.
6. Check that your real data is actually crawlable. Inventory, pricing and incentive detail that only renders inside a third-party JavaScript widget is invisible to the systems now deciding what to show. The advantage you have over a generated interface only counts if it can be seen.

The line to watch

Google's paper calls these interfaces ephemeral, generated for one prompt and then gone, and describes the system as a way to "spin up an instant (AI-based) product management, UX design, and engineering teams" for a single query. At I/O in May, Google also previewed standing mini apps and dashboards for subscribers. The distance between a one-off generated page and a permanent tool is shrinking.

Formula plus generic inputs is the part of your website that Google is absorbing. Real inventory, real pricing and a real offer from a real store is the part it cannot reach, and it is where the money was anyway.

If you want a straight read on which of your site's tools fall on which side of that line, book a walkthrough and we will go through them with you.