A Real A/B Test: What ChatGPT Knows vs. What It Guesses

October 2026 12 min read

We ran a real, controlled experiment. The same image, the same prompt, submitted to ChatGPT twice. The only variable that changed between the two runs was whether the file carried a VISID stamp.

The result wasn't a marginal improvement. It was the difference between an AI doing its best guess and an AI actually knowing.

The Setup

The image was a photorealistic architectural visualization of SkyLume, a real, 60-foot kinetic public sculpture installed in Huntsville, Alabama, rendered by Pixelhead Studio. We asked ChatGPT the same question both times: "Tell me everything you can about this image."

Run one used the file exactly as exported - no embedded identity information at all. Run two used the identical image, stamped with VISID: a structured, declared record naming the sculpture, its artist, the visualization studio that rendered it, and a link back to a public, verifiable record.

Nothing else changed. Same model, same prompt, same picture.

click to enlarge

Run One: No Stamp

Without a stamp, ChatGPT wrote nearly 1,500 words. Almost all of it was description, not knowledge: things anyone looking at the same picture could have said themselves. Curved metal forms. Colored elements arranged in rows. A low camera angle against a blue sky. Real observations, but observations anyone with eyes could make, not anything the model actually knew about the piece.

Where it reached further, it reached for speculation dressed up as insight. Asked about the sculpture's meaning, it offered a list of guesses: "the spiral resembles a shell, a curling leaf, an unfurling flower, or an ocean wave." None of that is wrong, exactly. None of it is right either. It's a plausible-sounding association, not information about what the piece actually is or why it exists.

The summary at the end makes the gap plain. Its own closing description: "futuristic, sculptural, kinetic-looking, biomorphic, colorful, and architectural." Six adjectives that could describe almost any abstract metal sculpture on earth. Nothing in that sentence is specific to this one.

Checking the file directly, it reported: "No embedded EXIF data found." It searched for a match against known public artworks and came up empty: "I couldn't establish a reliable match... It would be misleading to attribute it to an artist just because another artwork has similar curves and colors."

“The single biggest unanswered question is the artwork's identity.”

That was its own closing line, and it's the honest one. Everything above it was real effort spent describing what was already visible, not telling us anything new. The model did everything right with what it had. It simply didn't have anything real to work with.

click to enlarge

Run Two: Stamped

It started exactly the same way. The same visual read: a twisting metallic ribbon core, a sweeping circular canopy, the same cataloguing of color, lighting, composition. Same method, same effort, same starting point as run one.

Then it did one more thing. It checked the image's embedded metadata, and this time there was something there. It opened a new section, titled on its own: "I found the sculpture's identity."

That single move changed everything downstream. Where run one guessed at meaning, run two reported it. Asked what the design signifies, it didn't offer associations - it cited the artwork's actual, documented intent: the central ribbon represents energy, aspiration, and the forces that drive people forward. Not a resemblance someone might see in it. The real reason it was made.

Where run one closed with six adjectives that could describe almost any abstract sculpture, run two produced facts that describe exactly one. The artwork: SkyLume. The artist: Adam Buente of Project One Studio, an Indianapolis-based public art practice. The unveiling: May 2026, at the roundabout entrance to Joe Davis Stadium in John Hunt Park, commissioned by Arts Huntsville in partnership with the City of Huntsville - their largest public art commission to date. The specifications: 60 feet tall, a 12 x 12 foot footprint, painted steel and aluminum, kinetic elements driven by wind. The lighting: a separate, programmable system designed and programmed by Happen Interactive, who built the animated light sequences keyed to the sculpture's form and the control system that runs them.

It went further than lookup. It correctly separated what's confirmed from what's intended, distinguishing the sculpture's actual artist from the studio that created this specific visualization - crediting Adam Buente and Project One Studio for the real, physical artwork, and Pixelhead Studio for the rendering itself. That distinction requires understanding the structure of a creative record, not just reading a name off a label.

“Nothing about the image changed. The only difference was a declaration embedded directly in the file.”

None of this exists in the pixels. A camera cannot photograph a commission history, a funding partnership, or an artist's stated intent. Run one had the same picture and reasoned its way to nothing. Run two had the exact same picture, carrying one declaration inside it, and reasoned its way to an entire verifiable record.

click to enlarge

What Actually Changed

Nothing about the photograph changed. No new caption was written. No webpage was built around it. No extra context was typed into the conversation. The only difference between the two runs was a structured, machine-readable declaration embedded directly in the file - the kind of record VISID creates when an image is stamped.

This is the practical distinction between declared and inferred information. An AI system inferring context from pixels alone is doing genuine, sophisticated work - but it is still guessing, openly and honestly, as ChatGPT's own first answer shows. An AI system reading a declared record isn't guessing at all. It's reporting.

Why This Matters

Most of what happens to an image after it's published is out of a creator's hands. It gets reposted, cropped, screenshotted, redistributed by people and platforms with no obligation to preserve who made it or why. A caption gets lost. A webpage disappears.

“The record in the index survives what the file wrapper doesn't.”

What doesn't have to disappear is a structured declaration embedded directly in the file itself, backed by a public, verifiable record. That's what VISID does: it gives an image a persistent identity that travels with it, so that when an AI system is asked what it's looking at, it has something real to read instead of something to guess at.

The gap between these two runs is the gap this is built to close.