How to Test Multi-Turn AI Image Editing

How to Test Multi-Turn AI Image Editing

Test a multi-turn AI image editor by changing one variable at a time, naming what must stay fixed, saving every intermediate result, scoring unintended drift, and repeating the sequence in a fresh session.

A strong final image is not enough. Multi-turn editing is a control problem. The evaluator needs to know whether the requested change happened, whether unrelated content moved, and whether prior decisions survived the next instruction.

Step 1: Choose an input you can safely preserve

Use an image you own or have explicit permission to upload, transform, store, and publish. Avoid client material, private locations, confidential documents, children's images, and a stranger's face.

Google's current Gemini developer guide tells users to hold the necessary rights to uploaded images and not create material that deceives, harasses, or harms. Google image-generation guide Product terms, retention, human review, training use, and regional rules can change, so check the current service before each evaluation.

A useful test image contains one clear subject, several persistent details, readable text or a symbol, a textured background, directional light, and at least one object that can reveal accidental movement.

Step 2: Record the environment

The same brand can route to different models. Record the date, product, model label, model identifier when exposed, account tier, interface or API, region, settings, source-image dimensions, and any safety or watermark behavior.

Give the source and every output a stable identifier. Preserve original files rather than screenshots when the service permits. Compute a file hash if the evaluation must be audited later.

Step 3: Write the invariant contract

An invariant is something the editor must not change.

For a clothing edit, the contract might preserve face, hair, pose, hands, body geometry, pants, shoes, camera angle, crop, background objects, text, lighting direction, shadows, and color outside the garment.

The contract should be visible before the first request. If it is written after the output arrives, the evaluator can unconsciously ignore drift that makes the image look better.

flowchart LR
    A["Source and invariant contract"] --> B["One requested change"]
    B --> C["Save output and prompt"]
    C --> D["Score task success and preservation"]
    D --> E["Next single change"]
    E --> C
    D --> F["Fresh-session repeat"]

Step 4: Run a five-turn sequence

Turn zero is the untouched source.

In turn one, make a local attribute change. Change the shirt from white to navy while keeping every other invariant fixed.

In turn two, change the pattern while preserving the garment's shape and the rest of the image.

In turn three, replace that garment with a different garment. This tests a larger local edit without asking the system to rebuild the whole scene.

In turn four, ask the editor to restore the exact turn-two garment. This tests conversational memory and whether the system can return to an earlier state rather than approximate it.

In turn five, change one non-garment variable, such as the hand pose, while preserving the restored garment, identity, framing, and background.

Save the result after every turn. Do not hide a failed output by rerunning the same request until it looks good. If a retry is necessary, label it as another attempt.

Step 5: Score success and preservation separately

GIE-Bench separates functional correctness from preservation of content outside the intended region. Its reported comparison found that a model could follow the instruction well and still over-modify unrelated areas. GIE-Bench research record

That distinction produces a simple scorecard:

A zero on a critical invariant should remain visible even if the total score is high. A changed logo, face, legal label, or product feature can make an otherwise attractive output unusable.

[[How to Evaluate Identity and Detail Preservation in AI Image Editing]] expands the rubric and reporting rules.

Step 6: Compare each turn with two references

Compare an output with its immediate parent to see what the latest request changed. Then compare it with turn zero to see accumulated drift.

This catches a common failure. Each individual step may look close to the prior image while the fifth output has moved far from the original.

Use image overlays, crops, and pixel differences as inspection aids, not final truth. A small pixel change can be harmless color noise. A semantically important alteration can occupy very few pixels.

Google Research's EditInspector benchmark uses human-annotated dimensions such as accuracy, artifacts, visual quality, seamless integration, and common-sense consistency. Its authors also report that automated vision-language evaluators can struggle to assess edits comprehensively and may hallucinate when describing differences. EditInspector

Human review remains necessary.

Step 7: Reset and repeat

Run the same five-turn sequence in a new conversation with the same settings. If the service exposes a seed, record it. If it does not, treat variability as part of the product.

One successful chain shows that the workflow can work. Repeated chains begin to show how often it works under those conditions.

Do not compare two products from runs made months apart. Use the same date, source, prompts, sequence, output constraints, and scoring rules. Report refusals, timeouts, retries, and unusable outputs.

Step 8: Publish the whole chain

A credible report shows the source, full prompt sequence, every output, scores, test date, product and model identifiers, known account conditions, and rights boundary. It separates observation from preference.

Google's original Nano Banana launch specifically promoted multi-turn editing and preservation of the rest of the image. August 2025 launch record OpenAI has made the same kind of current product claim for subsequent edits in ChatGPT Images. ChatGPT Images product record

Those pages establish what the vendors claim their products are designed to do. The test establishes what happened in one documented sequence.

This protocol is Venture Step version 1. It is a practical editorial method, not a standardized scientific benchmark and not a high-stakes identity test.

AI assistance was used for research organization, drafting, and validation. Publication remains unauthorized.