How to Evaluate AI Image Identity Preservation

How to Evaluate Identity and Detail Preservation in AI Image Editing

Evaluate an AI image edit by separating whether the request succeeded from whether the subject, defining details, geometry, text, background, and untouched regions remained correct. A recognizable image can still contain a critical preservation failure.

Identity preservation is not one score. It changes with the subject and the use.

For a person, identity may depend on facial structure, hair, skin details, body proportions, age cues, and distinctive accessories. For a pet, it may include muzzle shape, ears, coat markings, wrinkles, and posture. For a product, it may depend on geometry, controls, labels, color, dimensions, and brand marks.

Define the edit and the invariants first

Every evaluation begins with three artifacts: the source image, the requested change, and the invariant contract.

The requested change says what should move. The invariant contract says what must not.

flowchart TD
    A["Source image"] --> D["Edited output"]
    B["Requested change"] --> D
    C["Invariant contract"] --> D
    D --> E["Instruction-success review"]
    D --> F["Preservation review"]
    D --> G["Critical-error review"]

Without the contract, an evaluator can praise an attractive output that solved a different task.

Use a dimensioned rubric

Score each dimension from zero to four.

Zero means the requirement failed or the change is unusable. One means major drift. Two means a mixed result requiring repair. Three means minor drift that may be acceptable for a low-risk use. Four means no material issue was found at the review scale.

The score describes the reviewed output, not the model as a whole.

Keep instruction success separate from preservation

An image editor can perform the requested change and still damage the rest of the image.

GIE-Bench evaluates functional correctness and content preservation as separate dimensions. Its object-aware method masks the intended edit region and examines areas that should remain untouched. The authors report a trade-off in which strong instruction following can coexist with over-editing. GIE-Bench

That distinction is the foundation of edit locality.

For "change the shirt to navy," the shirt is the authorized region. A small color reflection on nearby skin may be physically reasonable. A different face, watch, background sign, or camera crop is not required by the instruction.

Identity depends on the authorized change

Do not penalize a requested transformation as identity drift.

If the task turns a dog into a felt plush, fur texture and anatomical realism are meant to change. The evaluator can still inspect muzzle proportion, ear placement, coat color, wrinkles, and expression. Those remaining features support continuity with the source.

If the task changes a person's hairstyle, the hair cannot also be an invariant. Facial geometry, age cues, skin details, and other non-target features can be.

The evaluator should write the authorized exemption before scoring.

Use critical-error overrides

An average score can hide a failure that makes the image unsafe or useless.

A changed medication label, product control, identity document, legal notice, logo, measurement, safety sign, child's face, or factual diagram can be critical even when it occupies a small part of the frame.

Record critical errors beside the score and set the disposition separately:

Do not let a high total convert a reject into an accept.

Compare at three scales

Begin with the whole image. Does the output preserve composition, lighting, perspective, and subject relationships?

Then inspect the subject. Compare face or object shape, defining attributes, hands, edges, accessories, and contact with the scene.

Finally inspect details. Read every visible word and number. Check logos, buttons, jewelry, garment closures, pet markings, reflections, shadows, and background objects.

A side-by-side view supports semantic comparison. A rapid flicker or transparency overlay can reveal geometric movement. Magnified crops help with small details. None of those views should replace judgment about the intended use.

Compare to the source and the prior turn

For a single edit, compare source and output.

For a multi-turn chain, compare each output with its parent and with the original. Parent comparison shows what the latest request changed. Original comparison reveals accumulated drift.

The 2025 Google launch emphasized character likeness and changing specific parts while preserving the rest. Nano Banana launch record A current product claim can guide what to test, but it cannot score the output for you.

Do not automate away the hard judgment

Automated similarity scores can help locate change. They do not fully understand which change matters.

Google Research's EditInspector uses human annotations across accuracy, artifacts, visual quality, integration, common sense, and change description. The authors report that current automated evaluators can miss issues and hallucinate descriptions. EditInspector

Complex editing benchmarks also break instructions into location, appearance, dynamics, and objects because one global similarity measure cannot explain every failure. CompBench

For sensitive identity, product, documentary, medical, legal, or safety uses, involve a qualified human reviewer with access to the source and intended context.

Report uncertainty and rights

The rubric cannot prove legal identity, authorship, consent, authenticity, or ownership. It is not facial recognition and should not be used to decide whether a person is who they claim to be.

Record who owns the source, who consented to the transformation and publication, which product processed it, where files were stored, what disclosure is required, and which provenance signal is present. Keep the unmodified source, prompt, output, and review together.

E082 provides a useful lesson. Dalton felt the first figurine looked like him despite odd eyes, thought the Lulu plush preserved the dog well, watched several style prompts fail, and then saw a clothing sequence preserve much more of the scene. Those observations can motivate the rubric. The missing images prevent a retrospective score.

This is Venture Step rubric version 1. It is designed for transparent editorial evaluation, not a scientific benchmark or high-stakes identity system.

AI assistance was used for research organization, drafting, and validation. Publication remains unauthorized.