CHROMA · A controlled benchmark for machine map understanding

Toward AI-Friendly Cartography:Understanding How Color Design Influences Foundation Model Spatial Reasoning on Sequential Choropleth Maps

CHROMA measures how hue, color ordering, and lightness contrast shape foundation models' spatial reasoning on sequential choropleth maps.

Read the paper ↗ Explore the benchmark

Yonghe Sun · Zhenjia Liu · Hua Liao · Wenjia Xu · Nai Yang · Weihua Dong · Zhiwei Wei

Randomized encodingOrdinal structure broken

ClassesMixed

Sequential encodingMachine-readable

LowHigh

+9.2accuracy points
from ordered color

5,760controlled maps

28,800reasoning tasks

21multimodal FMs

5task dimensions

The question

Cartography was designed for people.
Do its visual rules transfer to machines?

CHROMA turns three classical color-design principles into controlled experiments, holding geography, values, questions, and rendering constant while changing only the visual encoding.

Hue palette

Do different sequential hue families carry stable semantic advantages for foundation models?

Limited, inconsistent effect

Ordinal organization

Does preserving the familiar light-to-dark sequence help models compare and rank regions?

Large, systematic effect

Lightness contrast

How much class separation is enough for reliable machine-readable thematic values?

Asymmetric threshold effect

CHROMA framework from controlled map construction to benchmark evaluation — The complete CHROMA pipeline: controlled hypotheses, map construction, multi-level tasks, human validation, and evaluation of 21 models.

The benchmark

Five levels of map reasoning

Every controlled map is paired with questions that move from reading a single region to recognizing higher-order spatial patterns.

United StatesFranceGermanySwitzerland

Attribute identification

Match regions to thematic classes and classes to regions.

Spatial recognition

Resolve direction and adjacency between geographic regions.

Comparison

Compare thematic magnitudes between specified regions.

Ranking

Find local or global extrema and rank regions by value.

Pattern delineation

Recognize clusters, trends, and spatial structures.

4 countries4 spatial structures4–7 classes18 palettes

Examples of CHROMA choropleth maps under standard, randomized, high-contrast, and low-contrast conditions

Core results

Structure beats decoration.

Models care far more about whether colors preserve magnitude and remain separable than about which hue family a map uses.

H2 · Color ordering−9.2 pts

Breaking the sequence breaks reasoning.

Mean accuracy falls from 53.7% to 44.5% when class colors are randomized. Ranking suffers most: D4 drops by 24.6 points.

Sequential53.7

Randomized44.5

H1 · Hue4.3 pt range

No universally best hue.

Across 18 sequential palettes, mean accuracy ranges from 51.2% to 55.5%. The winner changes across models and task types.

51.255.5

H3 · Contrast−2.0 pts

Avoid compressed lightness.

Average accuracy changes from 55.3% at standard contrast to 55.8% at high contrast and 53.3% at low contrast.

55.8High55.3Standard53.3Low

Main experiment · Accuracy (%)

Performance by model and color condition

H2 compares sequential and randomized ordering. H3 compares standard, increased, and reduced lightness contrast.

Model	H2 · Color ordering		H3 · Lightness contrast
Model	Sequential	Randomized	Standard	High	Low
Qwen3.5-9B	68.6	53.6	70.8	70.9	67.6
Qwen3.5-4B	62.6	49.1	63.6	63.6	60.2
Qwen3.5-2B	49.5	41.4	51.2	52.0	49.1
Qwen3-VL-8B-Instruct	62.1	48.0	63.3	64.1	60.3
Qwen3-VL-4B-Instruct	58.8	45.8	59.7	60.2	56.7
Qwen3-VL-2B-Instruct	43.4	36.3	44.5	44.7	42.6
Qwen2.5-VL-7B-Instruct	51.0	41.6	52.2	53.1	50.1
Qwen2.5-VL-3B-Instruct	44.2	38.0	46.2	47.4	45.1
GLM-4.6V-Flash	64.3	51.9	66.5	66.6	62.7
Gemma3-12B-IT	35.7	35.3	36.8	36.7	36.2
Gemma3-4B-IT	29.6	29.7	29.6	29.4	29.8
InternVL3.5-8B	59.1	46.3	62.2	64.2	60.4
InternVL3.5-4B	58.6	47.0	59.9	60.9	57.0
InternVL3.5-2B	41.2	35.4	41.0	41.9	40.2
Kimi-K2.6	78.9	65.1	80.9	81.8	79.4
Qwen3.6-plus	44.3	41.1	46.9	47.7	44.7
Doubao-Seed-2.0-lite	47.8	44.9	49.5	50.1	49.0
MiMo-V2.5	49.4	38.3	50.7	51.3	49.4
ERNIE 5.0	40.4	37.2	43.3	43.1	43.2
Gemini-3.5-Flash	77.9	58.9	80.4	80.5	76.7
GPT-5.5	59.7	49.2	61.4	61.6	58.6
Average	53.7	44.5	55.3	55.8	53.3

For H1, averages across the 18 hue palettes range from 51.2% (Purples) to 55.5% (YlGn); the complete palette-by-model table is available in the paper.

Ordering effect across models

The effect is broad, not model-specific.

Most tested systems lose accuracy when the same palette is detached from its natural ordinal progression. Human readers remain much more robust.

Contrast effect across models

Losses are asymmetric.

Reducing contrast hurts consistently, but extra contrast beyond an already separable standard condition adds comparatively little.

Design implications

Three practical rules for AI-readable choropleth maps

Preserve darker-means-more

Use the familiar directional convention, not merely any monotonic ordering. Reversing the convention can be especially harmful for comparison and ranking.

Protect class separability

Avoid compressed lightness steps. Once adjacent classes are clearly distinguishable, further expansion offers diminishing returns.

Choose hue for the audience

Because no hue family is consistently best for machines, hue can prioritize human accessibility, semantics, and context without sacrificing a universal model preference.

Matched 2×2 ablation

Where do the errors actually come from?

The same maps, questions, values, and answer choices are tested while attribute values and spatial relations are independently supplied as clean text. This separates color-and-legend decoding, spatial reasoning, and their integration without introducing OCR confounds.

Neither supplied64.0%Full visual task

Region values67.6%Spatial relations inferred

Spatial relations57.9%Values decoded from map

Both supplied68.9%Text-supported ceiling

What it shows

Color is useful, but the bottleneck is compositional.

Supplying values mainly improves attribute identification, comparison, and ranking.
Supplying spatial relations mainly helps topology-dependent questions.
Supplying both performs best, revealing an additional integration cost between the two information streams.

Can models learn around it?

Fine-tuning helps—design still matters.

LoRA adaptation substantially raises overall accuracy, yet the relative sensitivity to randomized ordering and insufficient contrast remains. Better models and better maps are complementary, not interchangeable.

93.2%Qwen3.5-9B after LoRA
on sequential maps

Open resources

Read, reproduce, and build on CHROMA.

PDF

Research paper

Methods, analyses, robustness checks, and appendices.

↗ { }

Benchmark repository

Maps, JSON annotations, and evaluation records.

↗ DOI

Persistent record

Use the DOI to reference the current arXiv release.

↗

Citation

Cite CHROMA

@article{sun2026chroma,
  title   = {Toward AI-Friendly Cartography: Understanding How Color Design Influences Foundation Model Spatial Reasoning on Sequential Choropleth Maps},
  author  = {Sun, Yonghe and Liu, Zhenjia and Liao, Hua and Xu, Wenjia and Yang, Nai and Dong, Weihua and Wei, Zhiwei},
  journal = {arXiv preprint arXiv:2608.15736},
  year    = {2026},
  doi     = {10.48550/arXiv.2608.15736}
}