Research Paper Analysis

The paper proposes the Tangram Construction Expression (TCE), a symbolic geometric framework that grounds tangram assemblies in exact, machine-verifiable coordinate specifications. This mitigates ambiguity from visual approximation and ensures geometric rigor. Each TCE instance includes components like instance_id, target_outline, initial_state, final_state, and adjacency_graph. Quantities are encoded as exact algebraic expressions in LATEX to prevent precision loss from floating-point representations. The final_state includes a transform_matrix for rigid motions (translation, rotation, reflection) of pieces.

TangramPuzzle features two complementary tasks: Outline Prediction (Task 1) requires models to infer global shapes from local components, selecting the correct silhouette from candidates. End-to-End Code Generation (Task 2) demands solving inverse geometric assembly problems, outputting a complete TCE in JSON format. Evaluation for Task 2 involves a hierarchical Constraint-based Evaluation Framework, checking for Syntax Error (TSE), Rigid Geometry Error (RGE), and Physical Error (PE). Shape similarity is measured by IoU and Hausdorff Distance. Human performance is also benchmarked to provide a realistic ceiling.

Experiments on various MLLMs (open-source and proprietary) reveal that models prioritize matching the target silhouette visually while often neglecting geometric constraints. This leads to distortions or deformations of pieces, or impermissible overlaps. Even top-tier models with high IoU scores often fail constraint validation (0% success in many cases for Task 2), highlighting a fundamental limitation in true compositional spatial reasoning. Gemini3-Pro stands out as an exception with robust geometric reasoning.