When a Crystal’s Shape Matters, Bigger Language Models Still Miss the Point

A material is more than a list of ingredients. Knowing that a crystal contains silicon, oxygen or iron is useful, but the way its atoms are arranged can determine whether the material bends, conducts heat or remains stable. That difference between what is present and where it sits is the focus of a new study testing whether large language models can predict material properties from structural data.

Researchers writing in Nature Computational Science report that the language models they tested learned categorical patterns, such as which types of atoms occur, but struggled to use coordinate information about positions and geometry. Their experiments included synthetic tasks designed to isolate those contributions and benchmark tests involving shear modulus, bulk modulus and formation energy.

Scale did not make geometry disappear

The team tested language models up to 70 billion parameters and datasets containing as many as two million structures. In the tasks studied, increasing model size or training data did not remove the difficulty. The researchers found that specialised geometric approaches, including graph neural networks, generally performed better when the prediction depended on spatial structure.

The result is a useful check on the rush to apply one model family to every scientific problem. Language models can be effective at text, and can help researchers search, summarise or extract information from scientific writing. But a model that reads prose well is not automatically the best tool for three-dimensional relationships between atoms.

What the study does not prove

The findings are bounded by the experiments. The authors tested selected properties and model architectures, including BERT-based and Llama-2 systems. They note that models with explicit geometric inputs or other multimodal designs were not covered. The study therefore does not show that language models are useless in materials science. It shows that the tested text-oriented models did not reliably capture spatial information in the tested representations.

This distinction matters because material discovery can be expensive. If a model confidently predicts a property from the wrong signal, researchers may spend time and resources pursuing an unpromising candidate. A benchmark that exposes what a model has learned can prevent a fluent answer from being mistaken for a physically grounded one.

The practical lesson is to match the tool to the data. Use language models where language and composition dominate. Use geometric models when spatial relationships are central, and test hybrid methods when both matter. In science, choosing a more specialised model is not a step backward. It is a way of respecting the structure of the question.

Sources

• The open-access research paper and its benchmark results

Image credits

Cover image: Max Planck Institute for Solid State Research, microscopy image from separate quantum-materials work. In-article image: Alampara, Miret and Jablonka, Nature Computational Science, 2026, Figure 1.