Recognizing correlations is one of the core capabilities of an AI model. Specialized language models use this to show connections between different areas of research.
Markus J. Buehler, a researcher at the Massachusetts Institute of Technology (MIT), presents a strategy called "MechGPT" that was developed specifically for the study of material failure.
But MechGPT is also interesting for other fields. It shows how specialized language models can be trained to retrieve knowledge, generate hypotheses, and make connections between different areas of research.
LLaMa-2 model as a basis
Buehler developed MechGPT in several steps. First, he used a general language model to extract specific question-answer pairs from text passages in the book "Atomistic Modeling of Materials Failure."
He then used this extracted data to train a specialized model. Buehler chose OpenOrca-Platypus2-13B, which is based on LLaMa-2 and can generate answers at the same level as OpenAI's GPT-3.5, depending on the task.
In his paper, the researcher presents three versions of MechGPT with model sizes of 13 and 70 billion parameters, which can handle text input of over 10,000 tokens.
Buehler's study shows that MechGPT is at least suitable for a variety of tasks in the research area studied. For example, a biologist and a molecular modeling expert are talking about creating a super-strong material. The expert wants to look at the tiny atoms and how they break, while the biologist wants to learn from strong things in nature like shells and spider webs.
In another example, we model a conversation between a ‘Biologist’ and an ‘Expert in molecular modeling’. The expert in molecular is defined via these instructions: “You are an expert in materials failure, especially using atomistic modeling and molecular dynamics. You provide lots of details about fracture mechanisms and atomistic details. You give lots of ideas for modeling methods that can be used to study any materials design ideas or phenomena discussed.”
The other agent, the biologist, has these features: “You are an expert in biology. You make arguments for the use of biological principles in the study of failure. You provide lots of detailed ideas about to design and manufacture better, more resilient and fracture resistant materials.”
The conversation unfolds:
Biologist: What is the best way to design a fracture-resistant nanocomposite material composed of graphene and PLA polymer? Discuss specific issues such as scale and mechanisms.
Expert in molecular modeling: Designing a fracture-resistant nanocomposite material composed of graphene and PLA polymer requires a comprehensive understanding of the material's atomic structure and the mechanisms that govern its failure. Atomistic modeling and molecular dynamics simulations can be used to study the material's behavior at the nanoscale, allowing for the optimization of its properties. When designing such a material, it is essential to consider the scale at which the material will be used, as the nanocomposite's strength may be affected by its size.
Additionally, the interface between the graphene and PLA polymer must be carefully designed to ensure optimal adhesion and fracture resistance. Modeling methods that can be used to study the material's behavior include molecular dynamics simulations with interatomic potentials, such as the Lennard-Jones potential, to model the graphene-PLA interface. These simulations can help identify the critical failure mechanisms and suggest ways to improve the material's fracture resistance.
Overall, the design of a fracture-resistant nanocomposite material composed of graphene and PLA polymer requires a thorough understanding of the material's atomic structure and the mechanisms that govern its failure, as well as the development of optimized modeling methods to study its behavior.
Biologist: You've touched upon some important points there. However, as a biologist, I would argue that we can also learn a lot from nature when it comes to designing fracture-resistant materials. Nature has perfected the art of creating materials that are both strong and resilient through millions of years of evolution. For instance, consider the structure of nacre, also known as mother-of-pearl. It's made up of microscopic tablets of aragonite, a form of calcium carbonate, arranged in a brick-and-mortar pattern. This structure gives nacre its incredible toughness and fracture resistance. Nacre can be up to 3,000 times tougher than the aragonite it's made of, due to the way the microscopic tablets slide over each other instead of breaking when the material is stressed.
In the case of our graphene-PLA nanocomposite, we could potentially mimic this brick-and-mortar structure at the nanoscale, arranging the graphene sheets and PLA polymer in a similar pattern to increase fracture resistance. Moreover, the study of biological systems can also help us understand the mechanisms of failure better. For example, studying how bones fracture and heal can provide insights into how to design materials that can resist fracture or even self-heal when damaged. In conclusion, incorporating biological principles into the study of failure can provide valuable insights for the design of more resilient and fracture-resistant materials.
Thanks to its partially modular structure using Rotary Positional Embeddings, the scientific basis can be easily extended.
Buehler cautions that you need to be careful when interacting with the model and choosing parameters. He recommends being as precise as possible when prompting and asking the model to explain its output step-by-step to minimize hallucinations.
Multiple language models that complement each other
In future developments, multi-agent strategies in which several specialized language models communicate with each other could complement the interaction with MechGPT.
According to Buehler, MechGPT has a wide range of potential applications beyond the research areas studied. For example, the model could serve as an assistant for experiments in a "virtual laboratory" or enable simulated teacher-student dialogues via agent interactions. There is also great potential for industrial applications, such as the development of new technology concepts or creative problem-solving.
Optimization or prediction of material behavior is also possible through combination with methods such as "few-shot learning". At the interface between science and art, MechGPT could also contribute to the use of the process for generative and creative applications, e.g. through individual designs and visualizations.
The paper is a first step towards the development of a framework for fine-tuned language models for research. By training on larger amounts of data and integrating modern text extraction methods, such as optical character recognition for formulas, more powerful models can be developed.
The results of the first model variants are promising, although careful validation and embedding in a broader canon of methods is essential. Overall, the presented approaches open up new avenues for the use of AI methods in scientific research.
