AI research | The Decoder

Matthias Bastian

May 28, 2024

Do large language models really need large context windows?

Some AI companies are marketing particularly large context windows for their models, which allow them to process a lot of data at the same time. But are they really the sensible next step?

Jonathan Kemper

May 26, 2024

TransAgents uses AI teamwork to tackle the complexities of literary translation

Matthias Bastian

May 22, 2024

Humans can barely distinguish AI-generated content from human-created content

Humans can hardly distinguish AI-generated content from real content

Matthias Bastian

May 22, 2024

Update

Anthropic's AI interpretability research shines a light into the black box of large language models

1/ Researchers at Anthropic have developed a special method to extract and analyze millions of features from the Claude 3 Sonnet language model. This allowed them to gain fascinating insights into the internal representations of the model. 2/ The method makes it possible to reconstruct activation patterns in the model's internal layers, allowing the researchers to identify concepts such as the Golden Gate Bridge or immunology that the model has learned. If the features are reinforced artificially, they strongly influence the generation. 3/ The extracted features cover an enormous range, from people and places to syntax elements in code and abstract concepts. Many features respond to both text and images. There is also evidence of a hierarchical arrangement of features.

Matthias Bastian

May 22, 2024

AI companies make progress on AI transparency, but sensitive issues remain shrouded in secrecy

A study by Stanford University shows that AI developers have revealed significantly more information about their models in the last six months. Nevertheless, there is still plenty of room for improvement.

Jonathan Kemper

May 20, 2024

EuroLingua aims to unite 45 European languages in a single open-source AI language model

Photo of the Barcelona Supercomputing Center.

Maximilian Schreiner

May 17, 2024

Meta's Chameleon AI model blends text and images, hinting at a future GPT-4o rival

Maximilian Schreiner

May 17, 2024

ORBIT-Surgical uses Nvidia's physics simulation and ray tracing to train surgical robots

Maximilian Schreiner

May 14, 2024

AI study sheds new light on the role of the hippocampus in learning

Maximilian Schreiner

May 14, 2024

BioHive-2 to speed up drug development by several years