Google's MatCha is a foundation model for understanding charts

Google's MatCha is a foundation model trained for both chart de-rendering and mathematical reasoning. Chart de-rendering explores the reverse engineering of charts, plots, or graphics to reveal their underlying data table or code, while math reasoning seeks to solve question-based problems on textual mathematical datasets. By combining these tasks, MatCha significantly outperforms existing models for visual language understanding of charts. The researchers also proposed DePlot, a model built on top of MatCha for improved reasoning on charts through translation to tables.