Google AI Introduces TSMixer: An Advanced Multivariate Model for Long-Term Forecasting That Leverages Linear Model Features for High Benchmark Performance

In recent years, the importance of accurate time series forecasting has become paramount in many real-world applications. Whether predicting demand trends or predicting the spread of epidemics, the ability to make accurate predictions is invaluable. When it comes to multivariate time series estimation, two categories of models have emerged: univariate and multivariate. Univariate models focus on inter-series interactions, capturing trends and seasonal patterns in single-variable time series. However, recent research has found that advanced multivariate models, despite their promise, often fall short of simple univariate linear models in long-term forecasting benchmarks. This raises important questions about the effectiveness of cross-variate information and whether multivariate models can still hold their own when such information is not as beneficial.

Transformer-based architectures have emerged in the time series forecasting landscape in recent years, thanks to their exceptional performance in sequencing tasks. However, their performance in long-term forecasting benchmarks has raised questions about their effectiveness compared to simpler linear models. In light of this, the Google AI team has introduced a groundbreaking solution: Time-Series Mixer (TSMixer). Developed after careful analysis of the advantages of univariate linear models, TSMixer represents a significant leap forward. It leverages the power of linear models while efficiently incorporating cross-variate information, culminating in a model that performs on par with the best univariate models on long-term forecasting benchmarks.

An important difference between linear models and transformers is how they capture temporal patterns. Linear models use fixed, time-step-dependent weights to capture stable temporal patterns, making them exceptionally effective at learning such patterns. In contrast, transformers rely on attentional mechanisms with dynamic, data-dependent weights to capture dynamic temporal patterns and enable the processing of cross-variate information. The TSMixer architecture combines these two approaches, ensuring that it retains the capabilities of temporal linear models while harnessing the power of cross-variate information.

Metrics don’t lie, and in TSMixer’s case, the results speak volumes. When evaluated on seven popular long-term forecast datasets, including electricity, traffic, and weather, TSMixer showed significant improvement in mean square error (MSE) compared to other multivariate and univariate models. This shows that when designed with precision and insight, multivariate models can perform as well as their univariate counterparts.

Finally, TSMixer represents a watershed moment in the field of multivariate time series forecasting. By deftly combining the strengths of linear models and transformer-based architectures, it not only outperforms other multivariate models, but also stands shoulder to shoulder with state-of-the-art univariate models. As the field of time series forecasting evolves, TSMixer paves the way for more powerful and effective models that can revolutionize applications across various domains.

check Papers and Google Articles. All credit for this research goes to the researchers in this project. Also, don’t forget to participate Our 30k+ ML SubReddit, 40k+ Facebook community, Discord ChannelAnd Email newsletterWhere we share the latest AI research news, cool AI projects and more.

If you like our work, you will like our newsletter.