Excarta

We recently published a preprint on Arxiv (https://lnkd.in/gHT_TXh3) that talks about HEAL-ViT, one of the AI weather models we've developed at Excarta. While the preprint has the details on the model, our blog post (https://lnkd.in/gTvWzDx7) provides a high-level overview of the key innovations. HEAL-ViT combines elements from two popular approaches in AI weather modeling: graph networks and transformers. We exploit the properties of the HEALPix mesh to adapt standard vision transformers to work on a spherical mesh. The result is a model that uses a spatially homogeneous latent representation, and attention mechanisms that are more computationally efficient than message-passing in graph networks. HEAL-ViT offers the RMSE/ACC performance now expected from AI weather models, in addition to improving on other models in metrics like bias and zonal energy spectra -- while using fewer features than other AI weather models. Just as impressively, HEAL-ViT is able to predict cyclone tracks accurately, as our blog post highlights with a case study on Hurricane Ian. Anyone looking at the weather forecasting space will know that AI weather models are already compelling, but there's still much to do. This makes it all the more important to be able to iterate and experiment quickly and cheaply. HEAL-ViT can be trained and fine-tuned in under a week on just 8 GPUs, making it much faster and cheaper to train than other AI weather models (8 A100GPUs for a week is ~$7,000 on GCP). This is an order of magnitude less than comparable models like Pangu-Weather (192 GPUs, 16 days) or GraphCast (32 TPUs, 4 weeks). Models based on HEAL-ViT are used in Excarta's operational weather forecasts (check out https://app.excarta.io/map to see our live forecasts visualized), also available through our APIs. If you're curious about our forecasts or our models -- get in touch :)