Researchers have discovered how to build reliable Machine Learning models that can understand complex equations in real-world situations whilst using less training data than expected.
Researchers from the University of Cambridge and Cornell University have discovered that for partial differential equations – a class of physical equations that describe how things in the natural world evolve in space and time – Machine Learning models can produce reliable results even when they are provided with limited data.
The study, ‘Elliptic PDE learning is provably data-efficient,’ was published in the Proceedings of the National Academy of Sciences.
The results could be useful for constructing more reliable and cost-efficient Machine Learning models for applications such as engineering and climate modelling.
How little data is needed for training reliable Machine Learning models?
Most Machine Learning models require large amounts of training data before they can return accurate results. To train the model, a human will usually annotate a large volume of data, such as a set of images.
“Using humans to train Machine Learning models is effective, but it’s also time-consuming and expensive,” said first author Dr Nicolas Boullé, from the Isaac Newton Institute for Mathematical Sciences.
“We’re interested to know exactly how little data we actually need to train these models and still get reliable results.”
Other researchers have been able to train reliable Machine Learning models with a small amount of data and get excellent results. However, how this was achieved has not been well-explained.
The team designed their AI models with partial differential equations
The team focused on partial differential equations (PDEs).
“PDEs are like the building blocks of physics: they can help explain the physical laws of nature, such as how the steady state is held in a melting block of ice,” said Boullé, who is an INI-Simons Foundation Postdoctoral Fellow.
“Since they are relatively simple models, we might be able to use them to make some generalisations about why these AI techniques have been so successful in physics.”
Why are PDEs useful in designing AI models?
The team discovered that PDEs that model diffusion have a structure that is useful for designing AI models.
“Using a simple model, you might be able to enforce some of the physics that you already know into the training data set to get better accuracy and performance,” said Boullé.
To build mathematical guarantees into the model, the researchers constructed an efficient algorithm for predicting the solutions of PDEs under different conditions. This was done by exploiting the short and long-range interactions happening.
From this, the team was able to determine how much training data was required to result in a reliable Machine Learning model.
“It depends on the field, but for physics, we found that you can actually do a lot with a very limited amount of data,” said Boullé.
“It’s surprising how little data you need to end up with a reliable model. Thanks to the mathematics of these equations, we can exploit their structure to make the models more efficient.”
The techniques will help data scientists design new Machine Learning models
The researchers believe that their techniques will allow data scientists to open the ‘black box’ of many Machine Learning models and design new ones that can be interpreted by humans.
However, further research is still required.
“We need to make sure that models are learning the right things, but Machine Learning for physics is an exciting field – there are lots of interesting maths and physics questions that AI can help us answer,” said Boullé.
