Not too big: Machine learning tames huge data sets

同样有效的笔记本电脑和超级计算机,the highly scalable algorithm solves hardware bottlenecks that prevent processing information from data-rich applications in cancer research, satellite imagery, social media networks, national security science and earthquake research, to name just a few.

"We developed an 'out-of-memory' implementation of the non-negative matrix factorization method that allows you to factorize larger data sets than previously possible on a given hardware," said Ismael Boureima, a computational physicist at Los Alamos National Laboratory. Boureima is first author of the paper inThe Journal of Supercomputingon the record-breaking algorithm. "Our implementation simply breaks down the big data into smaller units that can be processed with the available resources. Consequently, it's a useful tool for keeping up with exponentially growing data sets."

"Traditional data analysis demands that data fit within memory constraints. Our approach challenges this notion," said Manish Bhattarai, a machine learning scientist at Los Alamos and co-author of the paper. "We have introduced an out-of-memory solution. When the data volume exceeds the available memory, our algorithm breaks it down into smaller segments. It processes these segments one at a time, cycling them in and out of the memory. This technique equips us with the unique ability to manage and analyze extremely large data sets efficiently."

The distributed algorithm for modern and heterogeneous high-performance computer systems can be useful on hardware as small as a desktop computer, or as large and complex as Chicoma, Summit or the upcoming Venado supercomputers, Boureima said.

"The question is no longer whether it is possible to factorize a larger matrix, rather how long is the factorization going to take," Boureima said.

The Los Alamos implementation takes advantage of hardware features such as GPUs to accelerate computation and fast interconnect to efficiently move data between computers. At the same time, the algorithm efficiently gets multiple tasks done simultaneously.

Non-negative matrix factorization is another installment of the high-performance algorithms developed under the SmartTensors project at Los Alamos.

In machine learning, non-negative matrix factorization can be used as a form of unsupervised learning to pull meaning from data, Boureima said. "That's very important for machine learning and data analytics because the algorithm can identify explainable latent features in the data that have a particular meaning to the user."

The record-breaking run

In the record-breaking run by the Los Alamos team, the algorithm processed a 340-terabyte dense matrix and an 11-exabyte sparse matrix, using 25,000 GPUs.

"We're reaching exabyte factorization, which no one else has done, to our knowledge," said Boian Alexandrov, a co-author of the new paper and a theoretical physicist at Los Alamos who led the team that developed the SmartTensors artificial intelligence platform.

Decomposing or factoring data is a specialized data-mining technique aimed at extracting pertinent information, simplifying the data into understandable formats.

Bhattarai further emphasized the scalability of their algorithm, remarking, "In contrast, conventional methods often grapple with bottlenecks, mainly due to the lag in data transfer between a computer's processors and its memory."

“我们还展示了你不一定需要大的computers," Boureima said. "Scaling to 25,000 GPUs is great if you can afford it, but our algorithm will be useful on desktop computers for something you couldn't process before."

The funding:This research was funded by DNN R&D and by the Laboratory Directed Research and Development program at Los Alamos National Laboratory.

LA-UR-23-29923