Artificial intelligence is changing the way businesses store and access data. This is because traditional data storage systems were designed to process simple commands from a few users at a time, whereas today’s AI systems with millions of agents need to access and process massive amounts of data continuously and in parallel.
Traditional storage systems now have many layers of complexity, slowing down AI because data has to pass through many layers before reaching the GPU - the graphics processing unit, considered the "brain cell" of AI.
Cloudian — co-founded by Michael Tso (of MIT) and Hiroshi Ohta — is helping data storage keep up with the AI revolution. The company has developed a scalable enterprise storage system that helps data flow smoothly between storage and AI models.
The system reduces complexity by applying parallel computing to storage, consolidating AI and data functionality onto a single parallel processing platform capable of storing, retrieving, and processing large-scale datasets, with high-speed direct connectivity between storage and both GPUs and CPUs.
Cloudian's integrated compute-storage platform simplifies building AI tools at commercial scale, while providing enterprises with a storage infrastructure that can keep up with the AI explosion.
“One thing people often forget about AI is that it’s all about data,” says Tso. “You can’t get a 10% increase in AI performance with 10% more data, and even 10 times more data is not enough—you need 1,000 times more data. Storing data in a way that’s easy to manage, and embedding computation right there so you can do it as it comes in, without having to move it—that’s where the industry is going.”
Object Storage and AI
Currently, Cloudian’s platform uses an object storage architecture, where all types of data — documents, videos , sensor data — are stored as single objects with metadata. Object storage can manage massive amounts of data in a flat structure, making it ideal for unstructured data and AI systems, but previously it was impossible to send data directly to an AI model without first copying it to the computer’s memory — causing latency and wasting energy.
In July, Cloudian announced that it had expanded its object storage system with a vector database, which stores data in a form that is immediately available for AI use. As data is ingested, Cloudian computes the vector form of the data in real time to support AI tools such as recommendation engines, search, and AI assistants.
Cloudian also announced a partnership with NVIDIA to have its storage system work directly with its GPUs. Cloudian said the new system enables faster AI processing and lower computing costs.
“NVIDIA approached us about a year and a half ago because GPUs are only useful when there is data to feed them,” Tso said. “Now people realize that it is easier to bring AI to the data than to move huge volumes of data. Our storage system has a lot of AI capabilities built in, so we can pre-process and post-process the data close to where we collect and store it.”
AI Priority Storage
Cloudian is helping approximately 1,000 businesses worldwide get the most value out of their data, including large manufacturers, financial institutions, healthcare facilities, and government agencies.
For example, Cloudian’s storage platform is helping a major automaker use AI to determine when each of its manufacturing robots needs maintenance. Cloudian also partners with the U.S. National Library of Medicine to store research papers and patents, and with the National Cancer Database to store tumor DNA sequences—rich datasets that AI can process to aid in the development of new treatments or discoveries.
“GPUs are a great enabler,” says Tso. “Moore’s Law doubles computing power every two years, but GPUs can parallelize tasks on a chip, chain multiple GPUs together, and go beyond Moore’s Law. This scale is pushing AI to new levels of intelligence, but the only way to get GPUs to do their best is to feed data at the same rate as their computing power—and the only way to do that is to remove all the layers of intermediary between the GPU and your data.”
(According to MIT)
Source: https://vietnamnet.vn/cloudian-dua-du-lieu-den-gan-ai-hon-bao-gio-het-2433241.html
Comment (0)