Quobyte Releases Hadoop Native Driver to Unlock the Power of Enterprise Analytics, Machine Learning, Streaming, and Real-Time Applications

Update allows distributed Hadoop data processing framework to access small files, crucial for its ability to manage the current wave of real-time data-driven applications.

Quobyte® Inc., the leading developer of scale-out deploy-anywhere software-defined storage (SDS), today announced the availability of its Hadoop Driver. Quobyte's new native driver for Hadoop addresses the limitations of the Hadoop Distributed File System's (HDFS) high-capacity design within the enterprise. The new native driver brings significant benefits in optimizing Hadoop clusters for a much wider range of applications and workloads, and true file system sharing across object storage and applications.

"Today's analytics solutions allow enterprises to extract important insight from large volumes of data, but with the increasing prevalence of AI and machine learning in data analytics applications HDFS's batch processing limitations have been exposed," said Bjӧrn Kolbeck, CEO of Quobyte. "By deploying Quobyte's native Hadoop/HDFS driver, enterprises can now seamlessly share large amounts of file data with high performance across Hadoop/analytics, machine learning, and any Linux or Windows application."

Hadoop is a powerful framework for big data processing, distributed across large amounts of commodity hardware within the enterprise. It has proven critical in helping enterprises manage the exponential growth of data, but in today's data-driven environment, it lacks the ability to manage the real-time requirements from the new breed of analytic applications. Engineers have been working hard, with limited success, to enable small file processing by using NFS connectors.

Quobyte now offers the ability to seamlessly integrate small file processing into the Hadoop Framework, without the need for a new file system, to deliver the speedy and scalable file I/O that is needed for modern analytic and real-time applications, while integrating well with container technology.

Accelerated by the rapid evolution of analytic and real-time applications, Gartner predicts that by 2024, 50% of global storage capacity will be deployed as SDS, either on-premises or on the public cloud (up from less than 15% in 2020). With this announcement, Quobyte and Hadoop bring together the ability to run on any distributed commodity x86 hardware, which unlocks the benefits of Hadoop while removing the storage limitations of HDFS and NFS, ensuring these modern workloads can run efficiently on Hadoop clusters. 

In addition, the new native driver removes security complexities by leveraging an alternative to HDFS that allows for encryption at both the storage and networking levels.

Quobyte's deploy-anywhere software is truly hardware agnostic, running on both bare metal and the public cloud. For more information about the Free Edition of Quobyte for use in their data center with up to 150TBs of capacity, visit https://www.quobyte.com/

Follow Quobyte




About Quobyte -

Founded in 2013 by former Google employees and high-performance computing experts, Quobyte is dedicated to bringing HPC scale-out and hyperscaler operations to the enterprise. Backed by over a decade of research, Quobyte's next-generation parallel file system core serves low-latency and high-throughput workloads within a single system. Quobyte offers enterprise IT the benefits of modern storage for generation scale-out with unlimited performance, automatic transparent failover, and non-disruptive updates. Please visit www.quobyte.com for more information.

Carmen Mantalas

Source: Quobyte Inc.