MinIO has launched AIStor Table Sharing, a new feature in MinIO AIStor that enables businesses to make on-premises data directly accessible to the Databricks platform via the Delta Sharing open protocol. This feature enables Databricks to access current on-premises datasets immediately for analytics and AI tasks, without the need for traditional data movement or replication.
Addressing On-Premises Data Gravity for Databricks
Many companies use Databricks for analytics and AI, but most critical data remains stored on-premises due to scalability, performance, cost, and compliance reasons.
Connecting on-premises data to Databricks traditionally involved complex ETL pipelines, data duplication in cloud storage, and separate governance, leading to slower insights, higher costs, operational risks, and maintenance challenges.
AIStor Table Sharing integrates Delta Sharing into the object store, allowing seamless, secure data sharing across platforms, clouds, and regions. By embedding Databricks’ open-source protocol, MinIO enables federated analytics on live data without duplicating data or restricting formats.
MinIO co-founder and co-CEO AB Periasamy highlighted that businesses shouldn’t need to transfer large datasets to analyze them. He described “data gravity” as a continuous challenge as AI technology extends across both on-premises and cloud settings. Periasamy stated that AIStor Table Sharing, powered by Delta Sharing, enables data to be accessed where it is stored, aiming to improve insight speed, reduce risk, and streamline operations for hybrid AI deployments.
Stephen Orban, SVP of Product Ecosystem & Partnerships at Databricks, pointed out that customers often ask for ways to manage and share data both in the cloud and on-premises. He characterized the partnership between Databricks and MinIO as part of an open-data-ecosystem strategy. The integration of MinIO’s Delta Sharing is intended to help businesses securely link on-premises data to Databricks without complex replication, which speeds up insights for hybrid workloads.
Built on AIStor Tables and Apache Iceberg
AIStor Table Sharing is based on AIStor Tables, MinIO’s Apache Iceberg v3-native foundation for modern data lakehouse architectures that operate at enterprise scale in on-premises and hybrid environments. AIStor Tables include:
– High-performance, S3-compatible MinIO object storage
– Integrated Iceberg table catalogs and metadata
– A REST API for table operations
– Built-in support for open data sharing standards
Since table definitions, governance policies, and sharing protocols are all managed within the same data storage system, businesses can directly define and publish table shares from AIStor without requiring an external catalog or sharing layers.
The platform consolidates structured and unstructured data into a single storage and metadata environment. AIStor functions as an AI data store, enabling analytics and GPU-based AI tasks across a single data layer. Its purpose is to simplify architecture, prevent data duplication, and decrease operational complexity.
AIStor Table Sharing extends this framework to Databricks environments, allowing users to access on-premises AIStor data as first-class tables while preserving performance, cost efficiency, and local administrative control.
Key Features
AIStor Table Sharing emphasizes open protocols and hybrid architectures. Key features include:
- Open Sharing, Natively Implemented: AIStor uses Delta Sharing 1.0 in MinIO to enable standards-based data sharing, boosting compatibility and reducing vendor lock-in.
- In-Place Analytics on On-Premises Data: Databricks accesses on-premises data in AIStor via Delta Sharing without replication, cutting costs while preserving performance, security, and compliance.
- Multi-Format Table Support: AIStor Table Sharing supports both Delta and Apache Iceberg tables, allowing companies to adopt or coexist with different open formats over time without changing the sharing method.
- Native Databricks Integration: AIStor Table Sharing, based on Delta Sharing, enables direct access to on-premises tables from Databricks workspaces. Clusters can treat these shared tables as native, extending cloud analytics to on-premises data without extra ingestion or connectors.
- Hybrid and Multi-Cloud Design: This feature supports on-premises, hybrid, and multi-cloud environments, enabling data to remain local while accessible from Databricks across regions.
Focus on Enterprise
MinIO and Databricks serve a shared enterprise customer base across industries like manufacturing, financial services, energy, retail, and logistics. These organizations usually generate large amounts of operational data in on-premises settings. Regulations, latency challenges, and cost considerations make it impractical to move all data to the cloud.
AIStor Table Sharing targets scenarios where customers want to leverage Databricks’ analytics and AI features directly on in-place datasets, not just selectively replicated subsets. By keeping data local while enabling access through an open sharing protocol, businesses can uphold existing compliance, auditing, and sovereignty requirements while consolidating analytics and AI processes on Databricks.
Flexibility and Scale
AIStor Table Sharing also illustrates how large organizations manage diverse data across multiple formats and platforms at scale. By supporting both Delta and Apache Iceberg tables, MinIO prevents customers from being locked into a single lakehouse framework or a proprietary sharing method.
All MinIO AIStor editions share the same binary and core architecture, with differences limited to licensed features, scale constraints, and support levels. This structure allows organizations to adopt AIStor Table Sharing at an appropriate entry point and scale capacity, performance, and features as their needs grow, without changing the underlying software.
Availability
AIStor Table Sharing is now generally available with MinIO AIStor.
Amazon