Updates

TileDB has always supported NumPy-like slicing across multiple dimensions. These dimensions are defined in advance when creating TileDB arrays. Now, users have the option to further refine results at query-time by pushing down attribute filtering to the storage engine. Filter by any number of attribute values contained within the array.

This release adds a new configuration parameter to specify the server-side encryption algorithm to use when working with TileDB arrays on Amazon S3. Valid options are aes256 (Amazon S3-managed encryption keys) and kms (AWS Key Management Service). In a related update, this release also includes AWS IAM AssumeRole support with automatic token refresh.

NumPy datetime data types have long been supported as dimensions in sparse TileDB arrays. This release adds support for time data types (without associated dates) as both dimensions and attributes.

This release adds a new Hilbert layout for efficient space-filling-curve ordering of cells. It also fixes some previous issues with partitioning.

The TileDB adapter for Apache Arrow now supports zero-copy import/export for fast data exchange and efficient use of memory.

Working with missing data is now more efficient in TileDB and better aligns with modern dataframe conventions. Both fixed and variable-length attributes now support nullables. By default, fill values for nullable attributes will be null.

This release adds TileDB support for read-ahead caching on cloud-storage backends and upgrades cloud provider SDKs to their latest versions within TileDB.

Version 2.1 adds options for configuring concurrency levels and provides appropriate defaults.

Run Jupyter notebooks in seconds right from TileDB Cloud and jumpstart your work with examples on common use cases.

This release introduces task graphs to run parallel user-defined functions -completely serverless. Enjoy ease-of-use, superior scalability, and performance while eliminating idle compute costs.

TileDB 2.0 adds heterogeneous and string dimensions, now fully supporting dataframe use cases.

The R API in TileDB has been revamped for better performance and ease-of-use and a key building block for integrations to R packages such as tidyverse and Bioconductor.

TileDB 2.0 adds support for Google Cloud Storage and Azure Blob Storage to the existing AWS S3 support.

Today we are releasing TileDB Cloud, a cloud service that allows you to register and share your TileDB arrays with anyone in the cloud. You can define access policies and audit all access using simple and intuitive array semantics.

TileDB Cloud allows you to perform serverless SQL and Python UDFs, avoiding the deployment hassle and enjoying tremendous scalability.

TileDB Cloud comes with a pay-as-you-go model, allowing you to greatly reduce your operational costs.

We are releasing an open-source genomics customization called TileDB-VCF, which allows you to store arbitrarily large gVCF datasets as sparse 2D arrays in TileDB. With TileDB-VCF, you can solve the N+1 problem, achieving linear storage scalability, regardless your dataset size and number of updates.

TileDB-VCF reduces your collection of single-sample BCF files by 40%, leads to fast and scalable analysis on the cloud, and allows you to take advantage of the Data Science ecosystem via TileDB’s numerous integrations.

You can now enjoy the powerful integration of TileDB with MariaDB as an embeddable library, without having to install and maintain a MariaDB server, thus substantially simplifying your software builds.

We added TileDB as a pluggable storage engine to MariaDB. You are now able to run fast SQL queries via your MariaDB server on TileDB arrays, stored either on premises or on the cloud (e.g., AWS S3).

TileDB is integrated with PDAL, GDAL, and Rasterio, providing you a way to store your geospatial data in a single, cloud-optimized format.

We added support for key-value array metadata that you can attach to any array.

We removed KV objects that used to implement key-value functionality. We will soon introduce support for string dimensions, which will realize a full-fledged key-value store via TileDB arrays, allowing multi-dimensional and prefix-based string search.