Starburst MemSQL connector#
The following improvements are included:
The additional features of the connector require a valid Starburst Enterprise Presto license, unless otherwise noted.
The connector includes a number of performance improvements, detailed in the following sections.
This feature is available for free, and does not require a valid license.
The MemSQL connector supports table and column statistics to improve query processing performance based on the actual data in the data source.
The statistics are collected by MemSQL and retrieved by the connector.
Table-level statistics are based on MemSQL’s
MemSQL can automatically update its table and column statistics. In some cases, you may want to force a statistics update, for example after creating new columns or after changing data in the table. You can do that by executing the following statement in MemSQL Database:
ANALYZE TABLE table_name;
MemSQL statistics are estimates, and Presto and MemSQL may use statistics information in different ways. For this reason, the accuracy of table and column statistics returned by the MemSQL connector might be lower than that of others connectors.
Improving statistics accuracy
You can improve statistics accuracy and access column-level statistics with
histogram statistics. Column-level statistics are based on MemSQL’s column
INFORMATION_SCHEMA.ADVANCED_HISTOGRAMS table, if available. If
that table is not available, the information is based on the
INFORMATION_SCHEMA.OPTIMIZER_STATISTICS table instead.
ADVANCED_HISTOGRAMS table includes additional stats such as the
MAX values for a column, which are not available in the
LEGACY_HISTOGRAMS table. MemSQL’s
ADVANCED_HISTOGRAMS feature requires
version 6.5 or later, and a
cardinality_estimation_level >= 6.5.
To check for these conditions and determine whether
available to you, execute the following statement in MemSQL:
SELECT COLUMN_NAME, IF(RANGE_STATS=1, true, false) as histograms_available, IF(ADVANCED_HISTOGRAMS=1, 'Advanced', 'Legacy') as histogram_type FROM INFORMATION_SCHEMA.OPTIMIZER_STATISTICS WHERE DATABASE_NAME = 'db' AND TABLE_NAME = 'table';
If they are available in your MemSQL version, use the following statement to
ANALYZE TABLE table_name COLUMNS ALL ENABLE;
Refer to the MemSQL documentation for information about options, limitations and additional considerations.
The connector supports pushdown for the following aggregate functions:
Dynamic filtering is enabled by default. It causes the connector to wait for dynamic filtering to complete before starting a JDBC query.
You can disable dynamic filtering by setting the property
dynamic-filtering.enabled in your catalog properties file to
Caching table projections#
The connectors supports table scan redirection to improve performance and reduce load on the data source.