3.3. Data Source Management#

Data sources define the connection of Presto to the various external databases and other systems, that are queried by the Presto user. With Mission Control the creation and management of these data sources is available via a simple user interface.

Create and edit a data source#

The data source management is suitable to create, update, and delete these data sources. Any user can access the data source management by clicking on the Data Sources item in the left hand panel.

The data source management view displays a grid of all existing data sources.

Press the + symbol on the bottom right corner to add a new data source.

The following data sources can be configured with a rich user interface for the necessary properties.

  • Amazon S3
  • Amazon Redshift
  • Apache Kafka
  • Azure Storage
  • Delta Lake
  • HDFS
  • Microsoft SQL Server
  • MongoDB
  • MySQL
  • Oracle
  • PostgreSQL
  • Teradata

Creation and configuration of the data source and the related properties is performed in a tabbed user interface for basic information, connection details, advanced configuration and others. Specific sections differ based on the data source type.

If you can not find your desired data source in the list, try the custom data source configuration.

Data source configuration creates a catalog property file, which configures a specific connector and the relevant properties.

Clicking on a data source name, or using the Edit link in the actions drop down, allows you to edit the data source.

As a next step you can move on to create a cluster, in which the data source is used and available for Presto users to query against.

Data source properties#

Data sources are defined by the following properties:

Type:

Defines the type of external data source to connect to. Type is closely related to the connector used to access the data in the external data source.

Name:

The name of the data source. A well chosen name is important, since it details the data source to the user in Presto. For example, your data source is configured to access a specific database on a specific PostgreSQL server instance. Choosing a name postgresql is therefore not really sufficient. It can be more useful for your users to choose a name of postgresql-customerdb. Potentially, if your users do not really have to know anything about the underlying database, it can also be appropriate to just use a name such as customerdb.

Description:

The description allows you to compose a short paragraph that provides more details about the data source than the name alone.

Connection properties:

After choosing the data source type the most important, required properties to connect to the data sources are displayed. These include properties required by connector as well as properties that enable connection to the specified data source, such as access keys.

Data source types and connectors#

Data source types define the connector Presto uses for the specific data source. The connector in turn, defines the configuration properties to use.

Refer to the following table to determine what connector configuration properties are used for a specific data source type:

Data source type to connector mapping#
Data source type Connector
Amazon S3 Starburst Hive Connector
Amazon Redshift Redshift Connector
Apache Kafka Starburst Kafka Connector
Azure Storage Azure Storage Support for the Hive Connector
Delta Lake Starburst Delta Lake Connector
HDFS Starburst Hive Connector
Microsoft SQL Server Starburst SQL Server Connector
MongoDB MongoDB Connector
MySQL Starburst MySQL Connector
Oracle Starburst Oracle Connector
PostgreSQL Starburst PostgreSQL Connector
Teradata Starburst Teradata Connector

Custom data sources#

Custom data sources allow you to configure a catalog with a connector that Mission Control does not support. Only use this feature if you have expert knowledge of Presto connectors and catalog configuration.

You can use this interface in the following situations:

  • Configure a catalog using a connector with no related data source support in Mission Control.
  • Manually control the content of the catalog file as an alternative configuration method.
  • Configure a catalog for a custom connector.

Access the configuration with the following steps:

  • Press the + symbol on the bottom right corner to add a new data source.
  • Click on the Don’t see your data source? in the notes section beside the list of data source types
  • Click on the Add one manually link.
  • Confirm that you understand the risk.
  • Configure catalog name, description and connector name.
  • Add the content property definitions for the catalog file in the text input.
  • Press Create to finish.

The supported properties differ for each connector and are identical to the properties used in the data source configuration file, documented with the connectors.