6.5. Presto Kubernetes resource

Presto Kubernetes resources constitute particular Presto clusters. Each such a resource represents a single Presto cluster. A Presto Kubernetes resource YAML allows specifying various properties related to the Presto server, Presto connectors and Kubernetes. Properties are defined in the spec section of Presto Kubernetes resource YAML. The following snippet shows the structure of a Presto Kubernetes Resource YAML including all available properties (with defaults). When defining your particular Presto cluster resource you need to specify only the properties with non-default values.

apiVersion: starburstdata.com/v1
kind: Presto
metadata:
  name: presto-cluster-name
spec:
  nameOverride: ""
  environment: ""
  additionalJvmConfigProperties: ""
  additionalCatalogs: {}
  additionalEtcPrestoTextFiles: {}
  additionalEtcPrestoBinaryFiles: {}
  licenseSecretName: ""
  imageNamePrefix: ""
  additionalBootstrapScriptVolume: {}

  prometheus:
    enabled: false
    additionalRules: {}

  service:
    type: ClusterIP
    name: ""
    additionalSpecProperties: {}
    nodePort: 31234

  image:
    name: starburstdata/presto
    pullPolicy: IfNotPresent
    tag: 312-e.1.k8s-0.7

  memory:
    nodeMemoryHeadroom: 2Gi
    xmxToTotalMemoryRatio: 0.9
    heapHeadroomPerNodeRatio: 0.3
    queryMaxMemory: 1Pi
    queryMaxTotalMemoryPerNodePoolFraction: 0.333

  coordinator:
    cpuLimit: ""
    cpuRequest: 16
    memoryAllocation: 60Gi
    nodeSelector: {}
    affinity: {}
    additionalProperties: ""

  worker:
    count: 2
    autoscaling:
      enabled: false
      minReplicas: 1
      maxReplicas: 100
      targetCPUUtilizationPercentage: 80
    deploymentTerminationGracePeriodSeconds: 7200 # 2 hours
    cpuLimit: ""
    cpuRequest: 16
    memoryAllocation: 100Gi
    nodeSelector: {}
    affinity: {}
    additionalProperties: ""

  readinessProbe:
    initialDelaySeconds: 5
    periodSeconds: 5
    timeoutSeconds: 15

  livenessProbe:
    initialDelaySeconds: 300
    periodSeconds: 300
    failureThreshold: 1
    timeoutSeconds: 15

  hive:
    metastoreUri: ""
    awsSecretName: ""
    googleServiceAccountKeySecretName: ""
    additionalProperties: ""
    internalMetastore:
      mySql:
        jdbcUrl: ""
        username: ""
        password: ""
      postgreSql:
        jdbcUrl: ""
        username: ""
        password: ""
      internalPostgreSql:
        enabled: false
        image:
          name: postgres
          pullPolicy: IfNotPresent
          tag: 9.6.10
        storage:
          className: ""
          size: 1Gi
          claimSelector: {}
        memory: 8Gi
        cpu: 2
        nodeSelector: {}
        affinity: {}
      s3Endpoint: ""
      image:
        name: starburstdata/hive-metastore
        pullPolicy: IfNotPresent
        tag: k8s-0.3
      memory: 8Gi
      cpu: 2
      nodeSelector: {}
      affinity: {}

General properties

Property name Example Description
nameOverride nameOverride: presto-cluster Presto Operator by default assigns a unique name to various Kubernetes resources (e.g. Services). The name consists of the cluster name and a unique suffix. Use this property to assign a static name instead.
environment environment: production The name of the Presto cluster environment.
licenseSecretName licenseSecretName: license_secret Name of a Kubernetes Secret that contains a Presto license file. The license file within the secret should be named signed.license.
imageNamePrefix imageNamePrefix: gcr.io/project-name/ Specifies prefix of Docker image names used by the Presto cluster. This property enables using a private Docker registry.
image
image:
  name: starburstdata/presto-customized
  pullPolicy: IfNotPresent
  tag: 312-e.1.k8s-0.7
Image section allows you to specify a custom Docker image to be used by Presto cluster.
readinessProbe
readinessProbe:
  initialDelaySeconds: 5
  periodSeconds: 5
  timeoutSeconds: 15
Properties of Presto Coordinator and Worker readiness probes. For more information on readiness probes, refer to Kubernetes probes.
livenessProbe
livenessProbe:
  initialDelaySeconds: 300
  periodSeconds: 300
  failureThreshold: 1
  timeoutSeconds: 15
Properties of Presto Coordinator and Worker liveness probes. For more information on liveness probes, refer to Kubernetes probes.
additionalBootstrapScriptVolume
additionalBootstrapScriptVolume:
  configMap:
    name: my-bootstrap-script
Property of Presto Coordinator and Worker pod. Allows adding a custom bootstrap script to customize the Presto pods. Executed prior to staring Presto on the pod. Refer to Custom Bootstrap Script

Service properties

Access to Presto Coordinator is possible via the Kubernetes Service. By default the service is only accessible within the Kubernetes cluster at http://presto-coordinator-CLUSTER_NAME_UUID.NAMESPACE.svc.cluster.local:8080, where NAMESPACE is the Kubernetes namespace where the given Presto cluster is deployed and CLUSTER_NAME_UUID is the cluster name with unique suffix appended.

Use the service.name Presto resource parameter to make the service name more predictable. For example setting service.name=test-cluster will cause the Presto Coordinator service address to be http://presto-coordinator-test-cluster.NAMESPACE.svc.cluster.local:8080.

Use the nameOverride parameter in order to set CLUSTER_NAME_UUID to a different value.

You can also change type of the Presto Coordinator Service using the service.type parameter. For more information on Kubernetes Service types, refer to Kubernetes Services types.

You can add additional parameters to spec section of Service by using service.additionalSpecProperties Presto resource parameter, e.g.

service:
  additionalSpecProperties:
    loadBalancerIP: 78.11.24.19
  type: LoadBalancer

Use service.nodePort parameter to specify the port on which Presto Coordinator Service should be exposed when service.type is set to NodePort, e.g.

service:
  type: NodePort
  nodePort: 3001

Presto Coordinator properties

Property name Example Description

coordinator.cpuRequest

coordinator.cpuLimit

coordinator:
  cpuRequest: 6
  cpuLimit: 32
Specifies the Presto Coordinator pod’s CPU limit and request.
coordinator.memoryAllocation
coordinator:
  memoryAllocation: 60Gi
Specifies Presto Coordinator pod’s memory usage (both request and limit). If empty Presto Coordinator pod will utilize entire memory available on the node.
coordinator.additionalProperties
coordinator:
  additionalProperties: |
    resource-groups.config-file=filename
Specifies additional config.properties properties.

coordinator.nodeSelector

coordinator.affinity

coordinator:
  nodeSelector:
    role: presto
  affinity:
    podAffinity:
      ...
Specifies Presto Coordinator pod’s node selector and affinity.

Presto Worker properties

Property name Example Description
worker.count
worker:
  count: 3
Number of Presto worker pods.
worker.autoscaling
worker:
  autoscaling:
    enabled: true
    minReplicas: 1
    maxReplicas: 100
    targetCPUUtilizationPercentage: 80
Configuration of Presto Workers autoscaling.
worker.deploymentTerminationGracePeriodSeconds
worker:
  deploymentTerminationGracePeriodSeconds: 7200
Specifies termination grace period for Presto Workers pods. Presto Worker pods won’t be terminated until queries running on the pod are finished or grace period passes.

worker.cpuRequest

worker.cpuLimit

coordinator:
  cpuRequest: 6
  cpuLimit: 32
Specifies Presto Worker pod CPU limit and request.
worker.memoryAllocation
worker:
  memoryAllocation: 100Gi
Specifies Presto Worker pod memory usage (both request and limit). If empty Presto Worker pod will utilize entire memory available on the node.
worker.additionalProperties
worker:
  additionalProperties: |
    resource-groups.config-file=filename
Specifies additional config.properties properties.

worker.nodeSelector

worker.affinity

worker:
  nodeSelector:
    role: presto
  affinity:
    podAffinity:
      ...
Specifies Presto Worker pod’s node selector and affinity.

General memory properties

memory section specifies general Presto memory configuration

Property name Example Description
memory.nodeMemoryHeadroom
memory:
  nodeMemoryHeadroom: 2Gi
Memory headroom that Presto should leave on a node when Presto pods are configured to use entire node memory (empty memoryAllocation configuration property).
memory.xmxToTotalMemoryRatio
memory:
  xmxToTotalMemoryRatio: 0.9
Ratio between Presto JVM heap size and memory available for a Presto pod.
memory.heapHeadroomPerNodeRatio
memory:
  heapHeadroomPerNodeRatio: 0.3
Ratio between memory.heap-headroom-per-node Presto configuration property and Presto JVM heap size.
memory.queryMaxMemory
memory:
  queryMaxMemory: 1Pi
Value of the query.max-memory Presto configuration property.
memory.queryMaxTotalMemoryPerNodePoolFraction
memory:
  queryMaxTotalMemoryPerNodePoolFraction: 0.333
Value the query.max-total-memory-per-node Presto configuration property expressed as fraction of Presto JVM heap size.

Hive connector properties

Starburst Presto on Kubernetes provides automatic configuration of the Hive connector. Such a connector allows you to either access an external Metastore or use built-in internal Presto cluster Metastore as well.

External Metastore

You can configure Presto to use an external Hive Metastore by setting the hive.metastoreUri property, e.g.

hive:
  metastoreUri: thrift://hive-metastore:9083

Internal Metastore

You can configure Presto to use an internal Hive Metastore by setting the hive.internalMetastore.mySql or hive.internalMetastore.postgreSql properties, e.g.

hive:
  internalMetastore:
    mySql:
      jdbcUrl: jdbc:mysql://mysql-server/metastore-database
      username: hive
      password: hivePassword

or

hive:
  internalMetastore:
    mySql:
      postgreSql: jdbc:postgresql://postgresql-server/metastore-database
      username: hive
      password: hivePassword

In such a case, an additional Hive Metastore pod will be created which will store data in the provided PostgreSQL or MySQL database.

You can also make the internal Metastore to use an internal ephemeral PostgreSQL database. This can be enabled by setting the hive.internalMetastore.internalPostgreSql.enabled to true, e.g.

hive:
  internalMetastore:
    internalPostgreSql:
      enabled: true

In this case, an additional PostgreSQL pod will be created along with the Hive Metastore pod. This Hive Metastore will use the internal ephemeral PostgreSQL as a relational database.

Note: Using the internal ephemeral PostgreSQL is not recommended for production. Data stored within the internal PostgreSQL will be lost when the Presto cluster is terminated.

Note: You cannot use external and internal Metastore at the same time. You also cannot use external and internal relational database for Metastore at the same time.

Using internal Metastore with custom S3 endpoints

It possible to configure an internal Metastore with S3 like data sources. You can specify a custom S3 endpoint via the hive.internalMetastore.s3Endpoint property, e.g.

hive:
  internalMetastore:
    s3Endpoint: http://my-custom-s3-endpoint

Using AWS credentials

You can provide AWS credentials for the Presto cluster as a Kubernetes secret. The secret name can be configured using the hive.awsSecretName property, e.g.

hive:
  awsSecretName: aws-secret-name

Such secret should contain two files:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY

which contain corresponding AWS credentials.

Using Google Service Account key

You can provide a Google Service Account Key for the Presto Cluster as a Kubernetes secret. The secret name can be configured using the hive.googleServiceAccountKeySecretName property, e.g.

hive:
  googleServiceAccountKeySecretName: google-service-account-key-secret-name

Such secret should contain single file:

  • key.json

which contains Google Service Account Key in JSON format.

Specifying additional Hive connector properties

It is possible to specify additional Hive connector properties via the hive.additionalProperties property, e.g.

hive:
  additionalProperties: |
    hive.allow-drop-table=true

Customizing internal Hive Metastore pod

It possible to customize internal Hive Metastore pod’s Docker image via the hive.internalMetastore.image property, e.g.

hive:
  internalMetastore:
    image:
      name: customized-internal-hive-metastore
      tag: 0.1
      pullPolicy: IfNotPresent

It is possible to change the resources required by Hive Metastore’s pod via:

  • hive.internalMetastore.cpu
  • hive.internalMetastore.memory
  • hive.internalMetastore.nodeSelector
  • hive.internalMetastore.affinity

properties.

Customizing internal PostgreSQL Docker pod

It is possible to customize the internal PostgreSQL pod’s Docker image via the hive.internalMetastore.internalPostgreSql.image property, e.g.

hive:
  internalMetastore:
    internalPostgreSql:
      image:
        name: customized-internal-postgresql
        tag: 0.1
        pullPolicy: IfNotPresent

It is possible to change the resources required by PostgreSQL’s pod via:

  • hive.internalMetastore.internalPostgreSql.cpu
  • hive.internalMetastore.internalPostgreSql.memory
  • hive.internalMetastore.internalPostgreSql.nodeSelector
  • hive.internalMetastore.internalPostgreSql.affinity

properties.

It is possible to specify the storage used by PostgreSQL’s pod via: the hive.internalMetastore.internalPostgreSql.storage property, e.g.

hive:
  internalMetastore:
    internalPostgreSql:
      storage:
        className: db-class
        size: 20Gi
        claimSelector:
          matchLabels:
            release: "stable"
          matchExpressions:
            - {key: environment, operator: In, values: [dev]}

Prometheus support

Presto Kubernetes supports exposing Presto metrics to Prometheus. You can use the Prometheus Operator in order to set up Prometheus in your Kubernetes cluster and collect Presto metrics.

To enable Presto Prometheus metric endpoints set the prometheus.enabled property to true. This will cause the following additional Kubernetes services to be created:

  • prometheus-coordinator-CLUSTER_NAME_UUID
  • prometheus-worker-CLUSTER_NAME_UUID

where CLUSTER_NAME_UUID is the cluster name with a unique suffix. Use the nameOverride property in order to set CLUSTER_NAME_UUID to a different value.

Those services will expose Presto metrics in the Prometheus format. You can use ServiceMonitor resources from the Prometheus Operator to make Prometheus collect metrics from those endpoints. In order to match Presto Prometheus endpoints you can use labels, e.g.

matchLabels:
  instance: CLUSTER_NAME_UUID
  role: prometheus-coordinator

The following Presto and JVM metrics will be exported by default:

  • running_queries - number of currently running queries
  • queued_queries - number of currently queued queries
  • failed_queries - total count of failed queries
  • jvm_gc_collection_seconds_sum – total GC (young and old) time in seconds
  • jvm_gc_collection_seconds_count – total GC (young and old) count
  • jvm_memory_bytes_committed, jvm_memory_bytes_init, jvm_memory_bytes_max, jvm_memory_bytes_used – JVM memory usage metrics (heap and non-heap). For more information see MemoryMXBean.