StorageClass issue during working with Vertica Operator

Hello everybody,

I have a cluster with 3 nodes, everyone in master mode.
I want to run Vertica Operator with Storage Class: local-storage, based on the documentation

here is my simple deployment for create Storage Class, Persistent Volume Claim and Persistent Volume:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: vertica-pv
spec:
  storageClassName: local-storage
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/data/data_vertica"
    type: DirectoryOrCreate
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - node1
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: vertica-pvc
spec:
  storageClassName: local-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  volumeName: vertica-pv

and deployment for Custom Resource is next:

apiVersion: vertica.com/v1beta1
kind: VerticaDB
metadata:
  name: my-verticadb
spec:
  image: localhost:5000/vertica/vertica-k8s:11.1.1-0-minimal
  imagePullPolicy: Always
  initPolicy: Create
  kSafety: "1"
  superuserPasswordSecret: su-passwd
  dbName: my-vertica-db
  volumeMounts:
  - name: vertica-storage
    mountPath: /data/data_vertica
  volumes:
    - name: vertica-storage
      persistentVolumeClaim:
        claimName: vertica-pvc
  local:
    storageClass: vertica-class
    dataPath: /data
    depotPath: /depot
    requestSize: 10Gi
  subclusters:
  - isPrimary: true
    name: primary-subcluster
    size: 1
    resources:
      limits:
        memory: "2Gi"
        cpu: 2
      requests:
        memory: "2Gi"
        cpu: 2

But after apply deployments, pods for Vertica show me such errors:

Warning  FailedScheduling  3m43s (x14542 over 15d)  default-scheduler  0/3 nodes are available: 3 node(s) didn't find available persistent volumes to bind, 3 node(s) had volume node affinity conflict.

I tried to setup Vertica Operator in the managed kubernetes cluster with preconfigured StorageClass and it's working correct, but after reconfigure deployment to the Local one pods can not start with same error.

Is it possible to start Vertica Operator with simple Storage Class or there is any presequites for it ? Perhaps I missed something in the documentation

Best Answer

Answers

  • SruthiASruthiA Vertica Employee Administrator

    Could you please share me the output of kubectl get storageclass

  • evgheni_antropovevgheni_antropov Vertica Customer
    edited July 2022

    Sorry for delay, here result of command:

    kubectl get sc
    NAME                   PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
    kafka-log.class        kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  164d
    nms-data-sc            k8s.io/minikube-hostpath       Retain          Immediate              true                   158d
    registry-class         kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  164d
    vertica-class          kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  38d
    zookeeper-data.class   kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  164d
    zookeeper-log.class    kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  164d
    
    kubectl describe sc vertica-class
    Name:            vertica-class
    IsDefaultClass:  No
    Annotations:     kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"vertica-class"},"provisioner":"kubernetes.io/no-provisioner","volumeBindingMode":"WaitForFirstConsumer"}
    
    Provisioner:           kubernetes.io/no-provisioner
    Parameters:            <none>
    AllowVolumeExpansion:  <unset>
    MountOptions:          <none>
    ReclaimPolicy:         Delete
    VolumeBindingMode:     WaitForFirstConsumer
    Events:                <none>
    
  • evgheni_antropovevgheni_antropov Vertica Customer

    Thank you @SruthiA

    In documentation https://www.vertica.com/docs/11.0.x/HTML/Content/Authoring/Containers/Kubernetes/ContainerizedVerticaWithK8s.htm#7 we're not observing any troubles to work with local.storage class.

    Will check how to create a storage class with dynamic provisioner.

  • evgheni_antropovevgheni_antropov Vertica Customer
    edited July 2022

    dublicated

  • SruthiASruthiA Vertica Employee Administrator

    @evgheni_antropov Issue is mainly with provisioner used in the storage class not with local.storage parameter.

    Provisioner: kubernetes.io/no-provisioner

    Below are the storageclass requirements for vertica:

    StorageClass Requirements
    The StorageClass affects how the Vertica server environment and operator function. For optimum performance, consider the following:

    If you do not set the local.storageClassName configuration parameter, the operator uses the default storage class. If you use the default storage class, confirm that it meets storage requirements for a production workload.
    Select a StorageClass that uses a recommended storage format type as its fsType.
    Use dynamic volume provisioning. The operator requires on-demand volume provisioning to create PVs as needed.

Leave a Comment

BoldItalicStrikethroughOrdered listUnordered list
Emoji
Image
Align leftAlign centerAlign rightToggle HTML viewToggle full pageToggle lights
Drop image/file