Skip to main content


Q1: How does HwameiStor scheduler work in a Kubernetes platform?

The HwameiStor scheduler is deployed as a pod in the HwameiStor namespace.


Once the applications (Deployment or StatefulSet) are created, the pod will be scheduled to the worker nodes on which HwameiStor is already configured.

Q2: How does HwameiStor schedule applications with multi-replicas workloads and what are the differences compared to the traditional shared storage (NFS / block)?

We strongly recommend using StatefulSet for applications with multi-replica workloads.

StatefulSet will deploy replicas on the same worker node with the original pod, and will also create a PV data volume for each replica. If you need to deploy replicas on different worker nodes, you shall manually configure them with pod affinity.


We suggest using a single pod for deployment because the block data volumes can not be shared.

Q3: How to maintain a Kubernetes node?

HwameiStor provides the volume eviction/migration functions to keep the Pods and HwameiStor volumes' data running when retiring/rebooting a node.

Retire a node

Before remove a node from a Kubernetes cluster, the Pods and volumes on the node should be rescheduled and migrated to another available node, and keep the Pods/volumes running.

Follow these steps to retire a node:

  1. Drain node.

    kubectl drain NODE --ignore-daemonsets=true. --ignore-daemonsets=true

    Allows the above command to succeed even if pods managed by daemonset exist. If it stacks due to PodDisruptionBudgets or something, try --force option. The command will also trigger HwameiStor to migrate all the volumes' replicas to another available node. Make sure the migration to complete by following command before moving ahead.

  2. Get LocalStorageNode.

    kubectl get localstoragenode NODE -o yaml

    The output may look like:

    kind: LocalStorageNode
    name: NODE
    hostname: NODE
    region: default
    zone: default
    class: HDD
    - capacityBytes: 17175674880
    devPath: /dev/sdb
    state: InUse
    type: HDD
    freeCapacityBytes: 16101933056
    freeVolumeCount: 999
    name: LocalStorage_PoolHDD
    totalCapacityBytes: 17175674880
    totalVolumeCount: 1000
    type: REGULAR
    usedCapacityBytes: 1073741824
    usedVolumeCount: 1
    volumeCapacityBytesLimit: 17175674880
    ## **** make sure volumes is empty **** ##
    state: Ready

    At the same time, HwameiStor will automatically reschedule the evicted Pods to the other node which has the associated volume' replica, and continue to run.

  3. Remove the NODE from the cluster.

    kubectl delete nodes NODE

Reboot a node

It ususally takes a long time (~10mins) to reboot a node. All the Pods and volumes on the node will not work until the node is back online. For some applications like DataBase, the long downtime is very costly and even unacceptable.

HwameiStor can immediately reschedule the Pod to another available node with associated volume data and bring the Pod back to running in very short time (~ 10 seconds for the Pod using a HA volume, and longer time for the Pod with non-HA volume depends on the data size).

If user doesn't want to migrate the volumes during the node reboots, can add the following label to the node before draining it.

kubectl label node NODE

To reboot a node, the step 1 and 2 are same as above (in section of Retire a node).

After the node reboots and comes back online, the volumes on this node can still be avaiable for access.

Run the following command to bring the node back to normal

kubectl uncordon NODE

For the traditional shared storage

StatefulSet will deploy replicas to other worker nodes for workload distribution and will also create a PV data volume for each replica.

The Deployment will also deploy replicas to other worker nodes for workload distribution but will share the same PV data volume (only for NFS). We suggest using a single pod for block storage because the block data volumes can not be shared.