node-disk-manager (NDM) aims to make it easy to manage the disks attached to the node. It treats disks as resources that need to be monitored and managed just like other resources like CPU, Memory and Network. It contains a daemon which runs on each node, detects attached disks and loads them as BlockDevice objects (custom resource) into Kubernetes.
While PVs are well suited for stateful workloads, the BlockDevice objects are aimed towards helping hyper-converged Storage Operators by providing abilities like:
- Easy to access inventory of block devices available across the Kubernetes Cluster.
- Predict failures on the blockdevices, to help with taking preventive actions.
- Allow for dynamically attaching/detaching blockdevices to a Storage Pod, without requiring a restart.
NDM has 2 main components:
- node-disk-manager daemonset, which runs on each node and is responsible for device detection.
- node-disk-operator deployment, which acts as an inventory of block devices in the cluster.
and 2 optional components:
- ndm-cluster-exporter deployment, which fetches block device object from etcd and exposes it as prometheus metrics.
- ndm-node-exporter daemonset, which runs on each node, queries the disk for details like SMART and expose it as prometheus metrics.
The design of the project is covered under this design proposal
The feedback of users / organizations currently using OpenEBS can be found here
Currently, the NDM project is in beta.
A detailed usage documentation is maintained in the wiki.
- FAQ and Troubleshooting
K8S : 1.18+
- Edit ndm-operator.yaml to fit your environment: Set the
namespace,serviceAccount, configure filters in thenode-disk-manager-config-map. - Switch to Cluster Admin context and create the components with
kubectl create -f ndm-operator.yaml. - This will install the daemon, operator and the exporters
kubectl get blockdevicesdisplays the blockdevices across the cluster, withNODENAMEshowing the node to which disk is attached,CLAIMSTATEshowing whether the device is currently in use andSTATEshowing whether the device is connected to the node.kubectl get blockdevices -o widedisplays the blockdevice along with the path at which the device is attached on the node.kubectl get blockdevices <blockdevice-cr-name> -o yamldisplays all the details of the disk captured byndmfor given disk resource.
Before building the image locally, you need to setup your development environment. The detailed instructions for setting up development environment, building and testing are available here.
OpenEBS welcomes your feedback and contributions in any form possible.
- Thanks to Daniel for setting up the go-based SMART library.
- Thanks to Humble, Jan and other from the Kubernetes Storage Community for reviewing the approach and evaluating the use-case.