Version

Kadalu Storage Pools

Kadalu Storage pool is a collection of one or more export directories from multiple servers.

Pool Creation

Create a Kadalu Storage Pool with Storage units from multiple Storage nodes.

[root@server1 ~]# kadalu pool create pool1 server1.example.com:/exports/pool1/s1
Pool pool1 created successfully
ID: 02b6a3a4-e704-47ee-bb05-e541a561a921

Where

pool1 is the Pool name
server1.example.com is the node from which the Storage unit will be exported.
/exports/pool1/s1 is the mounted path of Storage Unit.

Automatically add nodes to the Cluster that are specified in the Pool create request by adding --auto-add-nodes.

[root@server1 ~]# kadalu pool create pool1 \
    server1.example.com:/exports/pool1/s1   \
    --auto-add-nodes
Pool pool1 created successfully
ID: 02b6a3a4-e704-47ee-bb05-e541a561a921

Create a Replicated or Mirrored Pool

[root@server1 ~]# kadalu pool create pool1 \
    replica server1.example.com:/exports/pool1/s1 \
            server2.example.com:/exports/pool1/s2 \
            server3.example.com:/exports/pool1/s3
Pool pool1 created successfully
ID: 02b6a3a4-e704-47ee-bb05-e541a561a921

Using mirror is also supported

[root@server1 ~]# kadalu pool create pool1 \
    mirror server1.example.com:/exports/pool1/s1 \
           server2.example.com:/exports/pool1/s2 \
           server3.example.com:/exports/pool1/s3
Pool pool1 created successfully
ID: 02b6a3a4-e704-47ee-bb05-e541a561a921

Alternatively, GlusterFS syntax is also supported

[root@server1 ~]# kadalu pool create pool1 replica 3 \
    server1.example.com:/exports/pool1/s1 \
    server2.example.com:/exports/pool1/s2 \
    server3.example.com:/exports/pool1/s3
Pool pool1 created successfully
ID: 02b6a3a4-e704-47ee-bb05-e541a561a921

By default, pools are started on create. Use --no-start to override this behavior.
If the Storage unit path is not a mounted directory then use --allow-root-partition option (Not yet implemented). This is not an optimal setup since the Kadalu Pool can fill up the root partition or other applications can fill up Kadalu Pool.
To import a Pool, Use --pool-id option to set the Same Pool ID to the imported Pool. If required Pool ID to be reused is not saved, it can be found in server volfile or at storage unit path as extended attribute in hexadecimal format.
```
[root@server1 ~]# kadalu pool create pool1 \
    server1.example.com:/exports/pool1/s1 \
   --pool-id=02b6a3a4-e704-47ee-bb05-e541a561a921
```
To reuse a Storage unit directory that was part of other Pool earlier, use --reuse-storage-unit (Not yet implemented, only applicable if the Storage unit directory is empty).
To allow using Replicate/Mirrored/Disperse Storage units from same node, use --from-same-node. This is not an optimal setup since it does not provide high availability for the distribute group if one node goes down (Not yet implemented).

Ports for the Storage Unit are assigned automatically (49252 to 49452) while creating the Pool. Restarting the Pool will not change the Ports of the Storage units. The Pool info command will show the assigned ports. It is also possible to manually specify the Port while creating the Pool.

[root@server1 ~]# kadalu pool create pool1 \
    mirror server1.example.com:4501:/exports/pool1/s1 \
           server2.example.com:4501:/exports/pool1/s2 \
           server3.example.com:4501:/exports/pool1/s3
Pool pool1 created successfully
ID: 02b6a3a4-e704-47ee-bb05-e541a561a921

Pool Expansion and Rebalance

Expand an already created Kadalu Storage Pool with storage units from multiple storage nodes.

Example,

Create a Replicated or Mirror Kadalu Storage Pool 'pool1' with 3 different storage nodes and 3 storage units.

[root@server1 ~]# kadalu pool create pool1 \
    mirror server1.example.com:/exports/pool1/s1 \
           server2.example.com:/exports/pool1/s2 \
           server3.example.com:/exports/pool1/s3
Pool pool1 created successfully
ID: 03c6a3a4-e704-47ee-bb05-e541a561a875

Expand Kadalu Storage Pool pool1 by adding more Storage units.

Note:

Pool type, distribute group count, replica count, data and redundant storage unit count should be the same as when the Kadalu Storage Pool was created.

For example above Kadalu Storage Pool being of type mirror 3, it has 1 distribute group with 3 storage units. Hence during pool expansion Kadalu Storage expects distribute group being a multiple of 1.

[root@server1 ~]# kadalu pool expand pool1   \
    mirror server1.example.com:/exports/pool1/s1_e \
           server2.example.com:/exports/pool1/s2_e \
           server3.example.com:/exports/pool1/s3_e
Pool pool1 expanded successfully
ID: 03c6a3a4-e704-47ee-bb05-e541a561a875

Proceed to the rebalancing of pool pool1 by following the below steps.
To start the rebalancing of pool: kadalu pool rebalance-start pool1.
To force stop the rebalancing of pool: kadalu pool rebalance-stop pool1.

Rebalance

During the expansion of pool, Kadalu Storage also fixes the layout of all expanded storage units, i.e., it triggers the creation of directories in the expanded storage units, to maintain the same directory structure as in storage units before expansion.

Once the layout of the expanded storage units is fixed, data migration can commence. Data migration involves recalculating the hash of files and distributing them across storage units.

To start and stop the rebalance (i.e., data migration), follow the steps displayed after the pool expansion.

[root@server1 ~]# kadalu rebalance start pool1
Rebalance of Pool pool1 started.

[root@server1 ~]# kadalu rebalance stop pool1
Rebalance of Pool pool1 started.

Note:

The Rebalance process, if restarted in the middle of completion, will start scanning storage units from the beginning and not from the point where it stopped

To view the status of Rebalance processes,

kadalu rebalance status pool1

[root@server1 ~]# kadalu rebalance status pool1

Name                       : pool1
Type                       : Distributed Mirror
ID                         : 03c6a3a4-e704-47ee-bb05-e541a561a875
Fix-Layout Status          : complete
Total Dirs Scanned         : 3
Duration                   : 0
Progress                   : 100.00 %
Estimate Seconds           : 0
Scanned                    : 1.13MiB / 1.13MiB

Pool pool1 Rebalance Status            : complete
Total Number of Rebalance Process       : 6
Number of Completed Rebalance Process   : 6
Number of Failed Rebalance Process      : 0

Where,

Fix-Layout Status is status of fix-layout process after pool expansion.
Total Dirs Scanned indicates total number of dirs which was scanned and added to newer storage units.
Duration is the time taken for fix-layout process to complete.
Progress is the percentage amount of progress made with migration of data (Rebalance).
Estimate Seconds is the estimated amount of time which might be taken for completion of migration of data.
Scanned indicates in Bytes, files scanned out of total files.
Rebalance Status can be of complete, partial, failed and not-started indicating the status of Rebalance at pool level.

To view the Rebalance Status of each storage-unit : kadalu rebalance status <pool> --detail.

Community

Community Slack Github kadalu.tech

Kadalu Storage Pools

Pool Creation

Pool Expansion and Rebalance

Rebalance

ON THIS PAGE

Community