Let’s start hacking!!
Most of the time, one needs to know the heketi configuration and topology he is running with. I do this at times and today thought of sharing with a wider audience, there are more hacking that can be done once you got a handle of the things, but I put a disclaimer that, you should not do it unless you know what you are doing. Please reach out to support channels if you are playing this in production.
Use `oc/kubectl get pods` to know the heketi pod which is running in your setup.
[terminal]
[mynode~]# oc get pods
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 423 17d
busybox2 1/1 Running 403 16d
glusterblock-provisioner-dc-9-ch2lr 1/1 Running 0 6d
glusterfs-bztdb 1/1 Running 0 23d
glusterfs-h6sfw 1/1 Running 0 23d
glusterfs-vl6l8 1/1 Running 0 23d
heketi-1-dqcp2 1/1 Running 1 6d
storage-project-router-2-x5dsv 1/1 Running 0 23d
[/terminal]
We got the name of the heketi pod, now let’s describe the pod spec.
[terminal]
[mynode ~]# oc describe pod heketi-1-dqcp2
Name: heketi-1-dqcp2
Namespace: storage-project
Node: example.com/10.70.47.1
Start Time: Fri, 02 Mar 2018 17:33:38 +0530
Labels: deployment=heketi-1
deploymentconfig=heketi
glusterfs=heketi-pod
heketi=pod
Annotations: openshift.io/deployment-config.latest-version=1
openshift.io/deployment-config.name=heketi
openshift.io/deployment.name=heketi-1
openshift.io/scc=privileged
Status: Running
IP: 10.128.0.41
Controlled By: ReplicationController/heketi-1
Containers:
heketi:
Container ID: docker://61e4ffd7d6adda7f1c9c663ddf4ba44d54bdd6197d2677684e87ef7bfa16da4f
Image: X.X.X.X
Image ID: docker-pullable://XXXX@sha256:6f7ab12b8d39e66dc186da3152c8dbf1e7682eb1cdecd0f80fa86b267b0221c1
Port: 8080/TCP
State: Running
Started: Fri, 02 Mar 2018 17:42:06 +0530
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Fri, 02 Mar 2018 17:34:16 +0530
Finished: Fri, 02 Mar 2018 17:42:00 +0530
Ready: True
Restart Count: 1
Liveness: http-get http://:8080/hello delay=30s timeout=3s period=10s #success=1 #failure=3
Readiness: http-get http://:8080/hello delay=3s timeout=3s period=10s #success=1 #failure=3
Environment:
HEKETI_USER_KEY:
HEKETI_ADMIN_KEY:
HEKETI_EXECUTOR: kubernetes
HEKETI_FSTAB: /var/lib/heketi/fstab
HEKETI_SNAPSHOT_LIMIT: 14
HEKETI_KUBE_GLUSTER_DAEMONSET: y
Mounts:
/etc/heketi from config (rw)
/var/lib/heketi from db (rw)
/var/run/secrets/kubernetes.io/serviceaccount from heketi-service-account-token-mstk7 (ro)
Conditions:
Type Status
Initialized True
Ready True
PodScheduled True
Volumes:
db:
Type: Glusterfs (a Glusterfs mount on the host that shares a pod’s lifetime)
EndpointsName: heketi-storage-endpoints
Path: heketidbstorage
ReadOnly: false
config:
Type: Secret (a volume populated by a Secret)
SecretName: heketi-config-secret
Optional: false
heketi-service-account-token-mstk7:
Type: Secret (a volume populated by a Secret)
SecretName: heketi-service-account-token-mstk7
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: <none>
Events: <none>
[/terminal]
Pay attention to `config` section of type `secret`. You got the `secret` name ie `heketi-config-secret` from it. Let’s see whats there in this secret.
[terminal]
[mynode ~]# oc get secret heketi-config-secret -o yaml
apiVersion: v1
data:
heketi.json: ewoJIl9wb3J0X2NvbW1lbnQiOiAiSGVrZXRpIFNlcnZlciBQb3J0IE51bWJlciIsCgkicG9ydCIgOiAiODA4MCIsCgoJIl91c2VfYXV0aCI6ICJFbmFibGUgSldUIGF1dGhvcml6YXRpb24uIFBsZWFzZSBlbmFibGUgZm9yIGRlcGxveW1lbnQiLAoJInVzZV9hdXRoIiA6IGZhbHNlLAoKCSJfand0IiA6ICJQcml2YXRlIGtleXMgZm9yIGFjY2VzcyIsCgkiand0IiA6IHsKCQkiX2FkbWluIiA6ICJBZG1pbiBoYXMgYWNjZXNzIHRvIGFsbCBBUElzIiwKCQkiYWRtaW4iIDogewoJCQkia2V5IiA6ICJNeSBTZWNyZXQiCgkJfSwKCQkiX3VzZXIiIDogIlVzZXIgb25seSBoYXMgYWNjZXNzIHRvIC92b2x1bWVzIGVuZHBvaW50IiwKCQkidXNlciIgOiB7CgkJCSJrZXkiIDogIk15IFNlY3JldCIKCQl9Cgl9LAoKCSJfZ2x1c3RlcmZzX2NvbW1lbnQiOiAiR2x1c3RlckZTIENvbmZpZ3VyYXRpb24iLAoJImdsdXN0ZXJmcyIgOiB7CgoJCSJfZXhlY3V0b3JfY29tbWVudCI6ICJFeGVjdXRlIHBsdWdpbi4gUG9zc2libGUgY2hvaWNlczogbW9jaywga3ViZXJuZXRlcywgc3NoIiwKCQkiZXhlY3V0b3IiIDogImt1YmVybmV0ZXMiLAoKCQkiX2RiX2NvbW1lbnQiOiAiRGF0YWJhc2UgZmlsZSBuYW1lIiwKCQkiZGIiIDogIi92YXIvbGliL2hla2V0aS9oZWtldGkuZGIiLAoKCQkia3ViZWV4ZWMiIDogewoJCQkicmViYWxhbmNlX29uX2V4cGFuc2lvbiI6IHRydWUKCQl9LAoKCQkic3NoZXhlYyIgOiB7CgkJCSJyZWJhbGFuY2Vfb25fZXhwYW5zaW9uIjogdHJ1ZSwKCQkJImtleWZpbGUiIDogIi9ldGMvaGVrZXRpL3ByaXZhdGVfa2V5IiwKCQkJInBvcnQiIDogIjIyIiwKCQkJInVzZXIiIDogInJvb3QiLAoJCQkic3VkbyIgOiBmYWxzZQoJCX0sCgoJCSJfYXV0b19jcmVhdGVfYmxvY2tfaG9zdGluZ192b2x1bWUiOiAiQ3JlYXRlcyBCbG9jayBIb3N0aW5nIHZvbHVtZXMgYXV0b21hdGljYWxseSBpZiBub3QgZm91bmQgb3IgZXhzaXN0aW5nIHZvbHVtZSBleGhhdXN0ZWQiLAoJCSJhdXRvX2NyZWF0ZV9ibG9ja19ob3N0aW5nX3ZvbHVtZSI6IHRydWUsCgoJCSJfYmxvY2tfaG9zdGluZ192b2x1bWVfc2l6ZSI6ICJOZXcgYmxvY2sgaG9zdGluZyB2b2x1bWUgd2lsbCBiZSBjcmVhdGVkIGluIHNpemUgbWVudGlvbmVkLCBUaGlzIGlzIGNvbnNpZGVyZWQgb25seSBpZiBhdXRvLWNyZWF0ZSBpcyBlbmFibGVkLiIsCgkJImJsb2NrX2hvc3Rpbmdfdm9sdW1lX3NpemUiOiA1MDAKCX0sCgoJImJhY2t1cF9kYl90b19rdWJlX3NlY3JldCI6IGZhbHNlCn0K
private_key: null
topology.json: ewogICAgImNsdXN0ZXJzIjogWwogICAgICAgIHsKICAgICAgICAgICAgIm5vZGVzIjogWwogICAgICAgICAgICAgICAgewogICAgICAgICAgICAgICAgICAgICJub2RlIjogewogICAgICAgICAgICAgICAgICAgICAgICAiaG9zdG5hbWVzIjogewogICAgICAgICAgICAgICAgICAgICAgICAgICAgIm1hbmFnZSI6IFsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiZGhjcDQ2LTE4Ny5sYWIuZW5nLmJsci5yZWRoYXQuY29tIgogICAgICAgICAgICAgICAgICAgICAgICAgICAgXSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICJzdG9yYWdlIjogWwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICIxMC43MC40Ni4xODciCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBdCiAgICAgICAgICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAgICAgICAgICJ6b25lIjogMQogICAgICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAgICAgImRldmljZXMiOiBbCiAgICAgICAgICAgICAgICAgICAgICAgICIvZGV2L3NkYyIsCiAgICAgICAgICAgICAgICAgICAgICAgICIvZGV2L3NkZCIKICAgICAgICAgICAgICAgICAgICBdCiAgICAgICAgICAgICAgICB9LAogICAgICAgICAgICAgICAgewogICAgICAgICAgICAgICAgICAgICJub2RlIjogewogICAgICAgICAgICAgICAgICAgICAgICAiaG9zdG5hbWVzIjogewogICAgICAgICAgICAgICAgICAgICAgICAgICAgIm1hbmFnZSI6IFsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiZGhjcDQ3LTEubGFiLmVuZy5ibHIucmVkaGF0LmNvbSIKICAgICAgICAgICAgICAgICAgICAgICAgICAgIF0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAic3RvcmFnZSI6IFsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiMTAuNzAuNDcuMSIKICAgICAgICAgICAgICAgICAgICAgICAgICAgIF0KICAgICAgICAgICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICAgICAgICAgInpvbmUiOiAyCiAgICAgICAgICAgICAgICAgICAgfSwKICAgICAgICAgICAgICAgICAgICAiZGV2aWNlcyI6IFsKICAgICAgICAgICAgICAgICAgICAgICAgIi9kZXYvc2RjIiwKICAgICAgICAgICAgICAgICAgICAgICAgIi9kZXYvc2RkIgogICAgICAgICAgICAgICAgICAgIF0KICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICB7CiAgICAgICAgICAgICAgICAgICAgIm5vZGUiOiB7CiAgICAgICAgICAgICAgICAgICAgICAgICJob3N0bmFtZXMiOiB7CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAibWFuYWdlIjogWwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJkaGNwNDYtOTgubGFiLmVuZy5ibHIucmVkaGF0LmNvbSIKICAgICAgICAgICAgICAgICAgICAgICAgICAgIF0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAic3RvcmFnZSI6IFsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiMTAuNzAuNDYuOTgiCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBdCiAgICAgICAgICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAgICAgICAgICJ6b25lIjogMQogICAgICAgICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgICAgICAgImRldmljZXMiOiBbCiAgICAgICAgICAgICAgICAgICAgICAgICIvZGV2L3NkYyIsCiAgICAgICAgICAgICAgICAgICAgICAgICIvZGV2L3NkZCIKICAgICAgICAgICAgICAgICAgICBdCiAgICAgICAgICAgICAgICB9CiAgICAgICAgICAgIF0KICAgICAgICB9CiAgICBdCn0K
kind: Secret
metadata:
creationTimestamp: 2018-02-13T13:00:48Z
labels:
glusterfs: heketi-config-secret
heketi: config-secret
name: heketi-config-secret
namespace: storage-project
resourceVersion: “28053”
selfLink: /api/v1/namespaces/storage-project/secrets/heketi-config-secret
uid: e969d95a-10bd-11e8-9c7b-005056a5a340
type: Opaque
[/terminal]
From the tags, we have `heketi.json and topology.json`.
The value of these tags/fields are base64 encoded values, so let us pick the heketi.json field and decode it as shown below.
[terminal][mynode ~]# echo `echo ewoJIl9wb3J0X2NvbW1lbnQiOiAiSGVrZXRpIFNlcnZlciBQb3J0IE51bWJlciIsCgkicG9ydCIgOiAiODA4MCIsCgoJIl91c2VfYXV0aCI6ICJFbmFibGUgSldUIGF1dGhvcml6YXRpb24uIFBsZWFzZSBlbmFibGUgZm9yIGRlcGxveW1lbnQiLAoJInVzZV9hdXRoIiA6IGZhbHNlLAoKCSJfand0IiA6ICJQcml2YXRlIGtleXMgZm9yIGFjY2VzcyIsCgkiand0IiA6IHsKCQkiX2FkbWluIiA6ICJBZG1pbiBoYXMgYWNjZXNzIHRvIGFsbCBBUElzIiwKCQkiYWRtaW4iIDogewoJCQkia2V5IiA6ICJNeSBTZWNyZXQiCgkJfSwKCQkiX3VzZXIiIDogIlVzZXIgb25seSBoYXMgYWNjZXNzIHRvIC92b2x1bWVzIGVuZHBvaW50IiwKCQkidXNlciIgOiB7CgkJCSJrZXkiIDogIk15IFNlY3JldCIKCQl9Cgl9LAoKCSJfZ2x1c3RlcmZzX2NvbW1lbnQiOiAiR2x1c3RlckZTIENvbmZpZ3VyYXRpb24iLAoJImdsdXN0ZXJmcyIgOiB7CgoJCSJfZXhlY3V0b3JfY29tbWVudCI6ICJFeGVjdXRlIHBsdWdpbi4gUG9zc2libGUgY2hvaWNlczogbW9jaywga3ViZXJuZXRlcywgc3NoIiwKCQkiZXhlY3V0b3IiIDogImt1YmVybmV0ZXMiLAoKCQkiX2RiX2NvbW1lbnQiOiAiRGF0YWJhc2UgZmlsZSBuYW1lIiwKCQkiZGIiIDogIi92YXIvbGliL2hla2V0aS9oZWtldGkuZGIiLAoKCQkia3ViZWV4ZWMiIDogewoJCQkicmViYWxhbmNlX29uX2V4cGFuc2lvbiI6IHRydWUKCQl9LAoKCQkic3NoZXhlYyIgOiB7CgkJCSJyZWJhbGFuY2Vfb25fZXhwYW5zaW9uIjogdHJ1ZSwKCQkJImtleWZpbGUiIDogIi9ldGMvaGVrZXRpL3ByaXZhdGVfa2V5IiwKCQkJInBvcnQiIDogIjIyIiwKCQkJInVzZXIiIDogInJvb3QiLAoJCQkic3VkbyIgOiBmYWxzZQoJCX0sCgoJCSJfYXV0b19jcmVhdGVfYmxvY2tfaG9zdGluZ192b2x1bWUiOiAiQ3JlYXRlcyBCbG9jayBIb3N0aW5nIHZvbHVtZXMgYXV0b21hdGljYWxseSBpZiBub3QgZm91bmQgb3IgZXhzaXN0aW5nIHZvbHVtZSBleGhhdXN0ZWQiLAoJCSJhdXRvX2NyZWF0ZV9ibG9ja19ob3N0aW5nX3ZvbHVtZSI6IHRydWUsCgoJCSJfYmxvY2tfaG9zdGluZ192b2x1bWVfc2l6ZSI6ICJOZXcgYmxvY2sgaG9zdGluZyB2b2x1bWUgd2lsbCBiZSBjcmVhdGVkIGluIHNpemUgbWVudGlvbmVkLCBUaGlzIGlzIGNvbnNpZGVyZWQgb25seSBpZiBhdXRvLWNyZWF0ZSBpcyBlbmFibGVkLiIsCgkJImJsb2NrX2hvc3Rpbmdfdm9sdW1lX3NpemUiOiA1MDAKCX0sCgoJImJhY2t1cF9kYl90b19rdWJlX3NlY3JldCI6IGZhbHNlCn0K| base64 –decode`
{ “_port_comment”: “Heketi Server Port Number”, “port” : “8080”, “_use_auth”: “Enable JWT authorization. Please enable for deployment”, “use_auth” : false, “_jwt” : “Private keys for access”, “jwt” : { “_admin” : “Admin has access to all APIs”, “admin” : { “key” : “My Secret” }, “_user” : “User only has access to /volumes endpoint”, “user” : { “key” : “My Secret” } }, “_glusterfs_comment”: “GlusterFS Configuration”, “glusterfs” : { “_executor_comment”: “Execute plugin. Possible choices: mock, kubernetes, ssh”, “executor” : “kubernetes”, “_db_comment”: “Database file name”, “db” : “/var/lib/heketi/heketi.db”, “kubeexec” : { “rebalance_on_expansion”: true }, “sshexec” : { “rebalance_on_expansion”: true, “keyfile” : “/etc/heketi/private_key”, “port” : “22”, “user” : “root”, “sudo” : false }, “_auto_create_block_hosting_volume”: “Creates Block Hosting volumes automatically if not found or exsisting volume exhausted”, “auto_create_block_hosting_volume”: true, “_block_hosting_volume_size”: “New block hosting volume will be created in size mentioned, This is considered only if auto-create is enabled.”, “block_hosting_volume_size”: 500 }, “backup_db_to_kube_secret”: false }
[/terminal]
Is this what you wanted?
Copyright secured by Digiprove © 2018-2020 Humble Chirammal