openebs not starting correctly
Description
Problem/Justification
Impact
Activity
Waqar Ahmed October 20, 2021 at 10:28 PM
Waqar Ahmed October 20, 2021 at 3:08 PM
can you please confirm the date with time when you are available ? ( I am not sure what tomorrow evening actually is )
I basically have an idea of a fix and want to test it in your environment to make sure it's really/actually accurate before making the required change - which means i would require ssh access temporarily to test it out. It should take approx 15-30 min to test the changes. My email is waqar@ixsystems.com, please feel free to email me team viewer details and we can sync accordingly. Thanks!
Walter Zimmermann October 19, 2021 at 10:01 PM
ps.: if I just could see at what exact time you posted your replies...
I tried to figure it out, but JIRA only shows - Yesterday .. which is kinda useless
in communication like this here ...
Walter Zimmermann October 19, 2021 at 9:58 PM
Ah, sorry, here you go.
Yes, ofc, I reside in Vienna, Europe, so its CET and the only time possible for me
is tomorrow evening, then I ll be gone till Sunday evening and then I am gone
from Wednesday till Sunday again. Maybe I can give you direct access via my IP and
port fowarding, that would be the most pragmatic solution I guess, just let me know by tomorrow evening.
greetings
Waqar Ahmed October 19, 2021 at 10:08 AM
I was referring to a system debug ( System -> Advanced -> Save Debug ). Btw can we schedule a team viewer session today perhaps and if yes, can you please let me know your available time slots ( with time zone ) ?
I have an idea as to what might be happening but probably would be nice to confirm it.
fresh install of 21.08 Beta2
wanted to install nextcloud, stayed in deploying 0/3 for two days.
nas02# k3s kubectl describe nodes
Name: ix-truenas
Roles: control-plane,master
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=ix-truenas
kubernetes.io/os=linux
node-role.kubernetes.io/control-plane=true
node-role.kubernetes.io/master=true
Annotations: k3s.io/node-args:
["server","--flannel-backend","none","--disable","traefik,metrics-server,local-storage","--disable-kube-proxy","--disable-network-policy",...
k3s.io/node-config-hash: J3VVR3YO2OA6PDZ46YL66JQEAUUT76Q3TEYIKGHD2DIUBZ25C2DA====
k3s.io/node-env: {"K3S_DATA_DIR":"/mnt/live/ix-applications/k3s/data/3e0ae04e9af018fef47e33431e5617a8f12b335366444149a0c8165f10f59438"}
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Wed, 13 Oct 2021 00:13:21 +0200
Taints:
Unschedulable: false
Lease:
HolderIdentity: ix-truenas
AcquireTime:
RenewTime: Thu, 14 Oct 2021 14:52:08 +0200
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
------ ----------------- ------------------ ------ ------- MemoryPressure False Thu, 14 Oct 2021 14:47:36 +0200 Wed, 13 Oct 2021 00:13:20 +0200 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Thu, 14 Oct 2021 14:47:36 +0200 Wed, 13 Oct 2021 00:13:20 +0200 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Thu, 14 Oct 2021 14:47:36 +0200 Wed, 13 Oct 2021 00:13:20 +0200 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Thu, 14 Oct 2021 14:47:36 +0200 Wed, 13 Oct 2021 00:13:31 +0200 KubeletReady kubelet is posting ready status. AppArmor enabled
Addresses:
InternalIP: 192.168.0.112
Hostname: ix-truenas
Capacity:
cpu: 4
ephemeral-storage: 434857600Ki
gpu.intel.com/i915: 0
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 16129380Ki
pods: 110
Allocatable:
cpu: 4
ephemeral-storage: 423029472949
gpu.intel.com/i915: 0
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 16129380Ki
pods: 110
System Info:
Machine ID: 6980d26ba26e429a86dc2adf2a90defe
System UUID: 1e7b879b-f24c-46ed-b33e-cd40ef21af09
Boot ID: 2bcf0fcb-7cd5-4d44-b503-89c95fbf5464
Kernel Version: 5.10.42+truenas
OS Image: Debian GNU/Linux 11 (bullseye)
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://20.10.8
Kubelet Version: v1.21.0-k3s1
Kube-Proxy Version: v1.21.0-k3s1
PodCIDR: 172.16.0.0/16
PodCIDRs: 172.16.0.0/16
Non-terminated Pods: (4 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- —
kube-system coredns-7448499f4d-7zwqc 100m (2%) 0 (0%) 70Mi (0%) 170Mi (1%) 38h
kube-system openebs-zfs-node-748nl 0 (0%) 0 (0%) 0 (0%) 0 (0%) 38h
kube-system intel-gpu-plugin-jzhjt 0 (0%) 0 (0%) 0 (0%) 0 (0%) 38h
kube-system openebs-zfs-controller-0 0 (0%) 0 (0%) 0 (0%) 0 (0%) 38h
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------ cpu 100m (2%) 0 (0%)
memory 70Mi (0%) 170Mi (1%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
gpu.intel.com/i915 0 0
Events:
and just for the record, the application events
bitnami/postgresql:13.4.0@sha256:d206387b26ef8716b2c8d30eb987b58a05cca5ab735248fe21515b0f18eb01d6Up to date
docker.io/bitnami/redis:6.2.6-debian-10-r0Up to date
docker.io/nextcloud:22.2.0@sha256:842b3bfd60633639be8d631cba82b3e0fab06bd98259c6ee1bce03ec1e84390dUp to date
0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.
0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.
0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.
0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.
2021-10-14 12:20:00
Not starting job because prior execution is running and concurrency policy is Forbid
2021-10-14 12:11:10
waiting for a volume to be created, either by external provisioner "zfs.csi.openebs.io" or manually created by system administrator
2021-10-14 12:11:10
waiting for a volume to be created, either by external provisioner "zfs.csi.openebs.io" or manually created by system administrator
2021-10-14 12:11:10
waiting for a volume to be created, either by external provisioner "zfs.csi.openebs.io" or manually created by system administrator
2021-10-14 12:11:10
waiting for a volume to be created, either by external provisioner "zfs.csi.openebs.io" or manually created by system administrator
2021-10-14 12:11:10
waiting for a volume to be created, either by external provisioner "zfs.csi.openebs.io" or manually created by system administrator