> ## Documentation Index
> Fetch the complete documentation index at: https://docs.domino.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Pods stuck in ContainerCreating with an event mentioning errno 524

#### ***Version/Environment (if relevant):***

All versions on AWS/EKS

## **Issue:**

Customers find Pods stuck in ContainerCreating and/or liveness probes fail with an event in the kubectl describe which mentions **`errno 524`**:

```
Warning FailedCreatePodSandBox 4m9s (x384 over 89m) kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: unable to init seccomp: error loading seccomp filter into kernel: error loading seccomp filter: errno 524: unknown
```

##

## **Root Cause:**

There is a linux kernel memory leak bug in the EKS optimized AMI based on linux kernel version 5.10.x. The linux bug can be verified by using *`cat /proc/sys/net/core/bpf_jit_limit`* and seeing what the limit currently is and by using *`cat /proc/vmallocinfo | grep bpf_jit | awk '{s+=$2} END {print s}'`*, to see if the limit has been exceeded.

Example:

```
[root@ip-10-0-35-23 /]# cat /proc/sys/net/core/bpf_jit_limit
264241152
[root@ip-10-0-35-23 /]# cat /proc/vmallocinfo | grep bpf_jit | awk '{s+=$2} END {print s}'
352337920
```

More details: [https://github.com/awslabs/amazon-eks-ami/issues/1219#issuecomment-1533797378](https://github.com/awslabs/amazon-eks-ami/issues/1219#issuecomment-1533797378)

"This issue is more likely to be encountered with kernel versions *`kernel-5.10.176-157.645.amzn2`thru*`kernel-5.10.177-158.645.amzn` where the rate of the memory leak is higher."

***

## **Resolution:**

AWS corrected the problem by patching by a new AMI [Release AMI Release v20230501 awslabs/amazon-eks-ami](https://github.com/awslabs/amazon-eks-ami/releases/tag/v20230501), released May 3rd 2023.

Temporary workaround is to run ***`sysctl net.core.bpf_jit_limit=452534528`*** on all **platform** nodes by ssh'ing directly onto them.

The permanent fix is to get your EKS to use this AMI: [Release AMI Release v20230501 awslabs/amazon-eks-ami](https://github.com/awslabs/amazon-eks-ami/releases/tag/v20230501)

## **Notes/Information:**

Note the *"temporary workaround"* is short-term only, since the kernel bug can continue leaking.


## Related topics

- [Troubleshooting](/knowledge-base/troubleshooting-overview.md)
- [How to Troubleshoot Kubernetes Autoscaling (ASG) Issues in Domino](/knowledge-base/how-to-troubleshoot-autoscaling-asg-issues.md)
- [Container & Kubernetes security](/cloud/admin/security-and-compliance/pod-security-policies.md)
- [FailedMount event "efs.csi.aws.com not found in the list of registered CSI drivers"](/knowledge-base/failedmount-event-efs-csi-aws-com-not-found-in-the-list-of-registered-csi-drivers.md)