[Bug] kyuubi report an error "Failed to initialize the embedded ZooKeeper server" during initialization. #7087
Description
Code of Conduct
- I agree to follow this project's Code of Conduct
Search before asking
- I have searched in the issues and found no similar issues.
Describe the bug
Describe
Port 2181 is occupied by the QuorumPeerMain process (ZooKeeper main process), causing kill -9 to fail and kyuubi to report an error "Failed to initialize the embedded ZooKeeper server" during initialization.
Solution:
Linux zombie processes or kernel-mode processes will ignore kill -9, and the parent process must be terminated to completely clear them. (Stop QuorumPeerMain via supervisorctl stop zookeeper)
kyuubi's built-in ZooKeeper depends on port 2181 to start. If it is occupied, the service that depends on it will fail to initialize.
Requirement:
Do you have any other solutions?
Problem reproduction:
sh-5.0# jps
72 QuorumPeerMain
sh-5.0# kill -9 72
sh-5.0# jps
5226 QuorumPeerMain
The root cause of the failure of kill -9
The process may be in a zombie state or kernel state blocking, causing the SIGKILL signal to be ignored.
Typical phenomenon: After forced termination, the process ID changes (such as 72 → 5226), indicating that the child process or new instance of the original process is pulled up
执行命令
/opt/kyuubi/bin/kyuubi restart
报错日志如下：
2025-06-08 10:06:04.601 INFO main org.apache.kyuubi.Utils: Loading Kyuubi properties from /etc/kyuubi/conf/kyuubi-defaults.conf
2025-06-08 10:06:04.779 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT
2025-06-08 10:06:04.779 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:host.name=hadoop-master1.orb.local
2025-06-08 10:06:04.779 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:java.version=1.8.0_432
2025-06-08 10:06:04.779 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:java.vendor=Azul Systems, Inc.
2025-06-08 10:06:04.779 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:java.home=/opt/zulu8.82.0.21-ca-jdk8.0.432-linux_aarch64/jre
2025-06-08 10:06:04.779 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:java.class.path=/etc/kyuubi/conf:/opt/kyuubi/web-ui:/opt/kyuubi/jars/jetcd-api-0.7.7.jar:/opt/kyuubi/jars/jersey
2025-06-08 10:06:04.780 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:java.library.path=/opt/hadoop/lib/native:/usr/java/packages/lib/aarch64:/lib:/usr/lib
2025-06-08 10:06:04.780 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:java.io.tmpdir=/tmp
2025-06-08 10:06:04.780 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:java.compiler=<NA>
2025-06-08 10:06:04.780 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:os.name=Linux
2025-06-08 10:06:04.780 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:os.arch=aarch64
2025-06-08 10:06:04.780 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:os.version=6.10.14-linuxkit
2025-06-08 10:06:04.780 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:user.name=root
2025-06-08 10:06:04.780 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:user.home=/root
2025-06-08 10:06:04.780 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Server environment:user.dir=/
2025-06-08 10:06:04.789 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: Created server with tickTime 3000 minSessionTimeout 6000 maxSessionTimeout 60000 datadir /opt/kyuubi/embedded_zookeeper/version-2 s
2025-06-08 10:06:04.789 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: minSessionTimeout set to 6000
2025-06-08 10:06:04.789 INFO main org.apache.kyuubi.shaded.zookeeper.server.ZooKeeperServer: maxSessionTimeout set to 60000
2025-06-08 10:06:04.796 INFO main org.apache.kyuubi.shaded.zookeeper.server.NIOServerCnxnFactory: binding to port /172.18.0.4:2181
Exception in thread "main" java.lang.RuntimeException: Failed to initialize the embedded ZooKeeper server, binding to 172.18.0.4:2181
at org.apache.kyuubi.zookeeper.EmbeddedZookeeper.initialize(EmbeddedZookeeper.scala:71)
at org.apache.kyuubi.server.KyuubiServer$.$anonfun$startServer$1(KyuubiServer.scala:51)
at scala.Option.foreach(Option.scala:407)
at org.apache.kyuubi.server.KyuubiServer$.startServer(KyuubiServer.scala:50)
at org.apache.kyuubi.server.KyuubiServer$.main(KyuubiServer.scala:103)
at org.apache.kyuubi.server.KyuubiServer.main(KyuubiServer.scala)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:461)
at sun.nio.ch.Net.bind(Net.java:453)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)
at org.apache.kyuubi.shaded.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:90)
at org.apache.kyuubi.zookeeper.EmbeddedZookeeper.initialize(EmbeddedZookeeper.scala:64)
... 5 more
Affects Version(s)
1.10.0
Kyuubi Server Log Output
Kyuubi Engine Log Output
Kyuubi Server Configurations
Kyuubi Engine Configurations
Additional context
No response
Are you willing to submit PR?
- Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
- No. I cannot submit a PR at this time.