乐趣区

关于java:记一次自定义Tomcat-ClassLoader问题排查

jiezi

问题形容

以后开发的我的项目须要隔离 spark 环境，因而自定义实现了 SparkClassLoader。然而真正打包在服务器上运行的时候，利用须要初始化 SparkSession，然而报出了如下谬误：

20/04/09 16:57:02 ERROR SparkContext: Error initializing SparkContext.
java.lang.ClassCastException: org.apache.spark.serializer.JavaSerializer cannot be cast to org.apache.spark.serializer.Serializer
    at org.apache.spark.SparkEnv$.create(SparkEnv.scala:295)
    at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:187)
    at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:257)
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:424)
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2523)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935)
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926)
    at org.apache.spark.sql.SparderContext$$anonfun$initSpark$1$$anon$4.run(SparderContext.scala:128)
    at java.lang.Thread.run(Thread.java:748)

解决过程

  1. 通过 JVM -verbose:class 参数查看两个类加载的 jar 包，看一下是不是包抵触的问题 
[Loaded org.apache.spark.serializer.Serializer from file:/root/wangrupeng/spark/jars/spark-core_2.11-2.4.1-os-kylin-r3.jar]
[Loaded org.apache.spark.serializer.JavaSerializer from file:/root/wangrupeng/spark/jars/spark-core_2.11-2.4.1-os-kylin-r3.jar]

后果发现两个类都是从同一个 jar 包中加载的，排除依赖抵触的起因。

  1. 借助阿里巴巴的强力工具 Arthas

官网网址 

curl -O https://alibaba.github.io/arthas/arthas-boot.jar
java -jar arthas-boot.jar
# 输出我的项目的过程 ID
sc classloader #发现咱们自定义的 classloader 有两个实例
sc -d org.apache.spark.serializer.JavaSerializer # 不出意外，改类被两个 classloader 实例别离加载了两次

起因找到了，是因为两个类被两个 classloader 实例加载了两次，而后 class cast 的时候是两个不同 classloader 加载的，所以导致了 ClassCastException

  1. 为什么 classloader 会初始化两次呢？

因为这个 SparkClassLoader 是咱们本人定义的，所以我在其构造函数中打印了一下 Stack 信息，这样就可能看到这个类实例的初始化过程了 

protected SparkClassLoader(ClassLoader parent) throws IOException {super(new URL[] {}, parent);
        init();
        Thread.dumpStack();}

最终日志中相干输入如下：

java.lang.Exception: Stack trace
        at java.lang.Thread.dumpStack(Thread.java:1336)
        at org.apache.kylin.spark.classloader.DebugTomcatClassLoader.<init>(DebugTomcatClassLoader.java:75)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.catalina.loader.WebappLoader.createClassLoader(WebappLoader.java:753)
        at org.apache.catalina.loader.WebappLoader.startInternal(WebappLoader.java:598)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5581)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1016)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:992)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:639)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1127)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:2020)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)


java.lang.Exception: Stack trace
        at java.lang.Thread.dumpStack(Thread.java:1336)
        at org.apache.kylin.spark.classloader.DebugTomcatClassLoader.<init>(DebugTomcatClassLoader.java:75)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.catalina.loader.WebappLoader.createClassLoader(WebappLoader.java:753)
        at org.apache.catalina.loader.WebappLoader.startInternal(WebappLoader.java:598)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5581)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:1016)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:992)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:639)
        at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1296)
        at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:2038)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

能够看到是 Tomcat 在部署 web 实例的时候初始化的 ClassLoader 实例，而后重点关注，为什么 deploy 了两次，别离是 DeployDirectory 和 DeployWar 各一次，然而通过查看 Tomcat 官网文档能够晓得部署一个 web 利用这两种形式只会抉择一个，但为什么呈现了两次？

  1. 最终排查

最终发现是因为 tomcat/webapp 目录下有两个 app 目录，删掉一个没有用的就能够啦。

退出移动版