Flume v1.9.0启动报错ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process

背景

hadoop 3.2.2,flume 1.9.0版本操作hdfs sink报错

2022-05-16 23:07:35,002 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.hdfs.HDFSDataStream.configure(HDFSDataStream.java:57)] Serializer = TEXT, UseRawLocalFileSystem = false
2022-05-16 23:07:35,048 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:459)] process failed
java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
	at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
	at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
	at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1679)
	at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:221)
	at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:572)
	at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:412)
	at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
	at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
	at java.lang.Thread.run(Thread.java:745)
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
	at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
	at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
	at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1679)
	at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:221)
	at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:572)
	at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:412)
	at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
	at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
	at java.lang.Thread.run(Thread.java:745)

内容：ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:459)
java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)

报错原因

Hadoop 3.2.2 中的 guava 版本和 Flume 1.9.0 中的版本不一致
经查Hadoop 3.2.2 中 guava-27.0-jre.jar

[hadoop@hadoop001 ~]$ ll app/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar 
-rw-r--r--. 1 hadoop hadoop 2747878 Jan  3  2021 app/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar

Flume 1.9.0 中 guava-11.0.2.jar

[hadoop@hadoop001 ~]$ ll app/flume/lib/guava-11.0.2.jar 
-rw-rw-r--. 1 hadoop hadoop 1648200 Sep 13  2018 app/flume/lib/guava-11.0.2.jar

解决方案

删除低版本的 guava ，把高版本的 guava 复制一份放在低版本处
删除 Flume 中的 guava,把 Hadoop 中的 guava 复制到 Flume 中

[hadoop@hadoop001 ~]$ rm app/flume/lib/guava-11.0.2.jar
[hadoop@hadoop001 ~]$ ll app/flume/lib/guava-11.0.2.jar 
ls: cannot access app/flume/lib/guava-11.0.2.jar: No such file or directory
[hadoop@hadoop001 ~]$ cp app/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar app/flume/lib
[hadoop@hadoop001 ~]$ ll app/flume/lib/guava-27.0-jre.jar 
-rw-r--r--. 1 hadoop hadoop 2747878 May 16 23:18 app/flume/lib/guava-27.0-jre.jar

重新启动 Flume 问题解决！

参考链接：https://blog.csdn.net/llwy1428/article/details/112169028