blog/BigData-Notes
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update Spark_Streaming整合Flume.md

Browse Source
This commit is contained in:
heibaiying 2019-05-24 10:17:40 +08:00 committed by GitHub
parent fa05e65fc9
commit 862ec14d34
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
notes/Spark_Streaming整合Flume.md
View File

@ -116,7 +116,6 @@ object PushBasedWordCount {
这里我采用的是第三种方式：使用`maven-shade-plugin`插件进行`ALL IN ONE`打包，把所有依赖的Jar一并打入最终包中。需要注意的是`spark-streaming`包在Spark安装目录的`jars`目录中已经提供，所以不需要打入。插件配置如下：
> 关于大数据应用常用打包方式单独整理至：[大数据应用常用打包方式](https://github.com/heibaiying/BigData-Notes/blob/master/notes/大数据应用常用打包方式.md)
>
> 本项目完整源码见：[spark-streaming-flume](https://github.com/heibaiying/BigData-Notes/tree/master/code/spark/spark-streaming-flume)
```xml
@ -169,8 +168,8 @@ object PushBasedWordCount {
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
</transformer>
</transformers>
</configuration>
@ -357,4 +356,4 @@ spark-submit \
## 参考资料
1. [streaming-flume-integration](https://spark.apache.org/docs/latest/streaming-flume-integration.html)
1. [streaming-flume-integration](https://spark.apache.org/docs/latest/streaming-flume-integration.html)