Compile the Flink 1.9.0 report, Flink FS Hadoop shaded cannot be found

Compile the Flink 1.9.0 report, Flink FS Hadoop shaded cannot be found

1.Flink source code download

git clone git@github.com:apache/flink.git

Then you can switch to different branches of the project and execute the following command to switch the code to release-1.9 branch:

git checkout release-1.9

2.Flink source compilation

mvn clean install -Dmaven.test.skip=true -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true
  • -Dmaven.test.skip: skip test code
  • -Dmaven.javadoc.skip: skip Javadoc check
  • -Dcheck style.skip: skip code style checking

By skipping these checks when maven compiles, you can save a lot of time and possibly errors.

Note: add the following to the mirror of your maven settings.xml file (so that you can download to some dependencies that cannot be downloaded).

<mirror>
  <id>nexus-aliyun</id>
  <mirrorOf>*,!jeecg,!jeecg-snapshots,!mapr-releases</mirrorOf>
  <name>Nexus aliyun</name>
  <url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>

<mirror>
  <id>mapr-public</id>
  <mirrorOf>mapr-releases</mirrorOf>
  <name>mapr-releases</name>
  <url>https://maven.aliyun.com/repository/mapr-public</url>
</mirror>

Compile and report errors

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:12 min
[INFO] Finished at: 2020-02-22T09:00:29+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project flink-s3-fs-hadoop: Could not resolve dependencies for project org.apache.flink:flink-s3-fs-hadoop:jar:1.9-SNAPSHOT: Could not find artifact org.apache.flink:flink-fs-hadoop-shaded:jar:tests:1.9-SNAPSHOT in nexus-aliyun (http://maven.aliyun.com/nexus/content/groups/public) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :flink-s3-fs-hadoop

Solution

At https://www.mvnjar.com/
Search the corresponding jar package

Add to local after download

192:flink lijun$ mvn install:install-file -DgroupId=org.apache.flink -DartifactId=flink-fs-hadoop-shaded -Dversion=1.9-SNAPSHOT -Dpackaging=jar  -Dfile=/Users/lijun/Downloads/flink-s3-fs-hadoop-1.9.0.jar
[INFO] Scanning for projects...
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for org.apache.flink:flink-scala_2.11:jar:1.9-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.flink:flink-scala_${scala.binary.version}:[unknown-version], /Users/lijun/IT/workspace/workspace_idea/bigData/flink/flink-scala/pom.xml, line 31, column 14
[WARNING] 'build.plugins.plugin.version' for org.codehaus.mojo:build-helper-maven-plugin is missing. @ org.apache.flink:flink-scala_${scala.binary.version}:[unknown-version], /Users/lijun/IT/workspace/workspace_idea/bigData/flink/flink-scala/pom.xml, line 215, column 12
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for org.apache.flink:flink-runtime_2.11:jar:1.9-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.flink:flink-runtime_${scala.binary.version}:[unknown-version], /Users/lijun/IT/workspace/workspace_idea/bigData/flink/flink-runtime/pom.xml, line 32, column 14
[WARNING] 'build.plugins.plugin.version' for org.codehaus.mojo:build-helper-maven-plugin is missing. @ org.apache.flink:flink-runtime_${scala.binary.version}:[unknown-version], /Users/lijun/IT/workspace/workspace_idea/bigData/flink/flink-runtime/pom.xml, line 424, column 12
[WARNING] Some problems were encountered while building the effective model for org.apache.flink:flink-table-planner_2.11:jar:1.9-SNAPSHOT
[WARNING] 'artifactId' contains an expression but should be a constant. @ org.apache.flink:flink-table-planner_${scala.binary.version}:[unknown-version], /Users/lijun/IT/workspace/workspace_idea/bigData/flink/flink-table/flink-table-planner/pom.xml, line 30, column 14
[WARNING] 

[WARNING] Some problems were encountered while building the effective model for or
[INFO] flink-plugins-test ................................. SKIPPED
[INFO] dummy-fs ........................................... SKIPPED
[INFO] another-dummy-fs ................................... SKIPPED
[INFO] flink-tpch-test .................................... SKIPPED
[INFO] flink-contrib ...................................... SKIPPED
[INFO] flink-connector-wikiedits .......................... SKIPPED
[INFO] flink-yarn-tests ................................... SKIPPED
[INFO] flink-fs-tests ..................................... SKIPPED
[INFO] flink-docs ......................................... SKIPPED
[INFO] flink-ml-parent .................................... SKIPPED
[INFO] flink-ml-api ....................................... SKIPPED
[INFO] flink-ml-lib 1.9-SNAPSHOT .......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.672 s
[INFO] Finished at: 2020-02-22T09:11:41+08:00
[INFO] ------------------------------------------------------------------------
192:flink lijun$ 

Problem solving

72 original articles published, praised 8, visited 310000+
His message board follow

Keywords: Apache Maven Hadoop snapshot

Added by Sekka on Sat, 22 Feb 2020 16:58:52 +0200