본문 바로가기
프로그래밍/Spark

IntelliJ Spark test fail(solved)

by 정리 습관(★arranging★) 2020. 8. 24.
728x90

intelliJ 를 이용해 spark 관련 업무를 진행하기 위해,
셋팅 중 아래와 같은 문제에 봉착했다.

D:\Disk\jdk1.8.0_144\bin\java.exe -Dmaven.multiModuleProjectDirectory=D:\test "-Dmaven.home=C:\Program Files\JetBrains\IntelliJ IDEA 2019.2.4\plugins\maven\lib\maven3" "-Dclassworlds.conf=C:\Program Files\JetBrains\IntelliJ IDEA 2019.2.4\plugins\maven\lib\maven3\bin\m2.conf" "-Dmaven.ext.class.path=C:\Program Files\JetBrains\IntelliJ IDEA 2019.2.4\plugins\maven\lib\maven-event-listener.jar" "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA 2019.2.4\lib\idea_rt.jar=55013:C:\Program Files\JetBrains\IntelliJ IDEA 2019.2.4\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\JetBrains\IntelliJ IDEA 2019.2.4\plugins\maven\lib\maven3\boot\plexus-classworlds-2.6.0.jar" org.codehaus.classworlds.Launcher -Didea.version2019.2.4 -s D:\mavenRepo\settings.xml -Dmaven.repo.local=D:\mavenRepo\repository package [INFO] Scanning for projects... [INFO] [INFO] ---------------------------< com.hmns:test >---------------------------- [INFO] Building test 1.0-SNAPSHOT [INFO] --------------------------------[ jar ]--------------------------------- Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.481 s [INFO] Finished at: 2020-08-24T15:17:58+09:00 [INFO] ------------------------------------------------------------------------ [ERROR] Plugin org.apache.maven.plugins:maven-resources-plugin:2.6 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-resources-plugin:jar:2.6: Could not transfer artifact org.apache.maven.plugins:maven-resources-plugin:pom:2.6 from/to central (https://repo.maven.apache.org/maven2): sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true


이슈에 대해 여러 자료를 찾아보고 이것 저것 해보았지만,
해결 방법은 의외로 간단했다.

원인은 JDK 인증 문제로 결론내었다.

intelliJ를 설치할때 JDK 13버전이 내장되어 있었고,
라이센스를 가지고 있는 상태여서 해당 버전의 JDK로 프로젝트를 package하고
다시 안되었던 동작을 수행 했더니, 잘 동작함을 알 수 있었다.


라이센스 버전을 사용하고 있지 않은 intelliJ 유저는 open JDK를 설치해서 
문제를 해결할 수 있을 것으로 보인다.

intelliJ build(package) Error solving
It about SDK certification problerm.

If you have licensed IntelliJ you just choose jdk 13 option by project setting(short cut Key F4).
you don't have licensed IntelliJ then you setting openJDK newest version.


이제 spark "hello world" 를 해볼 차례다

댓글