Upgrade Spark version

This article summarizes the phenomena, causes, and handling suggestions for problems related to Spark upgrade to facilitate quick troubleshooting.

question

The spark version used by the customer's customized job is too old and a customized spark version is required.

solution

1. Log in to the spark official website and download the corresponding spark version binary installation package.

• https://archive.apache.org/dist/spark/

2. Delete local spark-min

3. Modify the spark download link in the install.sh script

4. Execute the install command and re-download the spark installation package

5. Modify the global spark dependency version

6. Repackage using isx package