Spark Boy Mac OS
I have encountered lots of tutorials from 2019 on how to install Spark on MacOS, like this one. However, due to a recent update on the availability of Java through Homebrew, these commands do not work anymore.
- Spark is an incredible email app for Mac and iOS from the house of Readdle, one of the leading developers of productivity apps. Spark allows you to add multiple email accounts to your Mac. Adding your email account to Spark for Mac is a quick and easy process, that will help you get started with your email account in no time.
- The requirements include a Mac running OS X 10.7.x or above, a dual-core Intel processor, 1GB of RAM, and an Intel HD 3000 GPU. This implies that the user will be able to play the game on almost.
If you don’t have Homebrew, here’s the command:
Mac 2.66GHz Intel Quad, 4GB RAM /OSX.5.6 /30 inch cinema display+20 inch Cinema Display Finale 2008 2009 2010/Logic Pro Studio/Komplete/GPO/Kore 2/Max.msp/Pluggo Back to Top.
Spark requires Java8, and this is where I had to browse Github to find this alternative command:
Spark Boy Mac Os Catalina
You probably know it, but Apache-Spark is written in Scala, which is a requirement to run it.
We’re almost there. Let’s now install Spark:
You might want to write your Spark code in Python, and pySpark will be useful for that:
Whether you have bashrc or zshrc, modify your profile with the following commands. Adapt the commands to match your Python path (using which python3) and the folder in which Java has been installed:
Finally, source the profile using:
And you are all set!
Now, in your Jupyter notebook, you should be able to execute the following commands:
And observe the SparkUI on the following link: http://localhost:4040/.
Spark is an in-memory open source cluster computing system allowing for fast iterative and interactive analytics. Spark utilizes Scala – a type-safe objected oriented language with functional properties that is fully interoperable with Java. For more information about Spark, please refer to http://spark-project.org. To test out Spark, you can install the stand-alone version on Mac OSX.
Install Scala 2.9.2
The first thing you will need to do is to install Scala 2.9.2 as Spark 0.6.1 is dependent on it. As of this posting, the current version of Scala is 2.10 but there are some issues with Spark 0.6.1 and Scala 2.10 as noted in this thread.
1) A handy way to installing Scala is to use Home Brew; please reference Installing Hadoop on OSX Lion (10.7) for more information on how to use Home Brew as well installing Hadoop on Mac OSX. It may be handy to install Hadoop so that way you can use Spark against HDFS as well.
2) The current Home Brew scala formula installs Scala 2.10 but you will need to use Scala 2.9.2. A quick way to do to this is to modify the scala.rb formula (/usr/local/Library/Formula/scala.rb) to install Scala 2.9.2.
3) Installing Scala via HomeBrew by typing the command in a bash terminal:
Upon running this command, scala will be located in /usr/local/Cellar/scala
Install Git Command Line for Mac
Ensure you have Git for Mac installed (even if you have GitHub for Mac installed; need to install Git so you can run from the command line)
Ensure you have set the JAVA_HOME and SCALA_HOME variables
In my case, I have configured my .profile with the following:
Installing Spark 0.6.1
1) Obtain the pre-built Spark 0.6.1 package at http://spark-project.org/downloads/. The direct link for the prebuilt package is:
http://github.com/downloads/mesos/spark/spark-0.6.1-prebuilt.tgz
2) Open up the tgz file and place it into a folder where you will install Spark. For example, I placed mine in the HomeBrew Cellar location, i.e.
/usr/local/Cellar/spark-0.6.1
Configure and Build Spark 0.6.1
Follow the instructions as per the README.MD in /usr/local/Cellar/spark-0.6.1
1) Run the Simple Build Tool (SBT) package from /usr/local/Cellar/spark-0.6.1
2) Modify the conf/spark-env.sh
Ensure that SCALA_HOME variable has been set
Running Spark 0.6.1
From here, you can now run Spark examples. Just in case, run the conf/spark-env.sh to set the Scala enviornment variables.
and to run the spark shell:
Spark Boy Mac Os Download
where local indicates standalone (vs. EC2, cluster, mesos, etc.) and [x] is the number of cores.
Spark Boy Mac Os X
Enjoy!