How to configure Presto/Hive/HDFS on Mac

It is quite a pain to setup everything.
Here are some links which helped me significantly:

Tricks:

Use java 1.7 with newest hadoop/hdfs/hive 2.0.0

To create metastore – go to $HIVE_HOME/bin and run:

schematool -initSchema -dbType derby

Derby is java in-memory database. This option will not allow you to run simultaneously Hive metastore (required for Presto) and Hive itself and so consider using mysql for metastore.

Then install presto going through instructions on prestodb.io

So to use presto – you need to shutdown Hive CLI and start metastore service from same directory where your derby is being set with schematool. To start metastore:

hive --service metastore

To check which components of Hive/HDFS are running on machine, run:

jps

To start datanode:

hdfs datanode

Create 2 aliases in ~/.bashrc to start/stop hadoop/hdfs:

alias hstart="/usr/local/Cellar/hadoop/2.7.1/sbin/start-dfs.sh;/usr/local/Cellar/hadoop/2.7.1/sbin/start-yarn.sh"
alias hstop="/usr/local/Cellar/hadoop/2.7.1/sbin/stop-yarn.sh;/usr/local/Cellar/hadoop/2.7.1/sbin/stop-dfs.sh"

Leave a Reply

Your email address will not be published. Required fields are marked *