Source code for Hadoop in Practice, Second Edition
This project contains the source code that accompanies the book
Hadoop in Practice, Second Edition.
License
Apache version 2.0 (for more details look at the license).
Usage
Tarball
The easiest way to start working with the examples is to download a tarball distribution of this project.
Doing so will mean that running your first example is just three steps away:
-
Go to the releases and download the most recent tarball.
-
Extract the contents ot the tarball.
$ tar -xzvf hip-<version>-package.tar.gz -
The examples in the book all use the
hipscript located inbin/hipto
execute the examples. While it's not required, it's recommended that you
addhip-<version>/binto your
path so that you can simply executehipand execute the examples in the
book by directly copy-pasting the commands. -
Run the "hello world" example, which is
$ cd hip-<version>
# create two input files in HDFS
$ hadoop fs -mkdir -p hip1/input
$ echo "cat sat mat", hadoop fs -put - hip1/input/1.txt
$ echo "dog lay mat", hadoop fs -put - hip1/input/2.txt
# run the inverted index example
$ ./hip hip.ch1.InvertedIndexJob --input hip1/input --output hip1/output
# examine the results in HDFS
$ hadoop fs -cat hip1/output/part*
Done! The tarball also includes the sources and JavaDocs.
Building your own distribution
Here you're going to checkout the trunk and then use Maven to run a build.
-
Checkout the code.
$ git clone git@github.com:alexholmes/hiped2.git -
Build the code and distribution tarball.
$ cd hiped2
$ mvn clean
$ mvn validate
$ mvn package
The JAR's and tarball will be under the target directory. Now you can follow the instructions in the
"Tarball" section above to explode the tarball and run an example.
What's next?
At this point check out the book for more examples and how you can execute them. Or if you find any issues then
please go to the issues and open a new issue.