Overview of Apache Spark

Following is the link to the overview of Apache Spark, very helpful and effective:

http://stanford.edu/~rezab/sparkworkshop/slides/holden.pdf

Apache Storm WordCount Example by Hortonnworks

Please follow the simple and flawless guidelines for the setup and basic WordCount example in Apache Storm.

http://hortonworks.com/hadoop-tutorial/processing-streaming-data-near-real-time-apache-storm/

THE ONLY MISSING POINT: NOT STOPPING THE JOB i.e. TOPOLOGY
TURN OFF YOUR JOB AFTER A FEW MINUTES, OTHERWISE BEING A STREAM PROCESSING, YOUR WORKER LOGS /usr/lib/storm/logs/worker*.log WILL KEEP INCREASING.

To do so:
1) go to UI : http://localhost:8744/
2) Under Topology Summary, click on WordCount
3) On newly directed page, under Topology Actions, click on "Deactivate" or "Kill"

YARN : Complete picture of Apache Hadoop Ecosystem

Above schematic explains the complete overview of Apache Hadoop Ecosystem using YARN for:

- Batch

- Interactive

- Realtime

- Search

- In Memory

operations ...

Following image shows the broad view for data ingestion, operations and management for whole process...

Source: http://hortonworks.com/blog/pivotal-hortonworks-shared-vision-operations-enterprise-hadoop/