Joyent Manta: map/reduce for Unix natives

Joyent has released Manta, a computing environment that includes a powerful map/reduce system which leverages native Unix commands. Unlike a Hadoop world where you have to spin up a Java environment to get anything done, Manta makes map/reduce as simple as possible for those of us who know the Unix shell pretty well.

Some examples may be useful.

for n in $HOME/doc/books/*
wc -l $n >> $TMP/wc-tmp
awk '{ t += $1 } END { print t }' < $TMP/wc-tmp

has an equivalent in Manta as something like

mfind -t o /$MANTA_HOME/stor/books | \
mjob create -o -m "wc -l" -r "awk '{ t += \$1 } END { print t }' "

where the word count command "wc" is the "map" step, and the awk summation is the "reduce" step.

Some other commentary on the system, which I'm still learning, is scattered around the net.

Joyent: Big compute for Big Data with Manta (screencast)

Joyent: Hello Manta: Bringing Unix to Big Data

Wanelo: A Cost-effective Approach to Scaling Event-based Data Collection and Analysis

Thanks for pointing this out, Alan Gutierrez!

Related articles

Inside Manta: Distributing the Unix shell
Joyent Launches Manta Service For Cloud Computing, Storage
Joyent goes big on big-data computing and storage with Manta service

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s