Search This Blog

Thursday, May 3, 2012

HortonWorks distribution: secretly powered by iTest and BigTop?

As I've mentioned here, and here BigTop is a real neat concept that I and a couple of my friends have put together a couple of years ago. Interestingly enough, second incarnation of the concept (known as iTest) made (back then) my manager to accusing me of stealing software in favor of ASF, followed by forceful departure from Cloudera (kick in the ass, with 'I am sorry' kinda smile on their face). But this all this in the past. The present however is much more interesting...

Some evidences were found that BigTop project (see first two links above) is seen as a power behind commercial offerings of some of the leading Hadoop vendors. Here how interesting it gets:

  • you can get HortonWorks' stack AMI from this link to play with and learn about Hadoop and stuff.
  • now let's see what they are using to power-up their distro
$ grep -i bigtop install_oozie.sh:     --extra-dir=DIR    path to Bigtop distribution files
install_oozie.sh:     --build-dir=DIR    path to Bigtop distribution files
install_oozie.sh:if [ -e /usr/libexec/bigtop-detect-javahome ]; then
install_oozie.sh:  . /usr/libexec/bigtop-detect-javahome
install_oozie.sh:elif [ -e /usr/lib/bigtop-utils/bigtop-detect-javahome ]; then
install_oozie.sh:  . /usr/lib/bigtop-utils/bigtop-detect-javahome
oozie.init:if [ -e /usr/libexec/bigtop-detect-javahome ]; then
oozie.init:  . /usr/libexec/bigtop-detect-javahome
oozie.init:elif [ -e /usr/lib/bigtop-utils/bigtop-detect-javahome ]; then
oozie.init:  . /usr/lib/bigtop-utils/bigtop-detect-javahome
oozie.spec:#Requires: bigtop-utils
now if you look a bit further, there some more hints that BigTop has been used big time, but that hasn't been hidden away very well...
oozie.init:if [ -e /usr/libexec/bigtop-detect-javahome ]; then
oozie.spec:#Requires: bigtop-utils
Wow, I feel proud, guys, I really am. Now I can officially claim that my ideas are giving the push to some of the biggest vendors of what is considered the hottest technology on the market ;) Although, they are unlikely to admit this publicly, predicating on the above example.

I will keep you posted on the development.

No comments:

Post a Comment

Followers