20170109 – Twin Cities Spark and Hadoop User Group Meeting


I missed this meeting due to weather issues in the area.  Fortunately, the meeting was recorded and posted online.  Doug Cutting, Chief Architecture at Cloudera was the guest speaker.


This wasn’t a prepared speech, so there was some ah’s and um’s.  The content was good overview of the Hadoop ecosystem – some of the history of how we got here along with what potential systems could like in the near future.

 “We don’t know where we are going, but we are well on our way”

<<I just love that quote – seemed fitting for this topic.>>  There are many large, legacy IT systems that are operating with very high performance.  These systems combine many years of engineering, requirements, design, maintenance and enhancements to get to where they are at today.  Yet many IT departments will say the current legacy system that is business critical is end of life and needs to be replaced.  The cost and complexity to replace legacy is huge, and the question of “where to start” is a barrier that many IT departments aren’t able to overcome.

Cloud and Hadoop technologies offer new tools for an IT department’s toolbox.  Yet many use cases do not have a clear path whether to use new or existing technologies (ie whether to use a relational database or a NoSQL database).  This will lead to many architecture debates which is the best approach.  The business line will want tried and true technologies to get a product out to market faster.  IT folks will push for new frameworks when designing a new system.

A speech “looking ahead 10 years” is difficult to organize.  There are a wide range of technology options that are out on the market now.  These technologies/frameworks will be the basis for the next generation of applications.  Learning about project experiences rolling out Hadoop and the use cases the framework solved helps understand what is possible today that wasn’t possible 10 years ago.

Additional Information

In June of 2015, Mike Olson, CEO and cofounder of Cloudera, was a keynote speaker at University of St. Thomas Graduate Programs in Software (GPS) 30th anniversary dinner.  The video clip contained that speech.  The first twenty minutes is about the UST’s GPS program.  The remaining video is Mr. Olson’s speech.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: