Module 0. What is big data? What is data science?

 This week was very interesting. From reading multiple 60+ page papers to submitting a two question quiz. Big data. It's certainly a buzz word in the modern age. Everyone and their dog is talking about, taking ownership of and requesting big data. But does it live up to the hype? Well this week's readings certainly seem to point towards that.

Firstly, I wanted to abstract that I most recently took a class called Data Mining and I was blown away how easy it was to create a neural network in R that was achieving 90% accuracy rates for very complex problems. Technology, and more specifically software, is really catching up with some of these hard problems. It seems now-a-days we have an hunger for more and more data. We're pushing the boundary on what is and what isn't data privacy. Honestly at times, it's a bit gross.

First let's review the lectures.

1.

Blogging, facebook and linked-in oh my! This will certainly be an interesting class experience. I wonder how much data will LinkedIn, Google and Facebook be able to collect and sell off of us while we complete this course. I'll probably get some some web cloud infrastructure advertisements very soon. Not too much to analyze here so I'll keep it short, I'm really looking forward to this class.

2.

Diving into big data. The thing I found most interesting was the paradigm shifts. To me, atleast partially, it feels like in a sense that companies are finding ways to really squeeze profits out of everything they can. Somewhat like how a company that makes orange juice is going to use almost every part of the orange. Not this is is necessarily a bad thing, but if you're following my metaphor it means they will have to sift through a bunch of rind. I think it's smart, and now that computers have storage abilities to store all this data we'll certainly see it being used more often. Something interesting not being brought up in these lectures is how companies are also using this data nefariously. I think it's important to talk about the good and the bad.

The other part I found most interesting is the process of Business Intelligence. It makes sense that traditionally most data came from transnational data, as I imagine in the past most data wasn't recorded unless it was absolutely necessary.
This image really made sense to me.

 

This image truly shows how many sources one could use to generate data from. I'm personally most interested in Data Mining but I also understand the importance of all the other circles as-well.


Let's review the readings.

Firstly:

1. Big Data Gets Personal

Wow, this read was awesome, and chock full of fun stories. I wont talk about them all here but one of the ones I found most interesting was how they were able to determine malaria hot spots just by looking at cellphone tower usage. This is certainly something I didn't expect to read and can only commend the out of the box thinking that led to this. What a great step, and useful tool for fighting such an awful disease.

2. McKinsey Global Institute - Ten IT Enabled Business Trends for the Decade ahead

Does anybody ever get tired of lists, I think I've read enough buzzfeed articles to know that anytime a list was brought up it's probably just some arbitrary concoction and likely to have to forced/poor points. Hope I'm not offending anybody but a number of these points could of been combined and some could of been expanded. For example; the me + ease + free is talking about three different trends completely. They're actually all valid points and honestly a bit more valid than some of the other trends in this article in my opinion. I agree with the me + ease + free trend but I think they're all so separate they should be talked about independently. 

3.  Big Data Widens Analytic Talent Gap

Yay! If you're going into big data you're likely to get more pay than an average engineer. Very exciting for us involved in big data. I love big data but I have to wonder how many of these high paying jobs are also the same companies pushing boundaries on privacy. That would be a huge 'no thank you' from me. There are a ton of figures in this so I can't possibly go over them all. Still sad to see that the profession is very dominated by males. We should use big data to figure out how to fix this problem.

4. The Rise of Big Data Foreign Affairs

The concept of big data vs big brother is something I think of quite often. It's amazing to me that we're literally approaching the plot of 2002 blockbuster "The Minority Report". The idea of common sense and a human touch is really important in this article and I agree with it thoroughly.

5.  What_is_big_data?

An informative read. I think many of the points covered in this article were covered by the "Big Data Gets Personal" article. The growth of data doesn't surprise me, with computers everything seems to be growing exponentially. This article is certainly a no nonsense hitter as it clearly explains the who, what, when, where and why of big data. I agree with the author that businesses truly need to adapt or die. It's such an important field to be in at the moment.


I wanted to share an article from wired about the duality of big data. How it's being used to help healthcare and... additionally being used for pure evil. The article is a very short read and of course since it's from Wired it's entertaining as well.

Riedel, Daniel. “The Duality of Big Data: The Angel and the Demon.” Wired, Conde Nast, 7 Aug. 2015, www.wired.com/insights/2014/10/duality-big-data/.