Big Data Analytics with Hadoop and Spark
Last few years have seen significant advances in the technologies for data collection, data transmission and storage of data. For the first time in the history, thanks to these technologies, we are at a position where we can instrument a scientific phenomenon, or a business workflow, at a very fine granularity and the collect the required data for the relevant time period. This simple approach can be used for diverse phenomenon – evolving traffic patterns in a city, or how social media plays a part in enhancing/diminishing a brand, or how a protein will fold and impact their function. Think of it as the technology has given us a ringside seat to observe the phenomenon in an unprecedented detail.
Though we might have earned ringside seat, we still have to do some work to get a ringside view on the phenomenon. The data that is collected and stored is far from usable, and in its raw form will not give us the understanding we are looking for. There is a significant analysis that needs to be done on the data before we can learn from that data and benefit from that learning. This task of analyzing this large amount of data to extract insights is the basis the field of data science. A new set of technologies have mushroomed to support the use cases demanded by data science, these are the big data technologies.
For further information see the course page: