FutureLearn Week 2: Post 2 of 4

With big data becoming much more important in recent years the need for data scientists has also increased. These data scientists are educated in discovering, collecting, processing, analysing and presenting information gained from these large amounts of data and often compare and combine it with different sources.

The skills these data scientists must have are data visualisation skills, statistical skills, data processing and systems engineering skills, they must have an understanding or have a sufficient level of skill with programming languages as well as mathematical and analytical skills.

Comments

Post #3: Growth of Big Data

There was an incredible amount of internet growth in the 1990s, and personal computers became steadily more powerful and more flexible. Internet growth was based both on Tim Berners-Lee’s efforts, CERN’s free access, and access to individual personal computers. In 2005, Big Data, which had been used without a name, was labelled by Roger Mougalas. He was referring to a large set of data that, at the time, was almost impossible to manage and process using the traditional business intelligence tools available. Additionally, Hadoop, which could handle Big Data, was created in 2005. Hadoop was based on an open-sourced software framework called Nutch, and was merged with Google’s MapReduce. Hadoop is an Open Source software framework, and can process structured and unstructured data, from almost all digital sources. Because of this flexibility, Hadoop (and its sibling frameworks) can process Big Data. Big Data is revolutionising entire industries and changing hum...

FutureLearn Week 2: Post 3 of 4

Two of the biggest challenges of big data is Analysing and Visualising the data. Firstly with analysing the data, the size of big data files can sometimes be substantial, there are many things that must be considered before downloading the data, for example the file size, how long the data file will take to download, will all of it be necessary or will part of the file suffice and is there enough storage space within the system itself. Visualisation is way to represent the data in a way that is easier to understand such as word clouds and things of the like. This will aid users in seeing the prominent and key terms from the analysis of the data sets. The first step after downloading the data would be to quality check it to ensure that each field had the appropriate data types in each field and to ensure that the user understood the meaning of each field. Keeping a copy of the original data would be essential as well as each documented version change for each stage of visualisation....

Post #4: Reasons for the Growth of Big Data

Big Data is continuously growing, each and every organisation is dealing with more and more data with each passing day and this growth shows no signs of slowing down. There are various reasons for this swift increase in growth, I will now discuss a few of these reasons. Business models are one of the main reasons for this exponential growth through the aggressive and continuous acquisition and permanent retention of data. Google is a perfect example of a business that is retaining vast amounts of data and this is definitely working in their favour as can be seen by their company growth. Infrastructure capacity is another reason for the increase as the cost of data storage has become incredibly low over the past few years while the capacity seems to be increasing almost doubling in the space of a couple of years. Business analytics has also seen an increased acceleration in the past few years and is now over a 100 billion dollar market and continues to grow year to year. Regul...

Blog For Big Data

Search This Blog