20 Mar Big Data: Shaping The World, Agoda Outside Knowledge Event
What is Big Data?
Nicole Lim from Red Dot Payment attended Agoda Outside’s Big Data conference on 20 March 2017, to find out about Big Data – where it comes from, how it can be used, and the issues with using it. Conferences attendees and presenters included Social Data Lab, Facebook, Google Cloud Platform and many others.
Big Data is the large volume of information compiled by software that automatically collects, sorts and processes the information. Over time, the software learns to recognise patterns with the information it is fed. This ‘machine learning’ evolves into ‘deep learning’ as more advanced programs take over to process data more rigorously and accurately, allowing for systems to generate insights. Alluxia, Mesos and S3 are examples of such programs.
After data is processed and insights are churned out we get a better understanding of the characteristics of the source of the data. For example, the analysis of supermarket data helps to shed light on a consumer buying behaviour in relation to purchasing patterns. This is where businesses can take advantage of specific patterns to deliver promotions or adjust inventory.
Big Data is also being used to solve everyday problems. A case study presented at the conference described how data analysts with the LTA went through a large amount of transport data to find out the root cause of signalling delays on the local train system. Starting from scratch with many sources of data and different perspectives, the analysts had to identify patterns that could be useful in determining solutions to the problem. It was eventually found that a single train was the reason for the delays and once it was removed, the delays were effectively eliminated. A key takeaway is that data analysts need to keep an open mind, avoid assumptions and explore different angles before arriving at conclusions.
While data can be leveraged upon to produce positive and profitable outcomes, it can also be potentially invasive to an individual’s privacy. For instance, shopping newsletters sent to consumers based on their purchasing patterns which may be in breach of what data customers want the store to know, retain and re-use for marketing. It has become important for companies to be aware of consumer privacy issues and data privacy management with respect to how data analytics systems work to produce useful insights and actions.
With these learnings in mind, Red Dot Payment seeks to form and present insights gathered from various data sources linked to our payment gateway, with the key aim of benefiting our clients and informing our business planning.