Recently, as a speaker in Big Data World Show in Malaysia, I had an opportunity to meet diverse range of customers across all industries. The atmosphere was filled with enthusiasm and curiosity about Big Data. However, after talking to many of them, I noticed that there was more than just the curiosity. The air was pregnant with uncertainties and concerns on implementation of Big Data Strategies. The “Big” part of Big Data is acting like a double edged sword. On one hand, it creates Big Hopes on crucial pattern revelations and Big Insights on information that has been hidden in huge data piles. On the other hand, the very scale of huge data, both internal and external, make it appear as formidable challenge to start with and get right results. Below are few of the challenges that I sensed are widespread in adopting Big Data-
1) Identifying the right use case
It appears that Big Data buzzword has reached CEOs and CFO and which is putting pressure on CIOs to demonstrate their Big Data strategies. Hence many times Big Data technologies are being talked, experimented without having any business challenge to solve.
Eg. IT department wants to see what correlations they will find from the data they have. However, the problem is they don’t have right context or business need to support their case. The point here is that, if you look for something somewhere, you will likely find something. But the value of that ‘something’ might not be related to business objective. Here, businesses need to have at least intuitive expectations on results to be expected from such endeavors. Hence it is Important to find a business use case with strong chances of success to prove the business outcome.
2) Understand the “Big” in Big Data
Many companies are lured into thinking that running analytics on huge data sets or deploying Hadoop is what is Big Data Analytics and it will deliver value out of data. Definitely they will find some correlations from such analysis. But then they are lost into complexities of such large and varied data sets. To me, the “Big” in Big Data stands for all data that is available.
Combined point 1 and 2 above, customers can really start looking at use cases where –
a)Variety of data is limited, may be internally available data first
b)Data sets are limited, may be two or three data sources
c)Volume of data should be overwhelming to handle as compared to infrastructure
d)Business outcome is clear and effective to prove the results internally
3)Getting right technology
Big Data Analytics space has garnered pace very recently and the technologies to handle these problems are evolving and yet to mature fully. There are many players, small and big, claiming their territories in Big data space. There are new technologies, new players, new vendors emerging very rapidly. The choices of solutions are very wide and they address different problems and needs. There is consolidation yet to happen in this space and there is no one or handful of winners, as of yet. This has created some confusion in the market about which technologies to choose. My suggestion in this case is to avoid any big investments on any particular technology to start with, till the time big data needs are clearly identified. Keep it small and flexible thereby avoiding any costly lock-ins. Technologies based on open source solutions can be very handy in such situations.
So the key central idea from all these points is “Start Small, think Big” and don’t get lost in “Big” of Big data. In my next article, I will cover some aspects about Data governance, organisational challenges and skills related problems.

– Kaustubh Patwardhan, Director I Southeast Asia and Hong Kong, Ashnik

Kaustubh (KP) is the Head of Business Development and Strategy at Ashnik. His role comprises of heading Strategic Partnerships, Channels Management and Business Development for ASEAN. With his expansive experience in IT, he plays a pivotal role in strategic initiatives undertaken by Ashnik. Apart from his usual responsibilities at Ashnik, he is passionate about photography, cricket and other sports. He is also an enthusiastic participant in poetic circles and plays.