Big Data is becoming mainstream initiative in organizations around the globe. With exploding growth in data volumes and availability of data from multiple sources organizations are looking to leverage it to improve various business performance.

The key steps in initiating big data plan are capture, store, sort, share, analyze, and visualize the data. Emerging technologies like NoSQL DB and Hadoop along with the analytics tools are changing the way organizations are  handling data and analysing it compared to the past.

Big Data initiative is a part of long term process which needs to be evolved, it cannot bring results overnight. It is a business process and not just a technology initiative, hence success depends on not only technology & analytics skill set, but participation from business  teams and management buy-in is necessary.

What do organizations hope to accomplish with this big data buildup? According to IDG Enterprise’s survey,

• 59 percent of respondents want to improve the quality of decision-making,
• while 53 percent want to increase the speed of decision-making.
• Getting better planning and forecasting, and developing new products/services and revenue streams are goals of 47 percent of respondents.
• Keeping existing customers or acquiring new ones is a goal of 44 percent of those who took the survey.

Despite all the hype and urgency of handling the huge volume of data, there seems to be quite a confusion in getting started with big data initiatives. Some of the common challenges in getting started are :

• Data is present in silos across various departments,
• Ineffective coordination amongst the business teams that own this data and
• Lack of single business driver to arrive at consensus as to what to achieve through analytics.

Here are some suggestive approaches that organization may look at adopting to overcome these challenges and make big data initiatives successful :

• Big data is sum of all the data that you have, even your relational data also falls under big data. What is new is the capability to capture, store and manage enterprise data, analyse with business urgency and this is made possible due to new generation tools & technologies.
• Look beyond proprietary data solutions. Open source technologies such as Hadoop, MongoDB and other frameworks are providing the ability to manage and make large volumes of data accessible for analysis – data types that would be otherwise very costly to go through ETL process in traditional data warehouse.
• Ensure the data governance is adopted. With big data comes the structured and non traditional data (unstructured) from sources like social media, videos, log files, sensors etc these should also be included in existing governance process.
• Big data success comes in small chunks, versus attempting overnight enterprise wide transformations.
• Data and enterprise architects are also required to plan and maintain infrastructures that best serve organizations. Many of the individuals who can handle these new tasks are already in today’s data centers, but require retraining and getting their skills refreshed to be effective in their expanded roles.

 NoSQL and Hadoop : core building blocks of big data infrastructure

NoSQL and Hadoop have become one of the core and critical components of today’s big data infrastructure. Both the technologies compliment each other. Hadoop for typical offline analytical workloads and NoSQL for online operational workload. Hadoop and NoSQL together can provide to organizations both analytical and operational capabilities for real time big data solution.

We, at Ashnik, are happy to bring you key components of big data solution along with our solution architecting expertise in making your journey successful.

– Sandeep Khuperkar I CTO and Director, Ashnik