Big data is the new buzzword for organizations. Patterns become visible that would not otherwise be visible. Amazon is said to know what customers are going to buy before the customers know it. The potential of big data opens up new, promising opportunities. Companies hope big data will help them develop better products and services based on data analyses. Unfortunately, having big data is quite a different thing from using them in the right way. This is particularly so when innovation and creativity are at the heart of what these companies do. In order to leverage big data to these ends, it is important to understand three unique challenges.

This emerges from research I have conducted with Prof. Elizabeth Long-Lingo from the Worcester Polytechnic Institute in Massachusetts. Prof. Long-Lingo and I compare data from cancer research and music production. Both scientists and artists work toward a creative outcome. Both employ cutting-edge technology that enables collecting, managing and manipulating many data points. Cancer researchers employ various types of experimental and modeling techniques that they use to study and explain cancer evolution. In music production, instruments are recorded separately and then big data technology enables endless possibilities to arrange or re-record the tunes and apply artistic effects.

You have probably already heard about the first challenge of big data: an immense volume of data creates a data management challenge. Preparing and maintaining the data for use implies keeping data accessible to analysts but protected from others. It also implies accuracy across the entire data set down to the level of individual data points. Digitization comes with infinite possibilities of data treatments, e.g. in how to process, manage, and store data.

The second challenge is to use big data to create an authentic story. A song, an argument or a product all need to peak consumer interest. They need to offer a unique story with original value. Hence, storytelling is at the heart of interpreting big data. As big data offers unlimited options of selecting, presenting, and interpreting big data, much work goes into trying out different stories and creating the most appealing, authentic version.

Both the volume and authenticity challenges lead to a third challenge. Working with big data as we observed in cancer research and music production is very repetitive and tedious. It demands technical data management skills and also constant cognitive attention to great detail. Clearly, the nature of this work is going against the creative flow that is essential for companies that hope to keep innovating. Hence, the third challenge is to keep working creatively and striving for innovation when a lot of the groundwork is boring and tiring.

Working with big data is challenging for three reasons. The volume and authenticity challenges are labor intensive. A lot of work goes into data collection, storage, management, and protection. In addition, putting data together and coming up with a meaningful interpretation is also a lot of work. Indeed, data analysts are in great demand to help companies make sense of the data. Unfortunately, when companies have no clue how big data could help them, it is even more difficult to use big data correctly. Without the following three skills, the best experts will be unable to help companies in their quest to leverage big data. Experts must have the technical skills required to create, curate, and store big data sets. They must also have strong storytelling capacity. Lastly, they must know how to work with big data in such a way as to preserve their creative energy. In the age of big data, purely technical skills are no longer enough to master big datasets.

For further details, please contact:

• Hille Bruns