If you love data, if you love analytics, if you love big data…. Consumer Package Goods (CPG) is the place to be! But you scoff saying, "what about Financial Services, or Research, or Logistics, or Social Media, or Retail?" Nope! If you're looking for velocity, volume and variety, nothing beats CPG. If you're looking for large data mash-ups that require Sherlock Holmes like skills of deduction and inference, nothing beats CPG.

Yet CPG faces the same challenge that it has faced for decades, too much data and not enough integration, let me explain.

I have worked at two iconic CPG companies, one in household products the other in confectionary and each has shown me a hidden side to the products that end up on our shelves and in our stomach. To really understand CPG, you need to be clear that the value chain does NOT start at the manufacturing plant, it starts in the ground.

So I will lead you on a journey from harvest to your home and illustrate the marvelous complexity of the CPG data landscape.
The beginning of the data value chain starts with geographical information system (GIS) data; the confectionary company was one of the world's leading consumers of sugar, milk and cocoa. Price fluctuations in raw goods could be catastrophic to profitability so monitoring rainfall, sunshine, storm patterns, using satellite imagery to track field yields in Ghana, Brazil, Cameron the United States was vital.

Logistics plays a vital role in CPG, twice! Getting raw material from field to dock, then to the plant at the right time in the correct quantity is no trivial matter. Of course GPS data is important, but as important are cargo temperatures, or validation that you didn't put 11,000 gallons of milk into a tanker truck that had a toxic load, all data points play a key role to the CPG value chain.

We may have gotten the raw goods to the plant, but before we can do anything…we need a recipe! The R&D aspect of CPG can be as rigorous as any life science company. Product development, versioning, recipe evaluation data, developing new manufacturing processes, FDA compliance data are all vital from a production perspective but also for brand management.

Really big data starts here in the value chain, production planning, manufacturing, inventory and consumer off-take data are both referenced and generated. CPG has always had big data, even before the term was coined. As a matter of fact CPG data is so large that an entire industry was spawned around it with companies like IRI and Nielsen providing data services. The data in fact is so large that it has mostly been samples and not granular transaction data. Why is consumer off-take data so important to CPG? CPG at its most effective uses a hybrid Push/Pull system for production since one of CPG's distinguishing characteristic is that it creates public demand through publicity and promotion.

My first adventure in CPG Dataland was with the household products company architecting and developing a Sales and Marketing system that had a visual user experience… this was heady stuff back in the dark ages! The basic premise was to combine 3 elements out of the value chain; Order data, shipping data and Offtake data. Typically this data is represented in large data tables with an anemic and visually unpleasant graph, not a conducive format for intellectual processing and decision making. Imagine the impact of a screen that showed you a scrollable list of little pictures of laundry detergents, toothpaste, dryer sheets and the map of the United States divided into the major sales regions. Click a product and a region and within 3 seconds a table and a graph detailed the UPC breakdown for products at the state level pop up, one more click and you can see the same breakdown by channel within the state. The map would change to the state level and display the major markets within the state. The system allowed Planning, Sales and Marketing staff to visually drill down in several clicks to an individual product UPC code sold at the Kroger's at the corner of Broad and Main Street. Instant access to strategic data was a radical departure to the norm of data held at arm's length from the business, the business impact was a profound, and it enabled critical thinking, group harmony and reduced meeting times since everyone had access to the current data.

My next adventure in dataland was truly epic on a global scale. Brand Management is no laughing matter, some brands carry global revenue numbers and equity that rival GDP's of developed counties. Additionally, management of brands in developing and emerging markets plays a vital role in the growth of the brand and corporate revenue. Bad global brand management is a very real liability. The challenge faced by the confectionary company was that as the Global Brand Leads traveled the world they saw different data, displayed in different formats and NO synergies.

Brand Management datasets are complex, how complex? We were "mashing up" huge amounts of data-the "basic" sales, marketing and off-take data, couponing and promotion data, video from television commercials and radio broadcasts, panel and survey data, sponsorship, direct marketing and viral data from around the world! The technical aspects of this scale of data integration is daunting, but the real challenge is the "data science" of building the connective metadata to correlate disparate types of data and identifying causality. One final aspect to the herculean challenge – make it global!

A key driver for a Global Brand Leader is to create synergies between emerging and established markets. An example could be Portugal and Brazil, what worked in Portugal that could be applied to Brazil? What promotions yielded the best lift, which media buys generated the right consumer sentiment, which launch strategies were the most effective for various products and socio-economic groups. One of the interesting discoveries of this "integrated" global brand management platform was that emerging markets were more adept at developing "infrastructure light", "one to one" guerilla marketing campaigns that leveraged social media and social engineering. These synergies allowed Brand Managers in developing countries to create new classes of campaigns that included some of the first entries into mobile.

As I said in the begin of the article, the issue faced by the CPG sector is too much data and not enough integration - not just physical integration, but more importantly the integration of data from a causation perspective.

Today the CPG data landscape is even richer and larger. Granular off-take data, consigned inventory at big box retailers, traditional media and social media integration all contribute to creating big data requirements. I strongly believe that CPG has the most complex data science and information transformation needs due to the scope and complexity of the data value chain that goes from a seed to our shelves and our stomachs. If you feel that your adventures in Dataland are not meeting expectations, give us a call we can help you achieve measurable results.