In a recent very thoughtful post on data quality, Paul Erb plays out an analogy comparing data users with Don Quixote and data quality professionals with Sancho Panza, then reverses the analogy to cleverly coin the “Sancho Panza" test of data quality professionals. He encourages data quality professionals promoting the critical role of data quality to apply a what would Sancho say test to ensure that they are aligned with the needs and interests of data consumers.

Here's Paul's description of the Sancho Panza test:

Think of Don Quixote [DQ] as the data-quality specialist or even the data management specialist or software vendor, bringing to the world his specialist's perspective and vocabulary and enthusiasm, influenced by the books he's read, visioning everyday business practices, with his value added, as goldmines for the organization. Meanwhile Sancho Panza represents the person who does a practical job every day, who knows what works around here and what doesn't.

I advocate to Data Quality (let's call it DQ) consultants that they listen to this Sancho Panza, and consider themselves as Don Quixote. Sancho doesn't know much about data, but he knows what he likes… He's open to listening, but slow to change, and he'll tell you what he thinks.

Paul's article reminded me that as a child I thought the problem with Don Quixote was that he tilted at windmills and attempted to ambush acting troupes because of his bad eyesight. Of course this is not the case, but to me it provides a relevant perspective on data quality in many organizations.

Here's the problem I've seen play out on a number of IT application projects:

  1. A high level business study recommends replacement or improvement of a current application.
  2. The organization approves the project described in a business case citing benefits named in the business study and costs detailed for infrastructure, package software, and application development, but data-related costs are glossed over or left out entirely.
  3. The project begins with a requirements phase that collects hundreds of imperative statements ("The system shall…") from business people who will use the system.
  4. Late in the requirements phase, the team finds that data integration work in system interfaces will be more complex than expected. A common example: the project requires changes to a feeder application with no documentation and no in-house support expertise.
  5. Project leadership goes back to the sponsor seeking more money.

In these situations the business case was incorrect because it did not account for all of the costs of data integration. I've seen projects weather steps four and five well, but often discovery of previously unseen data complexity starts a disruptive chain of events. (Sadly for the project manager, such situations are often seen as a failure of project management and corrected accordingly, but that's a topic for another post.)

In my view the root cause of unforeseen data complexity on projects is the lack of a data constituency in current IT. It is only recently that success of companies like Google and Amazon have motivated emergence of data as a key business resource in the collective consciousness. Famous success stories notwithstanding (see this link), there are relatively few senior IT managers with data quality backgrounds. Conversely, many rose through the ranks of the infrastructure, application development, or business (process) analysis groups.

It will be a while before, for example, a Mobil CIO's predecessor jobs include definition of a metadata repository or elimination of multipurpose data, but in the meantime here's what we can do: add a business case to the application lifecycle as the last step in requirements. Stop the project when the real costs are known, recalculate the cost/benefit, and ask the sponsors if the project should continue. Give Sancho (in this case the project team) a chance to speak to the reality of the situation, and hand to Don Quixote (project sponsors) the eyeglasses of in-depth visibility into real costs. If the decision is to move ahead with the project, then all share the same vision and the sponsors have endorsed the actual project, not the fuzzy image from earlier on that might have been a windmill.