Unstructured Data


Got a question?  Somewhere, on some government computer, the information you need is probably available.  Information you paid for and the government would gladly share with you---if only they could find it.  There are thousands and thousands of documents stored on thousands and thousands of hard drives just in the State of Utah.  Throw in city governments, county governments,  school districts, universities, water districts, and other special use districts and the problem is staggering.  Multiply that by fifty states and add in the federal government and its mind boggling.  With all of the technology available to index, catalog, and store this data, what's wrong?

There's also some great search and concept indexing software available.  I wrote something on unstructured data last week and today I read an article in InfoWorld on the same topic.   In the end, I think the answer comes down to ROI.  As the InfoWorld article says: 

Although no one disputes the value contained in unstructured information, the work and expense required to add structure to the data or to make it consumable and searchable by workers is significant, Ramos says. And because the ROI is not necessarily solid, many organizations put their limited technology dollars toward financials or CRM applications instead.

While there are some great arguments for making all this unstructured data available for use, its hard to convince people to spend money on that instead of medicaid benefits for N more people or new highways.  I'd argue that governments do better than most organizations their size in at least making an effort.  The State of Utah has an entire program on locating government information, housed in the State Library division, which has responsibility for this topic.    They're experts at meta data and retrieving information.    What they don't have is a lot of money and to get that, we'll need more than a soft ROI.   We'll need real numbers on real savings or benefits.