The Problems with Siloed Databases Part 2

I received several comments on my last post, Please, Stop Creating Microsoft Access Databases and thought I'd use today's post to respond to some of them.

  • "It's more often poor planning, lack of knowledge transfer and / or changes of use over time." - I completely agree that this is what makes ongoing database and application support complex. I accept all of these as reality so the question is, are non-developers or business users developing databases with structures and documentation that simplify changes? If they are performing a one time data analysis, then maybe this isn't a concern but for databases that will be used and updated over time, they should be managed by database developers and dba's that are trained to support enhancements and changes.

  • "The underlying problem here isn't MS Access." - MS Access is unique in that it is widely available to business users, it easily allows saving databases to desktop hard drives that are difficult to administer, and has application functionality such as forms and reports. So yes, one of the underlying problems is MS Access because of its capabilities and how it is deployed.

  • "Can we arrange a process of promotion, where ad hoc dbs get promoted to proper data in due course?" - Yes, this is possible and ideal, but hard to govern and sometimes difficult to staff. It depends on how much database development is in practice and the size of the organization. My policy would read something like:
    • Register all non-IT database development in a directory.
    • Allow databases to be created for one time data analysis, but archive them in three months or less.
    • Prototype databases for single user use, but if multiple users need access or if form/reports are needed, then the prototype should be transferred to IT so that they can be properly developed and managed. 

  • "Non-developers create Access databases because they need to get some work done" - I agree, and relying on IT isn't always the answer. However, most non-developers don't have an objective to create a database - they are usually looking to develop a workflow or to perform some analytics or reporting and realize they need a database to store the data. To that end, I think it is better for IT organizations to provide "self service" tools to manage departmental workflows (see my post: In my CIO toolkit), or tools for self service analytics/reporting.

  • "It turns out it is about dark data and how organizations should better consider their enterprise data handling." "Dark data can be a problem." - Indeed, that is really what the post is about. My definition of dark data is "Data that isn't documented or easily understood, data that can't easily be connected to other data sources, or data that can't easily be used in analytics.". So when you have poorly planned, silo'ed databases, then this is a dark data issue.


  1. MS Access Post from 2003: Beware the Rogue ASP

    Our guideline is to not use Access for more than analysis, but unfortunately, people will take 'analysis' and stretch that," the CIO says.

  2. Yeah its a good article. According to you what we project managers do is communicating. And a lot of this communication is done during project meetings. It can sometimes feel like you are running from one meeting to another and that your time is often wasted. Meetings don’t start on time, the issues aren’t dealt with, there is no agenda, there is no focus, nobody assigns any follow ups or tasks and of course then they also don’t end on time. An efficient project manager is required for the good management of a project. I think a project manager should PMP certified. Looking forwards to apply what I learned in PMP classes in my company.

  3. The problem is not Microsoft Access. The problem is management and/or the IT department.

    Okay, you get a job to fix a database that is not a text book designed system. Get over it, you have a job for the next 6 - 12 months (obviously you would account for the issues in your quote, might even be a little open-ended). Would you rather a job for the next 6 - 12 months or waiting for a job?

    Surely a competent programmer/analyst could work through a mess, yes, there maybe additional costs for time but that's management's problem not the person initially creating the Access database.