The Problems with Siloed Databases Part 2

I received several comments on my last post, Please, Stop Creating Microsoft Access Databases and thought I'd use today's post to respond to some of them.

  • "It's more often poor planning, lack of knowledge transfer and/or changes of use over time." - I completely agree that this is what makes the ongoing database and application support complex. I accept all of these as reality so the question is, are non-developers or business users developing databases with structures and documentation that simplify changes? If they are performing one-time data analysis, then maybe this isn't a concern but for databases that will be used and updated over time, they should be managed by database developers and dba's that are trained to support enhancements and changes.

  • "The underlying problem here isn't MS Access." - MS Access is unique in that it is widely available to business users, it easily allows saving databases to desktop hard drives that are difficult to administer, and has application functionality such as forms and reports. So yes, one of the underlying problems is MS Access because of its capabilities and how it is deployed.

  • "Can we arrange a process of promotion, where ad hoc DBs get promoted to proper data in due course?" - Yes, this is possible and ideal, but hard to govern and sometimes difficult to staff. It depends on how much database development is in practice and the size of the organization. My policy would read something like:
    • Register all non-IT database development in a directory.
    • Allow databases to be created for one-time data analysis, but archive them in three months or less.
    • Prototype databases for single user use, but if multiple users need access or if form/reports are needed, then the prototype should be transferred to IT so that they can be properly developed and managed. 

  • "Non-developers create Access databases because they need to get some work done" - I agree, and relying on IT isn't always the answer. However, most non-developers don't have an objective to create a database - they are usually looking to develop a workflow or to perform some analytics or reporting and realize they need a database to store the data. To that end, I think it is better for IT organizations to provide "self-service" tools to manage departmental workflows (see my post: In my CIO toolkit), or tools for self-service analytics/reporting.

  • "It turns out it is about dark data and how organizations should better consider their enterprise data handling." "Dark data can be a problem." - Indeed, that is really what the post is about. My definition of dark data is "Data that isn't documented or easily understood, data that can't easily be connected to other data sources, or data that can't easily be used in analytics.". So when you have poorly planned, siloed databases, then this is a dark data issue.

4 comments:

  1. MS Access Post from 2003: Beware the Rogue ASP

    Our guideline is to not use Access for more than analysis, but unfortunately, people will take 'analysis' and stretch that," the CIO says.

    ReplyDelete
    Replies
    1. I tried coming to your defense in the initial post, where the dissenters were surprisingly nasty and supporters made great points (I tend to agree about the post title having qualifiers, like "Stop Using Access Unless You Are a DBA OR, Failing That, You Intend To Delete The Locally Made DB After You've Accomplished The Task." Just less blanket condemnation. Anyway the reason I'm even taking time to comment now is because I think my Google search string that brought up your post is a great hilariously apt real world example underpinning your article: "best way to take over someones former messy access database and vba reporting". Hahaha I was merely looking for some efficiency tips, as I am currently the sole analyst tasked with making sense of/continuing the reporting of a recently deceased database guru whose myriad databases and VBA processes--the latter in both Access and Excel--are one unending labyrinth of crappy conventions, failure to delete or archive prior bad tables/query attempts, lack of compartmentalization (so the Access and Excel files have more data sets and do more tasks than a person who cares about others' understandibg would EVER have implemented.

      I keep having to figuratively come up for air so I don't drown in the chaos (meaning I literally inch toward panic regularly throughout this untangling process).

      Starting from scratch has often been my approach here, but presently I fear I must salvage much of the former process in order to get results for my boss on time... Sigh.

      Delete
  2. Anonymous7:15 PM

    The problem is not Microsoft Access. The problem is management and/or the IT department.

    Okay, you get a job to fix a database that is not a text book designed system. Get over it, you have a job for the next 6 - 12 months (obviously you would account for the issues in your quote, might even be a little open-ended). Would you rather a job for the next 6 - 12 months or waiting for a job?

    Surely a competent programmer/analyst could work through a mess, yes, there maybe additional costs for time but that's management's problem not the person initially creating the Access database.

    ReplyDelete
  3. Thanks for the article- definitely something to think about. I would have loved to have seen a paragraph at the end of the first article "Please Stop Creating Microsoft Access Databases" with suggestions for what to do instead. (On an unrelated note, it would also be helpful to see dates instead of just times, so that I know how old this article is!) Thanks

    ReplyDelete

Comments on this blog are moderated and we do not accept comments that have links to other websites.

Share

About Isaac Sacolick

Isaac Sacolick is President of StarCIO, a technology leadership company that guides organizations on building digital transformation core competencies. He is the author of Digital Trailblazer and the Amazon bestseller Driving Digital and speaks about agile planning, devops, data science, product management, and other digital transformation best practices. Sacolick is a recognized top social CIO, a digital transformation influencer, and has over 900 articles published at InfoWorld, CIO.com, his blog Social, Agile, and Transformation, and other sites. You can find him sharing new insights @NYIke on Twitter, his Driving Digital Standup YouTube channel, or during the Coffee with Digital Trailblazers.