10 Steps to Develop Applications with Embedded Analytics

Designing and developing data-driven applications has to go through a modified agile planning, discovery, and user experience design compared to traditional applications. Why? Because the data is an input to the design and development process!

In other words, designers of the user experience cannot  do their work in a bubble. They have to review the underlying data, work with customers to understand what insights are important and how they put insights to action, and then consider how to best design the user experience.

Unfortunately, I've seen agile product owners and designers make the mistake of designing a pixel-perfect user experience of a data/analytics driven application without considering the underlying data!

Developing analytics and data applications

Designing and developing analytics applications should go through a process that looks something like this:

1. Select a data visualization tool to learn more about the underlying data. Use this tool to connect to data sources, explore the data, and prototype dashboards.
2. Interview end users to better understand their questions and how they want to use the resulting data and and analytics. Define personas and user roles to help flush out the end user experience, security requirements, and workflow optimizations.
3. Prototype one or more dashboards that answer questions and share with end users for feedback.  Iterate on dashboard design, visualizations, and underlying analytics until the dashboards achieve an MVP of usefulness.
4. Review data integration and data quality to determine if any automation, data cleansing, data enrichment, data validation and other data prep work is required before using the data. 
5. Reconsider how the end user makes the data actionable and conceive the requirements for modifying data, workflow, and application/api integrations that will enable end users to leverage the insights discovered through the application.
6. Establish the security model on who gets access to what data and what actions, where can data modifications be performed, and what workflows and integrations can be done by different user types.
7. Review and update the underlying databases and data architecture to ensure that the data model can support the underlying analytics.
8. Develop a testing strategy and consider how validating data, analytics, and data visualizations will be performed on the applications. Establish a testing strategy, select tools for performing quality assurance, and review the agile development process to ensure that testing is factored into the development and release protocols.
9. Convert the dashboards into an application and consider performance, scale, costs, security, and other factors when considering how to convert the dashboard developed in the data visualization tool into an application. See my recent post at InfoWorld on how to choose the right data visualization tools for your apps
10. Consider the deployment architecture that supports development, testing, production, and disaster recovery of the application. Also consider data how the application will be monitored, and what data flows will be required to support ongoing development.

Please note that while I'm sharing this as ten steps, they can't always be performed in the order outlined. For example, for large scale applications, the deployment architecture should be decided a lot earlier in the design and development process. For applications being developed on sensitive data, the security model is likely to be a factor in the user experience design. For applications that require high data quality, reviewing and cleansing data might be an upfront project before starting the application design process.

That's why developing applications with embedded analytics isn't trivial. Ask me, as I've been a part of developing analytics applications in several industries!