This repo is a collection of resources intended to help guide and improve the evaluation process of data visualizations. It is written by and for the Axis Design team, but can also act as a point of reference for all data vis practitioners. Again, none of the guidelines mentioned here should be followed to the tee but rather used along the lines of a "Know your guidelines before you break them" mindset.
The goal of this repo is to serve as a toolkit that helps deconstruct the testing process into manageable chunks. It aims to:
- Identify pitfalls in our current evaluation process
- Provide direction for self-guided evaluation by means of best practices
- Equip designers with the tools necessary for testing visualizations
Designers should be able to use this repo to determine success criteria for their projects, prioritize which attributes to test, and conduct more focused and productive user tests.
This is a WIP living document. Please read our Contribution Guidelines to help refine this repo.
The approach advocated by this repo is:
1. Follow best practices
2. Determine research questions
3. Plan the test
4. Conduct the test
5. Act on your findings
To break down the contents of each phase:
-
Before beginning the evaluation process, its's helpful to first:
- Understand the inherent complexity that comes with evaluating a data visualization,
- Understand the pitfalls with the current evaluation methodologies, in order to prevent yourself from falling into them, and
- Equip yourself with the testing mindset in order to properly set expectations.
-
At a strategic level, this serves as an initial self-directed reflection, intended to help you determine the most important design criteria for the success of your project, as well as to help you improve the current design based on known best practices. At a tactical level, it provides guidelines to help you evaluate specific UI elements such as typography, color, and arrangement (see Data Visualization Checklist prepared by Stephanie Evergreen and Ann K. Emery).
-
Next, you will need to determine what it is you specifically want to test and measure, and what attributes you want to prioritize for your dashboard. Do you want to test its usability, usefulness, desirability, or a combination of these attributes? This section provides
- A list of desirable attributes and the corresponding questions you can ask to test those attributes
- A list of unwanted attributes you can use to cross your visualization against
- Guidelines on determining the style of data you should capture (verbal, multiple choice, rating, written, etc.)
- The Axis Design Sprint Testing Template that can be used to begin documenting the evaluation process
-
Now that you have pinpointed the relevant questions and attributes, you can use this page to devise a testing plan and methodology. This page includes information on
- Recruiting users/evaluators for your test
- Choosing the right test (Attitudinal or behavioral? Qualitative or quantitative? What's the context of use?)
- Choosing the right tasks
-
After you've mapped out the type of test that would be most appropriate for collecting feedback, you can reference this page to see specific testing methods, and instructions on how to conduct them.
-
Now comes the redesign and iteration. After conducting your tests, you'll need to analyze your results, then translate them into action items. This section provides resources on
- Resources on analyzing both quantitative and qualitative data
- A Checklist of potential actions for design and usability changes you can make based on feedback
- Guidelines to conveying your findings
Throughout the sprint we want to track how research questions mature as the design sprint progresses and what methods were used to answer those questions. Copy over this Google Sheets document to your own drive to help guide and document your testing process.
Acknowledgements: Thanks to Arielle Cason, UX Researcher for sharing her personal test templat (developed while she was a graduate student at Georgia Institute of Technology) with us. We were able to use her template as a substrate to build our own.
Here is a snippet of her template-
Copyright © 2017 Axis Group, LLC. The information contained in this document is free, and may be redistributed under the terms specified in the license.