Uncertainty Visualization Study Group Notes – 10/1/2012

Administrative Info

---

• All Hands (All Institutions) Meeting 29–30th of October
  • need Agenda
    • what do we want to accomplish by the end and then work backwards - How can we help each other? Encourage collaboration. - What are everybody’s strengths? (All collaborators) - Goals for deliverables? - email other suggestions to Bill/Ross
    • First deliverable = first annual report; very specific format
      • Concretize what we’re putting in the report, how we’re doing it, and where do we hit inter-institutional considerations
    • Put together overview what we as a group have accomplished so far

Discussion

---

• This is the paper everyone points to in terms of taxonomy
  • more of a first, not necessarily the best; hasn’t been revisited
• What is uncertainty?
  • statistical
  • error
  • range
• Sources of uncertainty
  • collection / acquisition
  • derivation / transformation – error vs uncertainty (direct reproducibility means it’s not uncertain)
  • visualization
• Classification
  • Value (data + associated uncertainty)
  • Location (positional data + associated positional uncertainty)
  • Extent (range / distribution of valid) -continuous vs discrete
  • Visualization extent (?)
    • what does this mean?
    • discrete data points indicated vs continuous range shown?
  • Axis Mappings
    • experiential [spatial location used to show inherently spatial data?] vs abstract

• Classification Reference Tables – see paper

• Radiosity: how does it relate to uncertainty?
  • just an application: the paper describes different methods for conveying differences in radiosity, e.g. glyphs

• Ways of illustrating differences is not uncertainty

• Spatial data vs spatial indexed data
  • uncertainty associated with values in a height-field (uncertainty value indexed in 3d) vs. where’s the bridge? (inherent 3d uncertainty)
    • subtle difference that we don’t care about most of the time, but are different

• Bayesian Uncertainty is viewed as anything that encapsulates differences from certainty / ideal – What are the sources of that “error” distinguishing it from the ideal? (due to quantization, random processes in nature?)

• Belief systems vs probabilistic uncertainty – pdf as instrinsic uncertainty vs. what we believe about it (reliable?)

Uncertainty Visualization Study Group Notes – 10/5/2012

Bill’s Email(s)

---

Thoughts on what exactly is means to "represent uncertainty" in a
visualization

For the moment, let's limit the discussion to:

- The Bayesian view of uncertainty quantification, in which all the
sources of variability are modeled as a single probability distribution.

- A single (scalar) continuous random variable.

- A distribution presumed to be Gaussian.

While these are a significant restriction, they are a useful place to start.

From a mathematical perspective, everything knowable about the
uncertainty associated with this random variable is specified by the
mean and standard deviation of the distribution, or any other equivalent
two value parameterization.

Our question is how we should communicate information about the
uncertainty (i.e., the distribution) in a way that can be easily
comprehended. Three pieces of information need to be communicate: the
functional form of the distribution and the two parameters that define
an instantiation of the distribution.

Communicating the nature of the distribution is a complex question that
I would like to defer for the moment.

Communicating the two parameters of the distribution relates to the
previously discussed question of whether or not it makes sense to think
about the visualization of uncertainty as being the separate visual
encodings of a value and an indication of the variability of that value.
It is tempting to think of the mean of the distribution as the "value"
of the variable we want to describe and the standard deviation as
somehow related to the variability we want to describe. This may in
fact make some sense for normal distributions, where the mean is one
possible measure of the central tendency for the distribution. It does
not make sense, however, for some other distributions, particularly
those that are multi-modal.

Still, for normal distributions we could perhaps argue that an intuitive
visual representation encodes the mean in one visual channel and the
standard deviation in another visual channel. There are, however, other
representations that might be equally intuitive. For example, we might
instead encode a description of the distribution using a confidence
interval, without without any explicit specification of the mean value.
This is in fact what is done with the cone of uncertainty used in
hurricane forecasts.

Discussion questions:

- Is it correct to define our problem, as limited by the assumptions
above, as finding an effective way to visually encode some transform of
the two parameters of the pdf that is used to describe uncertainty?

- If so, what is a reasonable set of alternatives to consider?

Post Meeting Addendum - 10/7

I have been arguing that two parameter PDFs representing the uncertainty
of a value should probably be encoded in two perceptually independent
visual channels. I now think that this is wrong. At least for
parametrizations such as mean + standard deviation or min/max confidence
interval, the two parameters are specified in the same units. This
suggests using not two separate and independent visual encoding
channels, but rather something like a "split screen" approach using two
different instances of the same channel.

- Bill

Discussion

---

• Uncertainty viewed as Bayesian for the purposes of this study
  • all kinds of uncertainty “lumped together” into a single value, represented by a pdf (probability density function)
  • mentioned as such in the grant proposal
• Gaussian, unimodal, single-variable data sets can be represented parametrically–
  • represented by small number of values–in this case, two values
  • need to define what the values represent: in this case, mean and standard deviation
• Bill: For the first experiments, we should start with a simple and well-defined case (limit to 2 degree of freedom parametric distribution, probably Gaussian): the mean and standard deviation
  • How does one communicate what the distribution is?
  • What do people assume the distribution is in the absence of any explanation (i.e. if the distribution is not communicated)?
• Pushing off discrete probabilities for now, because the parameter space gets bigger
• Bill: We should pick what transformation of parameters to use, and how to visually present these data
• Some visualizations and many cognitive papers treat uncertainty as a discrete variable: for example, 80% likelihood of 20-knot winds
• Idea for task: pit two visual channels against each other, one representing mean and one representing variance
  • Interpreted cognitively as data and uncertainty
  • Position is tricky for encoding these types of data
• How best to encode the parameters may be dependent on the expertise of the audience
  • Engineers are trained to use mean and standard deviation, while naïve users may understand confidence intervals more easily
• Parameters expressing a standard deviation can be various:
  • two confidence intervals
  • mean and standard deviation
    • parameters of central tendency and variance map cognitively onto data and uncertainty
• Bill: Does a domain-independent, problem-independent, natural (intuitive or otherwise cognitively compelling) paramaterization exist? Should we investigate?
  • e.g. two confidence intervals vs. mean and standard deviation
• First experiment: pick one, hold 2
  • Test efficacy of different paramaterizations
  • Use mean and central tendency to design an experiment testing visual channel encoding
  • Investigate the dependence of visual channel effectiveness on task