Skip to content

Attribute Uncertainty Quantification

Building attributes derived from source data (e.g., OpenStreetMap) or machine learning models inherently carry degrees of uncertainty.

To make this transparent, every attribute is accompanied by a confidence metric. How that confidence is calculated depends on two things: whether the attribute is categorical (type, subtype) or numerical (height, floors, construction year), and whether it was merged from a source or estimated by a model. The four resulting cases are described below.

Type & Subtype

The confidence of the categorical attributes type and subtype is quantified as follows:

For Attributes Merged From OSM: Footprint Intersection Ratio

When a category is merged from a source dataset (i.e. OSM) onto a target building \(a\), the confidence is the Intersection over Area (IoA) between the target and the source footprints that share the assigned (dominant) category:

\[\text{Confidence} = \frac{\text{Area}\big(a \cap \bigcup_i b_i\big)}{\text{Area}(a)}\]

where \(a\) is the target footprint and the union is taken over the matching source footprints \(b_i\) whose category equals the assigned one.

  • 1.0: Perfect spatial overlap, the agreeing sources fully cover the target.
  • < 1.0: Partial overlap, suggesting misaligned geometries or sources that disagree on the category.

For Attributes Predicted using ML: Calibrated Classification Probabilities

For attributes generated by our classification models, the confidence is the calibrated probability of the predicted class:

  • Subtype Confidence: The output of the model after applying calibration to ensure the probability reflects real-world accuracy.

  • Type Confidence: Since type (residential/non-residential) is an aggregate, its confidence is the sum of the calibrated probabilities of all subtypes belonging to that category:

    \[P(\text{Type}) = \sum_{s \,\in\, \text{Type}} P(s)\]

    where the sum runs over the subtypes \(s\) belonging to that type.

Height, Floors, Construction Year

The confidence of the numerical attributes height, floors and construction year is quantified as follows:

For Attributes Merged From OSM: Value Extremes

If multiple source buildings match a single target footprint with different values, we report the spread of those values:

  • Lower Confidence: The minimum value among all matching sources.
  • Upper Confidence: The maximum value among all matching sources.

For Attributes Predicted using ML: Bootstrapped 95% CI

To quantify the uncertainty of our regression models, we use a bootstrap approach:

  1. Bootstrap Sampling: The model is run \(n=10\) times using different seeds or data subsamples, resulting in a set of predictions \(Y = \{y_1, y_2, \dots, y_{10}\}\).

  2. Standard Error Calculation: We calculate the sample mean \(\bar{y}\) and the standard error of the mean (SEM):

    \[\text{SEM} = \frac{s}{\sqrt{n}}\]

    where \(s\) is the sample standard deviation.

  3. Interval Calculation: The confidence bounds are defined using the \(t\)-statistic for \(n-1\) degrees of freedom (at \(\alpha = 0.05\)):

    Lower Confidence:

    \[\text{lower} = \bar{y} - t_{0.975, 9} \times \text{SEM}\]

    Upper Confidence:

    \[\text{upper} = \bar{y} + t_{0.975, 9} \times \text{SEM}\]