SME
Remote Sensing
Classification
Use this for “A forest map is checked at 100 points: 45 forest mapped as forest, 10 non-forest mapped as forest, 5 forest missed, 40 non-forest right. Find the accuracies and kappa.”
Table
100 in all: Mapped forest 55, Mapped non-forest 45
Conditional relative frequency: P(Mapped forest | Forest) = 45/50 = 0.9, out of the Forest total only
This is an example. Type your own values to start a new problem.
n₁₁
Forest mapped as forest
example
n₁₂
Non-forest mapped as forest
example
n₂₁
Forest mapped as non-forest
example
n₂₂
Non-forest mapped as non-forest
example
N
Points checked
calculated
OA
Overall accuracy
calculated · %
PA
Producer’s accuracy, forest
calculated · %
UA
User’s accuracy, forest
calculated · %
pₑ
Chance agreement
worked out (not typed)
κ
Kappa
calculated
Formulas
N
=
n₁₁
+
n₁₂
+
n₂₁
+
n₂₂
100 = 45 + 10 + 5 + 40
OA
=
100
×
n₁₁
+
n₂₂
N
85
=
100
×
45 + 40
100
PA
=
100
×
n₁₁
n₁₁
+
n₂₁
90
=
100
×
45
45 + 5
UA
=
100
×
n₁₁
n₁₁
+
n₁₂
81.8182
=
100
×
45
45 + 10
pₑ
=
((
n₁₁
+
n₁₂
)
×
(
n₁₁
+
n₂₁
)
+
(
n₂₁
+
n₂₂
)
×
(
n₁₂
+
n₂₂
))
÷
N
2
0.5
=
((45
+
10)
×
(45
+
5)
+
(5
+
40)
×
(10
+
40))
÷
100
2
κ
=
OA ÷ 100 − pₑ
1 − pₑ
0.7
=
85 ÷ 100 − 0.5
1 − 0.5
Assumptions
•
Rows are the map’s classes, columns the reference (ground truth); the diagonal holds the points the map got right.
•
Producer’s accuracy asks how much real forest the map found (omission); user’s accuracy asks how much mapped forest is real (commission).
•
Kappa (κ) compares the agreement with pₑ, the share two maps with the same row and column totals would agree by chance.
Step-by-step
WE KNOW
n₁₁ = 45, n₁₂ = 10, n₂₁ = 5, n₂₂ = 40
FIND
points checked (N), overall accuracy (OA), producer’s accuracy, forest (PA), user’s accuracy, forest (UA), chance agreement (pₑ), kappa (κ)
Step 1 · Find points checked (N)
Use N = n₁₁ + n₁₂ + n₂₁ + n₂₂
Every checked point sits in one of the four cells.
N
=
45
+
10
+
5
+
40
N
=
100
Step 2 · Find overall accuracy (OA)
Use OA = 100 × (n₁₁ + n₂₂) ÷ N
The diagonal, the points mapped right, as a percent of all points.
OA
=
100
×
45 + 40
100
45 + 40 = 85 correct
OA = 85%
Step 3 · Find producer’s accuracy, forest (PA)
Use PA = 100 × n₁₁ ÷ (n₁₁ + n₂₁)
Of the real forest (the forest column), the share the map called forest.
PA
=
100
×
45
45 + 5
Forest column: 45 + 5 = 50
PA = 90%
Step 4 · Find user’s accuracy, forest (UA)
Use UA = 100 × n₁₁ ÷ (n₁₁ + n₁₂)
Of the mapped forest (the forest row), the share that is really forest.
UA
=
100
×
45
45 + 10
Forest row: 45 + 10 = 55
UA = 81.8182%
Step 5 · Find chance agreement (pₑ)
Use pₑ = ((n₁₁ + n₁₂) × (n₁₁ + n₂₁) + (n₂₁ + n₂₂) × (n₁₂ + n₂₂)) ÷ N²
Each
row
total
times
its
column
total,
summed,
over
N
2
:
the
agreement
expected
by
chance.
pₑ
=
((45
+
10)
×
(45
+
5)
+
(5
+
40)
×
(10
+
40))
÷
100
2
Forest: 55 × 50 = 2,750
Non-forest: 45 × 50 = 2,250
100
2
=
10,000
pₑ = 0.5
Step 6 · Find kappa (κ)
Use κ = (OA ÷ 100 − pₑ) ÷ (1 − pₑ)
The agreement beyond chance, as a share of the most there could be beyond chance.
κ
=
85 ÷ 100 − 0.5
1 − 0.5
κ
=
0.85 − 0.5
0.5
κ
=
0.35
0.5
κ
=
0.7
Check
Put the numbers back in. Both sides should match.
100 = 45 + 10 + 5 + 40 ✓
85
=
100
×
45 + 40
100
✓
90
=
100
×
45
45 + 5
✓
81.8182
=
100
×
45
45 + 10
✓
0.5
=
((45
+
10)
×
(45
+
5)
+
(5
+
40)
×
(10
+
40))
÷
100
2
✓
0.7
=
85 ÷ 100 − 0.5
1 − 0.5
✓
Topic check
See what you know: 5 questions on this topic
New questions each time. Pass to mark it complete.
More problem types
Minimum-distance classifier
Use this for “A pixel reflects 0.10 in red and 0.40 in near infrared. Which class mean is nearest: water, vegetation or soil?”
Supervised or unsupervised
Use this for “An analyst runs k-means with 12 clusters, then names each cluster. Is that supervised or unsupervised classification?”