Menu

1 / 19

Majority Vote Flipped 33% of Ambiguous Digit Labels, Only 3% on MNIST

Reading 0:00
15s threshold

This article introduces a re-labelled, controlled dataset designed for uncertainty exploration. We briefly cover the following:

Motivation

Dataset Curation

Dataset Overview

Dataset Description

Quick access via Hugging Face

Multi-modal extension: text

For more details, please check out the paper itself.

Yet another digit dataset , in this day and age?

Deep learning research keeps scaling, but massive datasets can introduce confounding factors that make understanding fundamental principles of deep learning rather challenging. While modern neural networks do really well on clear, unambiguous examples, their behaviour on examples that humans themselves can’t agree upon is still poorly understood. When a datapoint is genuinely ambiguous (e.g. a digit that could be either a 4 or a 9) , how should a model learn and evaluate such data?

image by author

A model predicting a 50/50 split could look like indecision, but it might just be the model confidently reporting that the answer is ambiguous.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.