MNIST Dataset and SGDClassifier Explained
MNIST is a set of 70,000 small images of handwritten digits, 28 x 28 pixels each (784 features), and a standard first dataset for classification; SGDClassifier is a simple binary classifier trained with stochastic gradient descent.
MNIST contains 70,000 small images of digits handwritten by high school students and employees of the US Census Bureau. Each image is labelled with the digit it represents, not a letter. Each image is 28 x 28 pixels, so it has 784 features, each one a pixel intensity from 0 (white) to 255 (black). Scikit-Learn has helper functions to download popular datasets, including MNIST. The dataset object has a DESCR key describing the dataset, a data key with an array with one row per instance and one column per feature, and a target key with an array of the labels.
The dataset is already split into a training set (the first 60,000 images) and a test set (the last 10,000 images). The training set is shuffled, which guarantees cross-validation folds are similar and that some algorithms are not sensitive to the order of instances.
SGD stands for stochastic gradient descent. SGDClassifier handles large datasets efficiently because it deals with training instances independently, one at a time, which also makes it well suited to online learning. It relies on randomness during training (hence the name stochastic) so set random_state for reproducible results. It computes a score based on a decision function; if the score is greater than a threshold the instance goes to the positive class. Scikit-Learn does not let you set the threshold directly, but you can access the decision scores and apply your own threshold.
Key points
- 70,000 images, 28 x 28 pixels, 784 features, labels are digits.
- Split: first 60,000 training, last 10,000 test.
- Dictionary keys: DESCR, data, target.
- SGD = stochastic gradient descent; trains one instance at a time; good for large data and online learning.
- Set a threshold on the decision score, not directly on the classifier.
Worked example
To detect the digit 5, create target labels y_train_5 = (y_train == 5), train an SGDClassifier on them, and you get a binary 5-detector. Its accuracy can look high even for a model that always says not-5, since only about 10 percent of images are 5s, which is why we use a confusion matrix.
Common questions
How many features does each MNIST image have?
784, because each image is 28 x 28 pixels.
What does SGD stand for?
Stochastic gradient descent.
Practice questions on this topic
Try these, then open the app to check your answers with explanations.
- According to Hands-On ML, in the MNIST dataset it is the test set (not the training set) that is already shuffled.
- A. false
- B. true
- in MNIST Dataset, A.............. key containing an array with one row per instance and one column per feature.
- A. data
- B. DESCR
- C. target
- D. none of them
- MNIST Dataset is a set of small images of .................. by high school students and employees of the US Census Bureau.
- A. digits handwritten
- B. letters handwritten
- C. words handwritten
- D. none of them
- in MNIST Dataset, A ........... key containing an array with the labels
- A. target
- B. DESCR
- C. data
- D. none of them
- MNIST Dataset is a set of ................. small images of digits handwritten
- A. 70,000
- B. 75,000
- C. 7000
- D. 700,000
- In MNIST Dataset, A........... key describing the dataset.
- A. DESCR
- B. data
- C. target
- D. none of them
Related topics
- Confusion Matrix, Precision, Recall and F1 Score
- ROC Curve and AUC Explained
- K-Fold and Stratified K-Fold Cross-Validation
- Multiclass Classification: One-versus-One and One-versus-Rest
- What is Machine Learning?
- Supervised vs Unsupervised vs Semi-supervised vs Reinforcement Learning
- All topics: Machine Learning
BASMAGLY is an independent student project, not an official Delta University site. Check lectures and your instructor for the final word.