Practical Machine Learning

by Sven Mayer

Lecture 05: Full Practical Neural Network Walkthrough

The material is licensed under the Creative Commons Attribution-Share Alike 4.0 (CC BY-SA) license: https://creativecommons.org/licenses/by-sa/4.0

Classification

The following clasificaion example determines if a touch inout is performed by a human finger or by the knuckle. Therefore, we will use a subste of the data published by Schweigert et al. [1]

[1] Robin Schweigert, Jan Leusmann, Simon Hagenmayer, Maximilian Weiß, Huy Viet Le, Sven Mayer, and Andreas Bulling. 2019. KnuckleTouch: Enabling Knuckle Gestures on Capacitive Touchscreens using Deep Learning. In Proceedings of Mensch und Computer 2019 (MuC'19). Association for Computing Machinery, New York, NY, USA, 387–397. DOI: https://doi.org/10.1145/3340764.3340767 Video: https://www.youtube.com/watch?v=4U1daa7fCbY

Source Code: https://git.hcics.simtech.uni-stuttgart.de/public-projects/knuckletouch/

Downloading the subset

Data Exploration

Now that we have data we need to ensure we understand how the data is formated and what it contains

Thats not looking good, but the authors provided instructions on how to read the data. See https://git.hcics.simtech.uni-stuttgart.de/public-projects/knuckletouch/-/blob/master/python/Step_02_ReadData.ipynb and https://git.hcics.simtech.uni-stuttgart.de/public-projects/knuckletouch/-/blob/master/python/Step_05_CNN_PreprocessData.ipynb for full details

Finally, loading all data files

Prepare the input for Tensorflow

Well, for a NN model the inputs images need to have the same size. So we need to do this first

One-Hot-Encoding

ML models can work with names such as "Finger" and "Knuckle." Therfore, we need to encode the classes using one-hot-encoding, see https://en.wikipedia.org/wiki/One-hot

[0,0,0,1] Class 1

[0,0,1,0]

[0,1,0,0]

[1,0,0,0]

Building a classifier with Tensorflow

Complex model stuctures are part of a future session

Blob1D -> f(x) -> Label

x -> f(x) -> y

But how good is it in reality?

Therefore we need training, validation, and test set. For the general information see https://en.wikipedia.org/wiki/Training,_validation,_and_test_sets for the HCI domain see https://dl.acm.org/doi/10.1145/3436958

Training, validation, and test set -- is part of a future session

For data recorded from particapnts never use a random split - it will lead to "overfitting"

Over and underfitting -- is part of a future session

Strategies to improve training