Reset filter
2023
In progress
Automated Surveillance Recognition in Smart Environments
Supervisor:
Matthias Gazzari Frank Hessel
This topic is about implementing various models to recognize the presence of as many smart things as possible based on sensor or other time series data. The goal of this topic is to compare and evaluate these models against each other in certain settings like in a smart home environment.
2023
In progress
Limits of CSI-based keylogging on 10-digit number pads
Supervisor:
Matthias Gazzari Jakob Link
...
2022
In progress
Exploring a Digital Intermediary for Smart Home Privacy Communications
Supervisor:
Matthias Gazzari
...
2022
In progress
Deep Learning for CSI-based Keylogging Side Channel Attacks
Supervisor:
Matthias Gazzari Jakob Link
...
2022
Completed
ECG-PPG A Comparison of Biometric Identification
Supervisor:
Matthias Gazzari
With the rise of the IoT and the usage of mobile devices, the need for improved security for those devices becomes more critical. Beyond regular passwords several other forms of identification such as biometric identification, have been introduced. They can offer increased convenience and less vulnerability to spoofing attacks. Most common forms of applied biometric identification include iris, face and fingerprint scanners that see most use in smartphones. But there has been an increasing interested in methods that utilize physiological signals of the human body. electrocardiogram (ECG) and photoplethysmogram (PPG) are among them and are the main point of interest for this work. They come with inherent advantages like being difficult to reproduce and can not be forgotten like a password.
Gathering records of the two signal types has become easier over the years and can now be performed with wearables like the Apple Watch. This opens new options for this field of research.
My work focuses on analyzing and reimplementing existing approaches for ECG and PPG based biometric identification systems and comparing them to deduct similarities, differences, strengths and weaknesses.
To achieve this two convolutional neural network (CNN) based ECG implementations and one PPG implementation that utilizes handcrafted feature extraction were adapted to work on a shared dataset that contain synchronized ECG & PPG data from the private SAPE and the public BIDMC database. This database was then used for evaluation of the systems. In addition commonly used biometric methods and databases were analyzed to aid in the final evaluation. High rates of accuracy were reached and compared to literature that utilized similar datasets.
2022
Completed
Comparison of Side-Channel Touchlogging Attacks using Wearables
Supervisor:
Matthias Gazzari
Although many research papers about touchlogging attacks, which are leveraging wearable devices as a side-channel to log keys being typed on a smartphone, exist, there is no concise summary of those attacks, their advantages & limitations, and different scenarios and evaluation setups make comparisons difficult or unfair. Therefore, one has to sort through countless articles and papers to see if an approach has already been evaluated in a specific scenario or can not fairly compare two good performing approaches because the evaluation setup differs drastically between the two.
This thesis provides a framework combining five of the most common approaches for touchlogging attacks in four different typing scenarios and eight ways the user is wearing the wearable device. With this framework and its evaluation, a concise overview and quick, fair comparisons between the most common approaches to touchlogging are presented.
2022
Completed
Impact of Multi-Path Effects on Acoustic Keylogging Systems
Supervisor:
Matthias Gazzari Florentin Putz
...
2022
In progress
Machine Learning Aided Penetration Testing
Supervisor:
Matthias Gazzari
...
2022
Completed
Limits on Inferring Handwritten Characters using Wearables
Supervisor:
Matthias Gazzari
Recent studies have shown that handwritten characters can be distinguished from each other with a high accuracy leading to security threats such as impersonation, side-channel attacks or just building systems to mirror handwritten characters to digital space.
Most of these studies just focused on the character recording and building (complex) systems around the classification of these handwritten characters, resulting in sparse data sets with only specialized hardware in restricted settings.
With these specialized settings and hardware, it’s not clear what limitations might impact the accuracy of classification, let it be the type of sensor of the general writing style of a person and if these researches also apply to consumer hardware or general settings like writing with a simple pen on paper.
The results of this work aim to set clear limitations and settings for the recording of handwritten characters while using a simple pen and paper setting with multiple consumer devices.
Sampling a data set full of handwritten lower-case characters with the usage of multiple consumer wearables in different positions on the forearm, while limiting the speed and size of a character drawn, are processed and calculated into several time-domain and frequency-domain features to be classified by different machine learning methods resulting in accuracies of 20 % to 22 % for the IMU data, 15 % to 17 % for the EMG data and 16 % to 20 % for a mixed approach.
The results are in the range of current state-of-the-art findings adjusted for the size of classifiers used, so the defined limitations in this work might give a direction to which limitations are more useful in the scenario of classifying characters based on signal data using consumer devices.
2022
Available now
Privacy and Security Implications of Cross-Modal Transformations on Human-Centric Sensor Data
Supervisor:
Matthias Gazzari
This topic is about implementing a cross-modal transformation model on a chosen pair of human-centric sensors (sensors which are worn by or close to humans), for recreating one stream of sensor data based on the other one. The ultimate goal of this thesis is to evaluate the performance of such a model with respect to the privacy and/or security implications.
Contact me if you are interested and/or have a cool idea for a specific pair of sensors relevant for violating the privacy and/or the security of a human being.
Experience with machine learning and/or signal processing is required. A good understanding of sensors and their measured physical quantities is strongly recommended.
2021
Completed
Finger Detection of Keystrokes from RGB Video Streams
Supervisor:
Matthias Gazzari
To research the security impact of side-channel keylogging attacks, we need suitable datasets containing the sensor data and the pressed keys. However, when our side-channel targets the user through acceleration, EMG, or other wearable sensors, we might want additional ground truth about the users’ activity, e.g., a representation of which finger was used to type a certain key. This data makes it possible to directly correlate the sensor readings with the activity that caused them, which could help develop more accurate and robust keylogging models. Previous work in this area focused more on stand-alone virtual input devices that do not reflect real-world keyboards or require expensive motion tracking hardware to track finger positions. In this thesis, we design, implement and evaluate a system that can infer finger usage from a monocular RGB video of a user typing on an unmodified keyboard. Our evaluation shows that our implementation can accurately label the hand usage for over 96 % of keystrokes and the finger usage for over 97 % of keystrokes. As such, our system can be a helpful aid in the creation of new datasets for research into keylogging side-channels.
2021
Completed
Handwriting Recognition using IMU and EMG Sensor Data
Supervisor:
Matthias Gazzari
With the rise of wrist-worn devices like smartwatches and fitness trackers and the integration of Inertial Measurement Unit (IMU) sensors questions about the privacy impact of their recorded data arise which often gets little attention in privacy considerations. Worn on the wrist one possible impact is a possible eavesdropper inferring the handwriting done by the wearer of the device using the collected IMU data. Another use case is the deliberate digitizing of handwriting by users wearing such devices. In this case it is also feasible for the user to wear an additional device to improve the digitizing.
In this thesis we investigate both the possible privacy impact and the possibilities for a deliberate digitizing of handwriting done on paper based on IMU sensor data recorded on a smartwatch. Furthermore, we collect Electromyography (EMG) sensor data using an armlet worn on the lower arm to analyze if the original recognition results can be improved utilizing these data. We design and conduct a data study aimed at mirroring everyday circumstances using an Apple Watch and a Thalmic Myo armlet to record the necessary data. Additionally, the original handwriting of the study participants is digitized by writing on paper on top of a Wacom Bamboo Slate tablet. We use the recorded continuous streams of IMU and EMG data to classify the written letters using the 1-Nearest Neighbor (1NN) algorithm in combination with the Dynamic Time Warping (DTW) algorithm. Our model achieves widely varying results depending on the writer and an overall accuracy of 0.28. Very low accuracies for the classification based on EMG data prevent us from evaluating possible improvements when combining both data types. Our findings suggest that the recognition depends on the writing style of the individual user and more research is required to make the handwriting recognition based on IMU or EMG data applicable to writing in everyday life.
2020
Completed
Circumventing ECG Authentication with Deep Generative Models based on PPG Pulse Data
Supervisor:
Matthias Gazzari
Electrocardiogram (ECG) biometrics is a steadily growing and increasingly popular field of research. In this work, we propose a novel attack scenario in which we train a generative model to uncover and spoof the ECG of a victim by merely observing another cardiovascular signal of the victim: their photoplethysmogram (PPG). For the model, we propose a conditional generative adversarial network (cGAN) with a U-Net style generator and least-squares loss. Since current training datasets do not fall into the off-the-person category, we additionally collect a custom dataset of synchronized PPG and ECG measurements. It features 33 recordings by 31 participants with a median age of 28.
We evaluate the model against a baseline by Zhu et al. Our model has a lead over the baseline with a mean relative root-mean-square error (rRMSE) of 0.47 vs. 0.49 on the TBME-RR dataset but lacks behind on our own dataset with a mean rRMSE of 0.61 vs. 0.55. The evaluation demonstrates that the cGAN is able to properly recreate the overall characteristics and noise of the ground truth. In the proposed attack scenario, the model yields an overall success rate of up to 26 % against a neural-network-based authentication system.
2020
Completed
Keylogging Side-Channel Attacks on Bluetooth Timestamps: A Timing Analysis of Keystrokes on Apple Magic Keyboards
Supervisor:
Matthias Gazzari Jiska Classen
In the past several timing attacks have been applied to recover sensitive input on keyboards. If these kind of attacks could be migrated to the wireless communication of keyboards, this would make the use of wireless keyboards less secure. In this thesis we apply a timing attack on the Bluetooth communication of the Apple Magic Keyboard by recording the time between consecutive Bluetooth packets and recover the typing with a Hidden Markov Model (HMM). With this attack we are able to shrink the search space of random passwords by a factor of 5 to 10, which considerably speeds up exhaustive search.
2020
Completed
Prevalence Analysis of Dark Patterns in Newsletters
Supervisor:
Matthias Gazzari
The dependence on online shopping makes consumers to popular targets of malicious intents. With a vast understanding of the human psyche, dark patterns are capable of leading consumers to perform actions which they would not do under normal circumstances, such as evoking buying pressure or giving away sensitive data. In this thesis, we focus on the detection of dark patterns, especially the Social Proof, Misdirection, Scarcity, and Urgency patterns using multinomial naïve Bayes, support-vector machine, k-nearest neighbor, and random forest, as well as state-of-the-art transfer learning methods like ULMFiT and DistilBERT. For this purpose, we utilize a collection of 1818 classified dark patterns. First, we perform nested cross-validations for all algorithms for valuable insights that we need for the model selection. Overall we achieve a balanced accuracy of 0.926 on average, whereas all models, except for k-nearest neighbor, perform similarly well. Then, with the gained knowledge, we demonstrate that dark patterns can indeed be detected using machine learning techniques. At last, using our fine-tuned models, we reveal the existence of dark patterns in a collection of newsletter emails, with a performance of 0.436 balanced accuracy. Thus we conclude, that this work provides essential insights into the fact that dark patterns exist in hitherto unnoticed fields and how more sophisticated methods are crucial to combat such patterns.
2020
Completed
Implementation and Analysis of a Keystroke Dynamics Authentication System
Supervisor:
Matthias Gazzari
Password based authentication systems face many problems in today’s time. Data breaches and users selecting weak passwords raised the need for different authentication methods or a second factor. Popular methods include fingerprint or face detection and second factors like access or transaction codes. But there are less explored systems that use keystroke dynamics authentication.
In this bachelor thesis we analyze existing keystroke dynamics authentication systems. To get a better understanding we implement such a system. Using datasets that are publicly available our system reaches a false acceptance rate (FAR) of 14 % and a false rejection rate (FRR) of 28 %. Having an own keystroke dynamics authentication systems we can then perform an evaluation in terms of usability in practice. Based on this evaluation we discuss in which cases such a system is a suitable and secure way for authentication.
We conclude that in general keystroke dynamics authentication systems are a convenient and secure way for an additional security factor. But we also distinguish existing challenges like when users have different computers (with different keyboards) or use auto-fill functions of password managers. And we state ideas on how our system’s performance could be improved and challenges could be faced.