Access Control and Machine Learning: Evasion and Defenses

Machine learning (ML) and artificial intelligence (AI) systems have experienced significant proliferation during the recent years, for example in the new market of "machine learning as a service". ML is also increasingly being deployed in security-critical applications, such as access control systems. ML can be used to make security systems easier to use, or to defend against specific attacks, such as the "relay attack".  Such ML applications are particularly sensitive to the recent development of  "adversarial machine learning", where weaknesses in machine learning systems are exploited to undermine some security-critical property. For example, "evasion attacks" undermine a ML system's prediction integrity, while "model extraction attacks" undermine the system's confidentiality. It has become increasingly important to evaluate ML applications against such undesired behavior.  The work described in this dissertation is divided into three parts. In the first part, I evaluate how security properties in so-called transparent authentication systems can be improved using machine learning, and describe how to evaluate security against strong adversaries. In the second part, I present state-of-the-art evasion and model extraction attacks against image classification systems. In the third part, I evaluate state-of-the-art hate speech classifiers against evasion attacks, and present a method of artificially creating credible fake restaurant reviews. Finally, I present general observations and conclusions about both transparent authentication, and the feasibility of using ML for purposes such as moderation.