FLARE: Fingerprinting Deep Reinforcement Learning Agents using Universal Adversarial Masks

Buse G.A. Tekgul, N. Asokan

Tutkimustuotos: Artikkeli kirjassa/konferenssijulkaisussaConference article in proceedingsScientificvertaisarvioitu

Abstrakti

We propose FLARE, the first fingerprinting mechanism to verify whether a suspected Deep Reinforcement Learning (DRL) policy is an illegitimate copy of another (victim) policy. We first show that it is possible to find non-transferable, universal adversarial masks, i.e., perturbations, to generate adversarial examples that can successfully transfer from a victim policy to its modified versions but not to independently trained policies. FLARE employs these masks as fingerprints to verify the true ownership of stolen DRL policies by measuring an action agreement value over states perturbed by such masks. Our empirical evaluations show that FLARE is effective (100% action agreement on stolen copies) and does not falsely accuse independent policies (no false positives). FLARE is also robust to model modification attacks and cannot be easily evaded by more informed adversaries without negatively impacting agent performance. We also show that not all universal adversarial masks are suitable candidates for fingerprints due to the inherent characteristics of DRL policies. The spatio-temporal dynamics of DRL problems and sequential decision-making process make characterizing the decision boundary of DRL policies more difficult, as well as searching for universal masks that capture the geometry of it.

AlkuperäiskieliEnglanti
OtsikkoProceedings - 39th Annual Computer Security Applications Conference, ACSAC 2023
KustantajaACM
Sivut492-505
Sivumäärä14
ISBN (elektroninen)979-8-4007-0886-2
DOI - pysyväislinkit
TilaJulkaistu - 4 jouluk. 2023
OKM-julkaisutyyppiA4 Artikkeli konferenssijulkaisussa
TapahtumaAnnual Computer Security Applications Conference - Austin, Yhdysvallat
Kesto: 4 jouluk. 20238 jouluk. 2023
Konferenssinumero: 39

Conference

ConferenceAnnual Computer Security Applications Conference
LyhennettäACSAC
Maa/AlueYhdysvallat
KaupunkiAustin
Ajanjakso04/12/202308/12/2023

Sormenjälki

Sukella tutkimusaiheisiin 'FLARE: Fingerprinting Deep Reinforcement Learning Agents using Universal Adversarial Masks'. Ne muodostavat yhdessä ainutlaatuisen sormenjäljen.

Siteeraa tätä