Time Budget Management in Multifunction Radars Using Reinforcement Learning

Petteri Pulkkinen; Tuomas Aittomaki; Anders Strom; Visa Koivunen

doi:10.1109/RadarConf2147009.2021.9455344

Time Budget Management in Multifunction Radars Using Reinforcement Learning

Petteri Pulkkinen, Tuomas Aittomaki, Anders Strom, Visa Koivunen

Visa Koivunen Group

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

2 Citations (Scopus)

81 Downloads (Pure)

Abstract

An adaptive revisit interval selection (RIS) in multifunction radars is an integral part of efficient time budget management (TBM). In this paper, the RIS problem is formulated as a Markov decision process (MDP) with unknown state transition probabilities and reward distributions. A reward function is proposed to minimize the tracking load (TL) while maintaining the track loss probability (TLP) at a tolerable level. The reinforcement learning (RL) problem is solved using the Q-learning algorithm with an epsilon-greedy policy. Compared to a baseline algorithm, the RL approach was capable of maintaining the tracks while reducing the tracking load significantly.

Original language	English
Title of host publication	2021 IEEE Radar Conference
Subtitle of host publication	Radar on the Move, RadarConf 2021
Publisher	IEEE
Number of pages	6
ISBN (Electronic)	9781728176093
DOIs	https://doi.org/10.1109/RadarConf2147009.2021.9455344
Publication status	Published - 7 May 2021
MoE publication type	A4 Article in a conference publication
Event	IEEE Radar Conference - Atlanta, United States Duration: 8 May 2021 → 14 May 2021

Publication series

Name	Proceedings of the IEEE Radar Conference
Volume	2021-May
ISSN (Print)	1097-5659
ISSN (Electronic)	2375-5318

Conference

Conference	IEEE Radar Conference
Abbreviated title	RadarConf
Country/Territory	United States
City	Atlanta
Period	08/05/2021 → 14/05/2021

Keywords

adaptive update rate
Q-learning
radar
reinforcement learning
revisit interval selection
time budget management

Access to Document

10.1109/RadarConf2147009.2021.9455344

Pulkkinen_Time Budget Management in MultifunctionAccepted author manuscript, 559 KB

OpenUrl availability

Full text

Cite this

@inproceedings{a72044a1a27e45bc877b48161da60b80,

title = "Time Budget Management in Multifunction Radars Using Reinforcement Learning",

abstract = "An adaptive revisit interval selection (RIS) in multifunction radars is an integral part of efficient time budget management (TBM). In this paper, the RIS problem is formulated as a Markov decision process (MDP) with unknown state transition probabilities and reward distributions. A reward function is proposed to minimize the tracking load (TL) while maintaining the track loss probability (TLP) at a tolerable level. The reinforcement learning (RL) problem is solved using the Q-learning algorithm with an epsilon-greedy policy. Compared to a baseline algorithm, the RL approach was capable of maintaining the tracks while reducing the tracking load significantly.",

keywords = "adaptive update rate, Q-learning, radar, reinforcement learning, revisit interval selection, time budget management",

author = "Petteri Pulkkinen and Tuomas Aittomaki and Anders Strom and Visa Koivunen",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; IEEE Radar Conference, RadarConf ; Conference date: 08-05-2021 Through 14-05-2021",

year = "2021",

month = may,

day = "7",

doi = "10.1109/RadarConf2147009.2021.9455344",

language = "English",

series = "Proceedings of the IEEE Radar Conference",

publisher = "IEEE",

booktitle = "2021 IEEE Radar Conference",

address = "United States",

}

Pulkkinen, P, Aittomaki, T, Strom, A & Koivunen, V 2021, Time Budget Management in Multifunction Radars Using Reinforcement Learning. in 2021 IEEE Radar Conference: Radar on the Move, RadarConf 2021., 9455344, Proceedings of the IEEE Radar Conference, vol. 2021-May, IEEE, IEEE Radar Conference, Atlanta, United States, 08/05/2021. https://doi.org/10.1109/RadarConf2147009.2021.9455344

Time Budget Management in Multifunction Radars Using Reinforcement Learning. / Pulkkinen, Petteri; Aittomaki, Tuomas; Strom, Anders et al.
2021 IEEE Radar Conference: Radar on the Move, RadarConf 2021. IEEE, 2021. 9455344 (Proceedings of the IEEE Radar Conference; Vol. 2021-May).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Time Budget Management in Multifunction Radars Using Reinforcement Learning

AU - Pulkkinen, Petteri

AU - Aittomaki, Tuomas

AU - Strom, Anders

AU - Koivunen, Visa

PY - 2021/5/7

Y1 - 2021/5/7

N2 - An adaptive revisit interval selection (RIS) in multifunction radars is an integral part of efficient time budget management (TBM). In this paper, the RIS problem is formulated as a Markov decision process (MDP) with unknown state transition probabilities and reward distributions. A reward function is proposed to minimize the tracking load (TL) while maintaining the track loss probability (TLP) at a tolerable level. The reinforcement learning (RL) problem is solved using the Q-learning algorithm with an epsilon-greedy policy. Compared to a baseline algorithm, the RL approach was capable of maintaining the tracks while reducing the tracking load significantly.

AB - An adaptive revisit interval selection (RIS) in multifunction radars is an integral part of efficient time budget management (TBM). In this paper, the RIS problem is formulated as a Markov decision process (MDP) with unknown state transition probabilities and reward distributions. A reward function is proposed to minimize the tracking load (TL) while maintaining the track loss probability (TLP) at a tolerable level. The reinforcement learning (RL) problem is solved using the Q-learning algorithm with an epsilon-greedy policy. Compared to a baseline algorithm, the RL approach was capable of maintaining the tracks while reducing the tracking load significantly.

KW - adaptive update rate

KW - Q-learning

KW - radar

KW - reinforcement learning

KW - revisit interval selection

KW - time budget management

UR - http://www.scopus.com/inward/record.url?scp=85112456520&partnerID=8YFLogxK

U2 - 10.1109/RadarConf2147009.2021.9455344

DO - 10.1109/RadarConf2147009.2021.9455344

M3 - Conference contribution

AN - SCOPUS:85112456520

T3 - Proceedings of the IEEE Radar Conference

BT - 2021 IEEE Radar Conference

PB - IEEE

T2 - IEEE Radar Conference

Y2 - 8 May 2021 through 14 May 2021

ER -

Time Budget Management in Multifunction Radars Using Reinforcement Learning

Abstract

Publication series

Conference

Keywords

Access to Document

OpenUrl availability

Other files and links

Fingerprint

Cite this