### Abstract

We develop optimal sleeping and harvesting policies for radio frequency (RF) energy harvesting devices, formalizing the following intuition: when the ambient RF energy is low, devices consume more energy being awake than what can be harvested and should enter sleep mode; when the ambient RF energy is high, on the other hand, it is essential to wake up and harvest. Toward this end, we consider a scenario with intermittent energy arrivals described by a two-state Gilbert-Elliott Markov chain model. The challenge is that the state of the Markov chain can only be observed during the harvesting action, and not while in sleep mode. Two scenarios are studied under this model. In the first scenario, we assume that the transition probabilities of the Markov chain are known and formulate the problem as a partially observable Markov decision process (POMDP). We prove that the optimal policy has a threshold structure and derive the optimal decision parameters. In the practical scenario where the ratio between the reward and the penalty is neither too large nor too small, the POMDP framework and the threshold-based optimal policies are very useful for finding non-Trivial optimal sleeping times. In the second scenario, we assume that the Markov chain parameters are unknown and formulate the problem as a Bayesian adaptive POMDP and propose a heuristic posterior sampling algorithm to reduce the computational complexity. The performance of our approaches is demonstrated via numerical examples.

Original language | English |
---|---|

Article number | 7543484 |

Pages (from-to) | 3528-3539 |

Number of pages | 12 |

Journal | IEEE Journal on Selected Areas in Communications |

Volume | 34 |

Issue number | 12 |

DOIs | |

Publication status | Published - 1 Dec 2016 |

MoE publication type | A1 Journal article-refereed |

### Keywords

- ambient radio frequency energy
- Bayesian inference
- Energy harvesting
- learning
- partially observable Markov decision process

## Fingerprint Dive into the research topics of 'Optimal Radio Frequency Energy Harvesting with Limited Energy Arrival Knowledge'. Together they form a unique fingerprint.

## Cite this

*IEEE Journal on Selected Areas in Communications*,

*34*(12), 3528-3539. [7543484]. https://doi.org/10.1109/JSAC.2016.2600364