Thompson sampling in dynamic systems for contextual bandit problems

T Xu, Y Yu, J Turner, A Regan - arXiv preprint arXiv:1310.5008, 2013 - arxiv.org
arXiv preprint arXiv:1310.5008, 2013arxiv.org
We consider the multiarm bandit problems in the timevarying dynamic system for rich
structural features. For the nonlinear dynamic model, we propose the approximate inference
for the posterior distributions based on Laplace Approximation. For the context bandit
problems, Thompson Sampling is adopted based on the underlying posterior distributions of
the parameters. More specifically, we introduce the discount decays on the previous
samples impact and analyze the different decay rates with the underlying sample dynamics …
We consider the multiarm bandit problems in the timevarying dynamic system for rich structural features. For the nonlinear dynamic model, we propose the approximate inference for the posterior distributions based on Laplace Approximation. For the context bandit problems, Thompson Sampling is adopted based on the underlying posterior distributions of the parameters. More specifically, we introduce the discount decays on the previous samples impact and analyze the different decay rates with the underlying sample dynamics. Consequently, the exploration and exploitation is adaptively tradeoff according to the dynamics in the system.
arxiv.org