Policies.OCUCBH module

The Optimally Confident UCB (OC-UCB) policy for bounded stochastic bandits. Initial version (horizon-dependent).

• Reference: [Lattimore, 2015](https://arxiv.org/pdf/1507.07880.pdf)

• There is also a horizon-independent version, OCUCB.OCUCB, from [Lattimore, 2016](https://arxiv.org/pdf/1603.08661.pdf).

Policies.OCUCBH.PSI = 2

Default value for parameter \(\psi \geq 2\) for OCUCBH.

Policies.OCUCBH.ALPHA = 4

Default value for parameter \(\alpha \geq 2\) for OCUCBH.

class Policies.OCUCBH.OCUCBH(nbArms, horizon=None, psi=2, alpha=4, lower=0.0, amplitude=1.0)

Bases: Policies.OCUCB.OCUCB

The Optimally Confident UCB (OC-UCB) policy for bounded stochastic bandits. Initial version (horizon-dependent).

• Reference: [Lattimore, 2015](https://arxiv.org/pdf/1507.07880.pdf)

__init__(nbArms, horizon=None, psi=2, alpha=4, lower=0.0, amplitude=1.0)

New generic index policy.

• nbArms: the number of arms,

• lower, amplitude: lower value and known amplitude of the rewards.

psi = None

Parameter \(\psi \geq 2\).

alpha = None

Parameter \(\alpha \geq 2\).

horizon = None

Horizon T.

__str__()

-> str

computeIndex(arm)

Compute the current index, at time t and after \(N_k(t)\) pulls of arm k:

\[I_k(t) = \frac{X_k(t)}{N_k(t)} + \sqrt{\frac{\alpha}{N_k(t)} \log(\frac{\psi T}{t})}.\]

• Where \(\alpha\) and \(\psi\) are two parameters of the algorithm.

__module__ = 'Policies.OCUCBH'

class Policies.OCUCBH.AOCUCBH(nbArms, horizon=None, lower=0.0, amplitude=1.0)

Bases: Policies.OCUCBH.OCUCBH

The Almost Optimally Confident UCB (OC-UCB) policy for bounded stochastic bandits. Initial version (horizon-dependent).

• Reference: [Lattimore, 2015](https://arxiv.org/pdf/1507.07880.pdf)

__init__(nbArms, horizon=None, lower=0.0, amplitude=1.0)

New generic index policy.

• nbArms: the number of arms,

• lower, amplitude: lower value and known amplitude of the rewards.

__str__()

-> str

computeIndex(arm)

Compute the current index, at time t and after \(N_k(t)\) pulls of arm k:

\[I_k(t) = \frac{X_k(t)}{N_k(t)} + \sqrt{\frac{2}{N_k(t)} \log(\frac{T}{N_k(t)})}.\]

__module__ = 'Policies.OCUCBH'