Policies.Experimentals.klUCBlog10 module

The generic kl-UCB policy for one-parameter exponential distributions. By default, it assumes Bernoulli arms. Note: using \(\log10(t)\) and not \(\log(t)\) for the KL-UCB index. Reference: [Garivier & Cappé - COLT, 2011].

class Policies.Experimentals.klUCBlog10.klUCBlog10(nbArms, tolerance=0.0001, klucb=CPUDispatcher(<function klucbBern>), c=1.0, lower=0.0, amplitude=1.0)

Bases: klUCB.klUCB

The generic kl-UCB policy for one-parameter exponential distributions. By default, it assumes Bernoulli arms. Note: using \(\log10(t)\) and not \(\log(t)\) for the KL-UCB index. Reference: [Garivier & Cappé - COLT, 2011].

__str__()

-> str

computeIndex(arm)

Compute the current index, at time t and after \(N_k(t)\) pulls of arm k:

\[\begin{split}\hat{\mu}_k(t) &= \frac{X_k(t)}{N_k(t)}, \\ U_k(t) &= \sup\limits_{q \in [a, b]} \left\{ q : \mathrm{kl}(\hat{\mu}_k(t), q) \leq \frac{c \log_{10}(t)}{N_k(t)} \right\},\\ I_k(t) &= U_k(t).\end{split}\]

If rewards are in \([a, b]\) (default to \([0, 1]\)) and \(\mathrm{kl}(x, y)\) is the Kullback-Leibler divergence between two distributions of means x and y (see Arms.kullback), and c is the parameter (default to 1).

computeAllIndex()

Compute the current indexes for all arms, in a vectorized manner.

__module__ = 'Policies.Experimentals.klUCBlog10'