configuration_nonstationary module¶

Configuration for the simulations, for the piecewise stationary single-player case.

configuration_nonstationary.CPU_COUNT = 4¶: Number of CPU on the local machine

configuration_nonstationary.HORIZON = 5000¶: HORIZON : number of time steps of the experiments. Warning Should be >= 10000 to be interesting “asymptotically”.

configuration_nonstationary.DO_PARALLEL = True¶: To profile the code, turn down parallel computing

configuration_nonstationary.N_JOBS = -1¶: Number of jobs to use for the parallel computations. -1 means all the CPU cores, 1 means no parallelization.

configuration_nonstationary.REPETITIONS = 100¶: REPETITIONS : number of repetitions of the experiments. Warning: Should be >= 10 to be statistically trustworthy.

configuration_nonstationary.configuration = {'append_labels': {}, 'change_labels': {1: 'klUCB', 2: 'Thompson Sampling', 9: 'SW-klUCB', 10: 'DTS', 11: 'M-klUCB', 12: 'CUSUM-klUCB(Localization)', 13: 'GLR-klUCB(Local, Localization)', 14: 'GLR-klUCB(Local, Localization, increasing $\\alpha_t$)', 15: 'GLR-klUCB(Global, Localization)', 16: 'GLR-klUCB(Global, Localization, increasing $\\alpha_t$)'}, 'environment': [{'arm_type': <class 'Arms.Bernoulli.Bernoulli'>, 'params': {'listOfMeans': [[0.3, 0.5, 0.9], [0.3, 0.2, 0.9], [0.3, 0.2, 0.1], [0.7, 0.2, 0.1], [0.7, 0.5, 0.1]], 'changePoints': [0, 1000, 2000, 3000, 4000]}}, {'arm_type': <class 'Arms.Bernoulli.Bernoulli'>, 'params': {'listOfMeans': [[0.4, 0.5, 0.9], [0.5, 0.4, 0.7], [0.6, 0.3, 0.5], [0.7, 0.2, 0.3], [0.8, 0.1, 0.1]], 'changePoints': [0, 1000, 2000, 3000, 4000]}}], 'horizon': 5000, 'n_jobs': -1, 'nb_break_points': 4, 'plot_lowerbound': False, 'policies': [{'archtype': <class 'Policies.UCB.UCB'>, 'params': {}}, {'archtype': <class 'Policies.klUCB.klUCB'>, 'params': {'klucb': CPUDispatcher(<function klucbBern>)}, 'change_label': 'klUCB'}, {'archtype': <class 'Policies.Thompson.Thompson'>, 'params': {'posterior': <class 'Policies.Posterior.Beta.Beta'>}, 'change_label': 'Thompson Sampling'}, {'archtype': <class 'Policies.OracleSequentiallyRestartPolicy.OracleSequentiallyRestartPolicy'>, 'params': {'changePoints': [0, 1000, 2000, 3000, 4000], 'listOfMeans': [[0.3, 0.5, 0.9], [0.3, 0.2, 0.9], [0.3, 0.2, 0.1], [0.7, 0.2, 0.1], [0.7, 0.5, 0.1]], 'policy': <class 'Policies.UCB.UCB'>, 'reset_for_all_change': True, 'reset_for_suboptimal_change': False}}, {'archtype': <class 'Policies.OracleSequentiallyRestartPolicy.OracleSequentiallyRestartPolicy'>, 'params': {'changePoints': [0, 1000, 2000, 3000, 4000], 'listOfMeans': [[0.3, 0.5, 0.9], [0.3, 0.2, 0.9], [0.3, 0.2, 0.1], [0.7, 0.2, 0.1], [0.7, 0.5, 0.1]], 'policy': <class 'Policies.klUCB.klUCB'>, 'reset_for_all_change': True, 'reset_for_suboptimal_change': False}}, {'archtype': <class 'Policies.OracleSequentiallyRestartPolicy.OracleSequentiallyRestartPolicy'>, 'params': {'changePoints': [0, 1000, 2000, 3000, 4000], 'listOfMeans': [[0.3, 0.5, 0.9], [0.3, 0.2, 0.9], [0.3, 0.2, 0.1], [0.7, 0.2, 0.1], [0.7, 0.5, 0.1]], 'policy': <class 'Policies.Thompson.Thompson'>, 'reset_for_all_change': True, 'reset_for_suboptimal_change': False}}, {'archtype': <class 'Policies.Exp3S.Exp3S'>, 'params': {'horizon': 5000, 'max_nb_random_events': 4}}, {'archtype': <class 'Policies.Exp3R.Exp3R'>, 'params': {'horizon': 5000}}, {'archtype': <class 'Policies.DiscountedUCB.DiscountedklUCB'>, 'params': {'gamma': 0.95}}, {'archtype': <class 'Policies.SlidingWindowUCB.SWklUCB'>, 'params': {'tau': 206}, 'change_label': 'SW-klUCB'}, {'archtype': <class 'Policies.DiscountedThompson.DiscountedThompson'>, 'params': {'posterior': <class 'Policies.Posterior.DiscountedBeta.DiscountedBeta'>, 'gamma': 0.95}, 'change_label': 'DTS'}, {'archtype': <class 'Policies.Monitored_UCB.Monitored_IndexPolicy'>, 'params': {'policy': <class 'Policies.klUCB.klUCB'>, 'horizon': 5000, 'w': 150, 'delta': 0.1}, 'change_label': 'M-klUCB'}, {'archtype': <class 'Policies.CUSUM_UCB.CUSUM_IndexPolicy'>, 'params': {'horizon': 5000, 'policy': <class 'Policies.klUCB.klUCB'>, 'max_nb_random_events': 4, 'lazy_detect_change_only_x_steps': 20, 'epsilon': 0.1, 'use_localization': True}, 'change_label': 'CUSUM-klUCB(Localization)'}, {'archtype': <class 'Policies.GLR_UCB.BernoulliGLR_IndexPolicy_WithDeterministicExploration'>, 'params': {'policy': <class 'Policies.klUCB_forGLR.klUCB_forGLR'>, 'per_arm_restart': True, 'delta': 0.00408248290463863, 'alpha0': 0.007148647364956354, 'lazy_detect_change_only_x_steps': 10, 'lazy_try_value_s_only_x_steps': 10, 'use_localization': True, 'use_increasing_alpha': False}, 'change_label': 'GLR-klUCB(Local, Localization)'}, {'archtype': <class 'Policies.GLR_UCB.BernoulliGLR_IndexPolicy_WithDeterministicExploration'>, 'params': {'policy': <class 'Policies.klUCB_forGLR.klUCB_forGLR'>, 'per_arm_restart': True, 'delta': 0.00408248290463863, 'alpha0': 0.007148647364956354, 'lazy_detect_change_only_x_steps': 10, 'lazy_try_value_s_only_x_steps': 10, 'use_localization': True, 'use_increasing_alpha': True}, 'change_label': 'GLR-klUCB(Local, Localization, increasing $\\alpha_t$)'}, {'archtype': <class 'Policies.GLR_UCB.BernoulliGLR_IndexPolicy_WithDeterministicExploration'>, 'params': {'policy': <class 'Policies.klUCB_forGLR.klUCB_forGLR'>, 'per_arm_restart': False, 'delta': 0.007071067811865475, 'alpha0': 0.00412727348049926, 'lazy_detect_change_only_x_steps': 10, 'lazy_try_value_s_only_x_steps': 10, 'use_localization': True, 'use_increasing_alpha': False}, 'change_label': 'GLR-klUCB(Global, Localization)'}, {'archtype': <class 'Policies.GLR_UCB.BernoulliGLR_IndexPolicy_WithDeterministicExploration'>, 'params': {'policy': <class 'Policies.klUCB_forGLR.klUCB_forGLR'>, 'per_arm_restart': False, 'delta': 0.007071067811865475, 'alpha0': 0.00412727348049926, 'lazy_detect_change_only_x_steps': 10, 'lazy_try_value_s_only_x_steps': 10, 'use_localization': True, 'use_increasing_alpha': True}, 'change_label': 'GLR-klUCB(Global, Localization, increasing $\\alpha_t$)'}], 'repetitions': 100, 'verbosity': 6}¶: This dictionary configures the experiments

configuration_nonstationary.NB_ARMS = 3¶: Number of arms for non-hard-coded problems (Bayesian problems)

configuration_nonstationary.LOWER = 0.0¶: Default value for the lower value of means

configuration_nonstationary.AMPLITUDE = 1.0¶: Default value for the amplitude value of means

configuration_nonstationary.VARIANCE = 0.25¶: Variance of Gaussian arms, if needed

configuration_nonstationary.ARM_TYPE¶: alias of Arms.Bernoulli.Bernoulli

configuration_nonstationary.MEANS = [0.05, 0.49999999999999994, 0.95]¶: Means of arms for non-hard-coded problems (non Bayesian)

configuration_nonstationary.lowerbound_on_sequence_length(horizon, gap)[source]¶

A function that computes the lower-bound (we will find) on the sequence length to have a reasonable bound on the delay of our change-detection algorithm.

It returns the smallest possible sequence length $L = \tau_{m+1} - \tau_m$ satisfying:

\[L \geq \frac{8}{\Delta^2} \log(T).\]

configuration_nonstationary.check_condition_on_piecewise_stationary_problems(horizon, listOfMeans, changePoints)[source]¶: Check some conditions on the piecewise stationary problem.

configuration_nonstationary.NB_BREAK_POINTS = 4¶: Number of true breakpoints. They are uniformly spaced in time steps (and the first one at t=0 does not count).

configuration_nonstationary.nbArms = 3¶: Number of arms in the first environment

configuration_nonstationary.klucb¶: Warning: if using Exponential or Gaussian arms, gives klExp or klGauss to KL-UCB-like policies!

configuration_nonstationary.WINDOW_SIZE = 150¶: Default window size $w$ for the M-UCB and SW-UCB algorithm.