Experiments¶
Base class¶
-
class
schedy.
Experiment
(name, status='RUNNING')[source]¶ Base-class for all experiments.
Parameters: - name (str) – Name of the experiment. An experiment is uniquely identified by its name.
- status (str) – Status of the experiment. See Experiment status.
-
add_job
(**kwargs)[source]¶ Adds a new job to this experiment.
Parameters: - hyperparameters (dict) – A dictionnary of hyperparameters values.
- status (str) – Job status. See Job status. Default: QUEUED.
- results (dict) – A dictionnary of result values. Default: No results (empty dictionary).
Returns: The instance of the new job.
Return type:
-
next_job
()[source]¶ Returns a new job to be worked on. This job will be set in the
RUNNING
state. This function handles everything so that two workers never start working on the same job.Returns: The instance of the requested job. Return type: schedy.Job
-
all_jobs
()[source]¶ Retrieves all the jobs belonging to this experiment.
Returns: An iterator over all the jobs of this experiment. Return type: iterator of schedy.Job
-
get_job
(job_id)[source]¶ Retrieves a job by id.
Parameters: job_id (str) – Id of the job to retrieve. Returns: Instance of the requested job. Return type: schedy.Job
Experiment status¶
-
Experiment.
RUNNING
= 'RUNNING'¶ Status of a running experiment.
-
Experiment.
DONE
= 'DONE'¶ Status of a completed (or paused) experiment.
Manual search¶
-
class
schedy.
ManualSearch
(name, status='RUNNING')[source]¶ Bases:
schedy.experiments.Experiment
Represents a manual search, that is to say an experiment for which the only jobs returned by
schedy.Experiment.next_job()
are jobs that were queued beforehand (by usingschedy.Experiment.add_job()
for example).Base-class for all experiments.
Parameters: - name (str) – Name of the experiment. An experiment is uniquely identified by its name.
- status (str) – Status of the experiment. See Experiment status.
Random search¶
-
class
schedy.
RandomSearch
(name, distributions, status='RUNNING')[source]¶ Bases:
schedy.experiments.Experiment
Represents a random search, that is to say en experiment that returns jobs with random hyperparameters when no job was queued manually using
schedy.Experiment.add_job()
.If you create a job manually for this experiment, it must have only and all the hyperparameters specified in the
distributions
parameter.Parameters: - name (str) – Name of the experiment. An experiment is uniquely identified by its name.
- distributions (dict) – A dictionary of distributions (see
schedy.random
), whose keys are the names of the hyperparameters. - status (str) – Status of the experiment. See Experiment status.
Population Based Training¶
-
class
schedy.
PopulationBasedTraining
(name, objective, result_name, exploit, explore={}, initial_distributions={}, population_size=None, status='RUNNING', max_generations=None)[source]¶ Bases:
schedy.experiments.Experiment
Implements Population Based Training (see paper).
You have two ways to specify the initial jobs for Population Based training. You can create them manually using
schedy.Experiment.add_job()
, or you can specify theinitial_distributions
andpopulation_size
parameters.If you create a job manually for this experiment, it must have at least the hyperparameters specified in the
explore
parameter.Parameters: - name (str) – Name of the experiment. An experiment is uniquely identified by its name.
- objective (str) – The objective of the training, either
schedy.pbt.MINIMIZE
(to minimize a result) orschedy.pbt.MAXIMIZE
(to maximize a result). - result_name (str) – The name of the result to optimize. This result
must be present in the results of all
RUNNING
jobs of this experiment. - exploit (schedy.pbt.ExploitStrategy) – Strategy to use to exploit the results (i.e. to focus on the most promising jobs).
- explore (dict) – Strategy to use to explore new hyperparameter values.
The keys of the dictionary are the name of the
hyperparameters (str), and the values are the strategy
associated with the hyperparameter
(
schedy.pbt.ExploreStrategy
). Values for the omitted hyperparameters will not be explored. This parameter is optional: if you do not specify any explore strategy, only exploitation will be used. - initial_distributions (dict) – The initial distributions for the
hyperparameters, as dictionary of distributions (see
schedy.random
) whose keys are the names of the hyperparameters. This parameter optional, you can also create the initial jobs manually. If you use this parameter, make sure to usepopulation_size
as well. - population_size (int) – Number of initial jobs to create, before starting to exploit/explore (i.e. size of the population). It does not have to be the number of jobs you can process in parallel. The original paper used values between 10 and 80.
- status (str) – Status of the experiment. See Experiment status.
- max_generations (int) – Maximum number of generations to run before
marking the experiment the experiments as done
(Experiment status). When the maximum number of
generations is reached, subsequent calls to
schedy.Experiment.next_job()
will raiseschedy.errors.NoJobError
, to indicate that the job queue is empty.