Experiments

Base class

class schedy.Experiment(name, status='RUNNING')

Base-class for all experiments.

Parameters:

• name (str) – Name of the experiment. An experiment is uniquely identified by its name.
• status (str) – Status of the experiment. See Experiment status.

add_job(**kwargs)

Adds a new job to this experiment.

Parameters:

• hyperparameters (dict) – A dictionnary of hyperparameters values.
• status (str) – Job status. See Job status. Default: QUEUED.
• results (dict) – A dictionnary of result values. Default: No results (empty dictionary).

Returns:

The instance of the new job.

Return type:

schedy.Job

next_job()

Returns a new job to be worked on. This job will be set in the RUNNING state. This function handles everything so that two workers never start working on the same job.

Returns:The instance of the requested job. Return type:schedy.Job

all_jobs()

Retrieves all the jobs belonging to this experiment.

Returns:An iterator over all the jobs of this experiment. Return type:iterator of schedy.Job

get_job(job_id)

Retrieves a job by id.

Parameters:job_id (str) – Id of the job to retrieve. Returns:Instance of the requested job. Return type:schedy.Job

push_updates()

Push all the updates made to this experiment to the service.

delete(ensure=True)

Deletes this experiment.

Parameters:ensure (bool) – If true, an exception will be raised if the experiment was deleted before this call.

Experiment status

Experiment.RUNNING = 'RUNNING'

Status of a running experiment.

Experiment.DONE = 'DONE'

Status of a completed (or paused) experiment.

Manual search

class schedy.ManualSearch(name, status='RUNNING')

Bases: schedy.experiments.Experiment

Represents a manual search, that is to say an experiment for which the only jobs returned by schedy.Experiment.next_job() are jobs that were queued beforehand (by using schedy.Experiment.add_job() for example).

Base-class for all experiments.

Parameters:

• name (str) – Name of the experiment. An experiment is uniquely identified by its name.
• status (str) – Status of the experiment. See Experiment status.

Random search

class schedy.RandomSearch(name, distributions, status='RUNNING')

Bases: schedy.experiments.Experiment

Represents a random search, that is to say en experiment that returns jobs with random hyperparameters when no job was queued manually using schedy.Experiment.add_job().

If you create a job manually for this experiment, it must have only and all the hyperparameters specified in the distributions parameter.

Parameters:

• name (str) – Name of the experiment. An experiment is uniquely identified by its name.
• distributions (dict) – A dictionary of distributions (see schedy.random), whose keys are the names of the hyperparameters.
• status (str) – Status of the experiment. See Experiment status.

Population Based Training

class schedy.PopulationBasedTraining(name, objective, result_name, exploit, explore={}, initial_distributions={}, population_size=None, status='RUNNING', max_generations=None)

Bases: schedy.experiments.Experiment

Implements Population Based Training (see paper).

You have two ways to specify the initial jobs for Population Based training. You can create them manually using schedy.Experiment.add_job(), or you can specify the initial_distributions and population_size parameters.

If you create a job manually for this experiment, it must have at least the hyperparameters specified in the explore parameter.

Parameters:

• name (str) – Name of the experiment. An experiment is uniquely identified by its name.
• objective (str) – The objective of the training, either schedy.pbt.MINIMIZE (to minimize a result) or schedy.pbt.MAXIMIZE (to maximize a result).
• result_name (str) – The name of the result to optimize. This result must be present in the results of all RUNNING jobs of this experiment.
• exploit (schedy.pbt.ExploitStrategy) – Strategy to use to exploit the results (i.e. to focus on the most promising jobs).
• explore (dict) – Strategy to use to explore new hyperparameter values. The keys of the dictionary are the name of the hyperparameters (str), and the values are the strategy associated with the hyperparameter (schedy.pbt.ExploreStrategy). Values for the omitted hyperparameters will not be explored. This parameter is optional: if you do not specify any explore strategy, only exploitation will be used.
• initial_distributions (dict) – The initial distributions for the hyperparameters, as dictionary of distributions (see schedy.random) whose keys are the names of the hyperparameters. This parameter optional, you can also create the initial jobs manually. If you use this parameter, make sure to use population_size as well.
• population_size (int) – Number of initial jobs to create, before starting to exploit/explore (i.e. size of the population). It does not have to be the number of jobs you can process in parallel. The original paper used values between 10 and 80.
• status (str) – Status of the experiment. See Experiment status.
• max_generations (int) – Maximum number of generations to run before marking the experiment the experiments as done (Experiment status). When the maximum number of generations is reached, subsequent calls to schedy.Experiment.next_job() will raise schedy.errors.NoJobError, to indicate that the job queue is empty.