Grid Search is the process of choosing a set of parameters for a learning algorithm, usually with the goal of optimizing a measure of the algorithm's performance on an independent data set.

With cnvrg you can fire up multiple experiments in parallel with a single command. This feature is very useful for grid search and hyperparameter optimization.

To do so, you will need a hyperparameter YAML file that will contain information about the parameters, and their possible values.

In the YAML file, each parameter has a param_name that should match the argument that is fed to the experiment. So the parameter name: kernel will create a new experiment with this string is attached: "--kernel='rbf'" as rbf is one of the possible parameters.

To trigger the grid search, please run your file using the following command:
cnvrg run --medium --grid=src/hyper.yaml python myscript.py  

Below you can see an example yaml file:
(This file is automatically created when initializing a project via cnvrg CLI)

parameters:
    # Integer parameter is a range of possible values between a
    # minimum (inclusive) and maximum (not inclusive) values.
    # Values are round down (0.7 => 0)
    #
    - param_name: "epochs"
      type: "integer"
      min: 0 # inclusive
      max: 10 # not inclusive
      scale: "linear"
      steps: 4 # The number of linear steps to produce.

    # Float parameter is a range of possible values between a
    # minimum (inclusive) and maximum (not inclusive) values.
    #
    - param_name: "learning_rate"
      type: "float" # precision is 9 after period
      min: 0.00001
      max: 0.1
      scale: "log2" # Could be log10 as well
      steps: 2

    # Discrete parameter is an array of numerical values.
    #
    - param_name: "c"
      type: "discrete"
      values: [0, 0.1 ,0.001]

    # Categorical parameter is an array of string values
    #
    - param_name: "kernel"
      type: "categorical"
      values: ["linear", "rbf"]

Python Implementation Example 

Below you can find a python snippet that uses argparse library to parse input values from command line:

import argparse

parser = argparse.ArgumentParser(description='set input arguments')

parser.add_argument('--epochs', action="store", dest='epochs', type=int, default=10)
parser.add_argument('--dropout', action="store", dest='dropout', type=float, default=0.2)
parser.add_argument('--batch_size', action="store", dest='batch_size', type=int, default=128)
parser.add_argument('--hidden', action="store", dest='hidden', type=int, default=512)
parser.add_argument('--learning_rate', action="store", dest='learning_rate', type=float, default=0.0001)

args = parser.parse_args()

epochs        = args.epochs
batch_size    = args.batch_size
hidden_nodes  = args.hidden
dropout       = args.dropout
learning_rate = args.learning_rate

R Implementation Example 

Below you can find an R snippet that uses optparse library to parse input values from command line:

 install.packages('optparse', dependencies = TRUE)


 library("optparse")

 option_list = list(
         make_option(c("--epochs"), type="integer",     default=10,help="number of epochs to perform",metavar="number"),
         make_option(c("--learning_rate"), type="double", default=0.001,help="learning rate values",metavar="number"));

 opt_parser = OptionParser(option_list=option_list);
 opt = parse_args(opt_parser);

 #sample printing value

 sprintf("Number of epochs is: ( %s )", opt$epochs)
 sprintf("learning rate is: ( %s )", opt$learning_rate)
Did this answer your question?