HParams¶
-
class
shabda.hyperparams.HParams(hparams, default_hparams, allow_new_hparam=False)[source]¶ Bases:
objectA class to maintains hyperparameters for configing Shabda modules.
The class has several useful features:
Auto-completion of missing values. Users can specify only a subset of hyperparameters they care about. Other hyperparameters will automatically take the default values. The auto-completion performs recursively so that hyperparameters taking dict values will also be auto-completed All Shabda modules provide a
default_hparams()containing allowed hyperparameters and their default values. For example## Recursive auto-completion default_hparams = {"a": 1, "b": {"c": 2, "d": 3}} hparams = {"b": {"c": 22}} hparams_ = HParams(hparams, default_hparams) hparams_.todict() == {"a": 1, "b": {"c": 22, "d": 3}} # "a" and "d" are auto-completed ## All Shabda modules have built-in `default_hparams` hparams = {"dropout_rate": 0.1} emb = tx.modules.WordEmbedder(hparams=hparams, ) emb.hparams.todict() == { "dropout_rate": 0.1, # provided value "dim": 100 # default value }
Automatic typecheck. For most hyperparameters, provided value must have the same or compatible dtype with the default value. HParams does necessary typecheck, and raises Error if improper dtype is provided. Also, hyperparameters not listed in default_hparams are not allowed, except for “kwargs” as detailed below.
Flexible dtype for specified hyperparameters. Some hyperparameters may allow different dtypes of values.
- Hyperparameters named “type” are not typechecked. For example, in
get_rnn_cell(), hyperparameter “type” can take value of an RNNCell class, its string name of module path, or an RNNCell class instance. (String name or module path is allowd so that users can specify the value in YAML config files.) - For other hyperparameters, list them in the “@no_typecheck” field in default_hparams to skip typecheck. For example, in
get_rnn_cell(), hyperparameter “_keep_prob” can be set to either a float or a tf.placeholder.
- Hyperparameters named “type” are not typechecked. For example, in
Special flexibility of keyword argument hyparameters. Hyperparameters named “kwargs” are used as keyword arguments for a class constructor or a function call. Such hyperparameters take a dict, and users can add arbitrary valid keyword arguments to the dict. For example:
default_rnn_cell_hparams = { "type": "BasicLSTMCell", "kwargs": { "num_units": 256 } # Other hyperparameters } my_hparams = { "kwargs" { "num_units": 123, "forget_bias": 0.0 # Other valid keyword arguments "activation": "tf.nn.relu" # for BasicLSTMCell constructor } } _ = HParams(my_hparams, default_rnn_cell_hparams)
Rich interfaces. An HParams instance provides rich interfaces for accessing, updating, or adding hyperparameters.
hparams = HParams(my_hparams, default_hparams) # Access hparams.type == hparams["type"] # Update hparams.type = "GRUCell" hparams.kwargs = { "num_units": 100 } hparams.kwargs.num_units == 100 # Add new hparams.add_hparam("index", 1) hparams.index == 1 # Convert to `dict` (recursively) type(hparams.todic()) == dict # I/O pickle.dump(hparams, "hparams.dump") with open("hparams.dump", 'rb') as f: hparams_loaded = pickle.load(f)
- Args:
- hparams: A dict or an HParams instance containing hyperparameters.
- If None, all hyperparameters are set to default values.
- default_hparams (dict): Hyperparameters with default values. If None,
- Hyperparameters are fully defined by
hparams. - allow_new_hparam (bool): If False (default),
hparamscannot - contain hyperparameters that are not included in
default_hparams, except for the case of"kwargs"as above.