GsdEvaluationCV#

class mobgap.gait_sequences.evaluation.GsdEvaluationCV( dataset: ~mobgap.data.base.BaseGaitDatasetWithReference, cv_iterator: ~tpcp.validate._cross_val_helper.DatasetSplitter | int | ~sklearn.model_selection._split.BaseCrossValidator | ~collections.abc.Iterator | None, scoring: ~typing.Callable | None = <function gsd_evaluation_scorer>, *, cv_params: dict | None = None, )[source]#

Evaluation challenge for Gait Sequence Detection (GSD) algorithms using cross-validation.

This class will use cross_validate to evaluate the performance of a GSD algorithm wrapped in a GsdEmulationPipeline on a dataset with reference information. This is a suitable approach, when you want to evaluate and compare algorithms that are “trainable” in any way. This could be, because they are ML algorithms or because they have hyperparameters that can be optimized via Grid-Search.

The cross validation parameters can be modified by the user to adapt them to a given dataset.

Parameters:

dataset: A gait dataset with reference information. Evaluation is performed across all datapoints within the dataset.
scoring: A scoring function that evaluates the performance of the algorithm on a single datapoint. It should take a pipeline and a datapoint as input, run the pipeline on the datapoint and return a dictionary of performance metrics. These performance metrics are then aggregated across all datapoints.
cv_iterator: A valid cv_iterator. For complex CVs (e.g. stratified/grouped) this should be a DatasetSplitter instance. For more information see cross_validate.
cv_params: Dictionary with further parameters that are directly passed to cross_validate. This can overwrite all parameters except optimizable, dataset, scoring, and cv, which are directly set via the other parameters of this method. Typical usecase is to set n_jobs to activate multiprocessing.

Other Parameters:

optimizer: The tpcp optimizer passed to the run method.

Attributes:

cv_results_: Dictionary with all results of the cross-validation. The results are returned by cross_validate. You can control what information is provided via cv_params
start_datetime_utc_timestamp_: The start time of the evaluation as UTC timestamp.
start_datetime_: The start time of the evaluation as human readable string.
end_datetime_utc_timestamp_: The end time of the evaluation as UTC timestamp.
end_datetime_: The end time of the evaluation as human readable string.
runtime_s_: The runtime of the evaluation in seconds. Note, that the runtime might not be exactly the difference between the start and the end time. The runtime is independently calculated using time.perf_timer.

Methods

`clone`()	Create a new instance of the class with all parameters copied over.
`get_params`([deep])	Get parameters for this algorithm.
`run`(optimizer)	Run the evaluation challenge.
`set_params`(**params)	Set the parameters of this Algorithm.

__init__( dataset: ~mobgap.data.base.BaseGaitDatasetWithReference, cv_iterator: ~tpcp.validate._cross_val_helper.DatasetSplitter | int | ~sklearn.model_selection._split.BaseCrossValidator | ~collections.abc.Iterator | None, scoring: ~typing.Callable | None = <function gsd_evaluation_scorer>, *, cv_params: dict | None = None, ) → None[source]#

clone() → Self[source]#

Create a new instance of the class with all parameters copied over.

This will create a new instance of the class itself and all nested objects

get_params(deep: bool = True) → dict[str, Any][source]#

Get parameters for this algorithm.

Parameters:

deep: Only relevant if object contains nested algorithm objects. If this is the case and deep is True, the params of these nested objects are included in the output using a prefix like nested_object_name__ (Note the two “_” at the end)

Returns:

params: Parameter names mapped to their values.

run( optimizer: BaseOptimize[GsdEmulationPipeline, BaseGaitDatasetWithReference], ) → Self[source]#

Run the evaluation challenge.

This will call the optimizer for each train set and evaluate the performance on each test set defined by the cv_iterator on the dataset.

Parameters:

optimizer

A valid tpcp optimizer that wraps a pipeline that is compatible with the provided dataset and scorer. Usually that should be an optimizer wrapping a GsdEmulationPipeline. If you want to run without optimization, but still use the same test-folds, use DummyOptimize:

>>> from tpcp.optimize import DummyOptimize
>>> from mobgap.gait_sequences import GsdIluz
>>>
>>> dummy_optimizer = DummyOptimize(
...     pipeline=GsdEmulationPipeline(GsdIluz()),
...     ignore_potential_user_error_warning=True,
... )
>>> challenge = GsdEvaluationCV(dataset, cv_iterator, scoring=scoring)
>>> challenge.run(dummy_optimizer)

Returns:

self: The instance of the class with the cv_results_ attribute set to the results of the cross-validation.

set_params(**params: Any) → Self[source]#

Set the parameters of this Algorithm.

To set parameters of nested objects use nested_object_name__para_name=.

Examples using `mobgap.gait_sequences.evaluation.GsdEvaluationCV`#

GSD Evaluation Challenges

GsdEvaluationCV#

Examples using mobgap.gait_sequences.evaluation.GsdEvaluationCV#

Examples using `mobgap.gait_sequences.evaluation.GsdEvaluationCV`#