.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_revalidation/full_pipeline/_04_additional_experiments.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_revalidation_full_pipeline__04_additional_experiments.py: .. _pipeline_val_experiments: Additional Full Pipeline Experiments ==================================== The following analysis tries to awnser the question how much specific components of the pipeline contribute to the improvements that we see in the new implementation of the full pipeline. .. note:: If you are interested in how these results are calculated, head over to the :ref:`processing page `. .. GENERATED FROM PYTHON SOURCE LINES 14-17 .. code-block:: Python from typing import Optional .. GENERATED FROM PYTHON SOURCE LINES 18-21 Below the list of pipelines that are compared is shown. Note, that we use the prefix "new" to refer to the reimplemented python algorithms, and the prefix "old" to refer to the original Matlab-based implementation. .. GENERATED FROM PYTHON SOURCE LINES 21-37 .. code-block:: Python algorithms = { "Official_MobiliseD_Pipeline": ("Mobilise-D Pipeline", "MobGap"), "EScience_MobiliseD_Pipeline": ( "Mobilise-D Pipeline", "Original Implementation", ), "Official_MobiliseD_Pipeline__old_gs": ( "Mobilise-D Pipeline", "MobGap (with old GS)", ), "Official_MobiliseD_Pipeline__old_lrc": ( "Mobilise-D Pipeline", "MobGap (with old LRC)", ), } .. GENERATED FROM PYTHON SOURCE LINES 38-45 The code below loads the data and prepares it for the analysis. By default, the data will be downloaded from an online repository (and cached locally). If you want to use a local copy of the data, you can set the `MOBGAP_VALIDATION_DATA_PATH` environment variable. and the `MOBGAP_VALIDATION_USE_LOCA_DATA` to `1`. The file download will print a couple log information, which can usually be ignored. You can also change the `version` parameter to load a different version of the data. .. GENERATED FROM PYTHON SOURCE LINES 45-173 .. code-block:: Python from pathlib import Path import pandas as pd from mobgap.data.validation_results import ValidationResultLoader from mobgap.utils.misc import get_env_var def format_loaded_results( values: dict[tuple[str, str], pd.DataFrame], index_cols: list[str], col_prefix_filter: Optional[str], convert_rel_error: bool = False, ) -> pd.DataFrame: formatted = ( pd.concat(values, names=["algo", "version", *index_cols]) .pipe( lambda df: ( df.filter(like=col_prefix_filter) if col_prefix_filter else df ) ) .reset_index() .assign( algo_with_version=lambda df: ( df["algo"] + " (" + df["version"] + ")" ), _combined="combined", ) ) if col_prefix_filter: formatted.columns = formatted.columns.str.removeprefix( col_prefix_filter ) if convert_rel_error: rel_cols = [c for c in formatted.columns if "rel_error" in c] formatted[rel_cols] = formatted[rel_cols] * 100 return formatted local_data_path = ( Path(get_env_var("MOBGAP_VALIDATION_DATA_PATH")) / "results" if int(get_env_var("MOBGAP_VALIDATION_USE_LOCAL_DATA", 0)) else None ) __RESULT_VERSION = "v1.0.0" loader = ValidationResultLoader( "full_pipeline", result_path=local_data_path, version="main", ) # Loading free-living data free_living_index_cols = [ "cohort", "participant_id", "time_measure", "recording", "recording_name", "recording_name_pretty", ] _free_living_results = { # Matched and aggregate/combined per-recording results for the 2.5 h free-living recordings v: loader.load_single_results(k, "free_living") for k, v in algorithms.items() } _free_living_results_raw = { # Matched per-WB results for the 2.5 h free-living recordings v: loader.load_single_csv_file(k, "free_living", "raw_matched_errors.csv") for k, v in algorithms.items() } free_living_results_combined = format_loaded_results( _free_living_results, free_living_index_cols, "combined__", convert_rel_error=True, ) free_living_results_matched = format_loaded_results( _free_living_results, free_living_index_cols, "matched__", convert_rel_error=True, ) del _free_living_results, _free_living_results_raw # Loading laboratory data laboratory_index_cols = [ "cohort", "participant_id", "time_measure", "test", "trial", "test_name", "test_name_pretty", ] _laboratory_results = { # Matched and aggregate/combined per-recording results for the laboratory recordings v: loader.load_single_results(k, "laboratory") for k, v in algorithms.items() } _laboratory_results_raw = { # Matched per-WB results for the laboratory recordings v: loader.load_single_csv_file(k, "laboratory", "raw_matched_errors.csv") for k, v in algorithms.items() } laboratory_results_combined = format_loaded_results( _laboratory_results, laboratory_index_cols, "combined__", convert_rel_error=True, ) laboratory_results_matched = format_loaded_results( _laboratory_results, laboratory_index_cols, "matched__", convert_rel_error=True, ) laboratory_results_matched_raw = format_loaded_results( values=_laboratory_results_raw, index_cols=laboratory_index_cols, col_prefix_filter=None, convert_rel_error=True, ) del _laboratory_results, _laboratory_results_raw cohort_order = ["HA", "CHF", "COPD", "MS", "PD", "PFF"] .. rst-class:: sphx-glr-script-out .. code-block:: none 0%| | 0.00/7.30k [00:00 pd.DataFrame: return ( df.pipe(apply_transformations, format_transforms_combined) .rename(columns=final_names_combined) .loc[:, list(final_names_combined.values())] ) def format_tables_matched(df: pd.DataFrame) -> pd.DataFrame: return ( df.pipe(apply_transformations, format_transforms_matched) .rename(columns=final_names_matched) .loc[:, list(final_names_matched.values())] ) .. GENERATED FROM PYTHON SOURCE LINES 328-337 Free-living dataset ------------------- Combined/Aggregated Evaluation ****************************** Below a quick compressed version of the results without further explanation. For information on the metrics, see the main full pipeline analysis example. All results across all cohorts ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 337-375 .. code-block:: Python import matplotlib.pyplot as plt import seaborn as sns sns.set_context("talk") metrics = { "abs_rel_error": "Abs. Rel. Error (%)", "error": "Error (m/s)", "rel_error": "Rel. Error (%)", "abs_error": "Abs. Error (m/s)", } def multi_metric_plot(data, metrics, nrows, ncols): fig, axs = plt.subplots( nrows, ncols, sharex=True, figsize=(ncols * 6, nrows * 4 + 2) ) for ax, (metric, metric_label) in zip(axs.flatten(), metrics.items()): overall_df = data[["version", f"walking_speed_mps__{metric}"]].rename( columns={f"walking_speed_mps__{metric}": metric_label} ) sns.boxplot( data=overall_df, x="version", hue="version", y=metric_label, ax=ax ) ax.set_title(metric_label) ax.set_ylabel(metric_label) ax.tick_params(axis="both", which="major") ax.tick_params(axis="both", which="minor") ax.grid(True) plt.tight_layout() plt.show() free_living_results_combined.pipe(multi_metric_plot, metrics, 2, 2) .. image-sg:: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_001.png :alt: Abs. Rel. Error (%), Error (m/s), Rel. Error (%), Abs. Error (m/s) :srcset: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 376-385 .. code-block:: Python free_living_combined_perf_metrics_all = ( free_living_results_combined.groupby(["algo", "version"]) .apply(apply_aggregations, custom_aggs_combined, include_groups=False) .pipe(format_tables_combined) ) free_living_combined_perf_metrics_all.style.pipe( revalidation_table_styles, validation_thresholds, ["algo"] ) .. raw:: html
    # participants WD mean and CI [m/s] INDIP mean and CI [m/s] Bias and LoA [m/s] Abs. Error [m/s] Rel. Error [%] Abs. Rel. Error [%] ICC
algo version                
Mobilise-D Pipeline MobGap 101 0.60 [0.57, 0.62] 0.57 [0.54, 0.60] 0.03 [-0.19, 0.26] 0.10 [0.08, 0.11] 12.08 [6.27, 17.89] 21.20 [16.50, 25.90] 0.62 [0.47, 0.73]
MobGap (with old GS) 101 0.66 [0.64, 0.69] 0.57 [0.54, 0.60] 0.10 [-0.11, 0.31] 0.12 [0.10, 0.13] 22.98 [16.94, 29.02] 26.33 [20.84, 31.82] 0.58 [0.10, 0.79]
MobGap (with old LRC) 101 0.60 [0.57, 0.62] 0.57 [0.54, 0.60] 0.03 [-0.19, 0.26] 0.10 [0.08, 0.11] 12.13 [6.27, 17.98] 21.28 [16.53, 26.02] 0.61 [0.47, 0.73]
Original Implementation 101 0.68 [0.66, 0.71] 0.57 [0.54, 0.60] 0.12 [-0.10, 0.34] 0.13 [0.12, 0.15] 28.18 [21.77, 34.60] 30.11 [24.03, 36.18] 0.51 [-0.03, 0.76]


.. GENERATED FROM PYTHON SOURCE LINES 386-389 Per-cohort analysis ~~~~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 389-402 .. code-block:: Python fig, ax = plt.subplots(figsize=(12, 6)) sns.boxplot( data=free_living_results_combined, x="cohort", y="walking_speed_mps__abs_error", hue="version", order=cohort_order, showmeans=True, ax=ax, ).legend().set_title(None) ax.set_ylabel("Absolute Error [m/s]") ax.set_title("Absolute Error - Combined Analysis") fig.show() .. image-sg:: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_002.png :alt: Absolute Error - Combined Analysis :srcset: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 403-413 .. code-block:: Python free_living_combined_perf_metrics_cohort = ( free_living_results_combined.groupby(["cohort", "algo", "version"]) .apply(apply_aggregations, custom_aggs_combined, include_groups=False) .pipe(format_tables_combined) .loc[cohort_order] ) free_living_combined_perf_metrics_cohort.style.pipe( revalidation_table_styles, validation_thresholds, ["cohort", "algo"] ) .. raw:: html
      # participants WD mean and CI [m/s] INDIP mean and CI [m/s] Bias and LoA [m/s] Abs. Error [m/s] Rel. Error [%] Abs. Rel. Error [%] ICC
cohort algo version                
HA Mobilise-D Pipeline MobGap 20 0.55 [0.52, 0.58] 0.57 [0.53, 0.62] -0.02 [-0.17, 0.12] 0.06 [0.04, 0.08] -2.48 [-7.89, 2.93] 10.36 [7.39, 13.33] 0.65 [0.32, 0.85]
MobGap (with old GS) 20 0.69 [0.65, 0.73] 0.57 [0.53, 0.62] 0.11 [-0.07, 0.29] 0.12 [0.09, 0.16] 21.90 [12.98, 30.82] 23.64 [15.67, 31.60] 0.35 [-0.11, 0.70]
MobGap (with old LRC) 20 0.55 [0.52, 0.58] 0.57 [0.53, 0.62] -0.02 [-0.17, 0.12] 0.06 [0.04, 0.08] -2.74 [-8.09, 2.62] 10.23 [7.21, 13.24] 0.66 [0.33, 0.85]
Original Implementation 20 0.63 [0.59, 0.66] 0.57 [0.53, 0.62] 0.05 [-0.13, 0.23] 0.09 [0.07, 0.11] 11.58 [4.19, 18.97] 16.87 [11.95, 21.79] 0.45 [0.04, 0.73]
CHF Mobilise-D Pipeline MobGap 10 0.56 [0.50, 0.63] 0.64 [0.54, 0.74] -0.08 [-0.30, 0.14] 0.11 [0.06, 0.16] -9.20 [-19.96, 1.56] 16.46 [10.44, 22.49] 0.61 [0.05, 0.88]
MobGap (with old GS) 10 0.70 [0.59, 0.81] 0.64 [0.54, 0.74] 0.06 [-0.19, 0.30] 0.12 [0.08, 0.16] 11.64 [-1.97, 25.24] 20.47 [12.48, 28.47] 0.71 [0.24, 0.92]
MobGap (with old LRC) 10 0.57 [0.49, 0.64] 0.64 [0.54, 0.74] -0.07 [-0.29, 0.14] 0.11 [0.06, 0.15] -9.00 [-19.58, 1.57] 16.02 [9.94, 22.10] 0.64 [0.09, 0.90]
Original Implementation 10 0.82 [0.73, 0.91] 0.64 [0.54, 0.74] 0.18 [-0.01, 0.36] 0.18 [0.12, 0.24] 32.04 [18.50, 45.59] 32.15 [18.71, 45.59] 0.52 [-0.10, 0.88]
COPD Mobilise-D Pipeline MobGap 17 0.59 [0.55, 0.63] 0.57 [0.51, 0.63] 0.02 [-0.16, 0.20] 0.08 [0.05, 0.10] 7.68 [-4.26, 19.62] 15.92 [6.13, 25.71] 0.63 [0.23, 0.85]
MobGap (with old GS) 17 0.69 [0.66, 0.72] 0.57 [0.51, 0.63] 0.12 [-0.11, 0.34] 0.12 [0.07, 0.17] 28.41 [5.11, 51.72] 28.86 [5.69, 52.03] 0.21 [-0.12, 0.56]
MobGap (with old LRC) 17 0.60 [0.56, 0.64] 0.57 [0.51, 0.63] 0.02 [-0.17, 0.21] 0.08 [0.05, 0.11] 8.63 [-4.48, 21.74] 16.74 [5.67, 27.81] 0.58 [0.17, 0.82]
Original Implementation 17 0.66 [0.62, 0.69] 0.57 [0.51, 0.63] 0.08 [-0.09, 0.25] 0.09 [0.06, 0.13] 19.71 [4.98, 34.44] 21.12 [6.87, 35.37] 0.49 [-0.04, 0.79]
MS Mobilise-D Pipeline MobGap 18 0.66 [0.61, 0.71] 0.61 [0.55, 0.67] 0.06 [-0.19, 0.30] 0.11 [0.07, 0.15] 13.04 [1.81, 24.28] 20.83 [12.66, 28.99] 0.42 [-0.00, 0.72]
MobGap (with old GS) 18 0.68 [0.63, 0.72] 0.61 [0.55, 0.67] 0.07 [-0.16, 0.30] 0.11 [0.07, 0.15] 15.07 [4.50, 25.64] 20.88 [12.86, 28.91] 0.44 [0.01, 0.74]
MobGap (with old LRC) 18 0.66 [0.61, 0.71] 0.61 [0.55, 0.67] 0.06 [-0.19, 0.30] 0.11 [0.07, 0.15] 12.77 [1.36, 24.17] 20.80 [12.48, 29.11] 0.41 [-0.01, 0.72]
Original Implementation 18 0.74 [0.69, 0.79] 0.61 [0.55, 0.67] 0.13 [-0.11, 0.38] 0.15 [0.10, 0.19] 25.77 [13.34, 38.20] 27.98 [16.68, 39.28] 0.27 [-0.11, 0.62]
PD Mobilise-D Pipeline MobGap 19 0.66 [0.60, 0.72] 0.58 [0.49, 0.67] 0.08 [-0.17, 0.33] 0.13 [0.09, 0.16] 23.06 [7.01, 39.10] 29.32 [15.63, 43.01] 0.63 [0.21, 0.84]
MobGap (with old GS) 19 0.68 [0.62, 0.74] 0.58 [0.49, 0.67] 0.10 [-0.15, 0.34] 0.14 [0.10, 0.17] 25.62 [11.07, 40.17] 30.58 [18.25, 42.92] 0.61 [0.12, 0.84]
MobGap (with old LRC) 19 0.66 [0.60, 0.72] 0.58 [0.49, 0.67] 0.08 [-0.17, 0.33] 0.13 [0.09, 0.16] 22.70 [7.08, 38.31] 29.17 [16.05, 42.30] 0.63 [0.21, 0.84]
Original Implementation 19 0.73 [0.68, 0.79] 0.58 [0.49, 0.67] 0.15 [-0.09, 0.40] 0.16 [0.11, 0.21] 37.31 [18.56, 56.06] 38.34 [20.05, 56.64] 0.49 [-0.09, 0.80]
PFF Mobilise-D Pipeline MobGap 17 0.53 [0.48, 0.58] 0.44 [0.37, 0.51] 0.11 [-0.05, 0.26] 0.11 [0.07, 0.14] 34.11 [16.74, 51.48] 34.11 [16.74, 51.48] 0.62 [-0.08, 0.88]
MobGap (with old GS) 17 0.54 [0.48, 0.60] 0.44 [0.37, 0.51] 0.10 [-0.03, 0.24] 0.10 [0.07, 0.14] 31.34 [17.95, 44.73] 31.59 [18.34, 44.83] 0.68 [-0.08, 0.91]
MobGap (with old LRC) 17 0.53 [0.48, 0.58] 0.44 [0.37, 0.51] 0.11 [-0.05, 0.26] 0.11 [0.07, 0.14] 34.35 [16.83, 51.87] 34.35 [16.83, 51.87] 0.62 [-0.08, 0.88]
Original Implementation 17 0.59 [0.54, 0.64] 0.44 [0.37, 0.51] 0.16 [0.00, 0.32] 0.16 [0.12, 0.20] 47.65 [28.94, 66.37] 47.65 [28.94, 66.37] 0.47 [-0.09, 0.82]


.. GENERATED FROM PYTHON SOURCE LINES 414-416 Matched/True Positive Evaluation ******************************** .. GENERATED FROM PYTHON SOURCE LINES 416-418 .. code-block:: Python free_living_results_matched.pipe(multi_metric_plot, metrics, 2, 2) .. image-sg:: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_003.png :alt: Abs. Rel. Error (%), Error (m/s), Rel. Error (%), Abs. Error (m/s) :srcset: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 419-428 .. code-block:: Python free_living_matched_perf_metrics_all = ( free_living_results_matched.groupby(["algo", "version"]) .apply(apply_aggregations, custom_aggs_matched, include_groups=False) .pipe(format_tables_matched) ) free_living_matched_perf_metrics_all.style.pipe( revalidation_table_styles, validation_thresholds, ["algo"] ) .. raw:: html
    # participants WD mean and CI [m/s] INDIP mean and CI [m/s] Bias and LoA [m/s] Abs. Error [m/s] Rel. Error [%] Abs. Rel. Error [%] ICC # Matched WBs
algo version                  
Mobilise-D Pipeline MobGap 101 0.71 [0.68, 0.73] 0.67 [0.64, 0.70] 0.04 [-0.13, 0.21] 0.10 [0.09, 0.11] 11.84 [8.31, 15.37] 19.63 [16.94, 22.32] 0.81 [0.67, 0.88] 1984
MobGap (with old GS) 101 0.74 [0.71, 0.77] 0.70 [0.67, 0.72] 0.04 [-0.12, 0.21] 0.11 [0.10, 0.11] 12.64 [8.89, 16.39] 20.21 [17.06, 23.37] 0.78 [0.60, 0.87] 2023
MobGap (with old LRC) 101 0.71 [0.68, 0.74] 0.67 [0.64, 0.70] 0.04 [-0.13, 0.21] 0.10 [0.09, 0.11] 11.84 [8.28, 15.41] 19.67 [16.98, 22.37] 0.81 [0.68, 0.88] 1991
Original Implementation 101 0.77 [0.75, 0.80] 0.71 [0.68, 0.74] 0.07 [-0.13, 0.26] 0.12 [0.11, 0.13] 17.11 [12.22, 22.01] 22.89 [18.62, 27.17] 0.69 [0.37, 0.83] 1697


.. GENERATED FROM PYTHON SOURCE LINES 429-444 .. code-block:: Python fig, ax = plt.subplots(figsize=(12, 6)) sns.barplot( data=free_living_results_matched.groupby(["version", "cohort"])[ "n_matched_wbs" ] .sum() .reset_index(), hue="version", y="n_matched_wbs", x="cohort", order=cohort_order, hue_order=["new", "old", "new with old GS", "new with old LRC"], ax=ax, ) fig.show() .. image-sg:: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_004.png :alt: 04 additional experiments :srcset: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 445-457 .. code-block:: Python fig, ax = plt.subplots(figsize=(12, 6)) sns.boxplot( data=free_living_results_matched, x="cohort", y="walking_speed_mps__abs_error", hue="algo_with_version", order=cohort_order, ax=ax, ).legend().set_title(None) ax.set_ylabel("Absolute Error [m/s]") ax.set_title("Absolute Error - Matched Analysis") fig.show() .. image-sg:: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_005.png :alt: Absolute Error - Matched Analysis :srcset: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 458-468 .. code-block:: Python free_living_matched_perf_metrics_cohort = ( free_living_results_matched.groupby(["cohort", "algo", "version"]) .apply(apply_aggregations, custom_aggs_matched, include_groups=False) .pipe(format_tables_matched) .loc[cohort_order] ) free_living_matched_perf_metrics_cohort.style.pipe( revalidation_table_styles, validation_thresholds, ["cohort", "algo"] ) .. raw:: html
      # participants WD mean and CI [m/s] INDIP mean and CI [m/s] Bias and LoA [m/s] Abs. Error [m/s] Rel. Error [%] Abs. Rel. Error [%] ICC # Matched WBs
cohort algo version                  
HA Mobilise-D Pipeline MobGap 20 0.72 [0.65, 0.78] 0.69 [0.62, 0.76] 0.02 [-0.08, 0.12] 0.08 [0.06, 0.09] 7.22 [3.70, 10.74] 13.72 [11.51, 15.93] 0.93 [0.82, 0.97] 524
MobGap (with old GS) 20 0.78 [0.73, 0.83] 0.74 [0.68, 0.80] 0.04 [-0.07, 0.15] 0.09 [0.08, 0.10] 9.84 [5.79, 13.89] 15.60 [13.04, 18.16] 0.86 [0.55, 0.95] 486
MobGap (with old LRC) 20 0.72 [0.65, 0.78] 0.70 [0.62, 0.77] 0.02 [-0.08, 0.12] 0.08 [0.06, 0.09] 7.28 [3.60, 10.97] 13.90 [11.54, 16.26] 0.93 [0.82, 0.97] 529
Original Implementation 20 0.79 [0.74, 0.84] 0.75 [0.69, 0.81] 0.04 [-0.09, 0.17] 0.09 [0.08, 0.11] 9.45 [4.82, 14.08] 15.36 [12.39, 18.33] 0.85 [0.59, 0.94] 410
CHF Mobilise-D Pipeline MobGap 10 0.75 [0.67, 0.84] 0.78 [0.67, 0.89] -0.02 [-0.16, 0.12] 0.10 [0.07, 0.13] 2.72 [-4.40, 9.85] 15.32 [10.60, 20.03] 0.90 [0.67, 0.97] 220
MobGap (with old GS) 10 0.77 [0.67, 0.86] 0.77 [0.65, 0.89] -0.01 [-0.16, 0.15] 0.09 [0.06, 0.13] 5.52 [-5.31, 16.36] 16.14 [7.30, 24.97] 0.91 [0.67, 0.98] 255
MobGap (with old LRC) 10 0.76 [0.67, 0.85] 0.79 [0.68, 0.90] -0.02 [-0.16, 0.12] 0.10 [0.07, 0.13] 2.62 [-4.88, 10.12] 15.50 [10.55, 20.46] 0.90 [0.66, 0.97] 216
Original Implementation 10 0.83 [0.72, 0.93] 0.83 [0.70, 0.96] -0.00 [-0.19, 0.18] 0.10 [0.06, 0.14] 5.39 [-6.37, 17.15] 15.19 [5.79, 24.60] 0.89 [0.60, 0.98] 176
COPD Mobilise-D Pipeline MobGap 17 0.69 [0.66, 0.73] 0.62 [0.57, 0.66] 0.07 [-0.05, 0.20] 0.10 [0.09, 0.12] 16.51 [10.46, 22.56] 20.74 [16.55, 24.92] 0.53 [-0.08, 0.83] 410
MobGap (with old GS) 17 0.73 [0.69, 0.77] 0.65 [0.60, 0.69] 0.08 [-0.04, 0.21] 0.11 [0.10, 0.13] 18.62 [13.35, 23.88] 22.49 [19.13, 25.84] 0.55 [-0.10, 0.85] 452
MobGap (with old LRC) 17 0.69 [0.65, 0.73] 0.62 [0.57, 0.66] 0.07 [-0.06, 0.20] 0.10 [0.08, 0.12] 16.33 [10.15, 22.52] 20.60 [16.40, 24.79] 0.55 [-0.06, 0.84] 420
Original Implementation 17 0.75 [0.71, 0.79] 0.65 [0.60, 0.69] 0.10 [-0.03, 0.23] 0.13 [0.10, 0.15] 21.58 [15.75, 27.41] 24.30 [19.58, 29.02] 0.40 [-0.11, 0.77] 323
MS Mobilise-D Pipeline MobGap 18 0.77 [0.70, 0.83] 0.69 [0.63, 0.75] 0.08 [-0.10, 0.25] 0.12 [0.09, 0.15] 17.47 [9.38, 25.56] 22.69 [15.91, 29.48] 0.67 [0.12, 0.88] 327
MobGap (with old GS) 18 0.79 [0.71, 0.86] 0.71 [0.63, 0.78] 0.08 [-0.10, 0.26] 0.12 [0.10, 0.15] 20.23 [8.91, 31.55] 25.80 [13.85, 37.76] 0.75 [0.21, 0.92] 370
MobGap (with old LRC) 18 0.77 [0.70, 0.83] 0.69 [0.63, 0.76] 0.07 [-0.10, 0.25] 0.12 [0.09, 0.14] 16.85 [8.79, 24.90] 21.94 [15.14, 28.74] 0.71 [0.17, 0.90] 322
Original Implementation 18 0.82 [0.76, 0.88] 0.71 [0.64, 0.78] 0.11 [-0.10, 0.32] 0.15 [0.11, 0.18] 27.78 [11.34, 44.22] 31.46 [16.01, 46.90] 0.59 [-0.02, 0.85] 355
PD Mobilise-D Pipeline MobGap 19 0.73 [0.67, 0.79] 0.71 [0.64, 0.78] 0.02 [-0.20, 0.24] 0.11 [0.09, 0.14] 8.92 [-2.59, 20.44] 20.70 [12.08, 29.33] 0.72 [0.41, 0.88] 267
MobGap (with old GS) 19 0.74 [0.68, 0.80] 0.72 [0.65, 0.79] 0.02 [-0.20, 0.24] 0.11 [0.09, 0.13] 9.48 [-2.21, 21.16] 20.44 [11.50, 29.38] 0.69 [0.36, 0.87] 268
MobGap (with old LRC) 19 0.73 [0.67, 0.79] 0.71 [0.64, 0.78] 0.02 [-0.20, 0.24] 0.11 [0.09, 0.14] 9.75 [-1.83, 21.33] 21.52 [13.04, 30.01] 0.71 [0.40, 0.88] 269
Original Implementation 19 0.79 [0.74, 0.85] 0.73 [0.66, 0.81] 0.06 [-0.20, 0.32] 0.13 [0.09, 0.16] 17.07 [1.50, 32.64] 24.82 [11.17, 38.48] 0.53 [0.13, 0.78] 256
PFF Mobilise-D Pipeline MobGap 17 0.58 [0.52, 0.63] 0.53 [0.46, 0.60] 0.04 [-0.12, 0.20] 0.10 [0.08, 0.12] 15.71 [4.68, 26.74] 24.07 [15.37, 32.77] 0.79 [0.46, 0.92] 236
MobGap (with old GS) 17 0.61 [0.56, 0.66] 0.59 [0.53, 0.65] 0.02 [-0.14, 0.18] 0.09 [0.07, 0.11] 8.48 [-0.02, 16.98] 19.05 [13.08, 25.03] 0.75 [0.37, 0.91] 192
MobGap (with old LRC) 17 0.58 [0.52, 0.63] 0.54 [0.47, 0.61] 0.04 [-0.12, 0.20] 0.10 [0.08, 0.12] 15.63 [4.36, 26.90] 24.04 [15.12, 32.97] 0.78 [0.44, 0.92] 235
Original Implementation 17 0.66 [0.61, 0.70] 0.59 [0.54, 0.65] 0.06 [-0.11, 0.23] 0.11 [0.09, 0.14] 16.48 [7.25, 25.71] 23.29 [16.30, 30.29] 0.61 [0.09, 0.86] 177


.. GENERATED FROM PYTHON SOURCE LINES 469-478 Laboratory dataset ------------------ Combined/Aggregated Evaluation ****************************** Below a quick compressed version of the results without further explanation. For information on the metrics, see the main full pipeline analysis example. All results across all cohorts ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 478-516 .. code-block:: Python import matplotlib.pyplot as plt import seaborn as sns sns.set_context("talk") metrics = { "abs_rel_error": "Abs. Rel. Error (%)", "error": "Error (m/s)", "rel_error": "Rel. Error (%)", "abs_error": "Abs. Error (m/s)", } def multi_metric_plot(data, metrics, nrows, ncols): fig, axs = plt.subplots( nrows, ncols, sharex=True, figsize=(ncols * 6, nrows * 4 + 2) ) for ax, (metric, metric_label) in zip(axs.flatten(), metrics.items()): overall_df = data[["version", f"walking_speed_mps__{metric}"]].rename( columns={f"walking_speed_mps__{metric}": metric_label} ) sns.boxplot( data=overall_df, x="version", hue="version", y=metric_label, ax=ax ) ax.set_title(metric_label) ax.set_ylabel(metric_label) ax.tick_params(axis="both", which="major") ax.tick_params(axis="both", which="minor") ax.grid(True) plt.tight_layout() plt.show() laboratory_results_combined.pipe(multi_metric_plot, metrics, 2, 2) .. image-sg:: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_006.png :alt: Abs. Rel. Error (%), Error (m/s), Rel. Error (%), Abs. Error (m/s) :srcset: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_006.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 517-526 .. code-block:: Python laboratory_combined_perf_metrics_all = ( laboratory_results_combined.groupby(["algo", "version"]) .apply(apply_aggregations, custom_aggs_combined, include_groups=False) .pipe(format_tables_combined) ) laboratory_combined_perf_metrics_all.style.pipe( revalidation_table_styles, validation_thresholds, ["algo"] ) .. raw:: html
    # participants WD mean and CI [m/s] INDIP mean and CI [m/s] Bias and LoA [m/s] Abs. Error [m/s] Rel. Error [%] Abs. Rel. Error [%] ICC
algo version                
Mobilise-D Pipeline MobGap 1168 0.82 [0.80, 0.83] 0.83 [0.82, 0.85] -0.04 [-0.34, 0.27] 0.11 [0.10, 0.12] -0.40 [-1.64, 0.85] 14.70 [13.78, 15.62] 0.83 [0.80, 0.85]
MobGap (with old GS) 1168 0.86 [0.84, 0.87] 0.83 [0.82, 0.85] -0.01 [-0.30, 0.27] 0.10 [0.10, 0.11] 2.53 [1.26, 3.80] 14.29 [13.30, 15.27] 0.86 [0.84, 0.87]
MobGap (with old LRC) 1168 0.82 [0.80, 0.83] 0.83 [0.82, 0.85] -0.04 [-0.34, 0.27] 0.11 [0.10, 0.12] -0.50 [-1.75, 0.74] 14.70 [13.78, 15.62] 0.83 [0.79, 0.85]
Original Implementation 1168 0.84 [0.82, 0.85] 0.83 [0.82, 0.85] -0.00 [-0.31, 0.30] 0.11 [0.11, 0.12] 5.28 [3.78, 6.78] 16.71 [15.52, 17.91] 0.82 [0.80, 0.84]


.. GENERATED FROM PYTHON SOURCE LINES 527-530 Per-cohort analysis ~~~~~~~~~~~~~~~~~~~ .. GENERATED FROM PYTHON SOURCE LINES 530-543 .. code-block:: Python fig, ax = plt.subplots(figsize=(12, 6)) sns.boxplot( data=laboratory_results_combined, x="cohort", y="walking_speed_mps__abs_error", hue="version", order=cohort_order, showmeans=True, ax=ax, ).legend().set_title(None) ax.set_ylabel("Absolute Error [m/s]") ax.set_title("Absolute Error - Combined Analysis") fig.show() .. image-sg:: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_007.png :alt: Absolute Error - Combined Analysis :srcset: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_007.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 544-554 .. code-block:: Python laboratory_combined_perf_metrics_cohort = ( laboratory_results_combined.groupby(["cohort", "algo", "version"]) .apply(apply_aggregations, custom_aggs_combined, include_groups=False) .pipe(format_tables_combined) .loc[cohort_order] ) laboratory_combined_perf_metrics_cohort.style.pipe( revalidation_table_styles, validation_thresholds, ["cohort", "algo"] ) .. raw:: html
      # participants WD mean and CI [m/s] INDIP mean and CI [m/s] Bias and LoA [m/s] Abs. Error [m/s] Rel. Error [%] Abs. Rel. Error [%] ICC
cohort algo version                
HA Mobilise-D Pipeline MobGap 227 0.85 [0.82, 0.88] 0.92 [0.89, 0.96] -0.08 [-0.37, 0.20] 0.12 [0.11, 0.14] -7.70 [-9.76, -5.65] 13.56 [12.11, 15.01] 0.76 [0.56, 0.86]
MobGap (with old GS) 227 0.94 [0.90, 0.97] 0.92 [0.89, 0.96] -0.03 [-0.31, 0.25] 0.10 [0.09, 0.12] -1.64 [-3.81, 0.52] 11.96 [10.44, 13.47] 0.83 [0.78, 0.87]
MobGap (with old LRC) 227 0.85 [0.82, 0.88] 0.92 [0.89, 0.96] -0.08 [-0.37, 0.20] 0.12 [0.11, 0.14] -7.86 [-9.91, -5.82] 13.50 [12.04, 14.96] 0.76 [0.55, 0.86]
Original Implementation 227 0.84 [0.81, 0.86] 0.92 [0.89, 0.96] -0.07 [-0.36, 0.23] 0.12 [0.10, 0.13] -5.20 [-7.23, -3.18] 12.88 [11.56, 14.19] 0.74 [0.60, 0.82]
CHF Mobilise-D Pipeline MobGap 106 0.83 [0.78, 0.88] 0.90 [0.84, 0.96] -0.11 [-0.47, 0.24] 0.14 [0.11, 0.17] -9.42 [-12.28, -6.56] 13.52 [11.33, 15.70] 0.74 [0.47, 0.86]
MobGap (with old GS) 106 0.88 [0.82, 0.95] 0.90 [0.84, 0.96] -0.07 [-0.37, 0.22] 0.11 [0.09, 0.13] -5.06 [-8.02, -2.09] 11.98 [9.87, 14.09] 0.85 [0.72, 0.91]
MobGap (with old LRC) 106 0.83 [0.78, 0.88] 0.90 [0.84, 0.96] -0.12 [-0.47, 0.24] 0.14 [0.11, 0.17] -9.68 [-12.52, -6.84] 13.61 [11.44, 15.78] 0.73 [0.45, 0.86]
Original Implementation 106 0.93 [0.87, 0.98] 0.90 [0.84, 0.96] -0.06 [-0.38, 0.26] 0.13 [0.10, 0.15] -2.40 [-5.85, 1.04] 13.77 [11.51, 16.04] 0.83 [0.72, 0.89]
COPD Mobilise-D Pipeline MobGap 214 0.84 [0.82, 0.87] 0.90 [0.87, 0.94] -0.07 [-0.42, 0.29] 0.11 [0.09, 0.13] -5.70 [-8.05, -3.34] 12.70 [10.90, 14.50] 0.63 [0.49, 0.73]
MobGap (with old GS) 214 0.92 [0.89, 0.95] 0.90 [0.87, 0.94] -0.02 [-0.37, 0.32] 0.10 [0.08, 0.12] 0.02 [-2.51, 2.56] 11.56 [9.55, 13.56] 0.70 [0.62, 0.77]
MobGap (with old LRC) 214 0.84 [0.82, 0.87] 0.90 [0.87, 0.94] -0.07 [-0.42, 0.29] 0.11 [0.09, 0.13] -5.62 [-7.97, -3.28] 12.59 [10.80, 14.37] 0.63 [0.49, 0.73]
Original Implementation 214 0.86 [0.84, 0.89] 0.90 [0.87, 0.94] -0.02 [-0.32, 0.28] 0.09 [0.08, 0.11] 0.21 [-1.95, 2.36] 10.92 [9.33, 12.50] 0.74 [0.66, 0.80]
MS Mobilise-D Pipeline MobGap 228 0.86 [0.82, 0.89] 0.84 [0.80, 0.88] 0.00 [-0.26, 0.26] 0.10 [0.08, 0.11] 2.32 [0.07, 4.57] 12.46 [10.87, 14.05] 0.88 [0.85, 0.91]
MobGap (with old GS) 228 0.87 [0.83, 0.90] 0.84 [0.80, 0.88] 0.01 [-0.26, 0.27] 0.10 [0.09, 0.11] 3.32 [0.92, 5.72] 13.07 [11.32, 14.82] 0.88 [0.85, 0.91]
MobGap (with old LRC) 228 0.86 [0.82, 0.89] 0.84 [0.80, 0.88] 0.00 [-0.26, 0.26] 0.10 [0.08, 0.11] 2.11 [-0.14, 4.36] 12.52 [10.95, 14.10] 0.88 [0.85, 0.91]
Original Implementation 228 0.86 [0.83, 0.89] 0.84 [0.80, 0.88] 0.01 [-0.27, 0.30] 0.11 [0.10, 0.12] 4.99 [2.31, 7.68] 14.71 [12.71, 16.70] 0.84 [0.80, 0.88]
PD Mobilise-D Pipeline MobGap 224 0.81 [0.78, 0.85] 0.79 [0.75, 0.83] -0.01 [-0.26, 0.24] 0.10 [0.09, 0.11] 1.56 [-0.83, 3.96] 13.78 [12.20, 15.37] 0.88 [0.85, 0.91]
MobGap (with old GS) 224 0.82 [0.79, 0.85] 0.79 [0.75, 0.83] -0.01 [-0.26, 0.24] 0.10 [0.09, 0.11] 2.22 [-0.22, 4.66] 13.61 [11.92, 15.30] 0.88 [0.85, 0.91]
MobGap (with old LRC) 224 0.81 [0.78, 0.85] 0.79 [0.75, 0.83] -0.01 [-0.26, 0.24] 0.10 [0.09, 0.11] 1.52 [-0.89, 3.92] 13.84 [12.25, 15.43] 0.88 [0.84, 0.91]
Original Implementation 224 0.84 [0.81, 0.87] 0.79 [0.75, 0.83] 0.02 [-0.26, 0.30] 0.11 [0.10, 0.12] 7.01 [4.07, 9.95] 16.25 [14.03, 18.47] 0.83 [0.79, 0.87]
PFF Mobilise-D Pipeline MobGap 169 0.68 [0.65, 0.72] 0.67 [0.62, 0.72] 0.01 [-0.26, 0.29] 0.11 [0.10, 0.12] 12.64 [7.60, 17.67] 22.74 [18.60, 26.88] 0.89 [0.85, 0.91]
MobGap (with old GS) 169 0.69 [0.65, 0.72] 0.67 [0.62, 0.72] 0.01 [-0.27, 0.30] 0.11 [0.10, 0.13] 13.71 [8.46, 18.96] 23.57 [19.20, 27.95] 0.88 [0.84, 0.91]
MobGap (with old LRC) 169 0.68 [0.65, 0.72] 0.67 [0.62, 0.72] 0.01 [-0.27, 0.29] 0.11 [0.10, 0.12] 12.55 [7.51, 17.59] 22.69 [18.54, 26.84] 0.89 [0.85, 0.91]
Original Implementation 169 0.72 [0.69, 0.76] 0.67 [0.62, 0.72] 0.05 [-0.27, 0.37] 0.14 [0.12, 0.15] 22.36 [15.97, 28.75] 30.38 [24.80, 35.96] 0.83 [0.76, 0.88]


.. GENERATED FROM PYTHON SOURCE LINES 555-557 Matched/True Positive Evaluation ******************************** .. GENERATED FROM PYTHON SOURCE LINES 557-559 .. code-block:: Python laboratory_results_matched.pipe(multi_metric_plot, metrics, 2, 2) .. image-sg:: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_008.png :alt: Abs. Rel. Error (%), Error (m/s), Rel. Error (%), Abs. Error (m/s) :srcset: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_008.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 560-569 .. code-block:: Python laboratory_matched_perf_metrics_all = ( laboratory_results_matched.groupby(["algo", "version"]) .apply(apply_aggregations, custom_aggs_matched, include_groups=False) .pipe(format_tables_matched) ) laboratory_matched_perf_metrics_all.style.pipe( revalidation_table_styles, validation_thresholds, ["algo"] ) .. raw:: html
    # participants WD mean and CI [m/s] INDIP mean and CI [m/s] Bias and LoA [m/s] Abs. Error [m/s] Rel. Error [%] Abs. Rel. Error [%] ICC # Matched WBs
algo version                  
Mobilise-D Pipeline MobGap 1168 0.79 [0.78, 0.80] 0.79 [0.78, 0.81] -0.00 [-0.21, 0.21] 0.08 [0.08, 0.09] 2.34 [1.36, 3.33] 11.86 [11.12, 12.59] 0.90 [0.88, 0.91] 674
MobGap (with old GS) 1168 0.82 [0.81, 0.83] 0.82 [0.81, 0.84] -0.00 [-0.22, 0.21] 0.08 [0.08, 0.09] 1.62 [0.67, 2.57] 11.66 [10.97, 12.35] 0.89 [0.87, 0.90] 825
MobGap (with old LRC) 1168 0.79 [0.78, 0.80] 0.79 [0.78, 0.81] -0.00 [-0.21, 0.21] 0.08 [0.08, 0.09] 2.24 [1.24, 3.24] 12.04 [11.29, 12.78] 0.89 [0.88, 0.91] 679
Original Implementation 1168 0.84 [0.83, 0.86] 0.84 [0.82, 0.85] 0.00 [-0.24, 0.25] 0.10 [0.09, 0.10] 4.05 [2.92, 5.19] 13.31 [12.44, 14.18] 0.86 [0.83, 0.87] 714


.. GENERATED FROM PYTHON SOURCE LINES 570-585 .. code-block:: Python fig, ax = plt.subplots(figsize=(12, 6)) sns.barplot( data=laboratory_results_matched.groupby(["version", "cohort"])[ "n_matched_wbs" ] .sum() .reset_index(), hue="version", y="n_matched_wbs", x="cohort", order=cohort_order, hue_order=["new", "old", "new with old GS", "new with old LRC"], ax=ax, ) fig.show() .. image-sg:: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_009.png :alt: 04 additional experiments :srcset: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_009.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 586-598 .. code-block:: Python fig, ax = plt.subplots(figsize=(12, 6)) sns.boxplot( data=laboratory_results_matched, x="cohort", y="walking_speed_mps__abs_error", hue="algo_with_version", order=cohort_order, ax=ax, ).legend().set_title(None) ax.set_ylabel("Absolute Error [m/s]") ax.set_title("Absolute Error - Matched Analysis") fig.show() .. image-sg:: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_010.png :alt: Absolute Error - Matched Analysis :srcset: /auto_revalidation/full_pipeline/images/sphx_glr__04_additional_experiments_010.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 599-608 .. code-block:: Python laboratory_matched_perf_metrics_cohort = ( laboratory_results_matched.groupby(["cohort", "algo", "version"]) .apply(apply_aggregations, custom_aggs_matched, include_groups=False) .pipe(format_tables_matched) .loc[cohort_order] ) laboratory_matched_perf_metrics_cohort.style.pipe( revalidation_table_styles, validation_thresholds, ["cohort", "algo"] ) .. raw:: html
      # participants WD mean and CI [m/s] INDIP mean and CI [m/s] Bias and LoA [m/s] Abs. Error [m/s] Rel. Error [%] Abs. Rel. Error [%] ICC # Matched WBs
cohort algo version                  
HA Mobilise-D Pipeline MobGap 227 0.84 [0.81, 0.86] 0.83 [0.80, 0.87] 0.00 [-0.19, 0.20] 0.08 [0.07, 0.09] 3.45 [1.02, 5.88] 11.44 [9.47, 13.42] 0.90 [0.84, 0.93] 80
MobGap (with old GS) 227 0.87 [0.84, 0.90] 0.89 [0.86, 0.92] -0.02 [-0.22, 0.18] 0.09 [0.08, 0.09] -1.62 [-3.32, 0.08] 10.50 [9.39, 11.62] 0.89 [0.85, 0.92] 166
MobGap (with old LRC) 227 0.83 [0.80, 0.85] 0.83 [0.80, 0.86] -0.00 [-0.20, 0.20] 0.08 [0.07, 0.09] 2.87 [0.39, 5.35] 11.77 [9.78, 13.76] 0.89 [0.84, 0.93] 81
Original Implementation 227 0.86 [0.84, 0.89] 0.87 [0.85, 0.90] -0.01 [-0.20, 0.18] 0.08 [0.07, 0.09] -0.46 [-2.08, 1.17] 9.61 [8.57, 10.65] 0.87 [0.81, 0.91] 102
CHF Mobilise-D Pipeline MobGap 106 0.75 [0.71, 0.79] 0.78 [0.73, 0.83] -0.04 [-0.22, 0.14] 0.07 [0.06, 0.09] -3.98 [-6.18, -1.77] 9.28 [7.74, 10.82] 0.91 [0.81, 0.95] 53
MobGap (with old GS) 106 0.81 [0.76, 0.86] 0.85 [0.79, 0.91] -0.04 [-0.27, 0.19] 0.09 [0.08, 0.11] -2.09 [-5.55, 1.37] 12.08 [9.48, 14.69] 0.91 [0.84, 0.94] 72
MobGap (with old LRC) 106 0.75 [0.71, 0.79] 0.79 [0.74, 0.83] -0.04 [-0.22, 0.14] 0.08 [0.06, 0.09] -4.09 [-6.28, -1.91] 9.32 [7.81, 10.84] 0.91 [0.81, 0.95] 53
Original Implementation 106 0.89 [0.84, 0.93] 0.95 [0.89, 1.02] -0.07 [-0.37, 0.24] 0.12 [0.10, 0.14] -3.44 [-7.01, 0.13] 13.16 [10.54, 15.77] 0.83 [0.69, 0.90] 60
COPD Mobilise-D Pipeline MobGap 214 0.87 [0.84, 0.89] 0.88 [0.85, 0.90] -0.01 [-0.15, 0.14] 0.06 [0.05, 0.07] 0.27 [-1.10, 1.63] 7.47 [6.54, 8.40] 0.92 [0.89, 0.95] 93
MobGap (with old GS) 214 0.88 [0.86, 0.91] 0.88 [0.86, 0.91] -0.00 [-0.20, 0.20] 0.06 [0.05, 0.08] 0.93 [-0.57, 2.43] 8.04 [6.96, 9.12] 0.85 [0.80, 0.89] 159
MobGap (with old LRC) 214 0.86 [0.84, 0.89] 0.87 [0.85, 0.90] -0.01 [-0.15, 0.14] 0.06 [0.05, 0.07] 0.19 [-1.22, 1.60] 7.58 [6.61, 8.56] 0.92 [0.89, 0.95] 95
Original Implementation 214 0.90 [0.88, 0.93] 0.89 [0.86, 0.92] 0.01 [-0.14, 0.17] 0.06 [0.06, 0.07] 2.38 [0.82, 3.94] 7.76 [6.56, 8.96] 0.91 [0.87, 0.94] 106
MS Mobilise-D Pipeline MobGap 228 0.83 [0.80, 0.86] 0.82 [0.79, 0.85] 0.01 [-0.22, 0.24] 0.09 [0.08, 0.10] 3.37 [0.94, 5.80] 12.73 [10.87, 14.60] 0.88 [0.84, 0.91] 176
MobGap (with old GS) 228 0.83 [0.80, 0.87] 0.82 [0.79, 0.86] 0.01 [-0.22, 0.25] 0.09 [0.08, 0.10] 3.04 [0.84, 5.25] 12.40 [10.83, 13.96] 0.88 [0.84, 0.91] 174
MobGap (with old LRC) 228 0.83 [0.80, 0.86] 0.82 [0.79, 0.85] 0.01 [-0.22, 0.24] 0.09 [0.08, 0.10] 3.22 [0.83, 5.62] 12.79 [10.98, 14.60] 0.87 [0.83, 0.91] 177
Original Implementation 228 0.86 [0.83, 0.89] 0.85 [0.81, 0.88] 0.01 [-0.25, 0.28] 0.10 [0.09, 0.12] 4.54 [1.98, 7.10] 14.03 [12.14, 15.93] 0.84 [0.79, 0.88] 182
PD Mobilise-D Pipeline MobGap 224 0.77 [0.75, 0.80] 0.79 [0.76, 0.82] -0.01 [-0.23, 0.20] 0.09 [0.08, 0.10] 0.66 [-1.32, 2.65] 12.16 [10.97, 13.36] 0.87 [0.83, 0.91] 150
MobGap (with old GS) 224 0.78 [0.76, 0.81] 0.79 [0.76, 0.82] -0.01 [-0.22, 0.20] 0.09 [0.08, 0.10] 1.54 [-0.48, 3.55] 12.09 [10.81, 13.37] 0.88 [0.84, 0.92] 138
MobGap (with old LRC) 224 0.77 [0.75, 0.80] 0.79 [0.76, 0.82] -0.01 [-0.23, 0.21] 0.09 [0.08, 0.10] 0.92 [-1.27, 3.11] 12.72 [11.27, 14.16] 0.87 [0.82, 0.90] 152
Original Implementation 224 0.83 [0.80, 0.85] 0.82 [0.79, 0.85] 0.01 [-0.24, 0.26] 0.10 [0.09, 0.11] 4.74 [2.30, 7.18] 13.92 [12.16, 15.67] 0.84 [0.78, 0.88] 141
PFF Mobilise-D Pipeline MobGap 169 0.69 [0.66, 0.72] 0.68 [0.64, 0.72] 0.01 [-0.21, 0.23] 0.09 [0.08, 0.10] 6.55 [3.34, 9.77] 15.07 [12.59, 17.55] 0.89 [0.85, 0.93] 122
MobGap (with old GS) 169 0.69 [0.66, 0.72] 0.68 [0.64, 0.71] 0.01 [-0.21, 0.24] 0.09 [0.08, 0.10] 7.21 [3.69, 10.72] 15.90 [13.12, 18.68] 0.87 [0.82, 0.91] 116
MobGap (with old LRC) 169 0.69 [0.66, 0.72] 0.68 [0.64, 0.72] 0.01 [-0.21, 0.23] 0.09 [0.08, 0.10] 6.44 [3.23, 9.66] 15.07 [12.60, 17.54] 0.89 [0.85, 0.93] 121
Original Implementation 169 0.74 [0.71, 0.78] 0.72 [0.67, 0.76] 0.03 [-0.25, 0.31] 0.12 [0.10, 0.13] 11.65 [7.40, 15.90] 19.86 [16.38, 23.34] 0.84 [0.77, 0.89] 123


.. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 17.260 seconds) **Estimated memory usage:** 80 MB .. _sphx_glr_download_auto_revalidation_full_pipeline__04_additional_experiments.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: _04_additional_experiments.ipynb <_04_additional_experiments.ipynb>` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: _04_additional_experiments.py <_04_additional_experiments.py>` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: _04_additional_experiments.zip <_04_additional_experiments.zip>` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_