MDE Researcher

`AbstractMdeResearchBuilder`	Base class for Experiment Builders.
`BaseSplitElement`	Dataclass with data params for experiment calculations
`MdeAlphaExperiment`	Dataclass for I type error calculations.
`MdeBetaExperiment`	Dataclass for II type error calculations.
`MdeResearchBuilder`	Calculates I and II type errors for different group sizes and injects.
`MultipleSplitBuilder`	Columns with splits and injects will be added
`MdeParams`	MDE research experiment parameters class.

Abstract MDE Experiment

class abacus.mde_researcher.AbstractMdeResearchBuilder(guests: DataFrame, abtest_params: ABTestParams, experiment_params: MdeParams)[source]

Base class for Experiment Builders.

Attributes:

experiment_params
group_sizes

abacus.mde_researcher.AbstractMdeResearchBuilder._build_group_sizes(self)

Build list of groups sizes tuples.

Returns:: List of groups sizes pairs.
Return type:: List[int]

Experiment Structures

class abacus.mde_researcher.BaseSplitElement(group_sizes: tuple, split_number: int, control_group_size: int = FieldInfo(default=PydanticUndefined, extra={'init': False}), target_group_size: int = FieldInfo(default=PydanticUndefined, extra={'init': False}))[source]

Dataclass with data params for experiment calculations

Parameters:

group_sizes (tuple) – tuple with group sizes. Should have control group size on the 0 index position and target group size on the 1 index position
split_number (int) – params with number of split

class abacus.mde_researcher.MdeAlphaExperiment(group_sizes: tuple, split_number: int, control_group_size: int = FieldInfo(default=PydanticUndefined, extra={'init': False}), target_group_size: int = FieldInfo(default=PydanticUndefined, extra={'init': False}), metric_name: str = FieldInfo(default=PydanticUndefined, extra={'init': False}))[source]: Dataclass for I type error calculations.

class abacus.mde_researcher.MdeBetaExperiment(group_sizes: tuple, split_number: int, control_group_size: int = FieldInfo(default=PydanticUndefined, extra={'init': False}), target_group_size: int = FieldInfo(default=PydanticUndefined, extra={'init': False}), metric_name: str = FieldInfo(default=PydanticUndefined, extra={'init': False}), inject: float = FieldInfo(default=PydanticUndefined, extra={'init': False}))[source]: Dataclass for II type error calculations.

MDE Research Builder

class abacus.mde_researcher.MdeResearchBuilder(guests: DataFrame, abtest_params: ABTestParams, experiment_params: MdeParams, stratification_params: SplitBuilderParams)[source]

Calculates I and II type errors for different group sizes and injects.

Attributes:

experiment_params
group_sizes

Methods

`calc_alpha`(guests[, is_splitted])	Calculates I type error.
`collect`([fill_with_default])	Calculates I and II types error using prepilot parameters.

abacus.mde_researcher.MdeResearchBuilder.calc_alpha(self, guests: DataFrame, is_splitted: bool = False) → DataFrame

Calculates I type error.

Parameters:

guests (pandas.DataFrame) – Dataframe with guests.
is_splitted (bool) – If False guests must contain splits for calculation. Otherwise splits will be compute for guests.

Returns:

Pandas DataFrame with I type error.

Return type:

pandas.DataFrame

abacus.mde_researcher.MdeResearchBuilder.collect(self, fill_with_default: bool = True) → DataFrame

Calculates I and II types error using prepilot parameters.

Parameters:: fill_with_default (bool) – Fill calculated vaules with defaults.
Returns:: Pandas DataFrames with aggregated results of experiment.
Return type:: pandas.DataFrame

Multiple Split Builder

class abacus.mde_researcher.MultipleSplitBuilder(guests: DataFrame, metrics_names: List[str], injects: List[float], group_sizes: List[int], stratification_params: SplitBuilderParams, iterations_number: int = 10)[source]

Columns with splits and injects will be added

Methods

`calc_injected_metrics`(guests_for_injects)	Calculates injected metrics for guests df.
`collect`()	Calculate multiple split with stratification.

abacus.mde_researcher.MultipleSplitBuilder._build_splits_grid(self)

abacus.mde_researcher.MultipleSplitBuilder._update_strat_params(self): Update stratification columns, because of columns names duplicated problem.

abacus.mde_researcher.MultipleSplitBuilder._build_split(self, guests_with_strata: DataFrame, control_group_size: int, target_group_size: int, split_number: int = 1)

Calculate one split with stratification.

Parameters:

guests_with_strata (pandas.DataFrame) – Dataframe with stratas.
control_group_size (int) – Control group size.
target_group_size (int) – Target group size.
split_number (int, default = 1) – Number of split. Uses as suffix for new column.

Returns:

pandas DataFrame with split.

Return type:

pandas.DataFrame

abacus.mde_researcher.MultipleSplitBuilder.calc_injected_metrics(self, guests_for_injects: DataFrame) → DataFrame

Calculates injected metrics for guests df.

Parameters:: guests_for_injects (pandas.DataFrame) – Dataframe with metrics columns.
Returns:: Dataframe with injected metrics columns.
Return type:: pandas.DataFrame

abacus.mde_researcher.MultipleSplitBuilder.collect(self) → DataFrame

Calculate multiple split with stratification.

Returns:: Pandas DataFrame with split columns.
Return type:: pandas.DataFrame

Params

class abacus.mde_researcher.MdeParams(metrics_names: ~typing.List[str], injects: ~typing.List[float], min_group_size: int, max_group_size: int, step: int, variance_reduction: ~typing.Callable[[~abacus.auto_ab.abtest.ABTest], ~abacus.auto_ab.abtest.ABTest] | None = None, use_buckets: bool = False, transformations: ~typing.Any = None, stat_test: ~typing.Callable[[~abacus.auto_ab.abtest.ABTest], ~typing.Dict[str, int | float]] = <function ABTest.test_boot_confint>, iterations_number: int = 10, max_beta_score: float = 0.2, min_beta_score: float = 0.05)[source]

MDE research experiment parameters class.

Parameters:

metrics_names – Metrics which will be compare in experiments.
injects – Injects represent MDE values.
min_group_size – Minimal value of groups sizes.
max_group_size – Maximal value of groups sizes.
step – Spacing between min_group_size and max_group_size.
variance_reduction – ABTest methods for variance reduction.
use_buckets – Use bucketize method.
transformations – Pipeline of experiment. Will be calulted in __post_init__.
stat_test – Statistical test type.
iterations_number – Count of splits for each element in group_sizes.
max_beta_score – Maximum level of II type error.
min_beta_score – Minimum level of II type error.

Attributes:

transformations
variance_reduction

Methods

stat_test()

Performs bootstrap confidence interval and zero statistical significance.

groups_sizes_validator
stat_test_validator
variance_reduction_validator