A/B Testing fuer Fiori Apps in ABAP Cloud

kategorie

DevOps

Veröffentlicht

Feb 14, 2026

autor

A/B Testing ermoeglicht datengesteuerte Entscheidungen in der Produktentwicklung. Anstatt Vermutungen anzustellen, welche UI-Variante besser funktioniert, testest du beide Varianten mit echten Benutzern und misst die Ergebnisse.

Was ist A/B Testing?

A/B Testing (auch Split Testing genannt) vergleicht zwei Varianten eines Features, indem Benutzer zufaellig auf Variante A oder B verteilt werden:

┌─────────────────────────────────────────────────────────────────┐
│                     A/B Testing Ablauf                           │
├─────────────────────────────────────────────────────────────────┤
│                                                                   │
│  Benutzer          ┌──────────────┐                              │
│     │              │  Experiment  │                              │
│     │              │  Assignment  │                              │
│     ▼              └──────┬───────┘                              │
│  ┌─────┐                  │                                      │
│  │ 50% │─────────────────►│ Variante A (Control)                 │
│  └─────┘                  │    └─► Messen: Klicks, Zeit, etc.   │
│                           │                                      │
│  ┌─────┐                  │                                      │
│  │ 50% │─────────────────►│ Variante B (Treatment)               │
│  └─────┘                  │    └─► Messen: Klicks, Zeit, etc.   │
│                           │                                      │
│                    ┌──────▼───────┐                              │
│                    │  Statistische │                             │
│                    │   Auswertung  │                             │
│                    └──────────────┘                              │
│                           │                                      │
│                    ┌──────▼───────┐                              │
│                    │  Entscheidung │                             │
│                    │  A oder B?    │                             │
│                    └──────────────┘                              │
└─────────────────────────────────────────────────────────────────┘

Typische Metriken

Metrik	Beschreibung	Beispiel
Conversion Rate	Anteil der Benutzer, die eine Aktion abschliessen	15% Button-Klicks
Time on Task	Zeit bis zur Aufgabenerledigung	45 Sekunden
Error Rate	Anteil fehlerhafter Eingaben	3% Validierungsfehler
Engagement	Interaktionen pro Session	8 Klicks
Bounce Rate	Benutzer, die sofort abbrechen	12%

Architektur fuer A/B Testing in ABAP Cloud

Die Implementierung basiert auf Feature Flags erweitert um Tracking-Funktionalitaet:

┌─────────────────────────────────────────────────────────────────┐
│                   A/B Testing Architektur                        │
├─────────────────────────────────────────────────────────────────┤
│                                                                   │
│  ┌──────────────┐     ┌──────────────┐     ┌──────────────┐     │
│  │ Fiori App    │────►│ RAP Service  │────►│ Experiment   │     │
│  │              │     │              │     │ Assignment   │     │
│  └──────┬───────┘     └──────────────┘     └──────────────┘     │
│         │                                                        │
│         │ Benutzeraktionen                                       │
│         ▼                                                        │
│  ┌──────────────┐     ┌──────────────┐     ┌──────────────┐     │
│  │ Event        │────►│ Tracking     │────►│ Analytics    │     │
│  │ Tracking     │     │ Service      │     │ Auswertung   │     │
│  └──────────────┘     └──────────────┘     └──────────────┘     │
│                                                                   │
└─────────────────────────────────────────────────────────────────┘

Datenmodell fuer Experimente

Experiment-Konfiguration

@EndUserText.label : 'A/B Experiments'
@AbapCatalog.enhancement.category : #NOT_EXTENSIBLE
@AbapCatalog.tableCategory : #TRANSPARENT
@AbapCatalog.deliveryClass : #A
define table zab_experiment {
  key client         : abap.clnt not null;
  key experiment_id  : abap.char(40) not null;
  experiment_name    : abap.char(100);
  description        : abap.char(255);
  hypothesis         : abap.string(1000);
  status             : abap.char(10);       // DRAFT, RUNNING, COMPLETED, CANCELLED
  start_date         : abap.dats;
  end_date           : abap.dats;
  target_sample_size : abap.int4;
  control_percent    : abap.int2;           // z.B. 50 fuer 50%
  created_by         : abap.uname;
  created_at         : timestampl;
}

Varianten-Definition

@EndUserText.label : 'Experiment Variants'
@AbapCatalog.tableCategory : #TRANSPARENT
define table zab_variant {
  key client        : abap.clnt not null;
  key experiment_id : abap.char(40) not null;
  key variant_id    : abap.char(10) not null;  // A, B, C...
  variant_name      : abap.char(100);
  is_control        : abap_boolean;            // Control-Gruppe (A)
  allocation_weight : abap.int2;               // Gewichtung
  variant_config    : abap.string(4000);       // JSON-Konfiguration
}

Benutzer-Zuweisung

@EndUserText.label : 'User Experiment Assignment'
@AbapCatalog.tableCategory : #TRANSPARENT
define table zab_assignment {
  key client        : abap.clnt not null;
  key experiment_id : abap.char(40) not null;
  key user_id       : abap.char(40) not null;
  variant_id        : abap.char(10);
  assigned_at       : timestampl;
  assignment_hash   : abap.char(64);          // Fuer deterministische Zuweisung
}

Varianten-Zuweisung fuer Benutzer

Die Zuweisung muss deterministisch und konsistent sein - ein Benutzer soll immer dieselbe Variante sehen.

Experiment Service Klasse

CLASS zcl_ab_experiment_service DEFINITION
  PUBLIC
  FINAL
  CREATE PUBLIC.

  PUBLIC SECTION.
    TYPES:
      BEGIN OF ty_variant_config,
        variant_id   TYPE zab_variant-variant_id,
        variant_name TYPE zab_variant-variant_name,
        config       TYPE string,
      END OF ty_variant_config.

    METHODS get_variant_for_user
      IMPORTING
        iv_experiment_id TYPE zab_experiment-experiment_id
        iv_user_id       TYPE sy-uname DEFAULT sy-uname
      RETURNING
        VALUE(rs_result) TYPE ty_variant_config
      RAISING
        cx_static_check.

    METHODS is_experiment_active
      IMPORTING
        iv_experiment_id   TYPE zab_experiment-experiment_id
      RETURNING
        VALUE(rv_is_active) TYPE abap_bool.

  PRIVATE SECTION.
    METHODS calculate_bucket
      IMPORTING
        iv_experiment_id TYPE zab_experiment-experiment_id
        iv_user_id       TYPE sy-uname
      RETURNING
        VALUE(rv_bucket) TYPE i.

    METHODS assign_variant
      IMPORTING
        iv_experiment_id TYPE zab_experiment-experiment_id
        iv_user_id       TYPE sy-uname
        iv_bucket        TYPE i
      RETURNING
        VALUE(rv_variant_id) TYPE zab_variant-variant_id.

ENDCLASS.

CLASS zcl_ab_experiment_service IMPLEMENTATION.

  METHOD get_variant_for_user.
    " Pruefen ob Experiment aktiv ist
    IF is_experiment_active( iv_experiment_id ) = abap_false.
      RAISE EXCEPTION TYPE zcx_ab_experiment
        EXPORTING
          textid = zcx_ab_experiment=>experiment_not_active.
    ENDIF.

    " Bestehende Zuweisung pruefen
    SELECT SINGLE variant_id FROM zab_assignment
      WHERE experiment_id = @iv_experiment_id
        AND user_id = @iv_user_id
      INTO @DATA(lv_existing_variant).

    IF sy-subrc = 0.
      " Bereits zugewiesen - Varianten-Konfiguration laden
      SELECT SINGLE variant_id, variant_name, variant_config
        FROM zab_variant
        WHERE experiment_id = @iv_experiment_id
          AND variant_id = @lv_existing_variant
        INTO CORRESPONDING FIELDS OF @rs_result.
      RETURN.
    ENDIF.

    " Neue Zuweisung erstellen
    DATA(lv_bucket) = calculate_bucket(
      iv_experiment_id = iv_experiment_id
      iv_user_id       = iv_user_id
    ).

    DATA(lv_variant_id) = assign_variant(
      iv_experiment_id = iv_experiment_id
      iv_user_id       = iv_user_id
      iv_bucket        = lv_bucket
    ).

    " Varianten-Konfiguration laden
    SELECT SINGLE variant_id, variant_name, variant_config
      FROM zab_variant
      WHERE experiment_id = @iv_experiment_id
        AND variant_id = @lv_variant_id
      INTO CORRESPONDING FIELDS OF @rs_result.
  ENDMETHOD.

  METHOD is_experiment_active.
    SELECT SINGLE status, start_date, end_date
      FROM zab_experiment
      WHERE experiment_id = @iv_experiment_id
      INTO @DATA(ls_experiment).

    IF sy-subrc <> 0.
      rv_is_active = abap_false.
      RETURN.
    ENDIF.

    rv_is_active = xsdbool(
      ls_experiment-status = 'RUNNING' AND
      ls_experiment-start_date <= sy-datum AND
      ( ls_experiment-end_date >= sy-datum OR
        ls_experiment-end_date IS INITIAL )
    ).
  ENDMETHOD.

  METHOD calculate_bucket.
    " Deterministischer Hash basierend auf Experiment + User
    DATA(lv_input) = |{ iv_experiment_id }{ iv_user_id }|.

    " Hash berechnen (CL_ABAP_MESSAGE_DIGEST)
    DATA(lo_digest) = cl_abap_message_digest=>create(
      algorithm = 'SHA256'
    ).
    lo_digest->update(
      message = cl_abap_codepage=>convert_to( lv_input )
    ).
    DATA(lv_hash) = lo_digest->digest_hex_string( ).

    " Hash in Bucket 0-99 umwandeln
    DATA(lv_hash_part) = substring( val = lv_hash off = 0 len = 8 ).
    DATA(lv_number) = cl_abap_conv_in_ce=>conv_hex_to_int( lv_hash_part ).
    rv_bucket = lv_number MOD 100.
  ENDMETHOD.

  METHOD assign_variant.
    " Varianten mit Gewichtung laden
    SELECT variant_id, allocation_weight
      FROM zab_variant
      WHERE experiment_id = @iv_experiment_id
      ORDER BY variant_id
      INTO TABLE @DATA(lt_variants).

    " Bucket zu Variante zuordnen
    DATA(lv_cumulative) = 0.
    LOOP AT lt_variants INTO DATA(ls_variant).
      lv_cumulative = lv_cumulative + ls_variant-allocation_weight.
      IF iv_bucket < lv_cumulative.
        rv_variant_id = ls_variant-variant_id.
        EXIT.
      ENDIF.
    ENDLOOP.

    " Zuweisung persistieren
    INSERT INTO zab_assignment VALUES @(
      VALUE #(
        client        = sy-mandt
        experiment_id = iv_experiment_id
        user_id       = iv_user_id
        variant_id    = rv_variant_id
        assigned_at   = utclong_current( )
      )
    ).
  ENDMETHOD.

ENDCLASS.

Tracking von Benutzeraktionen

Das Tracking erfasst alle relevanten Benutzerinteraktionen fuer die spaetere Auswertung.

Event-Tabelle

@EndUserText.label : 'Experiment Events'
@AbapCatalog.tableCategory : #TRANSPARENT
define table zab_event {
  key client        : abap.clnt not null;
  key event_uuid    : sysuuid_x16 not null;
  experiment_id     : abap.char(40);
  variant_id        : abap.char(10);
  user_id           : abap.char(40);
  event_type        : abap.char(50);      // VIEW, CLICK, SUBMIT, ERROR
  event_name        : abap.char(100);     // button_primary, form_submit
  event_value       : abap.string(1000);  // JSON mit zusaetzlichen Daten
  page_url          : abap.string(500);
  session_id        : abap.char(64);
  timestamp         : timestampl;
  duration_ms       : abap.int4;
}

Tracking Service

CLASS zcl_ab_tracking_service DEFINITION
  PUBLIC
  FINAL
  CREATE PUBLIC.

  PUBLIC SECTION.
    TYPES:
      BEGIN OF ty_event,
        experiment_id TYPE zab_event-experiment_id,
        event_type    TYPE zab_event-event_type,
        event_name    TYPE zab_event-event_name,
        event_value   TYPE string,
        page_url      TYPE zab_event-page_url,
        session_id    TYPE zab_event-session_id,
        duration_ms   TYPE zab_event-duration_ms,
      END OF ty_event.

    METHODS track_event
      IMPORTING
        is_event TYPE ty_event.

    METHODS track_conversion
      IMPORTING
        iv_experiment_id TYPE zab_experiment-experiment_id
        iv_conversion_id TYPE string
        iv_value         TYPE decfloat34 OPTIONAL.

    METHODS track_page_view
      IMPORTING
        iv_experiment_id TYPE zab_experiment-experiment_id
        iv_page_url      TYPE string.

  PRIVATE SECTION.
    DATA mo_experiment_service TYPE REF TO zcl_ab_experiment_service.

    METHODS get_user_variant
      IMPORTING
        iv_experiment_id TYPE zab_experiment-experiment_id
      RETURNING
        VALUE(rv_variant_id) TYPE zab_variant-variant_id.

ENDCLASS.

CLASS zcl_ab_tracking_service IMPLEMENTATION.

  METHOD track_event.
    DATA(lv_variant_id) = get_user_variant( is_event-experiment_id ).

    DATA(ls_event) = VALUE zab_event(
      client        = sy-mandt
      event_uuid    = cl_system_uuid=>create_uuid_x16_static( )
      experiment_id = is_event-experiment_id
      variant_id    = lv_variant_id
      user_id       = sy-uname
      event_type    = is_event-event_type
      event_name    = is_event-event_name
      event_value   = is_event-event_value
      page_url      = is_event-page_url
      session_id    = is_event-session_id
      timestamp     = utclong_current( )
      duration_ms   = is_event-duration_ms
    ).

    INSERT INTO zab_event VALUES @ls_event.
  ENDMETHOD.

  METHOD track_conversion.
    DATA(lv_event_value) = |\{ "conversion_id": "{ iv_conversion_id }"|.
    IF iv_value IS NOT INITIAL.
      lv_event_value = |{ lv_event_value }, "value": { iv_value }|.
    ENDIF.
    lv_event_value = |{ lv_event_value } \}|.

    track_event( VALUE #(
      experiment_id = iv_experiment_id
      event_type    = 'CONVERSION'
      event_name    = iv_conversion_id
      event_value   = lv_event_value
    ) ).
  ENDMETHOD.

  METHOD track_page_view.
    track_event( VALUE #(
      experiment_id = iv_experiment_id
      event_type    = 'PAGE_VIEW'
      event_name    = 'page_view'
      page_url      = iv_page_url
    ) ).
  ENDMETHOD.

  METHOD get_user_variant.
    IF mo_experiment_service IS NOT BOUND.
      mo_experiment_service = NEW zcl_ab_experiment_service( ).
    ENDIF.

    TRY.
        DATA(ls_variant) = mo_experiment_service->get_variant_for_user(
          iv_experiment_id = iv_experiment_id
        ).
        rv_variant_id = ls_variant-variant_id.
      CATCH cx_static_check.
        rv_variant_id = 'UNKNOWN'.
    ENDTRY.
  ENDMETHOD.

ENDCLASS.

RAP Integration fuer Frontend-Tracking

" Behavior Definition
unmanaged implementation in class zbp_i_ab_tracking;

define behavior for ZI_AB_TRACKING_EVENT alias TrackingEvent
{
  static action trackEvent parameter ZA_TRACKING_EVENT_PARAM;
  static action trackConversion parameter ZA_CONVERSION_PARAM;
}

CLASS zbp_i_ab_tracking DEFINITION PUBLIC
  FINAL FOR BEHAVIOR OF zi_ab_tracking_event.

  PUBLIC SECTION.
    METHODS trackEvent FOR MODIFY
      IMPORTING it_event FOR ACTION TrackingEvent~trackEvent.

    METHODS trackConversion FOR MODIFY
      IMPORTING it_conv FOR ACTION TrackingEvent~trackConversion.

ENDCLASS.

CLASS zbp_i_ab_tracking IMPLEMENTATION.

  METHOD trackEvent.
    DATA(lo_tracking) = NEW zcl_ab_tracking_service( ).

    LOOP AT it_event INTO DATA(ls_event).
      lo_tracking->track_event( VALUE #(
        experiment_id = ls_event-%param-experiment_id
        event_type    = ls_event-%param-event_type
        event_name    = ls_event-%param-event_name
        event_value   = ls_event-%param-event_value
        page_url      = ls_event-%param-page_url
        session_id    = ls_event-%param-session_id
      ) ).
    ENDLOOP.
  ENDMETHOD.

  METHOD trackConversion.
    DATA(lo_tracking) = NEW zcl_ab_tracking_service( ).

    LOOP AT it_conv INTO DATA(ls_conv).
      lo_tracking->track_conversion(
        iv_experiment_id = ls_conv-%param-experiment_id
        iv_conversion_id = ls_conv-%param-conversion_id
        iv_value         = ls_conv-%param-value
      ).
    ENDLOOP.
  ENDMETHOD.

ENDCLASS.

Metriken sammeln und auswerten

CDS View fuer Experiment-Metriken

@AbapCatalog.viewEnhancementCategory: [#NONE]
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'Experiment Metrics'
@Analytics: { dataCategory: #CUBE }
define view entity ZI_AB_EXPERIMENT_METRICS
  as select from zab_event as Event
  inner join zab_assignment as Assignment
    on  Event.experiment_id = Assignment.experiment_id
    and Event.user_id       = Assignment.user_id
{
  key Event.experiment_id,
  key Event.variant_id,
  key Event.event_type,
  key Event.event_name,

  @EndUserText.label: 'Anzahl Events'
  @Aggregation.default: #SUM
  cast( 1 as abap.int4 ) as event_count,

  @EndUserText.label: 'Eindeutige Benutzer'
  @Aggregation.default: #COUNT_DISTINCT
  Event.user_id,

  @EndUserText.label: 'Durchschnittliche Dauer'
  @Aggregation.default: #AVG
  Event.duration_ms,

  @EndUserText.label: 'Event Datum'
  cast( Event.timestamp as abap.dats ) as event_date
}

Conversion Rate Berechnung

@AbapCatalog.viewEnhancementCategory: [#NONE]
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'Conversion Rates per Variant'
define view entity ZI_AB_CONVERSION_RATE
  as select from zab_assignment as Assignment
  left outer join zab_event as Conversion
    on  Assignment.experiment_id = Conversion.experiment_id
    and Assignment.user_id       = Conversion.user_id
    and Conversion.event_type    = 'CONVERSION'
{
  key Assignment.experiment_id,
  key Assignment.variant_id,

  @EndUserText.label: 'Zugewiesene Benutzer'
  count( distinct Assignment.user_id ) as assigned_users,

  @EndUserText.label: 'Konvertierte Benutzer'
  count( distinct Conversion.user_id ) as converted_users,

  @EndUserText.label: 'Conversion Rate'
  division(
    count( distinct Conversion.user_id ) * 100,
    count( distinct Assignment.user_id ),
    2
  ) as conversion_rate_percent
}
group by
  Assignment.experiment_id,
  Assignment.variant_id

Metriken-Service Klasse

CLASS zcl_ab_metrics_service DEFINITION
  PUBLIC
  FINAL
  CREATE PUBLIC.

  PUBLIC SECTION.
    TYPES:
      BEGIN OF ty_variant_metrics,
        variant_id       TYPE zab_variant-variant_id,
        sample_size      TYPE i,
        conversions      TYPE i,
        conversion_rate  TYPE decfloat34,
        avg_duration_ms  TYPE decfloat34,
        total_events     TYPE i,
      END OF ty_variant_metrics,
      tt_variant_metrics TYPE STANDARD TABLE OF ty_variant_metrics WITH KEY variant_id.

    TYPES:
      BEGIN OF ty_experiment_result,
        experiment_id       TYPE zab_experiment-experiment_id,
        status              TYPE string,
        is_significant      TYPE abap_bool,
        winning_variant     TYPE zab_variant-variant_id,
        confidence_level    TYPE decfloat34,
        variants            TYPE tt_variant_metrics,
      END OF ty_experiment_result.

    METHODS get_experiment_metrics
      IMPORTING
        iv_experiment_id TYPE zab_experiment-experiment_id
      RETURNING
        VALUE(rs_result) TYPE ty_experiment_result.

  PRIVATE SECTION.
    METHODS calculate_chi_square
      IMPORTING
        it_variants         TYPE tt_variant_metrics
      RETURNING
        VALUE(rv_chi_square) TYPE decfloat34.

    METHODS get_p_value
      IMPORTING
        iv_chi_square     TYPE decfloat34
        iv_degrees_freedom TYPE i
      RETURNING
        VALUE(rv_p_value) TYPE decfloat34.

ENDCLASS.

CLASS zcl_ab_metrics_service IMPLEMENTATION.

  METHOD get_experiment_metrics.
    rs_result-experiment_id = iv_experiment_id.

    " Metriken pro Variante laden
    SELECT variant_id,
           assigned_users AS sample_size,
           converted_users AS conversions,
           conversion_rate_percent AS conversion_rate
      FROM zi_ab_conversion_rate
      WHERE experiment_id = @iv_experiment_id
      INTO TABLE @DATA(lt_rates).

    LOOP AT lt_rates INTO DATA(ls_rate).
      APPEND VALUE #(
        variant_id      = ls_rate-variant_id
        sample_size     = ls_rate-sample_size
        conversions     = ls_rate-conversions
        conversion_rate = ls_rate-conversion_rate
      ) TO rs_result-variants.
    ENDLOOP.

    " Statistische Signifikanz berechnen
    IF lines( rs_result-variants ) >= 2.
      DATA(lv_chi_square) = calculate_chi_square( rs_result-variants ).
      DATA(lv_p_value) = get_p_value(
        iv_chi_square      = lv_chi_square
        iv_degrees_freedom = lines( rs_result-variants ) - 1
      ).

      rs_result-confidence_level = ( 1 - lv_p_value ) * 100.
      rs_result-is_significant = xsdbool( lv_p_value < '0.05' ).

      " Gewinner ermitteln
      DATA(lv_max_rate) = VALUE decfloat34( ).
      LOOP AT rs_result-variants INTO DATA(ls_variant).
        IF ls_variant-conversion_rate > lv_max_rate.
          lv_max_rate = ls_variant-conversion_rate.
          rs_result-winning_variant = ls_variant-variant_id.
        ENDIF.
      ENDLOOP.
    ENDIF.

    rs_result-status = COND #(
      WHEN rs_result-is_significant = abap_true THEN 'SIGNIFICANT'
      ELSE 'COLLECTING_DATA'
    ).
  ENDMETHOD.

  METHOD calculate_chi_square.
    " Chi-Quadrat-Test fuer Unabhaengigkeit
    DATA(lv_total_users) = REDUCE i( INIT sum = 0
      FOR variant IN it_variants NEXT sum = sum + variant-sample_size ).
    DATA(lv_total_conversions) = REDUCE i( INIT sum = 0
      FOR variant IN it_variants NEXT sum = sum + variant-conversions ).

    DATA(lv_expected_rate) = COND decfloat34(
      WHEN lv_total_users > 0
      THEN lv_total_conversions / lv_total_users
      ELSE 0
    ).

    rv_chi_square = 0.
    LOOP AT it_variants INTO DATA(ls_variant).
      DATA(lv_expected_conv) = ls_variant-sample_size * lv_expected_rate.
      DATA(lv_expected_non_conv) = ls_variant-sample_size * ( 1 - lv_expected_rate ).

      IF lv_expected_conv > 0.
        rv_chi_square = rv_chi_square +
          ( ( ls_variant-conversions - lv_expected_conv ) ** 2 ) / lv_expected_conv.
      ENDIF.

      IF lv_expected_non_conv > 0.
        DATA(lv_non_conv) = ls_variant-sample_size - ls_variant-conversions.
        rv_chi_square = rv_chi_square +
          ( ( lv_non_conv - lv_expected_non_conv ) ** 2 ) / lv_expected_non_conv.
      ENDIF.
    ENDLOOP.
  ENDMETHOD.

  METHOD get_p_value.
    " Vereinfachte p-Wert Approximation
    " Fuer produktive Nutzung: Statistische Bibliothek oder externe API verwenden

    " Kritische Werte fuer Chi-Quadrat (df=1)
    " p=0.10 -> chi=2.706
    " p=0.05 -> chi=3.841
    " p=0.01 -> chi=6.635

    rv_p_value = COND #(
      WHEN iv_chi_square >= '6.635' THEN '0.01'
      WHEN iv_chi_square >= '3.841' THEN '0.05'
      WHEN iv_chi_square >= '2.706' THEN '0.10'
      ELSE '0.50'
    ).
  ENDMETHOD.

ENDCLASS.

Statistische Signifikanz berechnen

Die statistische Signifikanz sagt aus, ob die beobachteten Unterschiede zwischen Varianten zufaellig sind oder tatsaechlich existieren.

Grundlegende Konzepte

Begriff	Bedeutung
p-Wert	Wahrscheinlichkeit, dass Unterschied zufaellig ist
Signifikanzniveau	Schwelle (typisch 0.05 = 5%)
Konfidenzintervall	Bereich, in dem wahrer Wert liegt
Sample Size	Anzahl Beobachtungen pro Variante
Effect Size	Groesse des Unterschieds

Sample Size Berechnung

CLASS zcl_ab_sample_calculator DEFINITION
  PUBLIC
  FINAL
  CREATE PUBLIC.

  PUBLIC SECTION.
    METHODS calculate_required_sample_size
      IMPORTING
        iv_baseline_rate      TYPE decfloat34  " z.B. 0.10 (10%)
        iv_min_detectable_effect TYPE decfloat34  " z.B. 0.02 (2%)
        iv_significance_level TYPE decfloat34 DEFAULT '0.05'
        iv_power              TYPE decfloat34 DEFAULT '0.80'
      RETURNING
        VALUE(rv_sample_size) TYPE i.

    METHODS estimate_test_duration
      IMPORTING
        iv_required_sample TYPE i
        iv_daily_users     TYPE i
      RETURNING
        VALUE(rv_days)     TYPE i.

ENDCLASS.

CLASS zcl_ab_sample_calculator IMPLEMENTATION.

  METHOD calculate_required_sample_size.
    " Vereinfachte Formel fuer zwei Proportionen
    " n = 2 * (z_alpha + z_beta)^2 * p * (1-p) / delta^2

    " Z-Werte fuer typische Parameter
    DATA(lv_z_alpha) = COND decfloat34(
      WHEN iv_significance_level = '0.05' THEN '1.96'
      WHEN iv_significance_level = '0.01' THEN '2.58'
      ELSE '1.64'
    ).

    DATA(lv_z_beta) = COND decfloat34(
      WHEN iv_power = '0.80' THEN '0.84'
      WHEN iv_power = '0.90' THEN '1.28'
      ELSE '0.84'
    ).

    DATA(lv_p) = iv_baseline_rate.
    DATA(lv_delta) = iv_min_detectable_effect.

    DATA(lv_numerator) = 2 * ( lv_z_alpha + lv_z_beta ) ** 2 * lv_p * ( 1 - lv_p ).
    DATA(lv_denominator) = lv_delta ** 2.

    IF lv_denominator > 0.
      rv_sample_size = ceil( lv_numerator / lv_denominator ).
    ENDIF.
  ENDMETHOD.

  METHOD estimate_test_duration.
    " Bei 50/50 Split brauchen wir doppelte Sample Size
    DATA(lv_total_needed) = iv_required_sample * 2.

    IF iv_daily_users > 0.
      rv_days = ceil( lv_total_needed / iv_daily_users ).
    ENDIF.
  ENDMETHOD.

ENDCLASS.

Ergebnis-Interpretation

CLASS zcl_ab_result_interpreter DEFINITION
  PUBLIC
  FINAL
  CREATE PUBLIC.

  PUBLIC SECTION.
    TYPES:
      BEGIN OF ty_recommendation,
        action      TYPE string,
        reasoning   TYPE string,
        confidence  TYPE string,
      END OF ty_recommendation.

    METHODS interpret_result
      IMPORTING
        is_result            TYPE zcl_ab_metrics_service=>ty_experiment_result
      RETURNING
        VALUE(rs_recommendation) TYPE ty_recommendation.

ENDCLASS.

CLASS zcl_ab_result_interpreter IMPLEMENTATION.

  METHOD interpret_result.
    DATA(lv_total_sample) = REDUCE i( INIT sum = 0
      FOR v IN is_result-variants NEXT sum = sum + v-sample_size ).

    " Mindestens 100 Samples pro Variante
    DATA(lv_min_sample) = 100 * lines( is_result-variants ).

    IF lv_total_sample < lv_min_sample.
      rs_recommendation = VALUE #(
        action     = 'WAIT'
        reasoning  = |Zu wenige Daten. { lv_total_sample } von { lv_min_sample } erforderlichen Samples.|
        confidence = 'LOW'
      ).
      RETURN.
    ENDIF.

    IF is_result-is_significant = abap_false.
      rs_recommendation = VALUE #(
        action     = 'CONTINUE'
        reasoning  = 'Kein signifikanter Unterschied erkannt. Test fortsetzen oder Hypothese ueberpruefen.'
        confidence = 'MEDIUM'
      ).
      RETURN.
    ENDIF.

    " Signifikant - Gewinner empfehlen
    READ TABLE is_result-variants INTO DATA(ls_control)
      WITH KEY variant_id = 'A'.
    READ TABLE is_result-variants INTO DATA(ls_winner)
      WITH KEY variant_id = is_result-winning_variant.

    DATA(lv_improvement) = COND decfloat34(
      WHEN ls_control-conversion_rate > 0
      THEN ( ls_winner-conversion_rate - ls_control-conversion_rate ) /
           ls_control-conversion_rate * 100
      ELSE 0
    ).

    rs_recommendation = VALUE #(
      action     = |IMPLEMENT_{ is_result-winning_variant }|
      reasoning  = |Variante { is_result-winning_variant } zeigt { lv_improvement }% Verbesserung bei { is_result-confidence_level }% Konfidenz.|
      confidence = COND #(
        WHEN is_result-confidence_level >= 99 THEN 'VERY_HIGH'
        WHEN is_result-confidence_level >= 95 THEN 'HIGH'
        ELSE 'MEDIUM'
      )
    ).
  ENDMETHOD.

ENDCLASS.

Integration mit SAP Analytics

Fuer erweiterte Analyse und Visualisierung kannst du die Daten an SAP Analytics Cloud senden.

Analytisches CDS View

@AbapCatalog.viewEnhancementCategory: [#NONE]
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'A/B Test Analytics'
@Analytics: { dataCategory: #CUBE, internalName: #LOCAL }
@ObjectModel.usageType: {
  serviceQuality: #D,
  sizeCategory: #L,
  dataClass: #MIXED
}
define view entity ZI_AB_ANALYTICS
  as select from zab_event as Event
  inner join zab_experiment as Experiment
    on Event.experiment_id = Experiment.experiment_id
  inner join zab_variant as Variant
    on  Event.experiment_id = Variant.experiment_id
    and Event.variant_id    = Variant.variant_id
{
  @Analytics.dimension: true
  @EndUserText.label: 'Experiment'
  key Event.experiment_id,

  @Analytics.dimension: true
  @EndUserText.label: 'Variante'
  key Event.variant_id,

  @Analytics.dimension: true
  @EndUserText.label: 'Event Typ'
  key Event.event_type,

  @Analytics.dimension: true
  @EndUserText.label: 'Event Datum'
  cast( Event.timestamp as abap.dats ) as event_date,

  @EndUserText.label: 'Experiment Name'
  Experiment.experiment_name,

  @EndUserText.label: 'Varianten Name'
  Variant.variant_name,

  @EndUserText.label: 'Ist Control'
  Variant.is_control,

  @Analytics.measure: true
  @Aggregation.default: #SUM
  @EndUserText.label: 'Anzahl Events'
  cast( 1 as abap.int4 ) as event_count,

  @Analytics.measure: true
  @Aggregation.default: #COUNT_DISTINCT
  @EndUserText.label: 'Unique Users'
  Event.user_id,

  @Analytics.measure: true
  @Aggregation.default: #AVG
  @EndUserText.label: 'Avg Duration'
  Event.duration_ms
}

Dashboard-Konfiguration

@Metadata.layer: #CORE
annotate view ZI_AB_ANALYTICS with
{
  @UI.chart: [{
    title: 'Conversion Rate pro Variante',
    chartType: #BAR,
    dimensions: ['variant_id'],
    measures: ['event_count'],
    qualifier: 'ConversionChart'
  }]
  @UI.presentationVariant: [{
    sortOrder: [{ by: 'event_date', direction: #DESC }],
    visualizations: [{
      type: #AS_CHART,
      qualifier: 'ConversionChart'
    }]
  }]
  experiment_id;
}

Ethische Ueberlegungen

A/B Testing sollte verantwortungsvoll durchgefuehrt werden:

Richtlinien

Aspekt	Richtlinie
Transparenz	Benutzer ueber Tests informieren (z.B. in Datenschutzerklaerung)
Keine Schaeden	Varianten duerfen Benutzer nicht benachteiligen
Datenminimierung	Nur notwendige Daten erfassen
Fairness	Kritische Features nicht testen (z.B. Preise)
Zeitbegrenzung	Tests nicht endlos laufen lassen
Dokumentation	Hypothese und Ergebnisse dokumentieren

Implementierung einer Ethik-Pruefung

CLASS zcl_ab_ethics_checker DEFINITION
  PUBLIC
  FINAL
  CREATE PUBLIC.

  PUBLIC SECTION.
    TYPES:
      BEGIN OF ty_ethics_result,
        is_approved   TYPE abap_bool,
        warnings      TYPE string_table,
        requires_review TYPE abap_bool,
      END OF ty_ethics_result.

    METHODS check_experiment
      IMPORTING
        is_experiment    TYPE zab_experiment
        it_variants      TYPE STANDARD TABLE
      RETURNING
        VALUE(rs_result) TYPE ty_ethics_result.

ENDCLASS.

CLASS zcl_ab_ethics_checker IMPLEMENTATION.

  METHOD check_experiment.
    rs_result-is_approved = abap_true.

    " Regel 1: Maximale Testdauer
    DATA(lv_duration) = is_experiment-end_date - is_experiment-start_date.
    IF lv_duration > 90.
      APPEND 'Test laeuft laenger als 90 Tage' TO rs_result-warnings.
      rs_result-requires_review = abap_true.
    ENDIF.

    " Regel 2: Sample Size nicht zu klein
    IF is_experiment-target_sample_size < 100.
      APPEND 'Sample Size unter 100 - statistisch nicht aussagekraeftig' TO rs_result-warnings.
    ENDIF.

    " Regel 3: Hypothese muss dokumentiert sein
    IF is_experiment-hypothesis IS INITIAL.
      APPEND 'Keine Hypothese dokumentiert' TO rs_result-warnings.
      rs_result-is_approved = abap_false.
    ENDIF.

    " Regel 4: Control-Variante muss existieren
    DATA(lv_has_control) = abap_false.
    LOOP AT it_variants INTO DATA(ls_variant).
      IF ls_variant-is_control = abap_true.
        lv_has_control = abap_true.
        EXIT.
      ENDIF.
    ENDLOOP.

    IF lv_has_control = abap_false.
      APPEND 'Keine Control-Gruppe definiert' TO rs_result-warnings.
      rs_result-is_approved = abap_false.
    ENDIF.
  ENDMETHOD.

ENDCLASS.

Best Practices

Dos and Don’ts

Do	Don’t
Klare Hypothese formulieren	”Mal schauen was passiert”
Eine Metrik als primaeres Ziel	Dutzende Metriken gleichzeitig
Ausreichend Sample Size warten	Frueh abbrechen bei positiven Trends
Segmente analysieren	Nur Durchschnitte betrachten
Ergebnisse dokumentieren	Learnings nicht teilen

Checkliste vor dem Start

Hypothese formuliert und dokumentiert?
Primaere Metrik definiert?
Sample Size berechnet?
Testdauer geplant?
Ethik-Check durchgefuehrt?
Tracking implementiert und getestet?
Rollback-Plan vorhanden?

Zusammenfassung

Komponente	Zweck
Experiment Service	Varianten-Zuweisung und -Verwaltung
Tracking Service	Event-Erfassung
Metrics Service	Auswertung und Statistik
Ethics Checker	Verantwortungsvolle Tests

A/B Testing ermoeglicht datenbasierte Entscheidungen in der Fiori-Entwicklung. Mit der richtigen Infrastruktur aus Experiment-Management, Tracking und statistischer Auswertung kannst du fundierte Produktentscheidungen treffen.