Skip to content

LLM:End to end testing

Jump to bottom
bayo.makanjuola edited this page Jul 1, 2025 · 10 revisions

Ledger Live Mobile end-to-end testing

This project uses Detox and Jest for end-to-end testing the LLM application. Detox is a mobile end-to-end testing tool developed by Wix, and is specifically built for React Native applications. Please refer to the documentation of those projects for the specifics. In this readme you will find the key setup and workflow steps for testing a flow in Ledger Live Mobile.

Quick start script to install iOS and Android debug versions from scratch

Ensure you have Android Studio and Xcode installed with the relevant development/emulator tools installed (see 'Local Environment Setup' below).

The following script will ensure a clean local environment for running detox tests in debug mode.

pnpm clean
pnpm i --filter="live-mobile..." --filter="ledger-live" --no-frozen-lockfile --unsafe-perm
pnpm mobile pod
pnpm build:llm:deps
pnpm mobile e2e:build -c android.emu.debug
pnpm mobile e2e:build -c ios.sim.debug

It's also a good idea to build the Dummy Live Apps as they are used in the Live SDK and Wallet API E2E tests for both LLM and LLD: pnpm test-utils dummy-apps:install && pnpm test-utils dummy-apps:build

Local Environment Setup

Writing and running Detox tests requires Xcode for iOS and Android Studio (along with the SDK and emulator tools) for Android. The best place to setup both Android and iOS is to follow the React Native's own documentation.

Next, follow the steps in the Detox Environment Setup section.

Prerequisites for all Detox tests:

  • Node is installed (currently we use v16)

Tips for iOS setup

Most of the setup is taken care of in the React Native docs, but you will have to do some additional installations, such as the Detox CLI and applesimutils (MacOS only). After following the above React Native and Detox steps, you should have the following setup:

  • XCode and XCode command line tools - run xcode-select -v and xcrun --version to make sure these are working
  • rbenv is installed and which ruby points to an rbenv shim, not usr/bin/ruby. Be sure to follow the steps in the RN guide to add rbenv to your shell profile.
  • An iPhone simulator for iPhone 13 - open Xcode > Window > Devices and Simulators > Simulators > Add a new device from the '+' sign in the bottom right corner.
  • applesimutils is installed. ( https://github.com/wix/AppleSimulatorUtils )

Tips for Android setup

The Android toolkit can be more complex than the iOS one. Once you've done the React Native and Detox setup steps, follow the Detox Android Environment Setup guide for further steps. The main things to make sure of are:

  • Java version 11 installed. Check with java -version
  • Android 12.0 (API Level 11) is installed.
  • Android SDK Build Tools, SDK Platform Tools, SDK Command Line Tools, Android Emulator, CMake 3.10.2 and NDK 21.4.7075529 are installed. You can do this through Android Studio > Tools > SDK Tools, or via the command line.
  • Your shell profile (for example ~/.zshrc) should have environmental variables setup something like this:
export JAVA_HOME=`/usr/libexec/java_home`
export ANDROID_HOME=$HOME/Library/Android/sdk
export PATH=$PATH:$ANDROID_HOME/emulator:$ANDROID_HOME/tools/bin/sdkmanager:$ANDROID_HOME/platform-tools:$ANDROID_HOME/cmdline-tools/latest/bin

Note: There is a bit of inconsistency in the documentation between React Native, Detox and Android themselves about whether to use ANDROID_ROOT or ANDROID_SDK_ROOT. The second one is now deprecated but both should work in either case.

Test Setup and Execution

Clean your local environment to remove node_modules and previous iOS and Android mobile app builds:

pnpm clean

Install dependencies:

pnpm i

There is a filtered version of this command which should be quicker and includes only dependencies needed for LLM: pnpm i --filter="live-mobile..." --filter="ledger-live".

Build dependencies for the mobile app:

pnpm build:llm:deps

Android Tests

Verify you have an emulator installed and have that match the Detox avdName (currently 'Android_Emulator') in the e2e/mobile/detox.config.js file. Be sure to make the device the correct architecture and system image. Currently this is x86_64 if you are on an Intel mac and arm64_v8a if you are on an M1 Mac (*info required for Windows and Linux*). If you are on an Intel Mac, you must run export CI=1 in the terminal session before

  • Build the apps
    • Debug: pnpm mobile e2e:build -c android.emu.debug
    • Release: pnpm mobile e2e:build -c android.emu.release
  • Run the tests
    • Debug: First, run pnpm mobile start to run Metro bundler, then in a separate terminal window run pnpm mobile e2e:test -c android.emu.debug. When developing locally, you may need to put the content of the .env.mock file in the app .env file to have the right test environment.
    • Release: pnpm mobile e2e:test -c android.emu.release

If you get an error for Android debug tests complaining that the emulator cannot find the bundled JS script, run adb reverse tcp:8081 tcp:8081 before starting the tests (but make sure the emulator is already started). This makes it possible for the emulator to access the Metro bundler on your local machine.

iOS Tests

Make sure you have the correct iPhone simulator that is listed in e2e/mobile/detox.config.js installed (currently 'iOS Simulator'). You can check if you do with applesimutils --list. Also make sure you have an iOS version installed for simulators by going to Xcode > Preferences > Components. You can try whichever version you like, but iOS 13.0 is known to work locally.

  • Build the apps
    • Debug: pnpm mobile e2e:build -c ios.sim.debug
    • Release: pnpm mobile e2e:build -c ios.sim.release
  • Run the tests
    • Debug: First, run pnpm mobile start to run Metro bundler, then in a separate terminal window run pnpm mobile e2e:test -c ios.sim.debug
    • Release: pnpm mobile e2e:test -c ios.sim.release

CI

⚠️ Android and iOS tests are currently switched off on the CI for PRs due to issues installing the app on the emulators and general flakiness with the runners. However the tests are running at midday and midnight daily

Triggering Release Runs from GitHub Actions

You can trigger mobile E2E tests manually using the GitHub Actions workflow. This is useful for running tests on specific branches, with different configurations, or for targeted testing scenarios.

Workflow Location

.github/workflows/test-mobile-e2e-reusable.yml

This is a reusable workflow designed to be called from another workflow via uses:.

How to Trigger

  1. Go to the GitHub Actions page for the ledger-live repository
  2. Select the "Mobile E2E Tests" workflow
  3. Click "Run workflow"
  4. Configure the inputs as needed
  5. Click "Run workflow" to start the execution

Workflow Inputs

Field Type Default Description
Workflow Branch String develop Use workflow from branch
Branch String develop Build flavor / scheme (e.g. debug, release).
Base branch String (empty) The branch into which your chosen head will be merged. Usually left blank for feature branch.
Tests type String E2E - iOS & Android Which platforms to run: E2E - iOS, E2E - Android, or E2E - iOS & Android.
Enable transaction broadcast Boolean false If checked, broadcast transactions will be enabled.
Target Firebase Production env Boolean false If checked, to use prod Firebase env setting.
Filter to execute only test suite String (empty) A substring/regex to match spec filenames (e.g. addAccount, or swap). Leave empty to run the full suite of tests.
Send tests results to Xray Boolean false If checked, post pass/fail and artifacts to Xray for 🐞 tracking.
🤖 Test Execution String (empty) Xray execution ID for Android tests (e.g. B2CQA-2461).
🍎 Test Execution String (empty) Xray execution ID for iOS tests (e.g. B2CQA-2462).
Device to be used String nanoSP Which Ledger Nano model to attach in Speculos (e.g. nanoS, nanoSP, blue).

Example Invocation

# .github/workflows/ci-mobile.yml
name: CI – Mobile E2E

on:
  workflow_dispatch:

jobs:
  e2e:
    uses: LedgerHQ/ledger-live/.github/workflows/test-mobile-e2e-reusable.yml@main
    with:
      workflowBranch: develop
      branch: release
      baseBranch:          # leave blank or set e.g. "develop"
      testType: 'E2E - iOS & Android'
      enableBroadcast: 'true'
      firebaseTestTarget: 'true'
      testSuiteFilter:  # leave blank or name of tests to run "swap"
      sendToXray:
      testExecAndroid: 'B2CQA-2461'
      testExeciOS:     'B2CQA-2462'
      device: 'nanoSP'

Detailed Field Reference

workflow branch

Use workflow from branch

branch

The branch which triggered this workflow, Git ref to check out. For manual PR runs use refs/pull/{PR_NUMBER}/merge to simulate the actual merge commit.

baseBranch

The "base" into which your branch will be merged for a PR‐style test. Leave blank to skip merging.

testType

Which platform(s) to target:

  • E2E - iOS
  • E2E - Android
  • E2E - iOS & Android

enableBroadcast

Toggles log/screenshot streaming to Detox Cloud or equivalent test‐reporting dashboard.

firebaseTestTarget

Name of the Firebase env to target ("prod" vs "stg").

testSuiteFilter

A substring or regex passed to Jest/Detox's --testNamePattern, e.g. addAccount only runs that suite.

sendToXray

When true, test results (pass/fail + artifacts) are posted to Xray (JIRA) for both platforms.

testExecAndroid / testExeciOS

Provide your Xray execution IDs so we can group Android/iOS results under the right test cycle.

device

The Speculos emulated Ledger device:

  • nanoS
  • nanoSP
  • blue
  • (etc.)

Project Structure

Most files for the tests are in the /e2e/mobile directory.

  • /bridge: This contains the code to setup a websocket which allows communication between the test process and the LLM app. This allows us to:

    • create different conditions for testing the app by setting the user data.
    • perform mock device actions.
    • do quick navigations around the app (useful for setting up tests).

  • /models: The models contain logic for interacting with elements on specific pages in the application. They roughly follow the Page Object Model that is standard in UI testing.

  • /userdata: This is the application data that will be used to start the session, it contains all the information regarding user settings, existing accounts, operations, balances, etc. It allows us to test different scenarios with independent configurations.

  • /specs: The test suites themselves. We make use of the helpers and combine the snippets from the flows to build the different test scenarios. Ideally we should be able to reuse parts from flows in the specs.

  • /jest.config.ts: Configuration for Detox. Contains settings like what the setup and teardown files are, how long the timeout is, what test runner to use, etc.

  • /helpers: Convenience methods for use in the models/tests to make writing tests easier.

  • /setup.ts: Run after the global setup. It starts the websocket bridge, sets up the emulators to be more consistent in the test run (for example sets the time to 12.00), and shuts down the websocket bridge. Any logic to be run before and after a test run would go here.

Other important files outside /e2e/mobile

  • e2e/mobile/detox.config.js: Contains the configurations for the emulators when running detox test and the build artifacts to be used when running detox build

  • .github/workflows/test-mobile.yml: The workflow file to kick off tests in the Github servers.

Development workflow

The workflow for adding new tests is similar to the desktop workflow. These are:

Step 1: Identify Elements

Detox has a simpler API than Playwright and other E2E test solutions like Appium and Webdriver. The easiest way to make elements usable by Detox is by adding a testId attribute to the element in the code. Make sure to put the testId at the lowest level possible in the DOM tree.

Ideally these are placed at development time so tests are easier to write in future.

For example:

<BottomDrawer
  testId="AddAccountsModal"
  isOpen={isOpened}
  onClose={onClose}
  title={t("portfolio.emptyState.addAccounts.addAccounts")}
>

Step 2: Create a page object

Page objects are methods that group together behaviours so that tests are more readable and correspond to actions users would take in the app.

To create them:

  • Use the existing helper methods in e2e/mobile/helpers for actions such as clicking, entering text...
  • Create a new .ts step file in the e2e/mobile/models directory. Make sure it is named logically.
  • Start creating methods using the following pattern:
import { getElementByText, tapByElement, /* ... */ } from "path/to/helpers";

class MyPageObjectModel {
  getSomeItemByText = () => getElementByText("Set up my Ledger");
  getSomeItemById = () => getElementById("continue");
}

async chooseToSetupLedger() {
  await tapByElement(this.getSomeItemByText());
  await tapByElement(this.getSomeItemById());
}

Step 3: Create a test file

Test files go in the e2e/mobile/specs directory. Import the relevant page object model files and follow the example to create new tests:

import { expect, waitFor /* ... */ } from "detox";
import OnboardingSteps from "../models/onboarding/onboardingSteps";
import PortfolioPage from "../models/portfolioPage";

let onboardingSteps: OnboardingSteps;
let portfolioPage: PortfolioPage;
describe("Onboarding", () => {
  beforeAll(async () => {
    // Load some configs and setup your pages here
    await loadConfig("1AccountBTC1AccountETH", true);
    onboardingSteps = new OnboardingSteps();
    onboardingSteps = new PortfolioPage();
  })

  it("onboarding step should be visible", async () => {
     // test assertions
    await expect(onboardingSteps.getSomeElement()).toBeVisible();
  });

  it("should be able to start onboarding", async () => {
    // test actions (tap on some element)
    await onboardingSteps.startOnboarding();
  });

  it("should do some other stuffs", async () => {
    await onboardingSteps.DoIOwnDevice(true);
    // ...
  })

Tools and Features

Coming soon... 🚧

Trace Viewer

Detox Recorder

Tips

Animations

Detox synchronization sometime can't handle well animations, especially looping ones. You could disable either the blocking animation while you are in MOCK env (preferred) or disable the synchronization by wrapping your test code between these lines :

await device.disableSynchronization();
...
await device.enableSynchronization();

https://wix.github.io/Detox/docs/api/device#devicedisablesynchronization

You will have to wait manually (waitFor) to replace the synchronization. But be really careful about it, as it might make these tests unstable.


Clone this wiki locally