Appetize.io Docs
HomeDemoUploadPricing
Search…
Getting started
Core features
Uploading apps
Playback options
Embed your app
App permissions
Cross-document messages
Custom launch pages
API
Overview
Create new app
Update existing app
Direct file uploads
Delete app
List apps
Usage summary
Devices and IPs
Sample code
Advanced features
Custom branding
Reserved devices
Single sign-on
ADB tunnel
App recorder (beta)
Overview
Record events
Playback events
Retrieving the user interface
Powered By GitBook
Playback events
To playback a recorded event, send a cross-document message with type field set to playEvent, and value set to a playback object. We will attempt to locate the element of the recorded event and repeat the action.
A playback object should have the fields:
  • event - the entire event object that was previously recorded
  • timeout - (default 10) the number of seconds to wait for a match of the recorded element
  • strictMatch - (default false) if true, element must match all recorded attributes exactly. Otherwise, we will attempt to find the best matching element
  • id - (optional) a string to identify this playback attempt, to help identify success or failure.
If playback is successful, the iFrame will post a cross-document message with type set to playbackFoundAndSent. Its value field will be an object with the fields:
  • id - the id string that was optionally provided to playEvent
  • playback - if id was not provided, the entire playback object
  • matchedElement - the element that matched the original recorded event
  • matchedOn - an array of attributes where the new element matched the recorded element
If the playback failed the iFrame will post a cross-document message with type set to playbackError. Its value field will be an object with the fields:
  • id - the id string that was optionally provided to playEvent
  • playback - if id was not provided, the entire playback object
  • errorId - a string that may be one of:
    • notFound - we timed out waiting for a matching element to appear
    • uiError - we were unable to retrieve the UI heirarchy for the app
    • unknown - unknown error
  • message - a string describing the error

Playback element matching

A central problem when playing back recorded events is to find the recorded element in the new UI during playback. Apps may differ between recording and playback because of changes to the content, design, device dimensions, or language.
When searching for correct element during playback, we attempt to find a unique match based on the rules listed below. Each rule defines the set of fields we will consider. If there are multiple elements that satisfy a rule, we will attempt to "break the tie" by considering defining characteristics of sibling and parent elements. If remain multiple matches or there are no matches, we move on to the next rule.
In scrollable views, the path can be misleading as the exact scroll position may differ between playback and recording. Therefore, path is removed from the rules if the element is inside of a scrollable view. (Not yet implemented on iOS, see Known Issues above.)
On Android, we match using these rules:
  1. 1.
    content-desc, text, path, class, resource-id
  2. 2.
    content-desc, path, class, resource-id
  3. 3.
    content-desc
  4. 4.
    text, class, resource-id
  5. 5.
    text, class
  6. 6.
    text
  7. 7.
    path, class, resource-id
  8. 8.
    path, class
  9. 9.
    path, resource-id
  10. 10.
    class, resource-id
  11. 11.
    class
On iOS, we match using these rules:
  1. 1.
    accessibilityIdentifier, text, path, class, baseClass
  2. 2.
    accessibilityIdentifier, text, class, baseClass
  3. 3.
    accessibilityIdentifier, path, class, baseClass
  4. 4.
    accessibilityIdentifier
  5. 5.
    text, path, class, baseClass
  6. 6.
    text, class, baseClass
  7. 7.
    text, class
  8. 8.
    path, class, baseClass
  9. 9.
    path, class
  10. 10.
    path, baseClass
  11. 11.
    class, baseClass
  12. 12.
    class
  13. 13.
    baseClass
App recorder (beta) - Previous
Record events
Next - App recorder (beta)
Retrieving the user interface
Last modified 1yr ago
Copy link
Contents
Playback element matching