The Fragility of Record/Replay Tests

The convenience of using record/replay web testing tools is often defeated by the brittleness of the tests they produce.  Mouna Hammoudi’s paper “Why Do Record/Replay Tests of Web Applications Break?” explores why these tests break so easily as the application under test changes.  Toffee overcomes many of these issues with record/replay testing.

Hammoudi used a record/replay tool called Selenium IDE, which allows a user to record themselves using the application under test and replay their actions at a later time to test functionality.  After creating a test for each of 300 releases across 5 web applications, Hammoudi found that roughly 73% of all breaks in her Record/Replay tests were due to locators.

Hammoudi explains that locator breaks occur when either an HTML element’s attributes or the HTML structure has changed.  In our experience, tests that use structure based locators such as XPath are especially vulnerable: as web applications are updated the structure changes regularly and even small changes can invalidate structural locators.

Toffee offers two solutions that make Toffee tests more robust and less expensive to maintain compared to record/replay tests.  First, Toffee makes it easy to write test steps using attribute based locators like id, name, label, and others.  These locators tend to be more stable because they are not affected by common structural changes. Instead, developers have to intentionally change the attribute value.

Second, Toffee provides users with the ability to create aliases.  Aliases are custom commands which are defined by the the tester and refer to other Toffee commands.  For example, the command “click button with id button-logout-user” can be aliased as “logout”.  Then the “logout” alias can be used throughout your tests, instead of the original command.  This minimizes the necessary repair after the button’s id changes because only the “logout” alias must be updated.

The drawback of record/replay tests is their brittleness.  Toffee tests are more robust and reusable without sacrificing the accessibility of record/replay tools.

 

For more information about Toffee, visit http://toffeetesting.io, or feel free to contact us with questions at info@toffeetesting.io.

Categories: Automated Functional Testing, Quality Assurance, Toffee
Trackback URL for this post: https://www.ksmpartners.com/2017/04/the-fragility-of-recordreplay-tests/trackback/

Leave a Reply

Your email address will not be published. Required fields are marked *