UI Automation

There are three methods UiPath uses for triggering a Click or a Type Into activity on an application. These are displayed as properties in all activities that deal with UI automation.

Sometimes the automatically generated selectors propose volatile attribute values to identify elements and manual intervention is required to calibrate the selectors. A reliable selector should successfully identify the same element every time in all conditions, in development, test and production environments and no matter the usernames logged on to the applications.

Here are some tips on how to improve a selector in the Selector Editor or UI Explorer:

In the selector above, we notice the page title has a reference to the time when the selector was recorded and also that some attributes have randomly looking IDs. Tweaking the attributes, we can come up with a better selector than UiPath recorder proposed.

Similar to file paths, selectors can be full or partial (relative). Full selectors start with a window or an HTML identifier and have all the necessary information to find an element on the whole desktop, while partial selectors work only inside an attach/container that specifies the top-level window where the elements belong:

Here is a bit more info on Full versus Partial Selectors.

There are several advantages to using containers with partial selectors instead of full selectors:

Image matching is sensitive to environment variations like desktop theme or screen resolution. When the application runs in Citrix, the resolution should be kept greater or equal than when recording the workflows. Otherwise, small image distortions can be compensated by slightly lowering the captured image Accuracy factor. Check how the application layout adjusts itself to different resolutions to ensure visual elements proximity, especially in the case of coordinate based techniques like relative click and relative scrape.

If the automation supports different resolutions, parallel recordings can be placed inside a PickBranch activity and the Robot uses either match.

If OCR returns good results for the application, text automation is a good alternative to minimize the environment influence. The Google Tesseract engine works better for smaller areas, while Microsoft MODI for larger ones.

Using the MODI engine in loop automations can sometimes create memory leaks. This is why it is recommended that scraping done with MODI be invoked via a separate workflow, using the Isolated property.

When executing web automation with Chrome/Edge/Firefox browsers in the child session of a PictureInPicture (PiP) mode and you want to continue using the browser in the main session, please know that both Chrome and Edge cannot be launched simultaneously, in different sessions, while using the same user data directory.

To bypass this limitation, the Modern UseApplicationBrowser and the Classic OpenBrowser activities provide an out-of-the-box solution: the UserDataFolderMode and UserDataFolderPath properties which instruct the activity on launching the browser with either its default folder, the one automatically generated by the activity, or a custom one, specified by the latter property.

Let’s take a closer look at how these properties work.