v23.10

We added support for the following SAP WebGUI elements: SAP Menu items (using the Select Item activity), SAP Toolbar buttons (using the Click Toolbar Button activity), and SAP Trees (including stable identification of the Trees and reliable automation across all browsers and all SAP FIORI versions). Note than you can use the existing capabilities to automate various other SAP WebGUI elements, such as ABAP lists, Context menus, Pictures, Textedit controls, Generic Object Services, Dialog boxes, and Table controls.

In this version of the activity package, we optimized core issues by accounting for each targeting method's relevancy in terms of targeting power and resilience to UI changes and we addressed the main limitations in the 23.4.x. As a result, UI Automation Table Extraction is now Computer Vision boosted when indicating full tables and table extraction is also enabled for pure Computer Vision scenarios; you can change the OCR engine as you can now select a custom OCR engine besides the embedded UiPath Screen OCR; Computer Vision is enabled by default in scopeless activities further boosting fully loaded targets (having both selectors and Computer Vision descriptors); and we optimized the experience of indicating UI elements by updating the reconciling logic between Driver and Computer Vision regarding bounding boxes. We have thus continued to pursue our vision of a unified UI Automation user experience where both pure Computer Vision and combined Computer Vision plus Driver powered automations can be created without the need for the user to be aware of the intricacies of each targeting method. More specific details can be found below.

A new hierarchical concept is now applied to Unified Target, taking into account each targeting method and its relevancy in terms of targeting power and resilience to UI changes. Hence, selectors are now considered the primary targeting method, Computer Vision is a secondary targeting method, and Image is a tertiary targeting method (and is now disabled by default). Before this, all targeting methods were trying to identify the target element simultaneously and whichever targeting method was able to find it first was used. Also, a new setting, Wait for primary targeting method until timeout, has been added in Project Settings > UI Automation Modern. The setting is set to True by default and allows the primary targeting method to consume its full timeout until any fallback targeting method can be used.

Computer Vision is now integrated in the Table Extraction wizard. You can now simply indicate full tables without having to indicate a relevant table cell first. Since Computer Vision is able to see entire tables, this functionality is now also used at design-time to correct faulty selector-based extraction when indicating a full table. At runtime, however, we don't have a Computer Vision fallback mechanism in place just yet when selector-based extraction has issues extracting full tables. You can extract pure Computer Vision tables as well without having to switch to a CV Screen Scope activity or a CV Extract Table activity for scenarios where selectors are not available at all, such as remote desktops without the remote run-time installed. On top of that, you can now also extract tables spanning across multiple pages in pure Computer Vision scenarios by indicating the Next button in the wizard.

Using the same logic as in the CV Extract Table activity, extracting scrollable tables in pure Computer Vision scenarios is now also possible in the Table Extraction wizard, as well as extracting scrollable text using CV Get Text. The Log Table Scroll Data project setting (Project Settings > Computer Vision > Robot Logging) has been renamed to Log Scrollable Content Extraction Data and it now covers Computer Vision Table Extraction, Table Extraction for scrollable tables in pure Computer Vision scenarios, and CV Get Text with scrollable text content.

In the Use Application/Browser activity, in addition to the Computer Vision embedded OCR engine, namely UiPath Screen OCR, you can now use a custom OCR engine to enhance the text extraction in Computer Vision when used in Unified Target. The OCR engine can be changed in each Use Application/Browser activity by going to the activity card > choosing the hamburger menu that appears next to the informative screenshot after indicating an UI element > selecting OCR Engine > Custom OCR Engine. When this option is selected, a panel is added in the activity card where you can drop a different OCR engine activity than the embedded one. This only affects the Computer Vision targets, as the rest of the OCR activities still use the default OCR set in Project Settings > OCR section.

As the happy path is having fully loaded targets, Computer Vision is now auto-enabled for indicating in scopeless activities (that can be used outside an Use Application/Browser scope) when it is also enabled in Project Settings.

When Computer Vision is enabled as a targeting method in Project Settings, the App/Web Recorder now captures Computer Vision descriptors as well.

We adapted our reconciling logic between Driver and Computer Vision regarding bounding boxes by matching the two targets to avoid indicating a Computer Vision-only target and to favor fully loaded targets. Selecting Computer Vision-only targets should now be reserved for scenarios like remote desktop with no access to selectors.

Strict selector failures are now displayed in the logs as warnings alongside ten suggestions of the closest selector matches that can be used to replace the faulty selectors. This feature is set to True by default in Project Settings > UI Automation Modern > Show closest matches in the 'Element not found' error message.

The Computer Vision screenshot no longer needs an initial delay, so now the start of an indicate session is faster.