activities
latest
false
UI Automation Activities
Last updated Oct 28, 2024

v23.4

v23.4.11

Release date: April 3, 2024

Improvements

For traceability and debugging speed purposes for Computer Vision runtime analysis, DesignTime ScreenId is now sent in the header of the HTTP request.

Bug fixes

  • The Appeared event type from the Application Event Trigger activity could not be monitored inside iframe HTML elements.

v23.4.10

Release date: October 26, 2023

Improvements

For this release, we brought minor improvements and bug fixes to the activity package.

v23.4.9

Release date: October 5, 2023

Bug fixes

  • Fixed the broken Active Accessibility support for web pages containing IFrame elements or PDFs in Chrome and Edge version 117, which caused <ctrl /> selectors to fail.
    Also added the option to pass extra command-line arguments to the browser opened using the Open Browser / Use Browser activities. This can be done by setting the appropriate environment variable using the Set Environment Variable activity:
    • UIPATH_EXTRA_CMD_ARGS_CHROME
    • UIPATH_EXTRA_CMD_ARGS_EDGE
    • UIPATH_EXTRA_CMD_ARGS_FIREFOX

    These variables can be used to force-enable the Active Accessibility support in Chrome and Edge to avoid issues like the broken Chrome/Edge Active Accessibility support from versions 114 and 117.

    For Chrome, set the environment variable UIPATH_EXTRA_CMD_ARGS_CHROME with the value --force-renderer-accessibility=complete.
    For Edge, set the environment variable UIPATH_EXTRA_CMD_ARGS_EDGE with the value --force-renderer-accessibility=complete.

v23.4.8

Release date: August 2, 2023

Bug fixes

  • Fixed a bug for the Send Hotkey activity that caused the .xaml file to be shown as unsaved.
  • Fixed a bug for the Use Application/Browser activity that threw an exception at design time in StudioX due to the asterisk indicating that a property field is required.
  • We added support for the CTable component of the Clarity Java application.

v23.4.7

Release date: June 7, 2023

Bug fixes

  • The latest Chrome and Edge updates to version 114 introduced a breaking change to the way support for Active Accessibility is enabled. This caused automation processes to fail when trying to interact with browser screens such as the Print pop-up or PDF viewer.

v23.4.6

Release date: 7 June 2023

Improvements

This package version includes various accessibility improvements.

v23.4.5

Release date: April 27, 2023

New features and improvements

UI Automation Browser Connections can now be used in Cross-platform projects.

You can create and edit a connection only in Studio Web (currently in preview) and consume it both in Studio Web and Studio Desktop.

The feature can be run only with Serverless Automation Cloud™ Robots; attempting to run the feature with local robots results in failure.

For further information, please refer to the dedicated UI Automation Browser Connection documentation page.

v23.4.4

Release date: April 27, 2023

General improvements

To decrease the package size for runtime, especially for cross-platform projects, the UiPath.UIAutomation.Activities package was split into design time package and runtime packages, as follows:
  • UiPath.UIAutomation.Activities - design time package
  • UiPath.UIAutomation.Activities.Runtime - runtime package; targets Windows and Windows-Legacy projects
  • UiPath.UIAutomation.Activities.Runtime.Portable - runtime for portable package; targets Cross-platform projects
Two new options are available in UI Automation Modern and UI Automation Classic Project Settings. Open browser timeout (seconds) allows you to configure the browser startup timeout, and Inject Js Script – Execution world applies the InjectJs.ExecutionWorld setting globally. The Open browser timeout value can also be configured through the UIPATH_OPEN_BROWSER_TIMEOUT_SECONDS environment variable.

You can now use the Open Application and Use Application/Browser activities to open the Power BI Desktop application.

We added Check support for more browser checkboxes.

We improved the Select Item activity to support complex HTML dropdowns, like non-native and non-Sap Fiori/Salesforce dropdowns.

The error prompt that indicates a version mismatch between the UiPath Remote Runtime component and the UiPath.UIAutomation.Activities package now contains a download link for the correct UiPathRemoteRuntime.msi version.

The UI Explorer selector no longer includes the hidden root for tree nodes.

UI Explorer no longer restores Studio window after indicating an element.

The Modifiers field for event data is now mapped to the NKeyModifier enum and returns a string (such as Shift, Alt, or Ctrl) instead of an integer when the toString() method is called.

If a Modern activity is used in a project, this information is now included in the published package, and is then available in Orchestrator. This feature requires Studio 2023.4 or later.

Browser extensions

We added the webRequest permission to the Chrome and Edge extensions to allow using the webRequest API. The Chrome and Edge extensions v2023.4 were published on the Web Stores with new extension IDs. The self-hosted extensions, used for Group Policy Offline installations, have also been updated with this permission, but they keep the same extension ID.

When installing extensions using the Native Host Only method, the file for the self-hosted extension is also generated, so that it can be added to the ExtensionInstallForceList policy.

Improved support for Chromium-based embedded browsers

We improved support for Chromium-based embedded browsers (Electron, CEF 103+) and added support for orphaned WebView2 windows.

We added support for Chromium Native versions 108, 109, and 110.

Cross-origin call forwarding for Chromium Native embedded browsers is now supported. This applies to CEF-native and Electron web controls.

We added support to automate CefSharp widgets embedded in applications with multiple UI threads.

We improved support for Electron applications that create multiple BrowserViews inside a single BrowserWindow. Each view can now be automated.

Trigger-based attended automation

In the context of the trigger-based attended automation initiative, where execution performance is critical, we improved event monitoring usability and performance by implementing new sets of events supported by different types of application technologies: Active Accessibility with <wnd> and <ctrl> tags, Web with <webctrl> tags, and Java with <java> tags. The events are available through the new Application Event Trigger activity, which helps set up a trigger based on a native event for any type of UI element.

Also in the context of trigger-based attended automation, we have migrated the most used Classic triggers to the Modern framework. The new Click Event Trigger and Keypress Event Trigger modern activities are designed to identify the elements via a unified target concept, use an Object Repository element and offer a friendlier user interface.

An important reason we provide them as part of the trigger-based attended automation solution is that these triggers can block the original event and replay it later, or simply ignore it. This specific capability allows implementation scenarios like a form submission being stopped until all the fields are correctly filled in.

The old classic activities Click Trigger and Key Press Trigger are still visible in this version of the package if the Classic filter is added in Activities tab.

For all trigger activities (Application Event Trigger, Click Event Trigger and Keypress Event Trigger) we have the following behavior:
  • When the activity is inside of a Trigger Scope, the activity doesn’t display the Scheduling mode and Enabled properties. In this case, the Scheduling mode property is available in the Trigger Scope activity.
  • When the activity is added without a Trigger Scope, the Trigger Scope is automatically generated and the Scheduling mode and Enabled properties are displayed for the trigger activity.
Some existing classic trigger activities can be used both within a triggered workflow (new model) or within a Trigger Scope activity (old model):
  • Hotkey Trigger
  • Mouse Trigger
  • Click Image Trigger
  • System Trigger
  • SAP Session Attributes Changed Trigger
  • Element State Change Trigger
  • Element Attribute Change Trigger

The Element State Change Trigger and Element Attribute Change Trigger activities, however, are now hidden from the activity package. If they are found in the workflow, they can be used, but they can no longer be added to the canvas.

The Set Text activity just got a modern version which can identify targets with the unified target concept and it can use an Object Repository stored element. Moreover, it can cache the target element to increase the execution speed when executing repeatedly, for example, when synchronizing text between multiple UI elements.

Native WPF support

WPF applications can be automated through Active Accessibility, which works out of the box unless accessibility support is disabled for that application. For example, accessibility support can be lost following an application update, in which case you might not be able to target individual elements in specific areas.

UiPath.UIAutomation.Activities v2023.4 provides access to UI Elements using native WPF API, instead of the generic Accessibility API currently used. This capability aims to reliably automate WPF applications using selectors with <wpf> attributes, resilient to change.

This feature is available through a new Project Setting: UI Automation Modern > Targeting methods - Desktop applications > Generate native WPF selectors (Preview). The default value for this option is False.

In v2023.4 this feature is under the Preview flag, as future versions may introduce breaking changes.

UI Automation Modern

The Inject JS Script activities, both Classic and Modern, have a new Execution world property field, which allows you to set the JavaScript world for a script to execute within. The Isolated option is for the activity's previous functionality, allowing the content script to make changes to its JavaScript environment without conflicting with the page. The new Page option allows the script to interact with the page as well.

Another modern activity that was added to the activity package is Check Element. You can now check the state of enabled or disabled elements in a curated way and perform actions based on their state, for example, when creating an automation that navigates a multi-page application to scrape data and you reach the last page, the next page button gets disabled and you need to stop scraping.

Modern activities require an application card/scope to minimize the searching area of the target element, thus, increasing the automation reliability. However, now no application card/scope is required when the activities receive an UiElement as input. So, all Modern activities have been updated accordingly.

Added support for writing a list of elements when using the Get Attribute activity in StudioX and saving it to Excel.

Improved matching for the Image selector by using the scale factor ratio (design time over runtime).

Improved the return of object types for the Get Attribute activity based on the selected attributes.

Added a red indicator for duplicate column names in the Table Extraction recorder.

SAP

We added support for reliable automation of SAP WebGUI using hardware events, including accurate identification of elements, stable attributes prefixed with sapweb-ls (for LightSpeed controls) or sapweb- (for SAP Scripting controls), and the ability to reliably use UI Automation activities with Hardware Events (Click, Type Into, Select Item, etc.). In this release, we added specific support for the following elements: SAP buttons (icons, push buttons, and radio buttons), SAP input fields, SAP dropdown lists, and SAP checkboxes.

We added specific support for SAP Fiori Calendars. Now, with the Select Dates in Calendar activity, you can easily automate calendars for all SAP Fiori-based systems, such as SAP S/4HANA or SAP SuccessFactors.

We added support for SAP Fiori Process Flow controls, including the ability to extract the status of these elements. We also introduced the concept of complex attributes, which have a ui5c- prefix and are represented in the JSON format. Note that the content of the new attributes is persistent, but the strings aren't reliable. For example, you could have {“a”:”value1”, “b”:”value2”} for the first call, and {“b”:”value2”, “a”:”value1”} for the next call.

Computer Vision

Computer Vision is now part of the Unified Target solution to help you reduce the complexity of building UI-based automations when you need both selectors and Computer Vision descriptors.

Adding Computer Vision to Unified Target increases the reliability and resilience to changes in UI-based automation: often, you do not control application UI changes; the UiPath robots must deal with these situations out of the box and not break, remaining uninfluenced by the UI deterministic changes.

Computer Vision can be a deep drill for targeting UI elements in the rare cases where the driver fails to ‘see’ certain UI elements or sometimes it can be the only viable technology like in the case of VDI automation. Other use cases of using Computer Vision span everywhere between these mentioned scenarios.

In terms of portability, any automation indicated in Native Windows can be reused in Remote Desktop environments by replacing the window selector.

In Project Settings > UI Automation Modern, the happy path for Computer Vision is to be enabled by default for Desktop applications, Java, and Web browsers and disabled by default for SAP as the happy path for SAP is using the strict selectors.

Computer Vision can be enabled as well from the Selection Options Helper new vertical bar design and it also has a keyboard shortcut (F8) to do so.

The CV Refresh activity is hidden from the package and can no longer be used in new projects.

Known issues

If the Enable WebView2 Native Automation setting (from Project Settings > UI Automation Classic > Browser) is set to True, web selectors are generated by default for trigger activities. Because the UI Automation trigger activities don't currently support embedded browser apps, the events aren't triggered if default generated selectors (web selectors) are used. As a workaround, AA selectors can be used by changing the UI Framework from Default to Active Accessibility while indicating the target UI Element. Support for embedded browser apps for UI Automation trigger activities will be added in a future version.

A Java Runtime Environment limitation causes unexpected modifiers behavior for Click and Mouse released events triggered for the middle and right buttons when used from the Application Event Trigger monitoring activity, as follows: an action with the middle button, without modifiers, reports an alt modifier; however, the same action executed with the middle button and an altmodifier reports correctly. An action with the right button, without modifiers, reports a win/meta modifier; the same action executed with the right button and a win/meta modifier reports correctly.
SendWindowMessages for WPF native is not currently supported.

Bug fixes

  • You can now automate Acrobat Reader with version newer than 2022.003.20258 even if you have the Enable Protected Mode option enabled by default. For details, see Security Option in Adobe Reader DC.
  • The Click activity failed on specific form elements when used with the Simulate input method. Now you can use the Enable workaround for simulate click with MV3 extension option in Project Settings.
  • With some websites, selector validation failed in Edge IE Mode. This issue occurred when a web page URL opened in Edge IE Mode was redirected to another URL compared to the one in Edge Chromium.
  • Browser automation activities intermittently failed to communicate with the Chrome extension, causing the following error: "Cannot communicate with the browser, please check the UiPath extension." This issue occurred on slower machines or machines with a high CPU load. To overcome this, the Chromium browser communication timeout was increased to 30 seconds.
  • Fixed a bug that prevented you from indicating tab elements in the Internet Explorer Developer Tools window.
  • With the UIAutomation Classic > Browser > Enable WebView2 Native Automation Project Setting set to True, in certain SAP Fiori pages opened inside SAP Business Client, WND and CTRL selectors were generated instead of HTML and WEBCTRL.
  • Fixed a bug that caused selector generation to fail for Remote Desktop when the Remote Desktop Client was published as a RemoteApp.
  • Fixed a bug that caused the UiPath Executor to crash when using a Get Visible Text activity on a Java application.
  • For Java Runtime Environment versions 13 to 19, indicating a table cell when the cell editor is opened at that cell now returns a valid element, with the role corresponding to the editor (such as text). Before, the resulting element was unusable.
  • Fixed an issue which caused the Click Trigger activity to slow down the target application when opening a web page in Edge IE mode with Protected Mode turned off.
  • Fixed an issue with Table Extraction which prevented extracting values from an input field. Before the fix, the default text attribute generated in the metadata used the innerText HTML property to retrieve the text. Starting with v2023.4, newly generated workflows and activities use fulltext as the default attribute. This new attribute uses the same approach as the full-text scraping method. To use the innerText property in new activities, you can manually edit the metadata and change the attribute name from fulltext to text.
  • Fixed a bug in Data Scraping that was causing the Join data table wizard to fail with ‘column not found’ error message due to column index being incorrectly marked as column name.
  • Fixed a bug that was causing, in some cases, the CV Extract Table activity to stop before the first scroll event when extracting scrollable tables.
  • Fixed a bug for the Click activity that was causing the activity to get stuck on a specific element when Input mode was set on Simulate. To solve the problem, the new UnblockInput property has been added to the classic Click activity, which should be used when Click triggers a modal dialog box or other blocking UI.
  • Fixed a bug for Modern activities that was causing them not to work properly when culture info interpreted comma as decimal separator.
  • Fixed a bug for the Attach Browser activity that was generating a memory leak when using a Browser variable.
  • Fixed a bug for the Find OCR Text Position activity that was causing the activity to find an incorrect position when the text includes a special character, such as %.
  • Fixed a bug in Object Repository that was causing the Design Data attribute not to be updated correctly when editing a target.
  • Fixed a bug for the Type Into, Keyboard Shortcuts, and Mouse Scroll activities that was throwing an invalid warning stating Click Offset is not supported on Simultate Input mode.
  • Fixed a bug in CV Screen Scope that was causing the local server to keep returning OCR boxes even when UiPath Screen OCR was removed from the activity card or from Project Settings.

Was this page helpful?

Support and Services
Get The Help You Need
UiPath Academy
Learning RPA - Automation Courses
UiPath Forum
UiPath Community Forum
Uipath Logo White
Trust and Security
Terms of Use
Privacy Policy
Cookies Policy
© 2005-2024 UiPath. All rights reserved.