- Getting Started
- Framework Components
- ML Packages
- Pipelines
- Data Manager
- OCR Services
- Document Understanding deployed in Automation Suite
- Document Understanding deployed in AI Center standalone
- Deep Learning
- Licensing
- References
- UiPath.Abbyy.Activities
- UiPath.AbbyyEmbedded.Activities
- UiPath.DocumentUnderstanding.ML.Activities
- UiPath.DocumentUnderstanding.OCR.LocalServer.Activities
- UiPath.IntelligentOCR.Activities
- UiPath.OCR.Activities
- UiPath.OCR.Contracts
- UiPath.DocumentProcessing.Contracts
- UiPath.OmniPage.Activities
- UiPath.PDF.Activities
About ML Packages
Using a Document Understanding ML Package involves these steps:
- Collect document samples and the requirements of the data points that need to be extracted.
- Label documents using Data
Manager.
Data Manager itself connects to an OCR Service.
- Download or export labeled documents as a Training dataset and upload that exported folder to AI Center Storage.
- Download or export labeled documents as an Evaluation dataset and upload that exported folder to AI Center Storage.
- Run a Training Pipeline on AI Center.
- Evaluate the model performance with an Evaluation Pipeline on AI Center.
- Deploy the trained model as an ML Skill in AI Center.
- Query the ML Skill from an RPA Workflow using the
UiPath.DocumentUnderstanding.ML activity package.
Note: Remember that using Document Understanding ML Packages requires that the machine on which AI Center is installed can access
https://du-metering.uipath.com.Important: When creating a UiPath.DocumentUnderstanding.ML.Activities Package in AI Center, the package name should not be any python reserved keyword, such asclass,break,from,finally,global,None, etc. Note that this list is not exhaustive since the package name is used forclass <pkg-name>andimport <pkg-name>.
These are out-of-the-box Machine Learning Models to classify and extract any commonly occurring data points from semi-structured or unstructured documents, including regular fields, table columns, and classification fields, in a template-less approach.
Out-of-the-box Machine Learning Packages that are delivered by UiPath have version 0 and are already available on your tenant, meaning that there is no need to download them.
Download is available only for versions 1 or higher, that were already trained by you.
Document Understanding contains multiple ML Packages split into six main categories:
This is a non-retrainable model which can be used with the UiPath Document OCR engine activity as part of the Digitize Document activity. To be used, the ML Skill must first be made public so that a URL can be copy-pasted into the UiPath Document OCR engine activity.
UiPathDocumentOCR requires access to the Document Understanding metering server at https://du.uipath.com/metering if the ML skill is running on an AI Center on-premises regular deployment. No internet access is needed on AI Center on-premises air-gapped deployments.
The UiPathDocumentOCR ML Package in AI Center is optimized for running on GPU, so we strongly recommend using it on GPU. If no GPU is available, we recommend using the standalone docker container for versions before 2021.10. Starting with 2021.10, the ML package can also be run in AI Center on-premises, but we advise having at least a 4-core CPU or ideally an 8-core CPU.
This ML Package can be deployed exactly the same way as the UiPathDocumentOCR ML Package, with the following differences:
- it is optimized to run on CPU, so you should see a 3-4x speedup when running in workflow, and 5-10x speedup when using it to import documents into Document Manager
- accuracy is slightly lower than the UiPathDocumentOCR ML Package, and it is similar to the UiPath.DocumentUnderstanding.OCR.LocalServer Studio package
- due to being faster, the CPU is also recommended when documents are large (over 20 pages per doc) in the absence of a GPU, which is ideal.
This is a generic, retrainable model for extracting any commonly occurring data points from any type of structured or semi-structured documents, building a model from scratch. This ML Package must be trained. If deployed without training first, deployment fails with an error stating that the model is not trained.
This is a generic, retrainable model for classifying any type of structured or semi-structured documents, building a model from scratch. This ML Package must be trained. If deployed without training first, deployment fails with an error stating that the model is not trained.
These are retrainable ML Packages that hold the knowledge of different Machine Learning Models.
They can be customized to extract additional fields or support additional languages using Pipeline runs. Using state-of-the-art transfer learning capabilities, this model can be retrained on additional labeled documents and tailored to specific usecases or expanded for additional Latin, Cyrillic or Greek language support.
The dataset used may have the same fields, a subset of the fields, or have additional fields. To benefit from the intelligence already contained in the pre-trained model, you need to use fields with the same names as in the out-of-the-box model itself.
These ML Packages are:
- Invoices: The fields extracted out-of-the-box can be found here.
- InvoicesAustralia: The fields extracted out-of-the-box can be found here.
- InvoicesIndia: The fields extracted out-of-the-box can be found here.
-
InvoicesJapan
Preview: The fields extracted out-of-the-box can be found here.Retraining using data from Validation Station is currently not supported.
-
InvoicesChina
Preview: The fields extracted out-of-the-box can be found here.Retraining using data from Validation Station is currently not supported.
- Receipts: The fields extracted out-of-the-box can be found here.
- Purchase Orders: The fields extracted out-of-the-box can be found here.
- Utility Bills
Preview: The fields extracted out-of-the-box can be found here. - ID Cards
Preview: The fields extracted out-of-the-box can be found here. - Passports
Preview: The fields extracted out-of-the-box can be found here. - RemittanceAdvices
Preview: The fields extracted out-of-the-box can be found here. - DeliveryNotes
Preview: The fields extracted out-of-the-box can be found here. - W2
Preview: The fields extracted out-of-the-box can be found here. - W9
Preview: The fields extracted out-of-the-box can be found here.
These models are deep learning architectures built by UiPath. A GPU can be used both at serving time and training time but is not mandatory. A GPU delivers>10x improvement in speed for Training in particular.
These are non-retrainable Packages that are required for non-ML components of the Document Understanding suite.
These ML Packages are:
- FormExtractor: Deploy as Public Skill and paste the URL into the Form Extractor activity.
- IntelligentFormExtractor: Deploy as Public Skill and paste the URL into the Intelligent Form Extractor activity. Make sure to first deploy the HandwritingRecognition ML Skill and configure that as OCR for the this package.
- IntelligentKeywordClassifier: Deploy as Public Skill and paste the URL into the Intelligent Keyword Classifier activity.
- HandwritingRecognition: Deploy as Public Skill and use as OCR when creating the IntelligentFormExtractor package.
