- Release notes
- Overview
- Setup and configuration
- Software requirements
- Hardware requirements
- Deploying the server
- Connecting to the server
- Data storage
Hardware requirements
This setup works on on-premises Nvidia GPUs, but also works with cloud providers such as AWS, Azure and GCP. Suggested GPU types include those from the RTX, Tesla, and Ampere family of products which have enough GPU memory and processing capability.
The main difference between these two types of GPUs is that the ones with virtualization usually have more GPU RAM and are offered by most cloud providers. Having more GPU RAM increases the maximum size of the image you can input to the model. In conclusion, virtualization GPUs are not significantly faster that the consumer GPUs.
You need a machine with the following hardware specifications:
Hardware specification | Requirements |
---|---|
Memory |
|
CPU |
|
GPU |
|
Storage |
|
Depending on the configuration you use (recommended GPU: Nvidia T4), you can expect the following performances (processing time is measured in seconds):
Resolution | Inference time | Total service time |
---|---|---|
1280x720 |
0.367 |
0.388 |
1440x900 |
0.487 |
0.515 |
1600x900 |
0.503 |
0.533 |
1920x1080 |
0.562 |
0.598 |
1920x1200 |
0.636 |
0.675 |
2560x1440 |
0.832 |
0.884 |
3840x2160 |
1.484 |
1.581 |