ROCm versus Metal Performance Tests
After installing Ubuntu 20.04.03 with ROCm 5.2.0 on my experimental stand with Mac OS Monterey 12.4 Metal API and Windows 11 with Adrenalin AMD GPU drivers I decided to do some performance Machine Learning tests and compare the same hardware versus different software platform.
Geekbench5 Results:
| Stand | Metal/Vulcan | Output | OpenGL | OpenGL Output |
|---|---|---|---|---|
| Monterey | 163223 Metal | output | 96870 | output |
| Ubuntu | 58474 Vulcan | output | 121388 | output |
| Windows 11 | 67802 Vulcan | output | 149357 | output |
| Ubuntu 6900 | 76636 Vulcan | output | 149107 | output |
Built on this hardware:
- Intel Core i9-10850K
- 64.00 GB DDR4 3200 MT/s
- AMD Radeon PRO W6800
- MSI 6900 XT
OpenGL graphics performance is incredibly fast on Windows 11. The next winner is Monterey OS and finally we have outsider Ubuntu with latest AMD drivers. For tests was installed only latest software available on current day.
Here is the list of 4 ML general purpose training tasks with the same frameworks:
- MLP Classifier: A 3 layer multilayer perceptron with ReLU activations, 2500 units in the intermediate layers and 10 output classes, trained on synthetic data. source
- ResNet50 CIFAR10: The standard ResNet50 computer vision model on the CIFAR10 dataset. source
- BERT IMDb: Implementation of the BERT natural language processing model from the Huggingface Transformers library trained on the IMDb dataset. source
- BERT Glue/Mrpc: Again BERT, but with the Glue/Mrpc dataset. source
For all tests the fastest performance per epoch was used for comparision. In Windows there is no Metal API or ROCm support, therefore no tests for Machine Learning are possible for now.
ML Results:
| Test | Metal | Metal Output | ROCm | ROCm Output | ROCm 6900 |
|---|---|---|---|---|---|
| MLP | 23s | output | 33s | output | 27s |
| ResNet50 | 14s | output | 17s | output | 12s |
| Bert | 87s | output | 58s | output | 44s |
| Bert-Mrpc | 90s | output | 63s | output | 49s |
Here we see that ROCm works signinicantly better than Metal for transformer-like models, whereas Metal outperfoms on CNN and ReLU models.
We do not have a clear winner between them, depending on your tasks you can switch to one or another.