Skip to content
This repository was archived by the owner on Oct 16, 2023. It is now read-only.

Commit cfb692a

Browse files
authored
Merge pull request #84 from hpcaitech/feature/trt
update readme
2 parents 5ca6d95 + db2231d commit cfb692a

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ Energon-AI provides 3 levels of abstraction for enabling the large-scale model i
1616
For models trained by [Colossal-AI](https://github.com/hpcaitech/ColossalAI), they can be seamlessly transferred to Energon-AI.
1717
For single-device models, they require manual coding works to introduce tensor parallelism and pipeline parallelism.
1818

19-
At present, we pre-build distributed Bert and GPT models.
19+
At present, we pre-build distributed Bert, GPT, and ViT models.
2020
For GPT, it extends to at most 175B parameters, which is called [GPT3](https://arxiv.org/abs/2005.14165).
2121
For Bert, Google reports a [super-large Bert with 481B parameters](https://mlcommons.org/en/training-normal-11/) in MLPerf-Training v1.1 open, indicating that Bert can also extend to large-scale.
2222

0 commit comments

Comments
 (0)