Floating point numbers in machine learning: Less is more for Intel, Nvidia and ARM

Retrieved on: 2022-09-16 07:03:59

Tags for this article:

Computer arithmetic

Binary arithmetic

Arithmetic

Computer architecture

Fields of mathematics

Bfloat16 floating-point format

IEEE 754

Half-precision floating-point format

Precision

Double-precision floating-point format

Single-precision floating-point format

Nan

Click the tags to see associated articles and topics

Floating point numbers in machine learning: Less is more for Intel, Nvidia and ARM. View article details on HISWAI: https://voonze.com/floating-point-numbers-in-machine-learning-less-is-more-for-intel-nvidia-and-arm/

Excerpt

It compares the use of 8-bit floating-point numbers (FP8) with that of FP16 values in the BERT and GPT-3 language and transformer models, ...

Article found on: voonze.com

View Original Article