Low precision arithmetic refers to using fewer bits than usual for representing numbers in mathematical operations and computations. In AI, this is useful for saving memory and increasing processing speed, albeit at the cost of some precision. For example, using 16-bit instead of 32-bit representation can speed up model training and deployment without significantly compromising performance. It's contrasted by high precision arithmetic, used when exact calculations are crucial.