ai scaling laws Just as there are widely understood empirical laws of nature — for example, what goes up must come down , or every action has an equal and opposite reaction — the field of AI was long defined by a single idea: that more compute, more training data and more parameters makes a better AI model. However, AI has since grown to need three distinct laws that describe how applying compute resources in different ways impacts model performance. Together, these AI scaling laws — pretraining scaling, post-training scaling and test-time scaling, also called long thinking — reflect how the field has evolved with techniques to use additional compute in a wide variety of increasingly complex AI use cases. The recent rise of test-time scaling — applying more […]
Original web page at blogs.nvidia.com
