x.com/SchmidhuberAI/status/2103019743477424167
1 correction found
Compute continues to become 10 times cheaper every 5 years.
Recent empirical studies do not show compute getting 10× cheaper every 5 years. Estimates for modern GPUs are closer to a 10× improvement in about 7 years, and some analyses find the past decade has been slower still.
Full reasoning
The claim says compute "continues" to get 10× cheaper every 5 years, which implies a very fast ongoing price-performance trend.
That is not what recent empirical analyses find:
- Epoch AI's 2023 hardware-trends study reports computational price-performance improving at 2× every 2.1 years for ML GPUs and 2× every 2.5 years for general GPUs. In the same table, it translates that to 10× every 7 years for ML hardware, not 5 years.
- Georgetown CSET's AI and Compute report says that while long-run historical averages can look faster, the past decade has been slower: compute per dollar has doubled only every 3 to 5 years over the past decade. A 3-to-5-year doubling time is far slower than a 10× drop in cost every 5 years.
- That same CSET report also notes a concrete recent example: Nvidia's A100 (2020) offered only about 5% more FLOPS per dollar than the V100 (2017) on Google Cloud, which is the opposite of a continuing 10×/5-year trend.
So the post overstates the current rate of compute-cost improvement. A more defensible summary would be that compute price-performance has improved substantially over time, but recent evidence does not support a blanket claim that it is still getting 10× cheaper every 5 years.
2 sources
- Trends in machine learning hardware | Epoch AI
Key hardware performance metrics ... include: computational price-performance [FLOP per $] doubling every 2.1 years for ML GPUs and 2.5 years for general GPUs ... Computational price-performance ... 10x every 7 [5; 9] years.
- AI and Compute: How Much Longer Can Computing Power Drive Artificial Intelligence Progress? | CSET
The price of computations in gigaFLOPS has not decreased since 2017 ... A100 GPU, released in 2020—[offers] only five percent more FLOPS per dollar than the V100 ... Some sources ... suggest that the amount of compute that could be purchased for a dollar ... has doubled roughly every 2.3 years on average since the 1940s, with a slower doubling rate of between three and five years over the past decade.