DRAM berdaya rendah Micron 'mengonsumsi sekitar sepertiga daya DDR5' dalam beban kerja pusat data AI, tapi saya tidak yakin itu kabar baik

AI's hunger for both power and memory, plus the resulting supply crisis of the latter, has led to many industry players engaging in a spot of creative problem solving. Case in point, Micron and Meta have collaborated on an experimental data centre roll out of low power DRAM.

Micron has developed SOCAMM2, a fresh form factor for LPDDR5X that is a "modular, serviceable module designed for servers." Testing used "Meta’s open-source DCPerf benchmark suite, which reflects production environments across a hyperscale fleet," and found that this low-power DRAM module "consumes approximately one-third the power of DDR5." According to Micron's white paper report, that accounts for under 7% of total system power.

The white paper goes on to elaborate that LPDDR5X's "reduced operating voltage and advanced signalling allows bandwidth to scale without a corresponding rise in power consumption, resulting in better bandwidth-per-watt performance than conventional server memory." Basically, data centres can install more low power DRAM without also hiking up power-related operating costs.

The AI industry potentially switching away from DDR5 may sound like good news in the short term for memory-starved PC gamers, but the full picture is a little more complex. For one thing, ramping up production of low power DRAM will likely divert already stretched resources away from DDR5 manufacturing, leaving PC gamers in largely the same position. For another, low power DRAM tends to be used in mobile devices like phones and tablets, so consumers could still feel the pinch there.

It's worth noting we have seen low power DRAM in at least one laptop, specifically in the form of LPCAMM2 memory modules in the Framework 13 Pro at Computex. However, the memory supply crisis likely means we won't see similar low power modules in many more laptops for the foreseeable.

Speaking of the memory shortage, there's little incentive for the AI industry to do anything other than use these developments in low power DRAM to continue to scale up, not slow down. Especially as Micron's white paper found that installing more low power DRAM modules "eliminated disk spilling in [the benchmark], delivering a dramatic speedup."

I also find it hard to not feel a smidge cynical about any power efficiency news coming out of the AI sector.

Previously, Google claimed Gemini sips a mere 'five drops' of water per text prompt, and OpenAI's Sam Altman said that a single ChatGPT query uses ‘one 15th of a teaspoon’ of water—but these are not meaningful metrics when it comes to, say, the industry's overall water consumption.

Similarly, there's no ignoring just how much power major players use, with analysts claiming Google's electricity consumption 'flipping from linear to exponential growth' over the last two years. So low-power DRAM for AI workloads is all well and good, but what will that mean in terms of environmental impact, and for the climate? Not to mention whether I'll be able to find two sticks of memory to rub together for a reasonable price in the near future.