AI is driving consumer and business shortages

The rise of artificial intelligence (AI) has created a host of new challenges for the ICT sector, including its enormous power requirements and - in water-scarce South Africa particularly - its data centres’ vast need for water for cooling.

One challenge, however, that may not have been quite as prominent, is the fact that many manufacturers are prioritising high-margin AI contracts. Since AI infrastructure is aggressively consuming the global supply of specialised components, particularly High-Bandwidth Memory (HBM), GPUs, and advanced packaging, this has led to shortages for laptops, smartphones, and traditional enterprise servers.

These ultra-high-value agreements between major public cloud providers - such as Microsoft or Oracle - and AI firms like Anthropic and OpenAI, have overwhelmed the supply chain for many types of hardware globally. The significant drop in the supply of consumer and enterprise hardware is largely due to massive demand for infrastructure used in AI training.

Demand for memory chips, built into the hardware and required for storage of the large data sets used for training and inferencing, is the chief culprit. These chips are utilised, very densely, in High Bandwidth Memory (HBM) for graphics processing units (GPUs) and Tensor Processing Units (TPUs) - the latter are specialised application-specific integrated circuits developed specifically to accelerate machine learning (ML) workloads.

These chips are equally vital for consumer electronics, and the shortage is further compounded by the fact that there are now only three major memory manufacturers in the world: SK Hynix, Micron and Samsung – and most of their existing and future inventory has already been set aside for AI projects.

The silicon underpinning memory chips is commonly used in consumer equipment such as laptops, and enterprise equipment such as servers, as dynamic random access memory (DRAM). It is also used for NAND (Not-AND) flash modules in solid state drives (SSDs), which have become the standard for storage over the last few years.

For consumers and businesses, one of the biggest challenges relating to these AI agreements is that a drop in the existing supply has the inevitable knock-on effect of significantly increased prices.

Here in South Africa, our market is so small that we essentially have no purchasing power, so price increases based on the US dollar - the currency that underpins all hardware and software purchases for cloud providers - hits us particularly hard. Worse still, consumers and businesses need to be aware that this is not a temporary ‘blip’ either, but in effect an ongoing structural change, which will not only continue to perpetuate but will, in all likelihood, escalate.

It is also not solely about the rising cost of hardware, but critically, also about the availability of inventory. Reports indicate that many original equipment manufacturers (OEMs), such as HPE and Lenovo, not only don't have stock available, but also do not know when they will again.

It is also worth noting that these price increases will not only affect the public when buying laptops and other IT equipment, but are also expected to affect both local and hyperscale public cloud providers with presence in South Africa.

Although cloud providers don't invest capital on a continuous basis, and certain providers may have sufficient capacity to avoid capital expenditure in the short-term, it is clear that the challenges already outlined are not going away anytime soon.

It thus goes without saying that if cloud customers have not yet experienced a price increase from their provider, they should expect one soon, and they should furthermore anticipate regular increases over the forthcoming period.

Unfortunately, even though South Africa has benefitted from a 20% improvement in the rate of exchange with the US dollar in recent months, hardware prices have increased in lockstep, due to structural global US dollar weakness. This means that there has only been a marginal benefit experienced.

So, where does that leave consumers and businesses that will require hardware refreshes in the next couple of years? evoila’s view is that we are all facing tough times, and - while there is little that we can do about it - these players can at least be better prepared for what is to come. To this end, our suggestion is for both consumers and businesses to be prepared for some cloud providers to restrict growth on their clouds, simply due to an inability to expand their infrastructure.

In addition, challenges around performance and reliability will intensify, as an increasing number of entities continue to leverage old equipment well beyond its supposed refresh date, and potentially even well past its useable life.

It also goes without saying that prices will continue to increase across the board, meaning that ultimately, only those with scale and growth and access to server inventory will be able to ameliorate these risks and reduce or eliminate price increases.

With our scale and mature financial models there is minimal market impact for customers. evoila intends to keep prices stable, buffering our client base and cloud platform from the  impact. Ends.

evoila Africa, previously known as Routed, is a division of evoila, a global company that is an integrated solutions provider specialising in the design, deployment and operating cloud Infrastructure for various markets. evoila Africa is a pinnacle Broadcom VMware Cloud Service Provider, specialising exclusively in platform engineering for secure, multi-tenanted, and private cloud solutions. The company partners with Cloud Service Providers and channel partners to deliver tailored VMware Cloud platforms that empower them to scale, innovate, and meet diverse customer needs. https://routed.co.za/

Samantha Hogg-Brandjes | GinjaNinja  | +27844584857 | samantha@ginjaninja.co.za