ResearchDen
Join us
← Back to blog
TechnicalAugust 11, 2026·5 min read

Data to Power: AI, the Invisible Compute Economy, and the Rise of Silicon and Energy Systems

A token is not a unit of text. It is a unit of work, and following one far enough down leads to a semiconductor fab and then to a power grid.

Miraj Uddin Chowdhury
Miraj Uddin Chowdhury
Research Associate · ResearchDen

Judging modern AI models and chipsets on algorithmic performance alone gives an incomplete picture. Recent work on AI integration and LLM deployment points at a plainer reality: the future of the field is no longer bound only to software design. It is tied to physical hardware and to energy constraints.

What follows traces how a single abstraction, the token, connects directly to global semiconductor supply chains and to power grid economics.

1. What a token is, and why APIs charge for one

When text is sent to a large language model API, the model does not process the raw string. The text is first segmented into numerical fragments called tokens.

The inference pipeline runs from text, to tokens, to embeddings or representations, to transformer computation, to output. The token step is highlighted and marked as the point that is counted and billed, described as a unit of work rather than a unit of text.
Figure 1. The token is the only step in the chain that is both countable and proportional to the compute behind it.

Why the token serves as the billing unit

A token is not a physical commodity. It is a measurable unit of computational workload.

  • Hardware utilisation. Processing each token during inference requires executing millions of matrix multiplications (A × B) across specialised hardware: GPUs, TPUs, or dedicated AI accelerators [1].
  • Memory bandwidth and overhead. As token volume rises, high bandwidth memory (HBM) access, networking overhead, and latency requirements scale non-linearly.
  • Capital and operating cost. Data centres, high-end server racks, gigawatt-scale electricity, and liquid cooling all have to be paid for somehow. Token counts are the usage metric that tracks them [3].

2. From an AI economy to a chip economy

As models become ubiquitous, the computational work required for both inference and training scales exponentially.

Modern AI infrastructure rests on integrated circuits holding tens of billions of transistors. The consequence is a rapid shift toward what is worth calling a "chip economy":

  1. Semiconductor geopolitics. The dominant entities in manufacturing, design, and lithography, meaning TSMC, NVIDIA, and ASML, have become central to global economic stability [2].
  2. Hardware as the binding constraint. Algorithmic progress is frequently limited not by software design but by hardware availability and silicon supply bottlenecks.

3. From silicon to the grid

Running high-density compute demands unprecedented power and advanced thermal management, enough that international energy authorities have revised their global forecasts.

A chain running from AI systems to tokens, compute, chips, electricity, cooling, and grid capacity. The first half is bracketed as the chip economy and the second half as the energy economy, with chips belonging to both. A note records a forecast of roughly 1,000 terawatt hours of data centre demand, close to Japan's annual consumption.
Figure 2. Chips belong to both halves of the chain, which is why a semiconductor shortage and a grid constraint look like the same problem from far enough back.
  • Grid strain. The International Energy Agency projects global data centre power consumption exceeding 1,000 terawatt-hours, roughly the total annual electricity consumption of Japan [4].
  • How that power gets supplied. Capital going into small modular reactors, nuclear, solar arrays, and grid storage is increasingly driven by the need to guarantee uninterruptible supply to AI hyperscalers.
  • Thermal management. Traditional air cooling is giving way to direct-to-chip liquid cooling and immersion cooling, out of physical necessity rather than preference.

4. Where the economic value has moved

The primary driver of value in technology has shifted distinctly across recent decades.

EraFocusWhere the value sat
2000 to 2015Software and dataApplications, cloud platforms, and data aggregation
2016 to nowAI and agentic computeAgent orchestration, real-time inference, compute capacity
AheadChips and energy infrastructureFabrication, grid density, capital-intensive clean energy

5. Where to build depth over the next five years

This shift suggests that specialisation should extend past high-level application code and API wrappers. Long-term value will increasingly sit at the intersection of software efficiency, physical systems, and hardware architecture. Four areas look strategically relevant:

  1. Hardware-aware software and optimisation. High-performance computing, CUDA and Triton kernel work, heterogeneous architectures, and quantisation schemes such as AWQ and FlashAttention.
  2. Semiconductor and systems engineering. VLSI design, advanced packaging, silicon photonics, and microarchitectural efficiency.
  3. Green compute and energy systems. Data centre power engineering, direct-to-chip liquid cooling, smart grid interconnection, and thermal management.
  4. AI infrastructure engineering. High-throughput orchestration, MLOps, low-latency distributed systems, and specialised vector and relational database engines.

Conclusion

Reading the trajectory of the field means looking at the whole system: AI, compute, chips, and energy together. The transformation of an abstract digital token into tangible demand for silicon fabrication and generating capacity shows how tightly software paradigms are tethered to physical resource constraints. The abstraction is only ever an abstraction at the top of the stack.

References

  1. Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems (NeurIPS). arXiv:1706.03762
  2. Miller, C. (2022). Chip War: The Fight for the World's Most Critical Technology. Scribner. bloomsbury.com
  3. Sheng, Y., Zheng, L., Yuan, B., et al. (2023). FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU. Proceedings of the 40th International Conference on Machine Learning (ICML), PMLR 202, 31094-31116. arXiv:2303.06865
  4. International Energy Agency (2024). Electricity 2024: Analysis and Forecast to 2026. IEA, Paris. iea.org

Want to share your own experience? Every member can write here: reach out and we'll help you publish your first post.