loop-unroller https://loop-unroller.eu/ quarto-1.10.19 Sat, 10 Oct 2026 00:00:00 GMT From prompt to token: LLM inference on a GPU, and on a thermodynamic chip Victor Bieszka https://loop-unroller.eu/posts/2026-10-10-llm-inference-thermodynamic.html In my primer on thermodynamic processors I argued that generative AI is, at its core, a sampling problem, and that hardware which samples natively could do that job far more efficiently than a GPU. That is a nice slogan. In this post I want to make it concrete by following a single request, “What is the capital of France?”, through today’s hardware and then through a hypothetical thermodynamic one.

For the second half I’ll be generous: assume the hard problems from the primer (scaling to millions of cells, precise couplings, fast I/O, models that match transformer quality) have been solved. The question is not whether that happens, but what an LLM server would look like if it does.

Part 1: Serving an LLM today

The life of a request

A modern LLM server is a CPU and one or more GPUs working as a team. The CPU does the bookkeeping, the GPU does the maths.

sequenceDiagram
    participant U as Client
    participant C as CPU (API server)
    participant G as GPU
    U->>C: prompt (text)
    C->>C: tokenize, schedule into a batch
    C->>G: token IDs
    G->>G: prefill: all prompt tokens in parallel, fill KV cache
    loop until end-of-sequence or max tokens
        G->>G: decode: one forward pass → logits
        G->>G: sample the next token
        G-->>C: token ID
        C-->>U: detokenized text (streamed)
    end

One request through a typical CPU + GPU serving stack.

  1. Tokenization (CPU). The text is split into tokens, integer IDs from a vocabulary of typically 30k–250k entries. “What is the capital of France?” becomes about eight of them.
  2. Scheduling (CPU). The server does not run one request at a time. A scheduler continuously merges requests into batches and manages GPU memory for them. Serving engines like vLLM page the per-request state much like an operating system pages virtual memory (Kwon et al. 2023).
  3. Prefill (GPU). All prompt tokens go through the network at once. Every layer produces a key and a value vector per token, which are stored in the KV cache so they never have to be recomputed. Prefill determines the time to first token.
  4. Decode (GPU). Now the model generates, one token per forward pass. Each pass looks at the KV cache, appends one entry and produces a vector of logits, one score per vocabulary entry.
  5. Sampling (GPU). The logits are turned into a probability distribution, and one token is drawn from it. That token is fed back in as the input of the next decode step.
  6. Detokenization and streaming (CPU). Token IDs are turned back into text and streamed to the client.

Steps 3 and 4 run the same network, but they stress the hardware in completely different ways. To see why, we need a bit of maths.

Prefill is compute-bound, decode is memory-bound

A transformer with parameters needs roughly floating-point operations per token: one multiply and one add per weight. For a prompt of tokens, prefill therefore costs

and the weights only have to be read from memory once for all tokens.

Decode is different. Every step produces a single token per sequence, but it still has to stream all weights from GPU memory (HBM) into the compute units. With a batch of sequences and 16-bit weights (2 bytes each), one decode step does about FLOPs while moving bytes. The ratio of the two is the arithmetic intensity :

The roofline model (Williams et al. 2009) tells us what a chip can deliver at a given intensity: either it is limited by memory bandwidth or by peak compute ,

An Nvidia H100 SXM has TFLOP/s (dense BF16) and TB/s. The ridge point, where the two limits meet, is at FLOP/byte. Anything below that leaves compute units idle while they wait for data.

Figure 1: Roofline of an H100 (log-log). Decode at small batch sizes sits far left on the memory-bound slope; prefill of a long prompt reaches the compute roof.

A back-of-the-envelope example with an 8B-parameter model in BF16, i.e. ~16 GB of weights:

  • Prefill of a 2,000-token prompt: FLOPs, about 32 ms at peak compute.
  • Decode at batch size 1: every token requires reading 16 GB, so the GPU can produce at most 210 tokens/s, using well under 1% of its compute.

This is why serving is all about batching: a larger moves decode to the right on the roofline and amortises each weight read over more users (Pope et al. 2022). It is also why tricks like speculative decoding exist, which guess several tokens cheaply and verify them in one pass (Leviathan et al. 2023).

The KV cache makes things worse as contexts grow. Per token it stores

For a Llama-3-8B-style model ( layers, KV heads, , BF16) that is bytes, i.e. 128 KiB per token. An 8k-token conversation holds 1 GiB of cache, which has to be read on every decode step as well.

Where the energy goes

The punchline for energy: moving a value from DRAM costs on the order of a hundred times more energy than doing arithmetic on it (Horowitz 2014). Decode is dominated by exactly that: shuffling weights and KV cache from HBM to the compute units, over and over, once per generated token. The actual “creative” step, picking the next token, is almost free in comparison.

The sampling step is a Boltzmann distribution

Let’s look at that last step more closely, because this is where the thermodynamic story begins. Given logits , the model samples token with probability

where is the temperature knob you may know from LLM APIs. Compare that to the Boltzmann distribution from the primer, . They are the same formula, with energy . Every LLM already ends with a Boltzmann sampler, just one that is simulated in floating point on a GPU.

Figure 2: Next-token distribution for “The capital of France is …” with made-up logits. Low temperatures freeze the distribution onto the most likely token; high temperatures flatten it.

So can we just replace the sampler with a thermodynamic chip? We could, but it would be pointless. A categorical distribution over 128k entries is a tiny computation; the expensive part is everything before it, the forward pass that produces the logits. To gain anything, the thermodynamic hardware has to take over the heavy lifting, not just the last step.

Part 2: Serving the same request on a thermodynamic processor

From here on we enter speculation. Assume a mature thermodynamic sampling unit (TSU): millions of p-bits, programmable couplings with enough precision, fast enough I/O, and, crucially, a language model that was designed for it and matches today’s quality.

A different kind of model

A TSU does not execute a transformer. It samples from an energy function over binary variables of the form

where are per-cell biases, are coupling strengths, and is the chip’s (sparse, local) wiring graph. Each cell repeatedly updates itself given its neighbours. The probability of a cell landing on 1 is a sigmoid of its local field:

This is exactly the Gibbs sampling loop from the primer, done by physics instead of by arithmetic. Cells that aren’t neighbours update simultaneously, so one sweep over millions of cells takes about as long as updating one.

A language model for this hardware therefore has to express “which tokens come next, given the context” as such an energy function:

where is a binary encoding of the next tokens (a vocabulary of 128k entries needs only bits per token) and is the context. The temperature knob is now literally a temperature, or rather the analog control that plays its role.

Two ingredients make this plausible:

  • Generate blocks, not single tokens. Diffusion language models already generate text by starting from a fully masked block and iteratively denoising it (Nie et al. 2025). Extropic’s denoising thermodynamic models apply the same idea to TSUs, chaining a handful of energy-based models, each of which the chip samples from directly (Jelinčič et al. 2026). Instead of one token per network pass, a block of tokens emerges after a few denoising steps.
  • Keep a digital front-end. Attention over a long context is not something local p-bit wiring is good at. A realistic design keeps a (smaller) digital network that reads the context and turns it into the biases for the TSU. That is the split Extropic’s Z1T models use, with a companion FPGA next to the thermodynamic chip (Extropic 2026).

The life of the same request

sequenceDiagram
    participant U as Client
    participant C as CPU (API server)
    participant D as Digital accelerator
    participant T as TSU
    U->>C: prompt (text)
    C->>C: tokenize, schedule
    C->>D: token IDs
    D->>D: encode context (prefill)
    loop for each block of k tokens
        D->>T: biases h(c), couplings already on chip
        loop denoising steps
            T->>T: Gibbs sweeps until equilibrium
        end
        T-->>D: k tokens as bits
        D->>D: extend context
        D-->>C: k token IDs
        C-->>U: detokenized text (streamed)
    end

The same request on a hypothetical hybrid server: the digital side encodes the context, the TSU samples blocks of tokens.

  1. Tokenization and scheduling (CPU) stay exactly the same.
  2. Prefill (digital). The context still has to be read. This remains compute-bound and fits GPU-style hardware well, and a smaller encoder makes it cheaper than today.
  3. Conditioning (digital → TSU). Instead of producing logits, the digital side produces the bias vector . The couplings are the model’s “weights” and stay resident on the chip. There is no weight streaming per token. This removes the memory-bandwidth wall from Part 1.
  4. Sampling (TSU). The chip runs a few denoising steps. In each, it performs Gibbs sweeps until it has equilibrated, and the read-out is a block of tokens. Sampling isn’t the final 1% of the work anymore. It is the work.
  5. Feedback and streaming. The new tokens extend the context; the digital side updates its state, and the loop continues with the next block.

A simple energy model

To compare the two worlds, let’s write down the energy per generated token. On the GPU it is dominated by data movement, roughly

with the energy per byte read from memory and the energy per operation. On the hybrid system, with denoising steps of Gibbs sweeps over p-bits each, and the energy of one p-bit update:

The TSU term can be tiny: no weights move, is small, and the cost is shared by tokens. But the first term doesn’t go away. This is Amdahl’s law in energy form. If a fraction of today’s energy moves to hardware that is times more efficient, the overall gain is

Figure 3: System-level energy gain G for different fractions φ of the work moved onto a TSU that is s times more efficient. The flat tails are the conventional parts of the system.

The plot makes the main point of this post visible: a 10,000× better sampler buys a 10× better system if 90% of the work moves to it. This is the gap between the headline numbers and system-level reality that the primer warned about. In Extropic’s own Z1T estimates the companion FPGA, not the thermodynamic chip, consumes over 95% of the energy (AI Wiki 2026). A thermodynamic LLM server only pays off if the model is designed so that almost all of the work happens in the sampler, and the digital front-end stays small.

What changes, and what doesn’t

GPU today Hybrid with TSU (hypothetical)
Model weights Streamed from HBM on every decode step Resident on chip as couplings
Bottleneck Memory bandwidth (decode) Mixing time, digital front-end, I/O
Unit of generation 1 token per forward pass Block of tokens per denoising chain
Randomness Pseudo-random numbers + softmax Thermal noise, native
Temperature A scalar in the softmax A physical control on the chip
Long context KV cache in HBM Still digital, still a KV-cache-like problem
Batching Essential to amortise weight reads Less critical, no weight reads to amortise

Some things are worth pointing out:

  • The memory wall disappears, and a new one appears. Instead of bytes per second, the limiting factor becomes how many sweeps the chip needs to reach equilibrium (its mixing time). Hard, multimodal distributions mix slowly, and a model that is cheap per sweep but needs a million sweeps gains nothing.
  • Batching becomes less important. Today, batching exists mostly to amortise weight reads. If the weights never move, a single user’s request is not a waste of the hardware, which could make local, low-latency inference much more attractive.
  • The compiler problem changes. Instead of tiling matrix multiplications for caches and tensor cores, a toolchain would have to map a sparse energy function onto a fixed physical wiring graph, choose update schedules (which cells may flip simultaneously) and trade sweeps for accuracy. This looks a lot more like place-and-route for FPGAs than like CUDA.

Wrapping up

Today, an LLM answers a prompt with a compute-bound prefill followed by a long, memory-bound decode loop, and it only touches randomness in its very last, cheapest step, through a softmax that is secretly a Boltzmann distribution. A thermodynamic processor turns this upside down: the weights stay put as physical couplings, sampling becomes the main event, and tokens can emerge in blocks from a denoising chain rather than one per pass.

Whether that future arrives depends on the problems we generously assumed away. But the exercise shows where the potential really is, and where it isn’t. Replacing the sampler alone does nothing. The gains only come from rebuilding the model around the hardware, and they are capped by whatever stays digital.

References

AI Wiki. 2026. “Extropic.” September.
Extropic. 2026. “Z1T: Sparse Transformer-Like Models for Probabilistic Hardware.” September.
Horowitz, Mark. 2014. “Computing’s Energy Problem (and What We Can Do about It).” IEEE International Solid-State Circuits Conference (ISSCC).
Jelinčič et al. 2026. “An Efficient Probabilistic Hardware Architecture for Diffusion-Like Models.” Npj Unconventional Computing.
Kwon, Woosuk, Zhuohan Li, Siyuan Zhuang, et al. 2023. “Efficient Memory Management for Large Language Model Serving with PagedAttention.” Proceedings of the 29th Symposium on Operating Systems Principles (SOSP).
Leviathan, Yaniv, Matan Kalman, and Yossi Matias. 2023. “Fast Inference from Transformers via Speculative Decoding.” Proceedings of the 40th International Conference on Machine Learning (ICML).
Nie, Shen, Fengqi Zhu, Zebin You, et al. 2025. “Large Language Diffusion Models.” arXiv Preprint arXiv:2502.09992.
Pope, Reiner, Sholto Douglas, Aakanksha Chowdhery, et al. 2022. “Efficiently Scaling Transformer Inference.” arXiv Preprint arXiv:2211.05102.
Williams, Samuel, Andrew Waterman, and David Patterson. 2009. “Roofline: An Insightful Visual Performance Model for Multicore Architectures.” Communications of the ACM 52 (4): 65–76.
]]> hardware ai thermodynamic-computing llm-inference https://loop-unroller.eu/posts/2026-10-10-llm-inference-thermodynamic.html Sat, 10 Oct 2026 00:00:00 GMT Thermodynamic Processors: A Primer Victor Bieszka https://loop-unroller.eu/posts/2026-10-09-thermodynamic-processors-primer.html Thermodynamic processors are chips that compute by letting random physical noise do the work, instead of spending energy to suppress it. They target one specific job that modern AI does constantly: drawing random samples from probability distributions. And they promise to do it with orders of magnitude less energy than a GPU.

Every chip you own is built on a single promise: a transistor is either on or off, and nothing in between ever leaks through. Keeping that promise is expensive. Engineers spend enormous effort holding voltages high enough above the electronic jitter of warm silicon that a 1 never flips to a 0. Then, ironically, a large share of today’s AI workloads takes those perfectly deterministic bits and uses them to simulate randomness.

Thermodynamic computing asks an obvious-sounding question: if the algorithm wants randomness, and the hardware is naturally random, why pay twice? This primer explains what that means, how the chips work, and where the idea stands as of late 2026.

The problem: AI is hitting an energy wall

Energy, not raw chip speed, is becoming the binding constraint on AI. Moore’s Law has slowed, energy per operation has largely plateaued, and data centers are now limited by how much power and cooling they can get. Building more GPUs scales the bill linearly with the ambition.

There is also a mismatch hiding inside the workload. Generative AI is, at its core, a sampling problem:

  • Training learns a probability distribution from data (what images, sentences or molecules tend to look like).
  • Inference draws samples from that distribution (a new image, the next token, a candidate molecule).

A digital chip has no native way to be random. It generates pseudo-random numbers with arithmetic, then pushes them through long chains of multiply-and-add to shape them into the right distribution. Diffusion models make this especially vivid: they start from pure noise and spend dozens to hundreds of network passes gradually turning it into an image.

So the stack looks like this: physics gives us noisy transistors, we spend energy making them perfectly deterministic, and then we spend more energy faking randomness on top. Thermodynamic computing tries to remove both layers of waste.

flowchart LR
    N([Thermal noise in silicon])
    subgraph D["Digital"]
        direction LR
        d1["Suppress noise<br/>(energy)"] --> d2[Deterministic bits] --> d3["Fake randomness<br/>with arithmetic (energy)"] --> d4([Sample])
    end
    subgraph T["Thermodynamic"]
        direction LR
        t1[Let noise drive the circuit] --> t2([Sample])
    end
    N --> d1
    N --> t1

The digital path spends energy twice, once removing noise and again faking it; the thermodynamic path skips both.

The core idea: let physics do the sampling

A physical system sitting in a warm environment already samples from a probability distribution for free. That is the whole trick.

Think of a ball in a hilly landscape that is being constantly shaken. It rolls around at random, but it spends most of its time in the valleys and rarely sits on the peaks. Watch it long enough and the fraction of time it spends at each spot follows a precise rule from 19th-century statistical physics, the Boltzmann distribution:

Here is the “energy” (the height of the landscape) at state , is temperature (how hard the shaking is), and is Boltzmann’s constant. Low-energy states are exponentially more likely.

Now flip it around. If you can design the landscape by setting so that its valleys correspond to the things you want (realistic images, good solutions to an optimization problem), then simply letting the system jiggle and taking snapshots gives you samples from exactly the distribution you care about. No pseudo-random number generator, no long chains of arithmetic.

This maps directly onto a family of machine-learning models called energy-based models, which describe data by an energy function rather than an explicit formula for probabilities. On a GPU these models are notoriously slow to sample from. On thermodynamic hardware, sampling is what the hardware does by default.

How a thermodynamic processor works

A thermodynamic processor takes the parameters of a probability distribution as input and returns samples from it as output. That is why Extropic calls its chips Thermodynamic Sampling Units (TSUs) rather than processing units (Extropic 2025).

The building block: a probabilistic bit

The simplest version starts with a p-bit (probabilistic bit). An ordinary bit is a light switch. A p-bit is a weighted coin that flips itself continuously.

In Extropic’s all-transistor design, a p-bit’s output voltage wanders randomly between high (1) and low (0), driven by the natural thermal noise in its transistors. A single control voltage sets the bias: turn it up and the p-bit spends, say, 80% of its time at 1. Reading the voltage at any moment gives you one random sample. The company reports that one X0 p-bit design settles in roughly 100 nanoseconds, and claims each flip costs far less energy than a single floating-point addition on a conventional chip (Extropic 2025).

From coins to a computer

One weighted coin is not useful. The power comes from wiring many of them together so that each one’s bias depends on its neighbours. The chip then runs a loop that statisticians call Gibbs sampling:

  1. Program the landscape. Load the model’s parameters onto the chip: a bias for each p-bit and a coupling strength for each wire between neighbours. Together these define the energy function .
  2. Each cell listens to its neighbours. It sums up their current states, weighted by the couplings, plus its own bias.
  3. Each cell flips its weighted coin. That sum sets the p-bit’s probability of landing on 1, and the noise does the rest.
  4. Repeat in parallel. Cells that aren’t neighbours update at the same time, so a bigger chip doesn’t make each step slower.
  5. Read out. After enough rounds, the pattern of 1s and 0s across the chip is a sample from the programmed distribution. Keep running to get more.

Two design choices make this efficient. Memory and compute live in the same place, so there is no shuttling of data between a processor and RAM, which is where much of a GPU’s energy goes. And each cell only talks to nearby cells, which keeps wires short and cheap.

Two flavours of hardware

The two best-known startups take different physical routes to the same idea.

Extropic Normal Computing
Basic unit p-bits (discrete 0/1), plus multi-level and Gaussian variants Continuous “s-units”: analog voltages in resonant circuits
Physics Thermal noise in ordinary CMOS transistors Coupled oscillators settling into thermal equilibrium (Langevin dynamics)
Native operation Sampling from energy-based models (Ising-style) Gaussian sampling and linear algebra, e.g. matrix inversion
Flagship chip Z1: 269,568 p-bits, under 1 W (taped out 2026) CN101 (taped out June 2025); CN201 and CN301 planned

Normal’s matrix-inversion trick is a good example of how strange and elegant this can be (Normal Computing, n.d.). You encode a matrix into the couplings between a set of oscillators, let the system reach thermal equilibrium, and measure how the voltages fluctuate together. That covariance turns out to be proportional to the inverse of the matrix you put in:

The physics solved the equation; you just watched.

What it’s good for (and what it isn’t)

Thermodynamic processors are accelerators for probabilistic work, not replacements for CPUs or GPUs. You will never run a web browser on one. The promising targets are:

  • Generative models built for sampling. Extropic’s Denoising Thermodynamic Model borrows the structure of a diffusion model, but instead of hundreds of small deterministic denoising steps, it chains a handful of energy-based models that the chip samples from directly (Jelinčič et al. 2026).
  • Sparse language models. In September 2026 Extropic published Z1T, a family of sparse, transformer-like models redesigned to fit the chip’s 16-neighbour wiring, with open weights on Hugging Face (Extropic 2026b).
  • Linear algebra. Normal Computing’s continuous-variable hardware can invert matrices and solve linear systems by letting oscillators equilibrate. These operations sit underneath much of scientific computing and machine learning.
  • Bayesian inference and uncertainty. Any model that wants a distribution of plausible answers rather than one best guess, such as weather forecasting, financial risk or robotics under uncertainty.
  • Optimisation. Many hard scheduling and routing problems can be written as “find the lowest valley in an energy landscape”, which is exactly what these systems explore.

The realistic picture is heterogeneous: a server where a CPU orchestrates, a GPU does dense matrix maths, and a thermodynamic chip handles the sampling-heavy parts. Both Extropic and Normal describe their chips as cards that sit alongside existing accelerators.

Who’s building it, and where things stand

As of October 2026, thermodynamic computing has real silicon but no commercial deployments yet. Two US startups lead, both founded in 2022 by alumni of Google’s quantum and AI research groups.

Extropic builds all-transistor p-bit chips in standard CMOS. Its Z1 chip packs 269,568 p-bits into a die under 12 mm per side drawing under 1 W, and is due in M.2 sticks, PCIe cards and a planned billion-p-bit cluster with early access in 2027 (Extropic 2026a). It publishes open-source software (THRML for simulation, Torx for programming) so developers can build for the hardware before it ships.

Normal Computing builds continuous-variable analog chips under what it calls the Carnot architecture, targeting up to 1000x energy efficiency on selected AI and scientific workloads. Its 2025 Nature Communications paper demonstrated an eight-cell circuit-board prototype (Melanson et al. 2025); CN101 is its first silicon (Normal Computing 2025).

Date Milestone
Sep 2026 Extropic releases Z1T sparse transformer-like models with open weights (Extropic 2026b)
Aug 2026 Extropic says Z1 has taped out; launches Torx framework and simulator API (Extropic 2026a)
Jul 2026 Extropic signs a non-binding letter of intent for up to $75M from the US Commerce Department’s CHIPS R&D office (I-Connect007 2026)
Jul 2026 Extropic’s diffusion-like-model paper published in npj Unconventional Computing (Jelinčič et al. 2026)
Mar 2026 Normal Computing raises $50M led by Samsung Catalyst (Fortune 2026)
Oct 2025 Extropic launches X0 prototype chip, XTR-0 dev platform and THRML library (Extropic 2025)
Aug 2025 Normal’s CN101 produces its first data in testing
Jun 2025 Normal tapes out CN101, billed as the first thermodynamic computing chip (Normal Computing 2025)
Apr 2025 Normal publishes its stochastic processing unit prototype in Nature Communications (Melanson et al. 2025)

The idea itself is not new. Boltzmann machines, the neural networks Geoffrey Hinton and colleagues described in 1985, are exactly the kind of model these chips run, and academic groups have built p-bit hardware for years. What changed is the push to make it scalable in mainstream semiconductor processes, and AI’s energy bill gave it a market.

Caveats: read the fine print on “10,000x”

The headline efficiency numbers are projections, not measurements of shipping products. Treat them as hypotheses that 2027 hardware will test (AI Wiki 2026).

  • The big multipliers come from simulation. Extropic’s widely quoted ~10,000x figure comes from a hardware model of a Z1-like chip generating low-resolution Fashion-MNIST clothing images, a simple benchmark far from a frontier image or language model. Its Z1T language-model estimates (about 14x to 139x over an Nvidia H100 per token) depend on assumed GPU utilisation and exclude parts of the workload.
  • The rest of the system still costs energy. In the Z1T estimates, the companion FPGA, not the thermodynamic chip, consumes over 95% of the energy. Gains on the chip can be swallowed by the conventional hardware around it.
  • Algorithms have to be rebuilt. Today’s models were co-designed with GPUs. To use these chips, models must be reshaped into sparse, locally connected, energy-based forms, and Extropic’s own study found its sparse models need roughly ten times more training compute to match a dense GPT-2.
  • Scaling is unproven. Small prototypes work; whether useful signals survive at millions of cells, across manufacturing variation and chip-to-chip communication, is the open question both companies are now trying to answer.
  • Analog is hard. Continuous-variable designs like Normal’s must cope with component tolerances and calibration drift, which is why the history of analog computing is full of promising prototypes that digital chips overtook.

None of this means the idea fails. It means the next 12 to 18 months, when independent benchmarks on real Z1 and CN-series hardware become possible, matter more than any press release.

The takeaway

Thermodynamic computing flips the oldest rule in chip design: instead of fighting noise, it uses noise as the engine. Because so much of modern AI is really about drawing samples from probability distributions, hardware that samples natively could do that job with far less energy than a GPU simulating randomness through arithmetic.

The physics is sound and the first chips exist. What remains unproven is whether the advantage survives at scale, on real workloads, inside full systems. If it does, the AI data center of the 2030s may look less like a wall of identical GPUs and more like a mixed rack, where some chips calculate and others simply let physics settle into the answer.

Sources

AI Wiki. 2026. “Extropic.” September.
ARIA. 2026. “Developing a World-First Thermodynamic Computing Chip.” March.
Extropic. 2025. “TSU 101: An Entirely New Type of Computing Hardware.” October.
Extropic. 2026a. “From One to One Billion: Torx, Thermalizers, and Z1.” August.
Extropic. 2026b. “Z1T: Sparse Transformer-Like Models for Probabilistic Hardware.” September.
Fortune. 2026. “Normal Computing Raises $50M from Samsung Catalyst.” March.
I-Connect007. 2026. “Extropic Signs $75M LOI with US Commerce Department.” July.
IEEE Spectrum. 2025. “Thermodynamic Computing Is the Hot New Trend.”
Jelinčič et al. 2026. “An Efficient Probabilistic Hardware Architecture for Diffusion-Like Models.” Npj Unconventional Computing.
Melanson et al. 2025. “Thermodynamic Computing System for AI Applications.” Nature Communications.
Normal Computing. 2025. “Normal Computing Announces Tape-Out of CN101.” June.
Normal Computing. n.d. “A First Demonstration of Thermodynamic Matrix Inversion.”
]]> hardware ai thermodynamic-computing energy-based-models https://loop-unroller.eu/posts/2026-10-09-thermodynamic-processors-primer.html Fri, 02 Oct 2026 00:00:00 GMT Two clocks start on 11 September Victor Bieszka https://loop-unroller.eu/posts/2026-07-04-cra-11-september.html Most companies I talk to have filed the Cyber Resilience Act under “December 2027 problem”. That is when the essential requirements bite, when CE marking needs a cybersecurity dimension, when products that don’t comply can no longer be placed on the EU market. Plenty of time, goes the reasoning, especially since the harmonised standards aren’t even finished.

The reasoning is wrong on one important point. The first enforceable obligation of the CRA does not arrive in December 2027. It arrives on 11 September 2026, and unlike almost everything else in the regulation, it comes with no grandfathering.

What actually changes on 11 September

From that date, Article 14 applies: any manufacturer of a product with digital elements sold in the EU must report two kinds of events.

The first is an actively exploited vulnerability in one of its products — a vulnerability that someone is actually using against real systems, not merely one that exists. The second is a severe incident having an impact on the security of the product, which the regulation defines roughly as anything that compromises the product’s ability to protect data or that enables the introduction or execution of malicious code.

Both trigger the same unforgiving cadence:

flowchart TB
    subgraph V["Actively exploited vulnerability"]
        direction LR
        v0([Awareness]) -- "24 h" --> v1[Early warning]
        v1 -- "72 h from awareness" --> v2[Vulnerability notification]
        v2 -- "14 days after a fix or mitigation exists" --> v3[Final report]
    end
    subgraph I["Severe incident"]
        direction LR
        i0([Awareness]) -- "24 h" --> i1[Early warning]
        i1 -- "72 h from awareness" --> i2[Incident notification]
        i2 -- "1 month after the notification" --> i3[Final report]
    end

Reports go simultaneously to the CSIRT designated as coordinator in your member state — for German manufacturers, the BSI — and to ENISA, through a new single reporting platform that ENISA is required to have operational on the same day the obligation starts. Nobody has production experience with that platform, for the simple reason that it doesn’t exist in production yet. On top of the regulator-facing reports, Article 14 also requires you to inform impacted users — and where appropriate, all users — about the vulnerability or incident and what they can do about it.

For orientation, here is where 11 September sits in the overall CRA schedule:

timeline
    title CRA application timeline
    10 Dec 2024 : Regulation (EU) 2024/2847 enters into force
    11 Jun 2026 : Rules for notified bodies and conformity assessment apply
    11 Sep 2026 : Article 14 reporting obligations apply
                : ENISA single reporting platform goes live
    11 Dec 2027 : Full application — essential requirements, CE marking, market surveillance

No grandfathering

This is the part that catches people. The CRA’s transitional provisions (Article 69) exempt products placed on the market before 11 December 2027 from the essential requirements, unless they are substantially modified. But the same article carves out an explicit derogation: the Article 14 reporting obligations apply to all in-scope products already on the market.

Your 2019 firmware counts. The legacy product line you stopped actively developing but still sell counts. The white-labelled device you rebadge and sell under your own name counts — under the CRA, putting your name on it makes you the manufacturer.

If a vulnerability in any of those products starts being exploited in the wild after 11 September, the 24-hour clock starts the moment you become aware of it. Not the moment you confirm it, reproduce it, or have a patch. Aware.

Why this is harder than it sounds

On paper, this looks like a paperwork exercise: fill in a form within a day. In practice, three things make it genuinely difficult.

The trigger is awareness of exploitation, and exploitation mostly happens in code you didn’t write. The phrase in Article 14 is a vulnerability “contained in the product”. That includes every third-party library, every open source dependency, every SoC vendor’s SDK baked into your product. Look at the vulnerabilities that have actually been exploited at scale in recent years — Log4Shell, the libwebp zero-day, MOVEit, the xz backdoor attempt — and notice that for most affected vendors, the vulnerable code was somebody else’s. The question you have to answer in hours, not weeks, is: is this exploited component in any product we ship, in an exploitable configuration? If your SBOMs are PDFs generated once at release and filed away, you cannot answer that question. If you have no structured intake for exploitation intelligence — CISA KEV, your national CSIRT’s advisories, researcher reports, customer tickets — you may not even know the clock has started.

“Severe incident” is a judgment call made under time pressure. Whether an event “negatively affects the ability of the product to protect the availability, authenticity, integrity or confidentiality of data” is the kind of question legal and engineering can debate for a week. You have a day. The only workable answer is to make the classification decision before the incident: written criteria, worked examples for your own product portfolio, and a named person empowered to make the call at 02:00 on a Saturday.

Reporting an unpatched, actively exploited vulnerability to authorities is uncomfortable — by design. The early warning will often describe a zero-day for which you have no fix yet. Industry raised exactly this concern during the legislative process — that the regime concentrates knowledge of exploitable vulnerabilities across a network of agencies — and the final text does contain safeguards allowing dissemination of a report to be limited on justified cybersecurity grounds. But the tension with coordinated vulnerability disclosure practice is real, and your disclosure policy, your PSIRT playbooks, and your legal review process all need to accommodate a regulator sitting in the loop from hour 24.

And this regime does not exist in isolation. A single event can simultaneously be a CRA product vulnerability report, a NIS2 incident for your own infrastructure, a GDPR personal data breach, and — if your customers are financial entities — the subject of frantic DORA-driven questionnaires from their side. Same facts, different clocks, different recipients, different severity thresholds. If those workflows live in four different teams that have never run an exercise together, September will be educational.

The supply-chain view

For people working in supply-chain security specifically, Article 14 is the visible tip of a larger shift: the CRA turns component-level visibility from a best practice into legal infrastructure.

Two provisions matter beyond the reporting duty itself. Article 13 obliges manufacturers to exercise due diligence when integrating third-party components, and — less noticed — to report vulnerabilities they discover in a component back to whoever maintains it, including open source maintainers. Vulnerability information is now legally expected to flow both up and down the chain. Meanwhile, open source projects themselves are largely out of scope unless commercialised, and the new “open-source steward” category (foundations and similar) carries a deliberately lighter regime.

The practical consequence is already observable: obligations propagate contractually. Manufacturers who must report within 24 hours are pushing 24-hour notification clauses onto their component suppliers, who push them onto theirs. If you sell software or hardware that ends up inside somebody else’s product, CRA-shaped clauses are coming to your contracts regardless of whether the regulation names you directly. Expect the same dynamic that DORA triggered in financial services — a wave of supplier questionnaires and register entries — this time for anything with a processor in it.

The honest test of your program is simple to state: when the next Log4Shell lands, how long does it take you to produce a complete, confident list of affected products and versions? If the answer is “a few hours”, Article 14 is a formality. If the answer is “we’d start by emailing product teams”, the regulation has just put a price on that gap — formally up to €15 million or 2.5% of global turnover, practically the chaos of doing archaeology on your own portfolio with a regulator waiting.

Ten weeks, concretely

Enforcement on day one will presumably focus on the unprepared rather than the imperfect, and the harmonised standards are still in the pipeline (the first of the prEN 40000 series are expected around Q3 2026). But the reporting duty doesn’t depend on any standard. Between now and September, a realistic minimum looks like this:

  1. Inventory what’s in scope. Every product with digital elements you place on the EU market, including legacy and white-labelled lines. This list is shorter to compile than you fear and longer than you expect.
  2. Establish one intake for exploitation awareness. KEV feeds, CSIRT advisories, researcher contact point, support-ticket escalation — all converging on one queue someone actually watches, weekends included.
  3. Make SBOMs queryable. Not documents; data. The question “which products contain component X at version Y” must be answerable by a query, not a project.
  4. Pre-write the reports. Templates for the 24-hour early warning and the 72-hour notification, with the fields the single reporting platform expects. Decide now who drafts, who reviews, who submits.
  5. Define “severe” for your products. Written classification criteria with worked examples, and a named decision-maker with a deputy.
  6. Run one exercise. Pick a past incident or a hypothetical exploited dependency and walk the full chain — detection to early warning to user notification — against the real clocks. The first dry run always fails. Better in July than in September.

The CRA’s December 2027 requirements will dominate compliance budgets for the next eighteen months. But 11 September is the date on which the regulation stops being a document and starts being a stopwatch — and it measures something no policy can fake: whether you actually know what is in the products you ship.

Further reading: the official CRA summary and reporting obligations overview from the European Commission, ENISA’s Single Reporting Platform page, the full text of Article 14, and the BSI’s CRA resources (German).

]]> cra regulation supply-chain-security sbom vulnerability-management https://loop-unroller.eu/posts/2026-07-04-cra-11-september.html Sat, 04 Jul 2026 00:00:00 GMT Embedding Wasmtime in C++ Victor Bieszka https://loop-unroller.eu/posts/2023-07-04-wasmtime-cpp.html This is a little guide on how to execute WebAssembly inside C++, and vice versa. First, we are going to embed Wasmtime in C++. This will allow us to call WebAssembly within any C++ project. Wasmtime is a fantastic standalone runtime for WebAssembly that comes equipped with its own compiler, called Cranelift. I might be a bit biased though, as I have contributed to Cranelift/Wasmtime. Nevertheless, give it a try, and if you have any questions, issues, etc., join us on Zulip.

Anyway, let’s get back to work. Calling C++ functions from WebAssembly is a little bit more tricky. This will require us to register host functions in Wasmtime. Lastly, if we are interested in passing more complex data to WebAssembly (such as C++ pointers or classes), we can utilize WebAssembly’s externref type.

Note

The examples were last tested with Wasmtime 49 and need C++17. The C++ API still changes between releases from time to time, so check the header that ships with your version if something doesn’t compile.

Setup

The C++ API is a single header, wasmtime.hh, that wraps Wasmtime’s C API. It used to live in a separate wasmtime-cpp repository, but nowadays it ships with every Wasmtime release. Download the C API archive for your platform from the releases page (e.g. wasmtime-v49.0.2-aarch64-macos-c-api.tar.xz) and unpack it. You get an include/ directory with the headers and a lib/ directory with the static and shared library.

Compiling our program is then a single command:

# macOS
clang++ -std=c++17 -I$WASMTIME/include main.cc $WASMTIME/lib/libwasmtime.a \
    -framework CoreFoundation -framework Security -o main

# Linux
g++ -std=c++17 -I$WASMTIME/include main.cc $WASMTIME/lib/libwasmtime.a \
    -lpthread -ldl -lm -o main

Throughout the examples I load the WebAssembly modules from disk with this little helper:

#include <fstream>
#include <sstream>
#include <string>

std::string readFile(const char *path) {
  std::ifstream file(path);
  std::stringstream buffer;
  buffer << file.rdbuf();
  return buffer.str();
}

Embedding WebAssembly

Now, let’s suppose we have a simple WebAssembly module called wasm.wat as follows:

(module
  (func (export "execute"))
)

This module contains one empty exported function that is correct, callable and yet does nothing. We now want to call this function from within our C++ project.

#include <wasmtime.hh>

using namespace wasmtime;

int main() {
  Engine engine;
  Store store(engine);

  auto module = Module::compile(engine, readFile("wasm.wat")).unwrap();
  auto instance = Instance::create(store, module, {}).unwrap();
  auto execute = std::get<Func>(*instance.get(store, "execute"));
  execute.call(store, {}).unwrap();
}

A quick tour of what is going on here:

  • The Engine holds the global configuration and the compiler. You usually need exactly one per process.
  • The Store owns all WebAssembly objects: instances, memories, functions. Everything a module does at runtime lives in a store, and it all gets freed when the store goes away.
  • Module::compile takes either the text format (.wat) or a binary (.wasm) and compiles it to native code with Cranelift. A module is stateless and can be instantiated many times.
  • Instance::create instantiates the module in our store. The empty {} is the list of imports, and our module has none.
  • instance.get looks up an export by name. It returns a std::optional<Extern>, where Extern is a std::variant over functions, globals, memories and tables, hence the std::get<Func>.

Almost every call returns a Result, which is either the value or an error. unwrap() returns the value or aborts with the error message, which keeps the examples short. In real code you would check the result instead.

Typed functions

Func::call takes and returns a std::vector<Val>, which is flexible but clumsy. If you know the signature up front, you can ask for a typed version of the function:

auto run = std::get<Func>(*instance.get(store, "run"))
               .typed<int32_t, int32_t>(store)
               .unwrap();
int32_t result = run.call(store, 20).unwrap();

typed checks the signature once, when you call it, so a mismatch shows up as an error right there, not as a crash later. Use std::tuple<...> for multiple parameters or results, and std::monostate for none.

Call C++ from WebAssembly

WebAssembly modules can’t do anything on their own. They can’t print, open files or read the clock. Everything the outside world offers has to be imported, and the embedder (that’s us) decides what gets passed in. In our case, these imports are plain C++ functions, called host functions.

Here is a module that imports two functions from a module named host, one to print a number and one to print a string:

(module
  (import "host" "print_i32" (func $print_i32 (param i32)))
  (import "host" "print_str" (func $print_str (param i32 i32)))

  (memory (export "memory") 1)
  (data (i32.const 0) "Hello from WebAssembly!")

  (func (export "run") (param $x i32) (result i32)
    (call $print_str (i32.const 0) (i32.const 23))
    (call $print_i32 (local.get $x))
    (i32.mul (local.get $x) (i32.const 2))
  )
)

To provide the imports, we use a Linker. It maps (module, name) pairs to definitions, and func_wrap turns any C++ callable into a WebAssembly function. Wasmtime derives the WebAssembly signature from the C++ parameter and return types, so a lambda taking an int32_t becomes a function with signature (param i32).

int main() {
  Engine engine;
  Store store(engine);
  Linker linker(engine);

  linker.func_wrap("host", "print_i32", [](int32_t value) {
    std::cout << "wasm says: " << value << "\n";
  }).unwrap();

  linker.func_wrap("host", "print_str",
                   [](Caller caller, int32_t ptr, int32_t len) {
    auto memory = std::get<Memory>(*caller.get_export("memory"));
    auto data = memory.data(caller);
    std::cout << std::string_view(
        reinterpret_cast<const char *>(data.data() + ptr), len) << "\n";
  }).unwrap();

  auto module = Module::compile(engine, readFile("host.wat")).unwrap();
  auto instance = linker.instantiate(store, module).unwrap();
  auto run = std::get<Func>(*instance.get(store, "run"))
                 .typed<int32_t, int32_t>(store)
                 .unwrap();

  int32_t result = run.call(store, 20).unwrap();
  std::cout << "result: " << result << "\n";
}

Running it prints:

Hello from WebAssembly!
wasm says: 20
result: 40

Two things are worth pointing out.

Strings don’t cross the boundary. WebAssembly only knows numbers, so the module passes a pointer and a length into its own linear memory. That’s why print_str takes a Caller as its first parameter. The Caller gives the host function access to the calling instance, so it can look up the exported memory and read the bytes. Note that memory.data() hands out a view into the instance’s memory. Don’t hold on to it across calls back into WebAssembly, since the memory may grow and move.

Instantiate through the linker. Instead of Instance::create with an explicit import list, we call linker.instantiate, which resolves every import by name. If an import is missing or has the wrong type, you get a readable error at this point instead of a trap later.

Externref for complex data

Passing numbers is easy, and passing strings through linear memory is manageable. But what if a host function needs a C++ object, say a database connection or a counter? We can’t copy it into the module’s memory, and handing out a raw pointer as an i32 would let the module forge pointers into our address space.

This is what externref is for. It is an opaque reference to a host value. WebAssembly can store it, pass it around and hand it back to the host, but it can’t look inside or fake one.

Here is a module that takes an externref and passes it back to the host twice:

(module
  (import "host" "increment" (func $increment (param externref)))

  (func (export "bump_twice") (param $counter externref)
    (call $increment (local.get $counter))
    (call $increment (local.get $counter))
  )
)

On the C++ side, an ExternRef wraps a std::any, so it can hold any copyable C++ value. In this example it holds a pointer to a Counter that lives on our stack:

struct Counter {
  std::string name;
  int value = 0;
};

int main() {
  Engine engine;
  Store store(engine);
  Linker linker(engine);

  linker.func_wrap("host", "increment",
                   [](Caller caller, std::optional<ExternRef> ref) {
    if (!ref) {
      return;
    }
    auto *counter = std::any_cast<Counter *>(ref->data(caller));
    counter->value++;
  }).unwrap();

  auto module = Module::compile(engine, readFile("externref.wat")).unwrap();
  auto instance = linker.instantiate(store, module).unwrap();
  auto bump = std::get<Func>(*instance.get(store, "bump_twice"))
                  .typed<std::optional<ExternRef>, std::monostate>(store)
                  .unwrap();

  Counter counter{"requests"};
  ExternRef ref(store, &counter);
  bump.call(store, ref).unwrap();

  std::cout << counter.name << " = " << counter.value << "\n";  // requests = 2
}

A few details:

  • An externref can be null, so on the C++ side it shows up as std::optional<ExternRef>. Always handle the empty case, because the module is free to pass ref.null extern.
  • ExternRef values are garbage-collected by the store. Creating one needs the store, and so does reading it back with data(). Inside a host function, the Caller stands in for the store.
  • The reference holds whatever you put into the std::any, here a raw pointer. Wasmtime keeps the std::any alive, but not the object the pointer points to. If the module might keep the reference around (e.g. in a global or a table), make sure the object outlives it, or store a std::shared_ptr instead.

Wrapping up

That covers the three building blocks of embedding Wasmtime in C++:

  1. Compile and instantiate a module, then call its exports, ideally through typed functions.
  2. Provide host functions with a Linker, and use the Caller to reach into the instance’s memory.
  3. Pass C++ objects into WebAssembly safely as opaque externrefs.

From here, the natural next steps are WASI, which gives modules a standard interface to files, clocks and the environment, and the component model, which replaces the pointer-and-length dance with real strings, records and lists. Both are covered in the Wasmtime documentation.

]]> webassembly wasmtime cpp https://loop-unroller.eu/posts/2023-07-04-wasmtime-cpp.html Tue, 04 Jul 2023 00:00:00 GMT