news?

I suppose it depends on what you include in the kernel. Our OS framework is a set of cooperating services, but the great majority of that is app code with no particular privileges – things like a math library, but including a lot that in legacy CPUs has to run in the kernel.

We define the real kernel as that code which has to be trusted because it can unilaterally change the state and condition of other code that is not trusted. This trust is different from code that is relied upon: if your app uses sqrt in some calculation, you rely on the math library to in fact give you a square root. But sqrt cannot change the state of its caller (courtesy the Mill protection model) so the app does not have to trust it in this sense.

So what has to be trusted? Not very much on a Mill: some initialization code for boot; the top-level interrupt handler; the dispatcher; a few allocators; and most importantly, the code that updates the protection state. We project ~3k LOC total. And nearly all of that code exists to deal with the way the Mill works, so it can’t be shared with any other platform, in either direction.

Of course, surrounding that microkernel there will be a ton of untrusted (but relied upon) libraries that we expect to lift from L4 and anywhere else. That will include a lot of what the original source thought was part of the kernel, but we don’t.

We anticipate considerable terminological confusion.

Positively antique! We’ve hoped for someone to join that wants to do it, but so far… You interested?

I don’t believe that I could deliver results commensurate with how exciting I find your work. As a consequent of turning a hobby into a bill-paying job, the job frequently saps my desire to perform my hobby (and given the paperwork that I signed for said job, I am frequently unsure of how to actually contribute to things of pith and moment). I’ll have to pass, for now.

Puke: Nevermind my tights, I thought we were talking about my pith.
Snot: That’s why you have to wash them out!

Thank you for your support and enthusiasm. The company has done fine over the past year; being already a global virtual company almost nothing changed for us. But it’s been a rough year personally for many of us, especially on of the compiler team who spent a month in hospital in Poland an is still not fully back.

About your recent fund-raising experiences: In your VC contacts, were you face-to-face or video? Had you been vaccinated?

Personally I’m in a high-risk group, and more than a little paranoid about the situation here in the US.

Come on, you know we can’t say 🙂

Nice to hear you’re still talking to them!

The lectures were truly inspiring and made tons of sense, while I was listening to them.
Still over the years I’ve forgotten so much, I couldn’t explain how the Belt works if I was asked me today 🙁

What I *do* remember is an order of magnitude better general performance from the same transistor budget.

But when I look at an Apple M1 vs. a Jetson Nano, or a AMD Ryzen 5800U vs. an AMD Bobcat, that’s also an order of magnitude in a decade, redoing architectures on a very conventional ISA.

It reminds me of i860 vs. i386 days when a novel “Cray-on-a-chip” ISA could deliver an order of magnitude of performance per clock, but never survived more than half an architecture refresh, while x86 still lives.

So I wonder how meaningful do “tin” to “gold” performance targets remain a decade after starting, when even x86 and ARM need to prove that they can continue to scale performance at static energy cost?

In theory a Belt ISA implementation should always remain ahead, but only if it could mobilise simlar budgets to keep scaling the implementation.

I am growing a little worried, that perhaps the Belt will wind up better than a comparable RISC-V at the same transistor budget, but that it won’t matter because it’s travelled downward to the embedded “sleep mostly” range where the cost of an extra ISA is much higher than the price for the extra die area.

You’d need to hit laptop or smartphone targets with significantly better performance and/or energy efficiency ratios to get enough sales traction to create an eco-system, so where would “tin” to “gold” fit today, when you imagined them a decade ago?

How would you grow to 256 Platinum cores for a server variant and can an ISA survive without planning for that league?

Had to look up Carpendale 🙂

We continue to feel little time pressure; our advantages (and drawbacks, such as they are) seem to persist. The industry and market have essentially given up on ISA advances; there have been process advances, but they apply to us as much as anyone. The work we can do in house results (eventually) in money to us instead of money to funders.

We know we can’t dawdle forever – patents time out, as do people. But for now, steady as she goes.

Has the industry given up on ISA improvements?

My impression is that a 10x efficiency improvement in general purpose code isn’t enough to make it change horses any more, because general purpose is becoming less important.

With wafer scale machine learning and quantum computing we are so down the road to special purpose architectures, that GP is really treated like orchestration code.

And RISC-V nicely fills that space were GP code and special purpose extensions make things happen in the embedded world, even if the European Processor Initiative is playing with HPC extensions, too. I can’t see the Mill compete there, because reduced entropy in its instruction space is at the heart of its design.

It’s extremely frustrating to know that with Mills on a current process mobiles, laptops and chromebooks could run just as fast but with much less CPU power, but with displays, RAM and storage already taking the wattage lion share (NPUs, DPUs, IPUs and GPUs the SoC real-estate) and few people having to survive days without charge it wouldn’t really matter that much any more.

At the high-end in cloud servers transistor budgets for cores and Watts to operate them seem much more compelling to pay for architecture switches, but I don’t know if you could scale the Mill meaningfully to dozens of cores in a die.

I fear that the Mill has missed its window of opportunity and I find that extremely sad, because it’s truly great and inspirational design.

abufrejoval writes:

With wafer scale machine learning and quantum computing we are so down the road to special purpose architectures, that GP is really treated like orchestration code.

The big market gap in machine learning is extreme sparsity and extreme sparsity requires indirect memory access due to the data structures required for sparse arrays. So the ML challenge is keeping shared, parallel, random memory access on-die. This is a gap that has existed ever since GPUs were repurposed for ML since graphics (up until ray trace) had little use for sparse matrix multiplies. This has, in turn, biased the entire field toward dense models — drunks looking for their keys under the lamp post. The hundreds of billions of parameters in the large models are dense but it’s been demonstrated that weight distillation can bring that down by at least a factor of 10 without loss of perplexity score. One case achieved 97% reduction. This is consistent with what we know about neocortical neuron connectivity. Moreover, Algorithmic Information Theory tells us that the gold standard for data-driven induction of models is parameter minimization approximating Kolmogorov Complexity of the data.

Quantum computing is a pig in a poke.

At the high-end in cloud servers transistor budgets for cores and Watts to operate them seem much more compelling to pay for architecture switches, but I don’t know if you could scale the Mill meaningfully to dozens of cores in a die.

It’s important to think in terms of 50e9 transistors per die as a way of keeping shared RAM access on-die. That’s where we are for both indirect memory access (sparsity in ML) and GP.

What you say rings true to my naive ears, but what does it mean for the Mill?

To my understanding RISC-V is a totally unremarkable architecture which on its own would come decades too late and offer none of Mill’s merits, but the easily extended instruction set punches exactly where it counts: special purpose acceleration seamlessly baked into general purpose outer loops. And here we are talking 3 or more orders of magnitude better than general purpose instructions, where the Mill might deliver one order of magnitude for a similar transistor budget.

But exactly because one way it achieves this advantage is by using a reduced encoding space for code, it loses ISA extensability AFAIK.

Of course, accelerators might just be memory mapped and orchestrators just need to ready the bits in the RAM that neural code might then decode as sparse. The Mill might still deliver 10:1 benefits on orchestration, but is that enough to motivate a switch of ISA?

That’s where I am looking for reassurance, because I just love the Mill. But loving it, doesn’t mean being convinced about the value it can deliver.

abufrejoval writes:

That’s where I am looking for reassurance, because I just love the Mill. But loving it, doesn’t mean being convinced about the value it can deliver.

Agreed and I’m not going to tell you I offer that reassurance in my very top-down technical market gap which, in the final analysis, is mainly about keeping shared RAM access on die given the fact that increased density has been outstripping increased clock rates for a decade or so.

But the machine learning world is not only an emerging market for silicon — it is breaking out of its drunken path-dependent stupor about dense models born of cheap GPUs to realize the value of sparse models — not only for better models (see Algorithmic Information Theory and Solomonoff Induction’s formalization of Occam’s Razor), but for a factor of 100 energy savings per inference. The large models everyone is so excited about are not just ridiculously wasteful of silicon, their energy costs dominate.

NVIDIA’s newest ML foray (Grace) at 80e9 transistors claims it supports “sparsity”. This is (to be _very_ kind) marketing puffery. Their “sparsity” is only about a factor of 2. In other words, each neuron is assumed to be connected to half of all the other neurons. All their use of that term tells us is that the market demands sparsity and that NVIDIA can’t deliver it but knows they need to. Actual graph clustering coefficients in neocortical neurons, and actual weight distillation metrics indicate you’re probably going to hit the broad side of the market’s barn by simply turning those 80e0 transistors into a cross-bar RAM where a large number of banks of phased access RAM are on one axis and a large number of simple GPs are on the other axis.

Can the Mill serve as the “simple GPs”? How many transistors does one Mill GP take if its architecture is biased toward sparse array matrix multiplies and/or sparse boolean array (with bit sum) operations?

As for as the “switch of ISA” is concerned, what do you think CUDA is? What I mean by that is there is a lot of work out there to adapt software to special purpose hardware motivated by machine learning. I don’t see why a pretty substantial amout of that couldn’t be peeled off to make the compilers more intelligent for matching the Mill ISA to the hardware market.

Can the Mill serve as the “simple GPs”? How many transistors does one Mill GP take if its architecture is biased toward sparse array matrix multiplies and/or sparse boolean array (with bit sum) operations?

We clearly have a home in the control-processor role, but the actual ML bit-banging sure seems like it needs a dedicated architecture, not a general purpose one. Mill can of course put in the same operations and accelerators as any other CPU. It can do it with a faster manufacturing turn too, because of the specification-based design. I’m pig-ignorant about ML, but my impression is that the problem is not the computation, it’s all about getting data from here to there. Bio is self-modifying and basically analog, which Mill is not, nor is anything else built with the tools and fabs used for CPUs today. We do have some NYF stuff in the pipeline that addresses on-chip distributed memory, but frankly that’s for conventional programs, not ML.

You clearly are deep into the subject – care to tell what you’d like to have?

Ivan Godard writes:

care to tell what you’d like to have?

See section 4.1 FPGA Implementation of “Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks” by Kevin Hunter et al of Numenta, Redwood City.

From the abstract:

Using Complementary Sparsity, we show up to 100X improvement in throughput and energy efficiency performing inference on FPGAs.

There are a couple of things to keep in mind here:

1) Numenta has been approaching computational neuroscience from the top down — starting with neuroscience and attempting to figure out how the neocortex’s fundamental block (“column”) operates in computational terms. So they’ve done almost the opposite of the rest of the ML community which started from a “What can we compute with the hardware at hand?” perspective. While the rest of the ML community is stuck with the path dependence of graphics hardware (which is, unsurprisingly, fine for a lot of low level image processing tasks), Numenta has been progressively refining its top-down computational neuroscience approach to the point that they’re getting state of the art results with FPGAs that model what they see as going on in the neocortex.

2) The word “inference” says nothing about how one learns — only how one takes what one has learned to make inferences. However, even if limited in this way, there are a lot of models that can be distilled down to very sparse connections without loss of performance and realize “up to 100X improvement in throughput and energy efficiency”.

I have no conflict of interest in this. My background, while it extends to the 1980s and my association with Charles Sinclair Smith of Systems Development Foundation who financed the PDP books that revived machine learning, my most significant role has been asking Marcus Hutter (PhD advisor to the founders of DeepMind) to establish The Hutter Prize for Lossless Compression of Human Knowledge — which takes a top-down mathematical approach to machine intelligence.

PS: Not to distract from the above, but since I cut my teeth on a CDC 6600, there is an idea about keeping RAM access on-die somewhat inspired by Cray’s shared memory architecture on that series, but it is wildly speculative — involving mixed signal design that’s probably beyond the state of the art IC CAD systems if it is at all physically realistic — so take it with a grain of salt.

• This reply was modified 2 years, 7 months ago by  jabowery.

Interesting bedtime reading; thank you.

I had one concern: the compaction phase is static preprocessing, which is fine for a fixed corpus but doesn’t really work when the a priori weights are unknown. Compaction looks to be the bin packing problem, and you shouldn’t (I surmise) stick a NP step in the processing. I wonder whether true isolation of the kernels is really necessary though – if the kernels are sparse enough, shouldn’t it be possible to just slam them together randomly and let any collisions be “learned around”in the style of a Bloom Filter?

Mill cores build on the same processes as any other core. If the limit of core count is the mm^2 occupied by cache, pin pads, and other sparable regular structures then the cores-per-die count will be the same as other ISAs using the same amount of cache etc. If the limit is yield per wafer then we expect 2X more Mill cores, because the un-sparable space occupied by a Mill core is half the size or smaller than a conventional OOO architecture of similar performance, and it’s the part of the chip that you can’t use sparing on that dominates yield.