I have 384TB of ECC DDR4 across two blades with 4 CPUs for a combined core count of 96.

@cursedsql @agowa338 @bob_zim @SecurityWriter I don't know what's typical for these things with solid state, but with spinning rust (and a few years ago) large arrays typically didn't use drives much bigger than about 600GB. The preference would be for more drives, rather than larger. The reason for that was to do with failure rates, rebuild times and bandwidth.
The maths might have changed with the technology, but I'd suggest that if you're using SSDs then your focus is probably response time and bandwidth rather than storage density, so I'd expect smaller rather than larger drives.

@GerardThornley @cursedsql @bob_zim @SecurityWriter

Or you want to place it in an environment where it has to deal with heavy vibrations. Like on a moving trolly or in a vehicle or ... there are multiple reasons for this. It may even just be because you need high random IO speeds...

And the sizing also depends on what you're using it for. Like e.g. if you get your data in to the system in infrequent busts but at multiple TB/s and you've to cache it until it is synced even to SSDs, well

@agowa338 Cards like that exist, but they don’t hold thousands of DIMMs.

@GerardThornley @cursedsql @bob_zim @SecurityWriter

Or you want to place it in an environment where it has to deal with heavy vibrations. Like on a moving trolly or in a vehicle or ... there are multiple reasons for this. It may even just be because you need high random IO speeds...

And the sizing also depends on what you're using it for. Like e.g. if you get your data in to the system in infrequent busts but at multiple TB/s and you've to cache it until it is synced even to SSDs, well

@GerardThornley @cursedsql @bob_zim @SecurityWriter

(The later was an example from scientific environments. I think it was CERN but I'm not sure...)

@agowa338 @cursedsql @bob_zim @SecurityWriter Yeah, that's sounds pretty plausible for things like the LHC experiments.

@bob_zim But PCIe lane splitters and extenders also exist. And I don't know what the highest archivable density of these cards currently is.

I so far have only had one old one in my hands and seen them in slides in class at my job training about 10 years ago (they were mentioned as accelerator cards primarily used for things like MS Dynamics and SAP databases)...

@agowa338 It’s more about the physical space for the cards. Most hold eight DIMMs. Holding 3072 would take 384 cards. That’s nearly a full rack just for the RAM cards, not counting the persistent drives. No way would you run that with even two entire blade frames, let alone two blades.

@thechris @SecurityWriter

Don't forget to mention AI ️

I have 384TB of ECC DDR4 across two blades with 4 CPUs for a combined core count of 96.

It powers a fully populated 192 disk solid state SAN.

I was told it was old and in need of replacing, but apparently now it’s worth more than the GDP of the UK.

Can’t afford to run it (or hear my thoughts when in the vicinity)… but I can sit atop it like a fucking dragon.

And I will.

@SecurityWriter

Don't Breath Fire !!!! you evil dragon !!!

@bob_zim

Hmm true. Makes it quite unlikely.

However I want to push back on one thing. It would still be theoretically possible, as PCIe fabrics and such also exist.

But yea, probably not then. I didn't do the math for the space requirements....

I have 384TB of ECC DDR4 across two blades with 4 CPUs for a combined core count of 96.

It powers a fully populated 192 disk solid state SAN.

I was told it was old and in need of replacing, but apparently now it’s worth more than the GDP of the UK.

Can’t afford to run it (or hear my thoughts when in the vicinity)… but I can sit atop it like a fucking dragon.

And I will.

I have 384TB of ECC DDR4 across two blades with 4 CPUs for a combined core count of 96.

It powers a fully populated 192 disk solid state SAN.

I was told it was old and in need of replacing, but apparently now it’s worth more than the GDP of the UK.

Can’t afford to run it (or hear my thoughts when in the vicinity)… but I can sit atop it like a fucking dragon.

And I will.

I have 384TB of ECC DDR4 across two blades with 4 CPUs for a combined core count of 96.

It powers a fully populated 192 disk solid state SAN.

I was told it was old and in need of replacing, but apparently now it’s worth more than the GDP of the UK.

Can’t afford to run it (or hear my thoughts when in the vicinity)… but I can sit atop it like a fucking dragon.

And I will.