dense
Dense vs. MoE Models: Active Parameters, Throughput, and When to Choose Each
How can a 30B-parameter model activate only 3B parameters per token, and still use the capacity of the larger model? Nemotron 3.5 Lightning illustrates the...
Efficient MoE Training for Biological Foundation Models
As language models grow, scaling dense architectures becomes increasingly expensive. In a dense transformer, every token passes through every layer, so adding..…
Faster Scientific Simulations with New FP64 Algorithms in NVIDIA cuBLAS
Many scientific simulations, from molecular chemistry to materials science, spend the majority of their execution time computing high-precision dense general...





