Try DGX Spark playbooks using Nix on DGX OS, or install NixOS on your DGX Spark for the full Nix experience. The repository provides USB images and a NixOS module with settings for DGX Spark systems.
This works on the NVIDIA DGX Spark itself and also on the Asus Ascent GX10.
See my 5 minute lightning talk from Planet Nix for an intro: https://youtu.be/AvK_gi_snJE?si=MPKv3iiuS9B5elIE
hamandcheese11 hours ago
Slightly off topic, but Claude Code (and likely other models/harnesses) are incredibly effective at Nix. It can trivially self-verify, without side effects, which is a perfect match for an LLM.
If you've ever been put off by the difficulty of the language, it's worth checking it out again with AI assistance.
HellsMaddy9 hours ago
Absolutely. Running NixOS is a wonderful experience for this reason. NixOS + LLMs make it a breeze to make changes to your system, install and configure new software, and debug issues. Having every aspect of your system defined in a git repo is the perfect fit for agent harnesses.
I'll admit, even with LLMs to help, the Nix language and NixOS did have a rather steep learning curve, because it's quite different from anything I'd experienced before. But after getting the hang of it, I can't imagine going back to a "normal" OS and I'm very happy I put in the time to get over the initial friction.
dhon_8 hours ago
You can also push deployments to remote systems which is great for servers and headless devices like raspberry pis. I use colmena and deploy-rs on different projects because they have some nice features but you can also do it with bare nix.
I have a repo for kiosk appliances that can build an SD card image for a specific device in a fleet (so no on device configuration is required) but it can also push out updates or changes at runtime. NixOS isn't perfect for embedded devices yet but works well for cases like this in my experience.
bergkvist5 hours ago
If you are using an Apple Silicon laptop and you want to deploy to an x86-64 Linux box, this can get a bit annoying due needing to deal with cross compilation in nix.
graham33op24 minutes ago
You can have an x86-64 remote builder, or use a service like nixbuild.net, rather than use cross-compilation.
miki1232117 hours ago
And to add to that, LLMs start with fresh context on each conversation, so they don't have the understanding of your system that human employees do. Nix lets you naturally encode that understanding in code (and in comments), making sure it never drifts from what the state of the system really is.
I never saw much point in Nix, but if you put it like that... now I want to try.
rgoulter5 hours ago
Nix (& NixOS) are very much in the same "as code" as Terraform. -- You put in extra effort now, to save effort later.
This isn't traditionally 'practical'. It's quicker to just install the package, compared to writing a declaration that says to make the package available. (And for ops, Docker images won out in terms of practicality).
Nix also quite a steep learning curve.. I expect a beginner just wants to learn how to write a package in Nix, and not learn about 'derivations', 'instantiation', etc.
Interesting to see LLM coding agents adjust that trade-off: significantly reducing the cost to "write something in Nix", so the benefits become much cheaper.
aomix10 hours ago
I got curious one Saturday and ported our monorepo to use Nix for build and test and release. It was a very pleasant process. I had to put aside to tackle more pressing things but I'm all in on using Nix in that capacity.
sohrob7 hours ago
I set up my homelab server running NixOS entirely using Claude and what would have likely taken me months (if not years) doing it the old fashioned way, got reduced to a few hours of back and forth iteration.
mbo8 hours ago
Same for Bazel. It has historically been a very difficult to understand/adopt/rollout tool for a lot of engineering orgs but is going swimmingly for the organization I currently work for.
arikrahman4 hours ago
Likewise, I don't know where my dotfiles would be without deepseek/reasonix model configuring my entire OS for almust free with cache hits.
Loeffelmann11 hours ago
I also love how with the right system prompt they can pull in tooling for what they currently need via a nix shell
sunaookami8 hours ago
Also incredibly useful with "comma" so Claude can directly use tools that are not installed yet or only one-off: https://github.com/nix-community/comma
graham33op10 hours ago
Agreed, Claude has helped a lot with this project, and being able to iterate without side effects for system configuration changes is really a game changer for agents.
colordrops11 hours ago
I've one-shotted custom distributions built with Nix using AI. It's crazy how well AI and Nix fit together.
redrove12 hours ago
Been running this on a few Asus GX10 machines with k3s on top, it’s been great. I’m running the new deepseek.
Thank you for your work!
pixelesque12 hours ago
What quant are you using, and what tps are you getting with K3?
redrove11 hours ago
The FP8 version from DeepSeek themselves [0], around 1800 tps prefill and 45 tokens per second decode.
I’ve been running a custom VLLM image with b12x as well as nvfp4_ds_mla.
I would say it’s quite fantastic in day to day, I use it mostly in Hermes and sometimes for coding.
I have qwen 3.6 27b on an rtx 6000 pro as well so I use that as a workhorse in pi with DS as a reviewer/planner.
[0] https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731
Edit: I think you may have misread my post. k3s is NOT kimi k3, and I did mention I was running deepseek.
pixelesque10 hours ago
Thanks - yeah, sorry, I mis-read that as you using both DS and K3...
reinitctxoffset6 hours ago
I'm doing K3 with SoL kernels, zero hassle, myelin to sweep up the crap, on shot deploy to vast or runpods. Free to the community.
I have to do like, low paying web dev to fund it, so I can't promise timelines, but it's coming and it will be free to anyone and fast as fuck.
I estimate about 20 bucks an hour at current spot rates in the hundreds if not thousands of tokens per second.
reinitctxoffset3 hours ago
@pg @dang
you're asking me how a watch works. let's just try to keep an eye on the time.
federal felony prosecution.
reinitctxoffset7 hours ago
i also used nixos as the base, but i went a little overboard with it, this is 7.1.2 supporting all devices and a custom driver that fixes the UMA page leak issues, and the wallpapers are all kernels at or above parity with what you can get out of the box.
i'm not quite ready to OSS the whole thing, it's got a few rough edges, but if anyone wants to alpha test, caveat emptor and it's yours.
ianlevesque7 hours ago
A fix for the leaks would be great. I’m rebooting them way too often when iterating.
reinitctxoffset6 hours ago
reinitctxoffset6 hours ago
That short got like, way more views than I thought. This is a little more of the complex plane math.
[deleted]6 hours agocollapsed
mkagenius9 hours ago
There is also a microvm.nix project which helped us support sandboxes with firecracker. So, whole ai workflow pipeline can now be nixos.
nixie-tubes13 hours ago
This has been amazingly helpful for managing my DGX Spark! Thank you for all your time and effort into this project!
graham33op13 hours ago
Good to hear, thanks!
ronef11 hours ago
Huge plus to anyone interested in the space to check out what Graham built here!
If anyone is also interested on Nix/CUDA/Capital Markets/Flox, we recently did another case study in the space - https://flox.dev/blog/deploying-hardened-flox-nvidia-cuda-st...
thenobsta11 hours ago
This is incredible. I have a Jetson lying around and will try to it out on this. I use it to play with vision models, not LLMs, and have been wanting a better way to manage the machine.
reasonabl_human3 hours ago
you probably want this if you have a jetson vs this spark repo:
haunter11 hours ago
Thanks for sharing, saving this for when I get a DGX Spark
hamza715911 hours ago
[dead]
legastenigga9 hours ago
[dead]