Arterialistop2 days ago
Hey guys, I had a few esp32 boards around for a while and always thought - how do we go beyond just one device, would it be possible to apply cloud-native thinking to distributed edge computing?
Result is Kube32. I got the cluster and a full-stack web app with real-time sync running on esp32 s3 dev board and cardputer - https://7f0c-180-74-71-21.ngrok-free.app/ (may go down) (should support at least a few concurrent users - break it)
You won't see one cool thing though, which is the hud pod that uses Cardputer as a keyboard that syncs keystrokes to all clients in real-time.
Honestly, it opens so many possibilities for distributed edge computing and multi-device systems. Let me know what use-cases do you think would benefit from kube32 the most.
ZenoArrow21 hours ago
Thank you for working on this. The most impressive part for me is getting Kubernetes running on such low spec hardware. I can't think of a use case I have for Kube32 at the moment, but I do have use cases for Kubernetes running on a small amount of RAM. Is there any chance of porting this back to x86/x86-64 PCs or ARM SBCs? For example, being able to run this on a cluster of Raspberry Pis would be interesting as it would free up more RAM for applications.
NewJazz20 hours ago
I feel like a gradualist approach may be better for those applications. Start with "vanilla" k8s, and iteratively target subsystems to shrink down. Either by porting to Rust or simply optimizing the Go code for low RAM environments.
Arterialistop13 hours ago
True, for RPI it makes more sense. For microcontrollers it's a different story :D
I think it'd be great to put some effort into this, as a separate project perhaps.
Arterialistop20 hours ago
Hmm, have you tried k3s? It promises low ram, but it could still be too much, I'm not sure about your requirements. In the meanwhile I'll see if it can be pushed further, but I guess we both know the answer :)
ahmeni19 hours ago
k3s does well on low ram but I've found it still uses a chunk of CPU.
gawacer19 hours ago
[dead]
everforward12 hours ago
Super cool! One thought that would make this way more useful to me (and probably a thousand times more complicated, sorry) is if it had a way to split out “GPIO” subroutines from the main app, and schedule them separately. So GPIO would become an addressable unit like pods or load balancers in k8s.
The idea is I could specify something like “I have this app that needs to pull data from a GPIO labeled as a ds18b20 and outdoors” and it takes care of scheduling my app, starting a ds18b20 reader, and networks them together. I never need to know if they’re on the same esp32, separate ones, or in a crazy alternate world the compute bit goes on a VM on a laptop with no GPIOs and the sensors go on a real esp32.
Having GPIOs be an addressable resource would be so cool. Like if nothing else, I could start a new esp32 project without wiring up a breadboard. Most of my stuff is some combination of the same 10 chips anyways.
Arterialistop12 hours ago
Yes! This is the philosophy. Omw to ship!
catlifeonmars20 hours ago
What kind of overhead does the Wasm approach add in terms of memory and cpu time? I also have way too many ESP32 boards lying around
Arterialistop20 hours ago
I'll measure and let you know later, would appreciate if can do measurements on your hardware too as I only have a few options to test on
vrighter21 hours ago
cloud native emedded devices are an oxymoron
wormik18 hours ago
[dead]
rayshena day ago
What's the benifit?
Arterialistop20 hours ago
there are few imo:
- trustless code execution on someone's hardware (kube32 supports gpio permissions, amongst other stuff) - your standard Kubernetes cluster can now reach the physical world natively - OTA updates at arbitrary scale - fault-tolerance in the physical world, orchestration of workloads based on physical proximity or other sensor data
those are just a few that come to mind
goodpoint16 hours ago
None of this needs the overhead of k8s
Arterialistop13 hours ago
Exactly, and I discarded the idea of porting k8s, building k8s-compatible system instead. It still carries some overhead, yes, and I'm yet to measure it fully, but it's a good trade-off for the applications it unlocks. If we remove the overhead form equation, do you think the capabilities I mentioned are unlocked with k8s-like system, or do you have a way to achieve all of them at the same time using something else in mind?
hbcdbff16 hours ago
Slop
I-overlay17 hours ago
[flagged]
professional_to18 hours ago
[dead]