Spinifex open-source AWS-compatible cloud for your own bare metal — EC2, S3, VPC, EKS, RDS

[serverguy12]

Disclaimer: I'm an engineer at Mulga, the company behind this, I'm a long-time STH reader but not a frequent forum contributor. Self-promo, but it's AGPL-3.0 and free to run.

Spinifex reimplements the AWS APIs on hardware you own: EC2, EBS, S3, VPC, IAM, ALB/NLB, EKS, ECS, ECR and RDS. It's real compute: instances are actual QEMU VMs, and VPCs are real OVN networks with security groups and elastic IPs. It's not an emulator like LocalStack.

Why not existing FOSS

Most infrastructure tooling, docs and skills today assume AWS. If you want that model on your own hardware, the existing options each miss something:

• LocalStack and similar emulators mock the API for testing. Nothing really runs, and nothing persists like a real cloud.
• OpenStack is a real private cloud, but it has its own APIs, so your AWS Terraform, SDK code and knowledge don't carry across.
• Proxmox is a great hypervisor, but it isn't a cloud API. There's no VPC, IAM, S3 or EKS layer, and the Terraform is completely different. And let's be honest, CEPH is a pain.

So we built Spinifex so the AWS API itself is the interface: the same CLI, SDKs and Terraform, pointed at your own box or multi-server cluster.

Why it's relevant

• Move workloads off AWS to your own infrastructure, while maintaining a "local" AWS cloud you own. Same SDKs/API endpoints run.
• Learn AWS without the bill. You can build the VPC-plus-EC2-plus-RDS stack from a cert course and leave it running for a month without a surprise invoice.
• Your Terraform works in both places. Keep one AWS profile for work and another for the rack in your home-lab, or a co-location provider for prod.
• Single-node install from a USB or ISO. It comes up as a working one-node cluster, and you can join more nodes later.

AI Disclosure

Spinifex is designed and built by infrastructure engineers, including former AWS team members. We use Claude Code to speed up certain tasks: boilerplate for AWS API handlers, documentation, generating test cases, and large refactors. Architecture, security (authentication, encryption, threat modelling), performance tuning and production operations are done by people.

Repo: GitHub - mulgadc/spinifex: Spinifex – Open source AWS-compatible platform for secure edge deployments. Run EC2, VPC, S3, and EBS-like services on bare metal with full control. Built for environments where running in the cloud isn’t an option. Whether you’re deploying to edge sites, private data-centers, or operating in low-connectivity or highly contested environments.

Site: Mulga | Run AWS workloads on any compute, zero re-write

--

I welcome feedback as a comment on the forum from the STH community for roadmap ideas and stories installing Spinifex on your home-lab or rack!

 

[MBastian]

While eyeballing your code and commit history, I think you're massively underselling AI usage. But for once I am intrigued, mainly because there are real people behind this. Let's see where this goes.

My main gripes right now (v1.20.0):
There is no live migration and no automatic restart of VMs on another node. If a host dies, its VMs are down until someone brings that host back.
Same for maintenance, you can't move VMs off a host first, they just get shut down.
Load balancers and RDS are also just single VMs with no failover. ECS and the EKS control plane do recover by themselves, that part works.
Currently there is no S3 versioning or bucket policies. Also no SAML/MFA/permission boundaries in IAM. No VPC peering, no managed VPN and no VPC endpoints or network ACLs. RDS is single instance only.

Imho fine for labs or edge sites where you can live with waiting for a fix but not for production if you expect AWS-style availability, unless you build the redundancy into the application yourself.

 

[serverguy12]

Thanks for the valuable feedback @MBastian and taking a deeper look at our implementation.

For our v1.X release we designed Spinifex for a single AZ deployment, with our goal to be feature parity with AWS Outposts for *core* features which we have recently reached.

The original idea to launch our startup and build Spinifex was "how do I move my AWS workloads to hardware I have at the edge, or operating in an air-gapped environment, without a rewrite?".

Our roadmap includes to extend Spinifex to support multi-AZ and more resilient Enterprise workloads, so stay tuned, we are busy building & shipping.

>There is no live migration and no automatic restart of VMs on another node. If a host dies, its VMs are down until someone brings that host back.

We will be launching the auto EC2 healing support in the next update, it's an important feature we are working on.

Recently we have made a number of enhancements that can support this feature, particularly our backing EBS/S3 implementation (GitHub - mulgadc/viperblock: Viperblock: High-performance block storage with S3 compatibility for edge to on-premise deployments. Features log-structured writes, block-to-object mapping, and configurable chunk sizes. Supports multiple backends with checksumming and intelligent caching for VM and container storage needs.) for improved fault protection and redundancy.

For example if a cluster is used for Spinifex (min 3 nodes required), EC2 instances EBS volume are correctly replicated between the N servers to provide a level of redundancy. If node-1 crashes or the hardware taken offline, we can move the EC2 resources to an available node so long as we have consensus in the cluster.

>Same for maintenance, you can't move VMs off a host first, they just get shut down.

Yes, in the current implementation if a hardware node is shutdown, the EC2 instances running on the hardware will be in the "stopped" state. However, if a cluster has consensus, these EC2 instances can be started again on an available node (manually, auto will be supported in the next update as above)

>Load balancers and RDS are also just single VMs with no failover. ECS and the EKS control plane do recover by themselves, that part works.

In the current single AZ design for Spinifex, we leverage the node-placement group in EC2 (spread) which ECS/EKS use to run its control plane across unique hardware.

We are aiming to have a similar design for a multi-AZ version of RDS/LB with the current constraint as a single-AZ.

>Currently there is no S3 versioning or bucket policies.

S3 versioning is something we'd like to support soon in our S3 implementation (GitHub - mulgadc/predastore: PredaStore is a high-performance, on-premise and edge-ready object storage platform, fully compatible with the Amazon S3 API. Designed for environments where speed, redundancy, and resiliency are critical, PredaStore is the ideal solution for edge data centers, private clouds, and hybrid deployments that demand low latency and high availability.) however not on the immediate roadmap.

For bucket policies, we support the basics via IAM.

>Also no SAML/MFA/permission boundaries in IAM.

To round out our v1 we will be supporting MFA and improved permission boundaries, and SAML is something we are considering given customer demand.

>No VPC peering, no managed VPN and no VPC endpoints or network ACLs.

VPC peering/VPC endpoints are important to round out our v1 on the roadmap, however we have not had as much demand for the managed VPN at this stage.

--

We are aiming for at least a fortnightly cadence to our release cycle, so keep an eye out for the next update that will include the EC2 auto healing!

 

[serverguy12]

Hi @MBastian we shipped Spinifex v1.21.0 today that includes the EC2 auto-heal feature across multiple nodes (minimum 3 to create a cluster)

Release v1.21.0 · mulgadc/spinifex

What's Changed GPU Compute Multi-GPU instance types, with up to 8 GPUs passed through to a single instance Multi-GPU sizes such as g5.48xlarge and p4d.24xlarge get the same GPU count as on AWS, up...

github.com

Each EC2 instance has a heartbeat check between nodes in an AZ cluster, after 3 failed heartbeats the instance is launched on a node in quorum that has available resources within a ~2 minute period.

To simulate an environment this is added as a test case for our e2e-nightly sweep on a bare-metal instance, e.g:

E2E Nightly · mulgadc/spinifex@8ab96b2 > Run Multi E2E Suite > Suite instancerecovery

Feedback welcome on the first implementation