Small business
Small Business Private AI
A local server that a team can rely on: redundancy, backup, UPS and 96 GB of ECC VRAM.
reliabilityredundancydata stays on-site
updated 16 Sept 2026Total price
$15,889
VRAM
96 GB
RAM
256 GB
Storage
12 TB
Est. power draw
1144 W
AI capability
Roughly ~40B params in fp16, ~80B in 8-bit, or ~147B in 4-bit quantization.
small-model inferencemid-size inferencelocal RAGlarge-model inferenceLoRA fine-tuningconcurrent serving
Components
- GPU2× NVIDIA RTX A6000memory: 48 GB GDDR6 ECC · bandwidth: 768 GB/s · interface: PCIe 4.0 x16$8,400Check price ↗
- CPUAMD Ryzen Threadripper 7960Xcores: 24 · threads: 48 · clock: 5.3 GHz boost$1,399Check price ↗
- MotherboardASUS Pro WS TRX50-SAGE WIFI (sTR5)chipset: TRX50 · memory: 4 x DDR5 RDIMM ECC, quad-channel · pcie: 4 x PCIe 5.0 x16$700Check price ↗
- RAMMicron 256 GB (8x32) DDR5-4800 ECC RDIMMcapacity: 256 GB (8 x 32) · type: DDR5-4800 RDIMM · ecc: ECC$1,050Check price ↗
- StorageKioxia CD8-R 3.84 TB U.2 NVMecapacity: 3.84 TB · bus: PCIe 4.0 U.2 · read: 7000 MB/s$620Check price ↗Sabrent Rocket 4 Plus 8 TB NVMecapacity: 8 TB · bus: PCIe 4.0 NVMe · read: 7100 MB/s$700Check price ↗
- PSUSuper Flower Leadex Titanium 1600 W 80+ Titaniumcapacity: 1600 W · rating: 80+ Titanium · standard: ATX 3.1$450Check price ↗
- CoolingArctic Liquid Freezer III 360 (AIO)type: liquid AIO 360 mm · dissipation: up to 350 W · noise: 22-33 dBA$110Check price ↗
- NetworkingIntel X710 10GbE SFP+ dual-port NICspeed: 10 GbE · ports: 2 (SFP+) · bus: PCIe 3.0 x8$180Check price ↗MikroTik CRS309 8x 10GbE SFP+ switchports: 8 x SFP+ 10 GbE + 1 GbE · throughput: non-blocking · form factor: desktop / 1U$280Check price ↗
- UPSEaton 9PX 3000 RT (online double-conversion, 2U)capacity: 3000 VA / 2700 W · topology: online double-conversion · runtime: ~7 min at full load$1,600Check price ↗
- ChassisSilverStone RM44 4U rackmount (4 dual-slot GPU)form factor: 4U rack · gpu slots: 4 dual-slot · drive bays: 8 x 3.5 + 2 x 2.5$400Check price ↗
Component prices are hand-sourced and dated (checked 2026-09-16), not scraped. Confirm on the vendor's site before buying.
Why this build
- · Two RTX A6000s: 96 GB of ECC VRAM in blower cards designed to sit next to each other and run 24/7. 70B in fp16, or several mid-size models resident at once.
- · Enterprise U.2 storage (1 DWPD endurance) for the shared dataset, plus an 8 TB NVMe for models and checkpoints.
- · A 3000 VA online UPS regenerates the mains: the server never sees a sag or a spike, which is what kills hardware early.
- · A 10 GbE switch lets the whole team hit one inference endpoint without saturating the link.
- · Everything is on-site - no data leaves the building, which is the reason most small teams self-host in the first place.
Pros
- + 96 GB ECC VRAM, built for continuous load
- + Redundant power, enterprise-endurance storage
- + Full data control - nothing leaves the premises
Limits
- − Single node: a motherboard failure takes the service down
- − A6000 is Ampere - a 4-card Ada server is faster for the same VRAM
- − Needs a ventilated cupboard or small server room
Alternatives
Cheaper
Four RTX 3090s reach the same 96 GB for far less, at the cost of blower cooling and warranty.
Balanced
Two RTX 6000 Ada: same 96 GB, roughly double the throughput, ~$5,000 more.
Buy vs rent vs hybrid
Over 36 months, at this build's power draw and the live cloud rate for its GPU.
Own it
- Hardware
- $15,889
- Electricity
- $2,480
- Total cost of ownership
- $18,369
Rent it (cloud)
- Equivalent cloud, monthly
- $475/mo
- Equivalent cloud, total
- $17,114
- Break-even
- 39 months
- · Break-even is 39 months - past the 36-month horizon, so renting wins over this period.
"Buy on Amazon" links are affiliate links - as an Amazon Associate, Obolith earns from qualifying purchases.