Deploy & compare local models
GGUF / llama.cpp / Ollama-style runtimes, quantisation choices, context, VRAM, latency and practical fit across NVIDIA and AMD hardware.
I’m a fintech founder, product architect and AI-native systems builder. I design and direct applied AI systems across hardware, software, institutional trust and product constraints — with a practical commitment to NVIDIA + AMD + Intel portability, not single-vendor lock-in.
AuthorityClaw — a working local-AI payments agent. The model reads invoices; a deterministic gate checks delegated authority before a simulated payment executes. Won first place and was invited to demonstrate at NVIDIA's Build-a-Claw London on 14 October.
I do not measure technical ability by whether I memorised the syntax. I measure it by whether I can frame the problem, specify constraints, direct the build, inspect the evidence, catch bad results and turn both results and failures into reusable systems and the next research question.
GGUF / llama.cpp / Ollama-style runtimes, quantisation choices, context, VRAM, latency and practical fit across NVIDIA and AMD hardware.
Use cloud reasoning as the planning/review layer while Linux-hosted CLI agents, local models and services do the execution.
Baselines, matched prompts, controlled conditions, synthetic cases, regression checks, raw evidence and explicit failure criteria.
Deterministic validation → First-Order Logic / Prolog → institutional policy / OPA, with AI kept outside the final authority path.
Local STT/TTS, image generation/editing, research pipelines and specialist model roles rather than forcing one LLM to do everything.
APIs, data models, onboarding logic, web/mobile surfaces, test suites and public technical artefacts — without pretending I hand-coded every layer.
The five numbered machines document equal parts of my professional, hands-on experience with inference, operating systems, hardware interoperability, memory constraints, agent interfaces and recovery. Each contributes a distinct architectural lesson to the same body of work.
Two independent NPU devices, each with a vendor-described 32 GB SoC memory pool and 48 GB dNPU memory pool. Black provided Qwen3.6 35B-A3B; White provided GPT-OSS-120B. Both were integrated as distinct AnythingLLM HQ endpoints, with inference and chat-history evidence.
The open research question is how SoC/dNPU memory placement changes actual useful work, not whether “80 GB” looks good on a specification sheet.
Original heterogeneous compute + Tiiny questions ↗Specialist models on AMD, NVIDIA and NPU endpoints connect through a shared workspace. The aim is to move perception and compute without moving authority: human mandates, deterministic checks and provenance remain separate from probabilistic output.
Five model workspaces and history readbacks were documented. That does not yet imply automatic routing, durable operations after all reboots or a financially self-sustaining autonomous organisation.
AuthorityClaw · separation of model and mandate ↗NVIDIA is a powerful reference platform, but I deliberately work across NVIDIA/CUDA, AMD/ROCm and Intel Arc/XPU as a portability research direction. The point is not to favour a logo: it is to learn which parts of the workload, memory and authority model can move between architectures — and where vendor-specific dependencies really bite.
Test the same useful pipeline across hardware and runtimes. Record API gaps and control-plane differences. A November CUDA-to-HIP softmax micro-sprint reached a CPU reference and a draft port, not GPU performance parity; that remains a precise open result.
Intel/Arc portability research agenda ↗Different memory arrangements change what can run, and at what latency or power. Experiments span FreeToken expert offloading, AMD dedicated VRAM, GB10 unified memory and Tiiny split SoC/dNPU pools. WiCi InferSSD, Strata and similar approaches are research targets, not claimed as fully validated on every device.
FreeToken measurements ↗My field notes do not stop at a PASS or a token-per-second chart. They preserve failures, falsifiers, unexplored dependencies, future tests and the practical human problem that would make the work worth repeating.
Six future research questions ↗The research boundary is intentional: running a proven model, drafting a port, verifying GPU equivalence and demonstrating commercial reuse are four different results.
My most useful technical thread so far: a reproducible synthetic demonstrator for regulated decisions, where model output must survive deterministic and formal checks before institutional policy can act.
A local demonstrator that treats the LLM as a perception layer — not as the final decision authority. The public repo includes a reproducible path, committed artefacts and CI.
Not a gallery of every page I have made. A short evidence set showing the range: infrastructure, evaluation, identity systems, formal reasoning and AI-assisted product builds.
A founder-led architecture and learning programme spanning the five numbered Olares, AMD/NVIDIA and DGX Spark machines, standalone Tiiny appliances, operating systems, AI runtimes, AnythingLLM workspaces and governed agents. My role is to define the problem, architecture, acceptance criteria and failure boundaries and direct AI-assisted implementation — not to pretend I hand-coded every component.
Why: to make AI tools reusable for real work and test whether a personal agent infrastructure can eventually become economically productive. The key obstacle is not token speed; it is reliable hand-off, persistent state, remote access and authority.
A local Qwen model reads synthetic supplier invoices. A separate deterministic authority gate checks principal, capacity, mandate, scope and revocation. Only a valid decision receipt can reach the simulated payment rail — including under bank-detail changes and prompt-injection attempts.
Hands-on investigation of heterogeneous inference across NVIDIA DGX Spark (GB10), Olares One, AMD Radeon and NVIDIA GPUs, plus Tiiny NPU appliances. I compare memory architecture, model offloading and practical usefulness — not merely token rates.
AI-assisted self-sovereign identity reference system with web + mobile surfaces, PostgreSQL, OpenAPI 3.1, generated clients and a branching KYC/KYB onboarding engine.
Experiments around whether natural-language model outputs can be translated into explicit, inspectable logic and then constrained by deterministic or institutional policy layers.
A set of experiments asking a harder question than “can an agent act?”: under what evidence, mandate, scope and revocation state should an agent be allowed to act?
Local speech recognition and TTS on a dedicated NVIDIA worker; ComfyUI image generation and editing on the main node; specialist workloads kept separate from the main LLM path.
The domain behind much of the technical work: reusable identity, compliance-grade architecture, proof-of-personhood, government wallets, KYC/KYB and AI-era institutional trust.
My useful skill is the loop around the code: turning a hard problem into an executable experiment, using specialist AI and tools as leverage, and refusing to accept a green screen without evidence.
Not a keyword cloud. These are tools, layers and concepts that have appeared in working experiments or current builds.
| Layer | Current toolbox | What I use it for |
|---|---|---|
| Local LLM | Qwen · Nemotron · Mistral · Gemma-class experiments · abliterated sandbox models | Inference, extraction, synthesis, coding, adversarial comparisons. |
| Runtime | llama.cpp · Ollama · LM Studio · FreeToken experiments | Local serving, GPU offload, quantised inference, heterogeneous hardware tests. |
| Agents | ChatGPT/Claude / Codex/Caude CLI · desktop/CLI hand-off patterns | Plan → execute → inspect → revise loops across cloud and local environments. |
| Formal | SWI-Prolog · First-Order Logic · OPA / Rego · Z3-direction experiments | Make assumptions explicit; keep policy and AI output separable. |
| Research | Local Deep Research · SearXNG-style search layer · local LLM synthesis | Evidence-led research without treating the base model as a current-information oracle. |
| Media | ComfyUI · FLUX-class image models · Whisper · local TTS | Local image generation/editing and speech workflows. |
| Product build | OpenAPI · React / Vite · React Native / Expo · PostgreSQL · Docker Compose | AI-assisted reference systems and prototypes with inspectable interfaces and tests. |
| Evidence | Git · GitHub Actions · hashes · manifests · raw outputs · regression suites | Turn a demo into something another person can inspect and rerun. |
Banking, fintech, compliance and digital identity taught me to care about who is allowed to do what, on whose evidence, under which rules.
Not a career change — the same profession.
Only the subject of trust changed: first the institution, now the model.
That is why my technical experiments keep converging on authority, evidence, provenance, identity, local control and regulated decision-making.
I am not trying to become a better programmer than programmers. I am learning to become very good at turning difficult institutional problems into systems that can be built, attacked, measured and inspected.
I’m most useful where product, AI, regulation, identity, infrastructure and institutional reality collide — and where a slide deck is not enough. I like turning the problem into something we can run.