TL;DR — The film above wasn’t edited by a human. I typed one sentence — “explain how hosted agents work on Azure” — and Video Gen Studio wrote it, storyboarded it, rendered every scene with Sora-2, narrated it, and finished it with captions, a music bed and brand bumpers. ~11 minutes, prompt to premiere, on the live studio. The nice twist: the studio’s agents themselves ran on Azure AI Foundry’s hosted Agent Service — so this is hosted agents explaining hosted agents.

The one line I typed

No shot list, no script, no assets. Just a sentence and a style (Rugged Paper-Cut Stop-Motion):

“Explain how hosted agents work on Azure AI Foundry’s Agent Service: you define an agent’s instructions, tools, and model, and Azure runs it server-side — hosting, scaling, and securing it with managed identity — so teams ship reliable agentic apps without operating the orchestration themselves.”

Everything below — the beats, the narration, the visuals, the voice, the cut — the agents decided.

Six agents, one film

Video Gen Studio isn’t a single model call. It’s a crew, and each member is a Microsoft Agent Framework agent you can watch work in the telemetry panel:

1. ScriptWriter breaks the idea into beats (hook → context → explain → turn → payoff) and writes a tight narration line per scene.
2. Director plans the shots and pauses for my approval — the one human gate.
3. Renderer paints each scene with Sora-2, locked to a reference frame so the look stays consistent.
4. Continuity/QA checks the rendered frames with gpt-4o vision.
5. DubbingArtist casts a voice and narrates each scene (Azure AI Speech, en-US-Andrew).
6. Distribution finishes with a deterministic ffmpeg pass — rolling captions, a ducked music bed, and brand intro/outro bumpers.

The storyboard it wrote

From that one sentence, ScriptWriter produced this — six beats, one crisp spoken line each (12–17 words, tuned so the voice fits the shot):

And then Sora-2 rendered each line as its own hand-made paper-craft shot:

Every scene moves — the pacing engineering

This is the part I’m proud of, and it’s why the film feels finished rather than generated.

Early cuts had a subtle disease: Sora renders a fixed-length clip, but a narration line is a different length. When the voice ran long, the studio froze the last frame and held it while the narrator kept talking. On an earlier six-scene film, 26% of the runtime was frozen — a third of every scene was a still image with a voice-over.

So the render pipeline now fits the clip to the voice: it estimates each line’s spoken length, picks the shortest Sora length that covers it, and trims the scene to end exactly when the narrator does. Same six scenes, measured with ffmpeg freezedetect:

Shorter, snappier, and the picture moves the whole way through. No held frames, no dead air.

Made on Azure — by agents, hosted by Azure

The whole crew runs keyless on Azure with a single managed identity: Microsoft Agent Framework for the agents, Sora-2 for motion, gpt-image-2 for keyframes and reference locking, Azure AI Speech for the voice, gpt-4o vision for QA, and Azure Container Apps (VNet-integrated, deployed with azd) for the studio itself.

And the agents in this run executed on Azure AI Foundry’s hosted Agent Service — so the film about hosted agents was, fittingly, directed by hosted agents.

The best part: I never opened a video editor. I typed a sentence, approved a storyboard, and watched six scenes stream in. That’s the whole pitch — you direct; the crew builds.

Read how Video Gen Studio works →

TL;DR — cordless is a tiny Node daemon on your dev box that owns real terminal sessions (a shell, or claude / codex), plus a phone app that attaches to them like browser tabs. Close the app, switch networks, come back later — your sessions are still running and replay exactly where you left off. I designed it in a running conversation with GPT-5.6 Sol on Azure, built it with GitHub Copilot CLI, and verified it in a real browser and a real Android emulator. Live: naveenneog.github.io/cordless.

I keep leaving long-running coding-agent sessions on my dev box — a claude chewing through a refactor, a build, a shell mid-task — and then walking away from the keyboard. I wanted to check on them and steer them from my phone, the way I flip between browser tabs. Not SSH-in-a-box; something that treats each agent session as a first-class tab that keeps living when my phone sleeps. So I built cordless.

What it is

• Persistent sessions. The PTYs live in the daemon. Your phone disconnecting, backgrounding, or hopping from Wi‑Fi to cellular doesn’t kill anything. Reconnecting replays from your last-seen byte — or a full-screen snapshot if you were away a while.
• Tabs for terminals. Run several Claude Code / Codex / shell sessions at once and switch instantly, with an unread dot when a background session produces output.
• Touch-first. A real terminal is unusable with thumbs without help, so there’s an on-screen key bar: Esc, Tab, Ctrl/Alt (one-shot latches), arrows, Ctrl‑C/D, pipes, and paste.
• Reach it from anywhere. Tailscale is the recommended path; same‑Wi‑Fi LAN also works. No ports exposed to the internet.
• Web or native. Install the PWA straight from your phone browser, or grab the Android APK.

How it was built

Designed in tandem with GPT‑5.6 Sol

The interesting part of this build is that I didn’t architect it alone. I kept a stateful conversation with GPT‑5.6 Sol (deployed on my Azure AI Foundry) open the entire time — not one-off prompts, but a running transcript so it stayed consistent with every prior decision. Sol produced the initial architecture, and then, crucially, reviewed my actual code. That review earned its keep.

The agent: node-pty, not tmux

Each session is a real pseudo-terminal spawned with node-pty (ConPTY on Windows — which, pleasingly, built on Node 26 first try; a Unix PTY on macOS/Linux). I deliberately did not use tmux: the daemon is already the multiplexer, and tmux adds nested-terminal state, sizing quirks, and a Windows install problem. Every session is a PTY plus a headless @xterm/headless mirror (for snapshots) plus an 8 MiB replay ring.

The subtle bug Sol caught here: @xterm/headless’s write() is asynchronous, and reset() is not ordered behind queued writes. So I route every terminal write and every snapshot through one per-session op queue, and assign a batch’s sequence number inside the write callback — which keeps the replay ring, the sequence counter, and serialize() perfectly consistent. Output is coalesced into ~16 ms / 32 KiB batches, one sequence number per batch.

// one op queue per session; seq assigned only after the bytes are parsed
this._queueOp(() => new Promise((resolve) => {
  this.term.write(batch, () => {
    const seq = this._nextSeq++;
    this._pushRing(seq, batch);   // ring, counter and headless mirror advance together
    this._broadcast(seq, batch);
    resolve();
  });
}));

That consistency is what makes reconnect-with-replay honest: attach with your last seq, and the server either replays the ring from there or, if it’s rolled past, serializes the headless terminal into a single reset snapshot. tmux-style survival, without tmux.

The client, and the 12 bugs the review caught

The app is Vite + React + xterm.js, served by the daemon itself so the PWA is same-origin. I wrote the connection layer, thought it was solid… and handed the whole file to Sol for review. It came back with twelve real issues — not style nits, actual races:

• stale-epoch writes duplicating output after a reconnect,
• the duplicate-frame check gating on the wrong counter (applied vs received seq),
• duplicate-attach and detach-during-attach races,
• an ack timer that leaked across reconnects and cleared a fresh ack,
• Ctrl/Alt latches that could get stuck,
• “close tab” silently undone by the next session-list poll.

The fix pattern throughout: give every socket a connection epoch that every handler, timer, and promise verifies; serialize each tab’s writes through an apply-chain; and use generation counters for attach/detach and resize. Boring, correct, and exactly the kind of thing a fresh reviewer spots that the author’s brain has already glossed over.

Security, baked in from the start

Because a paired device gets shell access to my machine, security couldn’t be an afterthought:

• Per-device tokens, only their SHA‑256 hashes stored on the box; revoke any device by id.
• Single-use, rate-limited QR pairing — and the pairing secret rides in the URL fragment, so it’s never sent to (or logged by) the server.
• An Origin allowlist on the WebSocket and pairing endpoints (blocks malicious web pages / DNS-rebinding), a strict CSP (script-src 'self', no inline JS), and no-store on every credential-bearing response.
• The daemon warns if you run it as root/Administrator and binds with a clear least-privilege note.

There’s a small automated suite that asserts all of this (cross-origin pairing → 403, cross-origin WebSocket → rejected, headers present) so it can’t quietly regress.

The bug only a real emulator could find

Here’s my favorite part. Everything worked perfectly as a PWA — because the browser served the app and the agent from the same origin. Then I packaged the Android APK with Capacitor and installed it on an emulator. Pairing failed instantly: “Failed to fetch.”

The cause was genuinely educational. Inside a Capacitor WebView the app’s origin is http://localhost, so talking to the agent is cross-origin — which triggers a CORS preflight my server had never needed to handle, because the same-origin PWA never sent one. The fix was to add CORS scoped to the existing Origin allowlist (echo Access-Control-Allow-Origin for allowed origins, answer the OPTIONS preflight, and — Sol’s catch — answer the Private Network Access preflight that Chromium sends when reaching a LAN/Tailscale address). A bug that would have broken every native-app user, invisible until I drove the real APK.

After the fix, the emulator paired, connected over WebSocket, attached a session, rendered PowerShell output, and replayed full session history on reconnect — the whole loop, natively.

Shipping it

The Android APK is built by GitHub Actions on every v* tag (debug-signed, androidScheme: http + cleartext so ws:// to a Tailscale IP works) and attached straight to the release. The landing page is GitHub Pages. Networking is Tailscale-first: WireGuard encrypts the hop, you get a stable *.ts.net name, and port 7443 never touches the public internet.

The good

Sessions that outlive the client turn a phone from a read-only status screen into a real remote control for long-running agents. Check a claude refactor from the sofa, unstick a build on the train, tap Ctrl‑C on a runaway process — then walk back to your desk and the exact same sessions are there in the browser. And because the whole thing is token-gated, Tailscale-scoped, and never exposed publicly, it stays yours.

The meta-lesson: pairing a builder (me, via Copilot CLI) with a dedicated reviewer (GPT‑5.6 Sol) in a persistent conversation produced noticeably better engineering than either alone — the 12-bug review and the CORS catch are things I’d have shipped without. And testing on a real emulator, not just a browser is what turned “looks done” into “actually works.”

Try it

• ▶️ Live / install the PWA: naveenneog.github.io/cordless
• 📦 Android APK: github.com/naveenneog/cordless/releases/latest
• 💻 Source: github.com/naveenneog/cordless

Part of the #AI4Good series. Built one day at a time. — @naveenneog

TL;DR — Type one sentence. A crew of AI agents scripts it, storyboards it, and waits for your approval — then Sora-2 renders every scene, Azure AI Speech narrates it, and a deterministic ffmpeg finishing pass burns in rolling captions, ducks a music bed under the voice, and tops-and-tails it with brand bumpers. Choose from nine cinematic styles — or upload any clip and it mints a brand-new style from the look. Everything lands in a real editing timeline. Built on Azure, driven end-to-end by GitHub Copilot CLI in autopilot.

Prompt in, premiere out

Making a short explainer or story film is a pipeline problem, not a single-model problem. You need a script, a consistent visual style, scenes that actually match the narration, a voice, captions, music, and a bit of brand polish so it doesn’t look like a tech demo. Today you either hire that out or stitch together six tools by hand.

Video Gen Studio collapses the whole pipeline into one screen. You give it a sentence and a look. A team of agents turns it into a finished, narrated, brand-consistent film — and hands you a real timeline to recut it.

Here’s a single frame, rendered by Sora-2 in the Cinematic style from the prompt “a lone lighthouse keeper watches a storm roll in at dusk” — no stock footage, no camera, one line of text:

Meet the crew: an agentic film studio

The studio runs on the Microsoft Agent Framework. Each stage is a real agent with a job, and there’s a human gate in the middle — you approve the plan before a single expensive frame is rendered.

• ScriptWriter breaks your prompt into a beat-by-beat storyboard with per-scene narration.
• Director reviews it for coherence and picture-voice lock, then pauses for you. Approve, or send it back.
• Character-Consistency locks a reference so a subject looks the same across scenes.
• Renderer calls Sora-2 to render each scene (2 concurrent, so it’s honest about time).
• Continuity / QA looks at the actual rendered frames with gpt-4o vision and flags drift.
• DubbingArtist casts a voice and narrates every scene with Azure AI Speech neural voices.
• Distribution stitches, thumbnails, and hands off to finishing.

Scenes stream into the UI the moment each one finishes — you watch the film assemble itself.

The studio you actually edit in

This isn’t a “generate and pray” box. The output lands in a real editor: a proper transport (play / pause / scrub with time), a scene timeline with a playhead, a voice/waveform lane, and per-scene recut — reorder, cut, trim, re-voice (including Microsoft’s newest MAI-Voice-2 neural voices), or regenerate a single scene from an edited prompt while keeping the rest.

Nine looks — or bring your own

Style is a drop-in profile, so the same story can be a photoreal film, a cel-shaded anime, a claymation short, a neon cyberpunk piece, a watercolour storybook, or a black-and-white noir. Nine curated styles ship in the box, each with a few creative controls (lens, film stock, light, camera move…) that get injected straight into the render prompt.

Bring Your Own Style

Have a look you love? Upload any short video and the studio mints a brand-new style from it. A Video Analyser samples frames, gpt-4o vision extracts a transferable StyleDescriptor (palette, lighting, texture, motion — never the people or logos in the clip), a Style Researcher writes the master look prompt and its controls, and it’s compiled into a durable, reusable style tagged “yours.” It captures the look, not the copyrighted content — and flags any IP it spots.

Finish like a brand, not a demo

The finishing pass is pure, deterministic ffmpeg — no image quota, seconds not minutes — so branding is free and repeatable:

It respects trademarks

Because “make a video with <famous character>” is a real temptation, an IP / trademark guard agent screens every prompt and storyboard before rendering. It surfaces the risk right at the human gate — with the specific items it found, a recommendation (allow / abstract / refuse), and a ready-made generic rewrite — and leaves the final call to you.

Know the cost before you hit “go”

Generative video isn’t free, and the real bottleneck (Sora throughput, gpt-image-2’s 2-images-per-minute limit) is invisible until it bites. So the studio shows an honest ETA band and rough cost up front, with the bottleneck named — right in the render screen and in the Advanced tab.

The honest build journey

This whole thing was built on Azure, with GitHub Copilot CLI in autopilot driving the code, deploys, and tests. The interesting part isn’t that it worked — it’s the three bugs that only surfaced because of how it was tested:

• The studio was silently blank — and every API test passed. A React hooks-order slip (a useState after an early return) crashed the UI on opening any project. TypeScript builds and API checks were all green; it only showed up when a headless Playwright pass screenshotted the deployed app. Lesson: for UI, a real screenshot is worth a thousand green checks.
• Sora refused to render a person. Photoreal styles were using a generated character sheet as an image reference, and Sora’s moderation blocks reference images containing identifiable people (people-in-user-uploads). The fix: fall back to text-to-video for human/photoreal looks.
• Captions “hung” for minutes on a fresh deploy. Not a hang — the first libass subtitle burn was cold-scanning the container’s giant Noto font set. Baking fc-cache into the image at build time cut a >3-minute first render to ~30 seconds.

None of these show up in a happy-path demo. All three were caught by testing the deployed thing the way a user would.

Under the hood

Finishing (captions, music ducking, bumpers, watermark, aspect reframes) is all deterministic ffmpeg — which keeps branding free of any model quota and perfectly repeatable.

Why this is AI4Good

A film crew, a voice-over artist, a colourist, an editor, and a motion-graphics designer used to be the price of entry for a good explainer or story film. That priced out most teachers, tiny non-profits, heritage projects, and solo creators.

Video Gen Studio puts that whole crew behind one prompt — with the guardrails (a human approval gate, IP screening, honest cost estimates) that make it safe to hand to anyone. The same engine that renders a lighthouse at blue hour can render a folk tale for a classroom, a health explainer in a local language, or a heritage story in a shadow-puppet style. That’s the point: good stories, told well, should not require a studio budget.

*Rough estimate for a ~30-second, 6-scene film — a planning aid, not billing. Trailer footage is real Sora-2 output from the studio; on-page art is gpt-image-2. Built and documented with GitHub Copilot CLI in autopilot. #AI4Good

#AI4Good started as a dare to myself: can I use AI to build and ship something genuinely useful — for kids, for health, for our culture — one build at a time? Eighteen days later there were eight live apps, each free, each on GitHub Pages. This is the story of how they came to be, in the order they were born — and each one now has its own deep-dive post (linked by date in the recap table below).

I’m Naveen Gopalakrishna — an AI Global Black Belt at Microsoft by day, a compulsive builder by night. The tooling that made this pace possible was GitHub Copilot CLI in autopilot mode (the agent researches, writes code, runs builds, and renders assets autonomously) sitting on top of Azure AI — gpt-image-2 for art, Azure Speech for narration, and open models from Hugging Face for the heavy lifting. But tools are just tools. The thread that ties these eight together is a question I kept asking: who does this help?

Let’s walk the timeline.

Jun 22 — PrimeBeats: your music, beautifully local

Where it started. PrimeBeats is an Amazon-Prime-Music-style player for the songs already on your phone — background playback, lock-screen controls, playlists, an on-device Smart Radio, and a real equalizer. No account, no ads, 100% offline. Built with React Native + Expo. → read the full story

The good: ownership and privacy — your music, zero data, zero tracking, zero cost.

Jun 23 — ActionCut: put a pro video studio in everyone’s pocket

The next build set the tone: take something normally locked behind subscriptions and make it free and open. ActionCut is a CapCut-class Android video editor — a real multi-track timeline, LUT color filters, GPU effects, audio mixing, and one-tap export presets for every platform. Built in Kotlin + Jetpack Compose + Media3.

The good: creators on entry-level Android phones get studio-grade editing without a paywall or a watermark. Storytelling shouldn’t require a subscription.

Jun 26 — KidKat: only the good stuff, for curious kids

This is the app that named the whole campaign. KidKat plays a finite, parent-approved stream of short educational videos — inside the official YouTube player, using the official Data API, fully ToS-compliant. No infinite feed. No algorithmic rabbit holes. No doomscroll. A parent allowlist decides what’s on the menu. Built with Flutter for Android and iOS. → read the full story

The good: it turns the most anxiety-inducing part of modern parenting — handing a child a screen — into something calm and intentional. AI for good starts with protecting the youngest users.

Jun 27 — NeoFit: health that speaks your language

“Sehat, simple banayi.” NeoFit is a science-informed Indian health and fitness app in 22 Indian languages — calorie tracking that actually knows dosa, biryani, and gulab jamun, on-device food recognition, AI-generated food photos, AI exercise videos, and Health Connect sync. Offline-first, powered by Azure AI, Kotlin/Compose. → read the full story

The good: most fitness apps assume you eat like the West and read English fluently. NeoFit meets 1.4 billion people where they are — in their language, with their food. Inclusion is a health outcome.

Jul 04 — GpsCamera: proof you can trust

GpsCamera is a fast, native Android camera that burns your exact location and a live mini-map onto every photo, writes standards-compliant GPS EXIF, and files each shot into its own album. Kotlin, Compose, CameraX. → read the full story

The good: field engineers, insurance surveyors, community reporters, and site inspectors need verifiable, tamper-evident documentation. A trustworthy timestamped-and-geotagged photo is a small tool that quietly protects people.

Jul 05 — The Lamp & the Machine: shadow-puppet theatre as an AI film studio

The Lamp & the Machine turns Karnataka’s 500-year-old leather shadow-puppet theatre, Togalu Gombeyaata, into an AI film studio — cinematic scenes from Sora 2, narration from Azure AI, finished with Demucs + FFmpeg, retelling the story of Kempegowda, the founder of Bengaluru. → read the full story

The good: generative AI as a preservation tool — putting a fading folk art in front of a new generation.

Jul 06 — Sopāna: a Snakes & Ladders that teaches

Sopāna reclaims the game the West calls Snakes & Ladders and restores its origin — the ancient Indian Moksha Patam, where every snake is a vice and every ladder a virtue. Land on one and the game animates and reads its meaning aloud. Play three ways (a 2D board, a 2.5D cinematic mode, and a 3D camera), across four worlds, with up to four players on one screen. A web PWA plus an Android APK. → read the full story

The good: it turns a children’s game back into what it was designed to be — a gentle lesson in ethics and consequence, wrapped in cultural heritage. AI narration and art make an 800-year-old teaching tool feel alive again.

Jul 09 — Chaturanga: ancient chess, reborn in glowing 3D

Chaturanga is the ancient Indian “game of the four divisions,” played with modern chess rules but authentic piece identities — Raja, Mantri, Gaja, Ashva, Ratha, Padati — carved in real-time glowing 3D. It ships with a teaching AI (five levels), a coach that reviews your blunders, an openings trainer, a rotating piece inspector, and a Warrior’s Eye camera. Four worlds, each teaching a moral lesson. → read the full story

The pieces themselves are an AI-for-good story: I generated each one from a single concept image using a free Hugging Face Space and headless Blender — the full image-to-3D pipeline is its own post here.

The good: a free, offline, no-sign-in chess tutor that also carries culture — learning and heritage in one board.

What #AI4Good actually means

Look at all eight together and a pattern appears — not “AI apps,” but apps that use AI to do something good:

The method: one build at a time, from a single laptop, with Copilot CLI autopilot doing the heavy lifting and Azure AI + Hugging Face models supplying the intelligence. The mission: put that capability toward education, health, culture, and trust — and give it away.

This is just the beginning of the campaign. Next up the runway: Indian Tales, a multilingual Togalu Gombe (shadow-puppet) storytelling app to keep a fading folk art alive.

If you build things, I’d gently challenge you to the same dare: pick the “who does this help?” first, then let AI help you ship it. That’s #AI4Good.

Follow the journey — GitHub · LinkedIn. Built one day at a time.

TL;DR — One gpt-image-2 concept image per piece → a free Hugging Face Space (tencent/Hunyuan3D-2) turns it into a raw 3D mesh → headless Blender projects the concept back on as texture → a web-ready GLB that glows in Three.js. No local GPU. All of it — the research, the dozens of scripts, the Hugging Face calls, the Blender runs, and this very blog post and its auto-publish to DEV — was orchestrated by GitHub Copilot CLI in autopilot mode.

I’ve been building Chaturanga — the ancient Indian “game of the four divisions,” played with modern chess moves but authentic piece identities (Raja, Mantri, Gaja, Ashva, Ratha, Padati), where every world teaches a moral lesson. The board above is the payoff. The hard part was the pieces: I wanted carved-ivory war figures with real silhouettes, in four themed armies, that spin in a WebGL inspector — and I wanted them for free, on a laptop with no GPU, in a fully scriptable, reproducible way.

This is the story of how that pipeline came together, including the parts that didn’t work.

The pipeline at a glance

Four stages, each free and headless:

1. Concept — Azure gpt-image-2 renders a themed, carved-ivory concept per piece (front 3/4, white background).
2. Image → 3D — the concept goes to a Hugging Face Space that runs image-to-3D on donated GPU and hands back a mesh.
3. Texture — Blender 5.1, headless on CPU, projects the concept image onto the mesh as its surface.
4. Ship — export a decimated, textured GLB and load it in Three.js with a little emissive glow.

Step 1 — Concept art with gpt-image-2

Each piece starts as a prompt describing a museum figurine. The key is a clean, front-facing 3/4 view on a plain background — that is exactly what image-to-3D models want.

pieces = {
  "raja":   "A carved ivory Indian chess piece: a standing king with crown, parasol and mace, Chaturanga style, white background, front 3/4 view, studio lighting",
  "ashva":  "A carved ivory Indian chess piece: Hanuman mid-leap raising a mace, ornate, Chaturanga style, white background, front 3/4 view",
  "gaja":   "A carved ivory Indian chess piece: a war elephant with a howdah canopy, Chaturanga style, white background, front 3/4 view",
  # ... mantri, ratha, padati
}
# -> Azure AI Foundry: POST /openai/deployments/gpt-image-2/images/generations

Step 2 — Image → 3D, on a free Hugging Face Space

This is the heart of it. Instead of buying GPU time, I call the public tencent/Hunyuan3D-2 Space through its Gradio API. The Space runs the heavy diffusion-to-mesh step on free ZeroGPU, and I just pull back a .glb:

from gradio_client import Client, handle_file

client = Client("tencent/Hunyuan3D-2", hf_token=HF_TOKEN)   # token optional; helps with rate limits
result = client.predict(
    image=handle_file("refs/raja.jpg"),
    steps=30, guidance_scale=5.0, seed=1234,
    octree_resolution=256, check_box_rembg=True,
    num_chunks=8000, randomize_seed=False,
    api_name="/shape_generation",     # geometry only — fast; texture is done later in Blender
)
# result -> a raw .glb mesh, generated on someone else's GPU, for free

I batch all six pieces over one reused client connection, with retries for transient ZeroGPU queue errors. Geometry-only (/shape_generation) is the fast path; I deliberately skip the Space’s texture step because I get sharper, on-brand results projecting my own concept image in Blender (Step 3).

No-signup fallback: for a path that needs zero accounts, I keep stabilityai/TripoSR (MIT) running locally on CPU. It’s the only open image-to-3D model I found that’s genuinely CPU-feasible on Windows — a few minutes per piece, softer surface, no PBR, but a reliable offline safety net.

python tooling/triposr_run.py refs/raja.jpg --out out/ --resolution 256
# TripoSR on CPU: marching cubes via PyMCubes (no native build), exports a vertex-coloured GLB

Bonus: discovering Spaces from inside the terminal

Copilot CLI now has the Hugging Face MCP server wired in, so I can search the Hub without leaving my shell. Asking it for image-to-3D Spaces returns exactly the shortlist this project lives on:

That turns “which model should I use?” into a two-second query instead of an afternoon of tab-hopping.

From flat concept to spinning figure

Here’s the whole point in one frame — the gpt-image-2 concept on the left, the Hugging Face + Blender result on the right:

And because a chess piece has to survive being spun around in a 3D inspector, here they are on the turntable:

Step 3 — Where Blender earns its keep (and where it fights back)

The Hugging Face mesh has a correct silhouette but a soft, under-detailed surface. All the crisp carving detail lives in the original concept image. So Blender’s job is concept-texture projection: orient the raw mesh, frame an orthographic camera on its “front,” and set each vertex’s UV to that camera’s projection — so the concept image becomes the surface, razor-sharp from the front.

blender -b --python tooling/blender/texture_project.py -- \
    raw/raja_hunyuan.glb refs/raja.jpg web/raja.glb preview.png
# headless, CPU-only: orient -> project concept as texture -> decimate to ~28k faces -> export GLB

Blender is fantastic here, but the road was bumpy. The honest limitations:

• Back faces get a mirrored projection. Single-camera projection is perfect head-on; the reverse side is a mirror of the front. For pieces viewed from above on a board, it’s an acceptable trade — but it is a trade.
• Blender 4 → 5 broke my material scripts. The Principled BSDF input sockets were renamed: it’s Subsurface Weight not Subsurface, Specular IOR Level not Specular. Scripts that ran on 4.x threw KeyError on 5.1 until I re-bound every socket. (Lesson: print(bsdf.inputs.keys()) before you touch anything.)
• Eevee Next won’t render headless on Windows. blender -b with the new Eevee crashes with an access violation — it wants a GPU context that a background process doesn’t have. Cycles on CPU renders fine headless; every image in this post (including the turntables) is Cycles-CPU.
• No native build allowed. To keep the whole thing laptop-reproducible, marching cubes runs through PyMCubes rather than a compiled isosurface extension.

The detour: “why not just build them in Blender?”

Early on I seriously tried Blender-only procedural modeling — skin modifier over a skeleton, geometry nodes, PBR node graphs baked from the concept. It’s fully scriptable and GPU-free… and it produced primitive blobs after hours of fiddling. Verdict: great for finishing, wrong tool for creating organic figures. That failure is exactly why the Hugging Face step exists.

The tools I tried — and where they hit a wall

Before settling, I benchmarked the field. The constraints were brutal on purpose: free, Windows, no GPU, scriptable.

The pattern is clear: the hosted API services are excellent but metered, and the best local models need a GPU I don’t have. The one escape hatch is a community-hosted Hugging Face Space — the model runs on donated GPU, and I pay nothing but a queue wait.

The meta-story: this was all built in Copilot CLI autopilot mode

Here’s the part I keep coming back to. I didn’t hand-write most of this. I drove GitHub Copilot CLI in autopilot mode — where the agent runs shell commands, scripts, and tools autonomously, multi-step, without stopping for approval at each move. In practice that looked like:

• a research sub-agent fanning out across 13 image-to-3D tools and returning a sourced comparison table;
• the agent writing and running hf_batch.py, invoking the Hunyuan3D-2 Space, and retrying on ZeroGPU hiccups;
• headless Blender runs for texture projection and the turntable renders you see above — launched, inspected, and re-tuned by the agent (it caught the Eevee-headless crash and switched to Cycles itself);
• and finally, this blog — rendered GIFs, pipeline diagram, prose — plus a GitHub Actions workflow that auto-publishes it to DEV on every push.

Autopilot turns “I have an idea for a pipeline” into “the pipeline, its assets, and its write-up all exist,” with me steering rather than typing every command.

Results & what’s next

The payoff is a full four-world army — Kurukshetra, Ramayana, Kalinga, Devasura — each piece a real 3D figure with a themed material and a bit of emissive bloom so the Devas literally glow.

Try it: the game is live at naveenneog.github.io/Chaturanga (3D + 2D, AI opponent, coach, openings trainer, Android APK).

Steal the pipeline: gpt-image-2 → tencent/Hunyuan3D-2 Space → Blender projection → GLB. It costs nothing, needs no GPU, and every step is a script. If you’re generating game assets, product mockups, or avatars, a free Hugging Face Space is the cheat code — and Copilot CLI autopilot is the thing that stitches it all together.

Built one day at a time. — @naveenneog

Part of the #AI4Good series — one app a day, each built for good.

Chess was born in India as Chaturanga — the “game of the four divisions.” Chaturanga the app plays by modern chess rules but keeps the ancient identities — Raja, Mantri, Gaja, Ashva, Ratha, Padati — and lets you actually learn the game instead of just losing it.

What it is

Chaturanga is ancient chess in real-time glowing 3D, with a full teach-and-play layer:

• Play the Guru — an alpha-beta chess AI with five difficulty levels (Padati → Mantri), running in a Web Worker so the board stays smooth on phones.
• A coach — a Hint that names the best move and why, plus a blunder review that gently flags mistakes and shows the stronger move.
• Openings trainer — six classic openings (Italian, Ruy López, Sicilian, French, Queen’s Gambit, King’s Indian) walked move-by-move with a narrated lesson.
• Piece inspector — tap a piece for a rotating 3D render and a diagram of how it moves and captures; a Warrior’s Eye camera looks across the board from a piece’s point of view.
• Four themed worlds, each with its own army, board art, teachings and a portrait cinematic intro. Local hotseat, undo, captured-pieces tray, under-promotion, read-aloud narration — no backend, works offline.

How it was built

The web app is deliberately buildless — vanilla ES modules, chess.js for the rules, three.js for rendering, Capacitor for Android. The interesting parts:

• A real chess engine, in the browser. The AI is alpha-beta negamax with quiescence search, MVV-LVA move ordering, and piece-square evaluation over chess.js, exposed as analyze() / bestMove() / classifyMove(). It runs in a Web Worker (with a main-thread fallback) so search never janks the render loop, and the five levels scale depth, blunder-rate and time cap. Root moves are searched full-window so every move gets an exact score — which is what makes the coach’s blunder detection possible.
• The pieces are AI-reconstructed, not hand-modelled. Each one starts as a themed gpt-image-2 concept, becomes a mesh via the free Hunyuan3D-2 Hugging Face Space, gets its concept projected back on as texture in headless Blender, and ships as a small web GLB. Portrait intros are generated with Azure Sora-2. The full image-to-3D pipeline is its own post here.
• Tested with node:test (rules + engine + coach/openings + all-worlds validation) and shipped as a debug-signed Capacitor APK (npm run apk, JDK 21).

The good

A patient, free, offline chess tutor — no sign-in, no subscription — that also carries heritage in every piece. Learning and culture on one board. That’s #AI4Good.

Play it

• ▶️ Play in your browser: naveenneog.github.io/Chaturanga
• 📦 Download the APK: latest release
• 💻 Source: github.com/naveenneog/Chaturanga

Previous → Sopāna · Read the whole journey → #AI4Good build story. #AI4Good

Part of the #AI4Good series — one app a day, each built for good.

Before it was a plastic race to 100, Snakes & Ladders was Moksha Patam — an Indian teaching game where the board was a map of the soul. Somewhere along the way we kept the dice and threw away the meaning. Sopāna puts it back.

What it is

Sopāna is a Snakes & Ladders that teaches. Open it and you land in a lobby — pick a theme, a mode, 1–4 players and a character for each, then play local pass-and-play multiplayer. The same game is drawn three ways:

• Board — a crisp 2D board with a live turn roster, animated themed snakes & ladders, and coloured tokens.
• Cinematic — a 2.5D backlit shadow-puppet ascent with a per-theme Sora-2 intro film and an adaptive procedural score.
• 3D — a real 3D board with an orbit camera (drag to spin, scroll to zoom), a themed environment and player-coloured pawns.

The signature is the Meaning Reveal: land on a snake or ladder and the board dims, the connector highlights, a themed card shows the name + one-line teaching, and a narrator reads it aloud with word-by-word highlight before your token travels. Four worlds ship — Moksha Path (vices vs virtues), Founder’s Climb (startup pitfalls vs wins), Panchatantra Trail (fables), and Habit Heroes (kids’ habits).

How it was built

Sopāna is radically data-driven: a new world is essentially one JSON file in web/worlds/*.json over a single pure rules engine (web/js/logic.js). Every snake/ladder is just { from, to, name, en, meaning }, and meaning is the line read aloud. The three renderers — Board, Cinematic (PixiJS), and 3D (Three.js) — all draw the same manifest, so content never lives in a renderer.

• Every asset is generated, per theme. Azure (AAD-only via az login) drives it: gpt-image-2 renders the board backdrop, the player figurine, and one illustration per snake/ladder in each world’s art style (Moksha = authentic Togalu Gombe leather shadow-puppet); Azure Neural TTS narrates each meaning (Moksha = en-IN-Arjun:DragonHDLatestNeural). Files follow a <type>-<from>.{png,mp3} convention the app loads automatically, falling back to the browser’s SpeechSynthesis when a clip is missing — and npm test includes an asset-sync check that every entry has matching art + narration.
• The Cinematic mode is a little engine of its own (PixiJS + Web Audio): the pilgrim walks a winding staircase step-by-step (each step ignites), landing on a vice/virtue triggers an animated light-sweep or a serpent strike with screen-shake, the 100 steps span five lokas with curtain-wipe realm transitions, and a procedural raga (Bhūpāli) plays an ascending melody as you climb, over an adaptive drone that brightens per realm.
• Zero-dependency tests (node:test) plus Playwright visual QA (tooling/shoot.mjs); shipped to GitHub Pages and an Android Capacitor APK (CI, JDK 21). Licensed PolyForm Noncommercial.

The good

Kids absorb values through play, not lectures. By restoring the why to every square — narrated, animated, and beautiful — Sopāna turns a car-trip game into a gentle lesson in ethics and consequence, and keeps a piece of cultural heritage alive. That’s #AI4Good.

Play it

• ▶️ Play in your browser: naveenneog.github.io/Sopana
• 📦 Download the APK: latest release
• 💻 Source: github.com/naveenneog/Sopana

Previous → The Lamp & the Machine · Next → Chaturanga. #AI4Good

Part of the #AI4Good series. This one is about heritage — using new machines to keep an old art breathing.

Under a banyan tree, a storyteller lights the brass oil lamps and, on a stretched cotton screen backlit by fire, a legend moves. This is Togalu Gombeyaata — Karnataka’s leather shadow-puppet theatre — and it is quietly disappearing. The Lamp & the Machine is my attempt to hand it a new stage: an AI-generated shadow-puppet film that retells the story of Kempegowda, the 16th-century chieftain who, in 1537, ploughed the four streets that became Bengaluru.

What it is

The Lamp & the Machine is both a film and a build log — a single themed page that walks through turning a folk theatre into an AI film studio, told in three acts. The film exists in four different narration cuts of the same visuals, and the project produced two reusable skills along the way.

How it was built

This is the richest build story of the series, so here’s the honest version — bugs and all.

Act I — the images (Sora 2 on Azure AI Foundry). The hard part of a multi-scene film isn’t generating a clip, it’s stopping every clip from drifting. Independent generations wander — the hero’s turban changes, the palette shifts, the backlit screen becomes a photoreal sunset. The fix was a locked “Style Bible” and a five-beat grammar — Invocation → Character → Journey → Conflict → Resolution → Moral — injected verbatim into all ~25 scenes, rendered at 12 seconds a clip and stitched with a shared warm-firelight grade and crossfades. The lesson came from one bug: scene 3 drifted into a photoreal landscape because a single line said “movement past Indian landscapes, palaces, forests, rivers.” Sora read it as paint a real location. Rewording it to “flat cut-leather silhouettes on the same backlit screen” killed the drift.

Act II — the voice (the editing problem in four words: keep the music, change the voice). The first cut narrated itself beautifully… in Kannada — because the Style Bible’s audio field said “a wise elder Kannada storyteller” (confirmed by transcribing with gpt-4o-transcribe). I wanted an English cut without losing the veena, mridangam and temple bells Sora had baked into the same track. So I ran the audio through Demucs (htdemucs) source separation to split vocals from music + ambience, dropped the old narration, kept the music bed, laid a fresh Azure Neural voice on top with side-chain ducking, and remuxed onto the untouched video.

Act III — the cast. One narrator became a company: a warm male elder (en-IN-Prabhat), a female voice (en-IN-Neerja, with empathetic and cheerful styles), and a unison finale. The first multi-voice cut taught two lessons — it sounded robotic (I’d pushed prosody too far, -8% rate / -2st pitch, adding artifacts) and cut out after 90 seconds (a real offset bug). Softening the prosody and switching to Azure’s ultra-natural DragonHD voices (en-IN-Arjun, en-IN-Neerja) — so lifelike no time-stretching was needed — produced the flagship cut.

The pipeline, all on Azure with Microsoft Entra (AAD) auth — no API keys, orchestrated from the terminal: Sora 2 (video) · gpt-4o-transcribe (audio QA) · Azure AI Speech (en-IN neural + DragonHD) · Demucs (separation) · FFmpeg (stitch, duck, mix, remux) · Python 3.14. It even spun off three reusable skills — togalu-gombe-video, voice-dub, and togalu-brand-bumpers (opening jingle + end credits).

The good

Folk arts don’t die because they stop being beautiful — they die because they stop being seen. Pointing generative video at a living tradition, respectfully, can put it in front of a generation that would otherwise never meet it. AI as a preservation tool for culture — that’s #AI4Good.

Explore it

• ▶️ Live site / build log: naveenneog.github.io/Sora-Azure-MultiPart-Video-Editing
• 💻 Source: github.com/naveenneog/Sora-Azure-MultiPart-Video-Editing

Previous → GpsCamera · Next → Sopāna. #AI4Good

Part of the #AI4Good series — one app a day, each built for good.

Not everything that does good is glamorous. Sometimes it’s a field engineer proving they inspected a transformer, a surveyor documenting flood damage, or a community reporter capturing a pothole with a location no one can dispute. They all need the same thing: a photo you can trust.

What it is

GpsCamera is a fast, native Android camera that makes every shot verifiable. Straight from the live site:

• Geotagged capture — every photo is stamped with the reverse-geocoded address, decimal coordinates, altitude, accuracy and a timestamp.
• Live mini-map — a real OpenStreetMap thumbnail of your exact spot, shown live on the viewfinder and burned onto the shot.
• Open in Maps — tap the map (or the link embedded in the photo) to jump straight to the coordinates in Google Maps.
• Photo & video, pinch-to-zoom, and a full-screen gallery (swipe, pinch-zoom, open-in-Maps from EXIF, share).
• Move & resize the stamp — drag the info block anywhere and pinch to resize before you shoot; it’s burned exactly as arranged.
• Standards-compliant EXIF — GPS lat/lon/alt/timestamp + a clickable Maps URL written into the JPEG, so Google Photos and Lightroom place it on a map automatically.
• Day & night themes, portrait and landscape reflow, and a dedicated Pictures/GPSCamera album.

How it was built

GpsCamera is deliberately native and dependency-light — Kotlin + Jetpack Compose (Material 3) + CameraX, min/target SDK 26/35, no heavyweight frameworks. The engineering value is in a few careful choices:

• Resilient location. It merges Google Play Services fused location and the platform GPS/network providers, so it still works on devices without Google Play Services — exactly the rural/field scenarios it’s built for.
• No-API-key maps. The mini-map is stitched from OpenStreetMap raster tiles with pure web-mercator (“slippy map”) tile math — no Maps SDK, no key, no quota. StaticMapProvider fetches and stitches the tiles and draws the pin; PhotoStamper burns the info panel + map onto the bitmap.
• Testable core. The tricky bits are pure, unit-tested Kotlin — GpsFormat (DMS / EXIF-rational / stamp text) and SlippyMap (tile math + Maps URLs) — plus on-device instrumented tests for what can only be verified for real: ExifWriter (EXIF read-back), PhotoStamper, and PhotoSaver (MediaStore storage).
• Clean module split: model/GeoFix · location/LocationRepository (fused + platform + reverse geocoding) · map/StaticMapProvider · camera/{PhotoStamper, ExifWriter, PhotoSaver, GalleryRepository} · Compose ui/. The app icon itself was generated with Azure AI Foundry gpt-image-2.

The good

A timestamped, geotagged, tamper-evident photo is a small piece of infrastructure for accountability — for insurance, field work, journalism, and public services. That it works without Google Play Services or a Maps API key means it reaches exactly the low-connectivity places that need proof the most. Trust is a public good, and GpsCamera hands it to anyone, free.

Try it

• ▶️ Live site: naveenneog.github.io/GpsCamera
• 📦 Download the APK: latest release
• 💻 Source: github.com/naveenneog/GpsCamera

Previous → NeoFit · Next → The Lamp & the Machine. #AI4Good

How to read this guide. Statements are tagged [Fact] when they reflect documented Azure behavior and [Assumption] when they are reasonable defaults you must confirm in your tenant. Where docs do not confirm something, it says so.

1. Executive Summary

Azure AI Foundry supports open-source models through three paths. [Fact] The Model Catalog offers curated models (Mistral, Qwen, Llama, Phi, and more) you deploy as serverless (pay-as-you-go) APIs or to managed online endpoints with no infra to manage. The Hugging Face collection in the catalog lets you deploy thousands of HF models to a managed endpoint backed by your own GPU compute. For anything else, Bring Your Own Model (BYOM) runs the model on Azure ML, AKS, or a VM, then registers it back into Foundry as a connection so agents can call it.

Use serverless catalog when the model is listed and you want fastest time-to-value and no GPU quota worries. Use Hugging Face → managed endpoint when the model is on HF but not serverless, and you accept managing GPU SKUs. Use BYOM for gated, quantized, custom-architecture, or compliance-isolated models. Cost and operational burden rise left-to-right; control rises with them.

2. Decision Tree

Is the model in the Foundry Model Catalog?
├─ YES → Serverless API available?
│        ├─ YES → Deploy serverless (pay per token). [easiest]
│        └─ NO  → Deploy to Managed Online Endpoint (pick GPU SKU).
└─ NO  → Is it on Hugging Face + supported task/architecture?
         ├─ YES → Deploy via Foundry HF collection / Azure ML to Managed Endpoint.
         └─ NO  → BYOM: containerize → AML/AKS/VM endpoint → register connection.

3. Step-by-Step Guides

Path 1 — Foundry Model Catalog

1. Open ai.azure.com → your project → Model catalog.
2. Filter by collection (e.g., Mistral, Qwen) and search the model.
3. Open the card → Deploy. Choose Serverless API or Managed compute.
4. For managed compute, pick the GPU SKU and instance count; create the endpoint.
5. Validate: card shows Succeeded; the Chat playground returns a response.

Path 2 — Hugging Face → Azure

1. In the catalog, open the Hugging Face collection; search by repo id.
2. Deploy to a managed online endpoint; select a GPU SKU sized to the model.
3. [Fact] Endpoint exposes a scoring URL + key. [Assumption] API shape depends on the serving stack (vLLM → OpenAI-compatible /chat/completions).
4. Validate: curl the endpoint with a sample payload; check 200 + tokens.

Path 3 — Bring Your Own Model (BYOM)

1. Containerize with vLLM/TGI; register the model in an Azure ML workspace.
2. Deploy to AML managed endpoint or AKS; expose HTTPS.
3. In Foundry, add a Connection (custom/serverless) pointing at the URL + key.
4. Link the connection to an Agent as a tool/model. Docs do not confirm every custom stack auto-registers — verify the connection type your model exposes.

4. Validation Framework

# OpenAI-compatible managed endpoint
curl -sX POST "$ENDPOINT/chat/completions" -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d '{
    "messages":[{"role":"user","content":"Reply with: OK"}],
    "max_tokens":16,"temperature":0}'

Verify: deployment Succeeded, /health 200, output contains expected text. Failure modes: 401 (key/identity), 429 (quota), 503 (cold start), garbage (wrong template).

5–7: Failures, Enterprise, Automation

Quota, region GPU shortages, VLM/MoE serving, and timeouts are the usual blockers. Govern with catalog allow-lists + Azure Policy; control cost via serverless vs reserved GPU; pipe logs to App Insights. CLI: az ml online-endpoint create. Python: azure-ai-ml.

GPU rule of thumb [Assumption]: ≤8B → L4/A10; 24B → A100-80GB; 35B+/FP4 → H100/B200.