Building an LLM Robot with My Son — EP 0. Why We Started an AI Robot Project

This series isn't a finished project showcase. As I write EP 0, there's a half-assembled acrylic chassis sitting on my son's desk with one wheel still spinning in reverse. That's where we're starting.

If you're in a similar situation — you want to do something technical with your kid but don't know where to begin — read this like you're watching someone else figure it out in real time.

---

We Both Like Building Things

My son loves LEGO. He's twelve, and he has the patience to sit with a Technic set spread across the floor and build it piece by piece from the manual. When it finally works, he gets that quiet satisfaction that doesn't need words. He figures out gear ratios by feel — just turning the pieces until something clicks.

I was the same way as a kid. Radio kits, model kits, anything with parts that went together. I started building LEGO with him a few years ago, and somewhere along the way I realized I was more into it than he was.

I've been tinkering with Arduino for a while too. Temperature sensors, distance sensors, small OLED displays. Nothing ambitious — just something to decompress with after work. I'd been vaguely thinking I should teach him to code eventually, but "eventually" kept getting pushed back.

Then this spring, something shifted.

---

Rethinking How to Teach Coding

I showed him Arduino once. He saw the LED blink, said "oh cool," and moved on. Which makes sense — a blinking LED isn't interesting.

Teaching Python from scratch felt wrong too. Variables, loops, conditionals — without something concrete at the end, a 12-year-old has no reason to care.

What kept hitting me at work was how much the gap had widened between people who can use AI tools well and people who can't. It's not about knowing which model to use. It's about knowing how to explain what you want and actually get it. That skill — describing intent, iterating through conversation — is becoming more valuable than writing code line by line.

So what should I teach a 12-year-old first? I kept coming back to the same answer: not Python syntax. The ability to work with AI to build what you have in mind. That matters more going forward.

And there's no better learning environment for that than a robot. Code something, and it moves in the physical world. The feedback is immediate and real.

---

"A Robot!" — His Answer

I asked him. "Want to build a robot with me? One with a camera and sensors, where AI decides what to do?"

He thought for maybe two seconds. "Yes."

That evening, without me asking, he went and watched YouTube videos about robots on his own. Wheeled ones, arm ones, LEGO Technic ones. The next day: "I'll be in charge of the hardware."

That's how the roles got set.

---

The Architecture

Three layers:

[Robot body (edge)]
  ├── Camera (captures video)
  ├── Ultrasonic sensor (distance/obstacle detection)
  ├── Motors + driver (movement)
  └── Arduino → Raspberry Pi (planned upgrade)

         ↕  WiFi (home LAN)

[LLM server]
  ├── Receives camera frames + sensor data
  ├── Local LLM reasons and generates commands
  └── Sends commands back to the robot

We're not using cloud APIs (ChatGPT, Claude API) because every real-time decision would have to make a round trip to the internet. With a local LLM server on the home LAN, that latency nearly disappears.

The LLM server runs on three Macs we already own:

Device	Memory	Role
Mac mini M1	16GB	Benchmark comparison
Mac mini M4	24GB	Always-on home server candidate
MacBook Pro M4 Pro 14"	24GB	Main server

Apple Silicon's unified memory architecture turns out to be well-suited for local LLM inference. The M4 and M4 Pro both have 24GB, but the memory bandwidth is more than double — that difference shows up in token generation speed. We'll benchmark that in a later episode.

---

Roles and the Harness

Here's the division of labor:

• My son: Hardware — assembly, wiring, sensors, robot body
• Me: Software — LLM server, agent harness design, communication layer
• Together: Vibe coding — my son learning to command the agent

"Agent harness" is a term I've written about separately (see the harness engineering post). The short version: everything outside the model. CLAUDE.md, tool definitions, hooks, memory, feedback loops. The formula is Agent = Model + Harness.

For this robot project, the harness holds:

• Sensor specs, pin maps, voltage tolerances
• Arduino/Pi library choices and why
• ROS2 node structure and topic naming conventions
• LLM server communication protocol
• Robot safety constraints (e.g., "stop if obstacle within 20cm")

When this is compiled into CLAUDE.md, the AI coding agent (Claude Code) works with the project already in context. It doesn't guess pin numbers or choose the wrong library. The harness catches that.

And my son learns to command this agent through vibe coding — he says "stop if the sensor reads under 20cm" and the agent writes the code. He uploads it, tests it, and if something's off, he describes it again. He's not memorizing syntax. He's learning to describe intent and iterate.

---

What We Have Now

• Arduino Uno
• HC-SR04 ultrasonic distance sensor
• USB webcam
• L298N motor driver + 2 DC motors
• Jumper wires, breadboard
• Acrylic robot chassis with wheels

We'll get the basics working on Arduino first, then migrate to Raspberry Pi. The Pi 4 vs Pi 5 vs Banana Pi decision is for a later episode. Pi 5 isn't on the immediate budget — the plan is to buy it if the blog generates enough revenue, which may or may not happen.

There's already talk of a version 2. My son wants to design the robot body himself using LEGO Technic — he's already thinking about gear ratios. That's the season 2 seed, for now we're focused on getting this one working.

---

Projects We Referenced

None of this is designed from scratch. What we looked at:

• ROS-LLM: Open source framework connecting ROS and LLM. Used for overall architecture direction.
• huawei-noah/HEBO: Agent learning framework. Ideas for agent structure.
• ROSClaw: ROS-based robot control. Communication layer reference.

None of these were copy-pasted. They confirmed we were headed in the right direction. Our actual implementation is custom-built for our environment (Mac LLM server, Pi edge, Korean language model).

I should note: I've only glanced at ROS2 before. This project is my real introduction to it. We're starting from uncertainty, and that's fine.

---

What This Series Covers

Nine episodes total:

#	Topic	Hardware needed
EP 0	Why we started (this post)	✗
EP 1	Designing the AI coding agent harness	✗
EP 2	First AI coding experiment	✗
EP 3	Local LLM speed: M1 vs M4 vs M4 Pro	✗
EP 4	Choosing the right local LLM	✗
EP 5	My son's first day coding with AI	✗
EP 6	Connecting robot to LLM server over LAN	△
EP 7	Upgrading from Arduino to Raspberry Pi	✓
EP 8	My son gives the robot its first real command	✓
EP 9	4-month retrospective, and what's next	—

Through EP 6, the series can continue regardless of hardware progress. If something comes up with the physical build, we ask Claude and keep going — the series doesn't stop.

---

Publishing this kind of in-progress journal is new for me. It's not a polished project reveal. It's a record of every conversation that happened, every turn that got us closer. That felt more useful than another "here's the finished thing" post. There aren't many of these for someone just starting out alongside their kid.

Next episode: designing the agent harness. How to teach an AI coding agent the domain knowledge for this specific robot project, and how to make the interface accessible enough that a 12-year-old can walk straight into it.