Building an LLM Robot with My Son — EP 9. 4-Month Retrospective, and What Comes Next

-

Building an LLM Robot with My Son — EP 9. 4-Month Retrospective, and What Comes Next

When I wrote EP 0, there was a half-assembled acrylic chassis on the desk with one wheel turning in reverse.

Now that robot walks to the kitchen on its own and finds a water glass. Can't pick it up — but it finds it.

Four months.

What Worked

The agent harness approach held up

Injecting domain knowledge through CLAUDE.md worked better than expected. The repeated context fatigue disappeared — AI wrote code that respected the project's rules without having to be reminded every session. The file started at ten lines and grew to 120. That growth is a record of the project itself.

My son completed a behavior through eight prompts

The scene from EP 5 is the one that stays with me. He sat down alone, iterated eight prompts, and built a working obstacle avoidance behavior. Never touched a line of code. "I made this" was not wrong.

Apple Silicon local LLM actually works

112 tok/s on M4 Pro. Sub-500ms round trips over home LAN, no cloud API latency. Running a local LLM server at home turned out to be more practical than I expected.

What Surprised Me

The Arduino → Pi migration came sooner than planned

The moment I built the laptop bridge in EP 6, it was obvious that was temporary. The series structure had Pi migration planned for EP 7, but honestly I wanted to do it by EP 4–5. The episode-by-episode format made me wait.

Vision LLM worked better than expected

When "go to the kitchen and find the water glass" actually succeeded on the third try in EP 8 — I was genuinely surprised. No hardcoded map, no coordinates, just a camera feed and a goal. A 7B parameter model doing that was not something I'd have predicted two years ago.

He started reading error messages

Early in the project, any error meant coming to find me. Now when the terminal shows red, he reads it first. He doesn't understand everything, but he finds the line number. I didn't teach him that. It just happened through doing.

What Didn't Work

He still doesn't understand code structure

What a function is, why there's a loop — he doesn't know. He can build behaviors through vibe coding but can't explain why the code works. I don't think that's a problem right now, but there will be a point where fundamentals need to come in.

Sensor noise was never fully resolved

HC-SR04 returning 0cm or 400cm with no obstacle present. We filtered it multiple times. It never went completely away. Hardware-level noise that software can't fully suppress — would need a sensor swap or a reworked level-shifter circuit. Didn't get there this season.

Didn't buy the Pi 5

Honest admission: I wrote in EP 0 that I'd buy Pi 5 if blog revenue materialized. Four months later, revenue did materialize. Pi 4 works well enough that I didn't buy it. Didn't need to.

What He Learned in Four Months

The original goal was "teach him to code." More accurately, we did it together.

What he can do now:
- Describe a desired behavior to AI in plain language
- Narrow down what's wrong when something behaves unexpectedly
- Read Serial monitor output and notice when sensor values are off
- Context check prompt (the routine he invented in EP 2)
- Hardware wiring — power rail separation, checking voltage before connecting

What he still can't do:
- Modify code directly
- Interpret compile errors
- Design circuits

The can't list is longer. Of course — four months, twelve years old.

But the instinct for describing what you want, working through conversation with AI, and getting a result is there. That's what this project set out to build. The rest can follow.

Season 2: LEGO

He already knows.

"Next time let's build it in LEGO." That came up in EP 0 and kept resurfacing for four months. Every time he watched the acrylic chassis move: "I think it would work better in LEGO Technic."

Season 2: my son designs the robot body himself in LEGO Technic. Gear ratios, torque, wheel configuration — this is territory he already understands intuitively from years of building. LEGO Mindstorms or Spike Prime integration means programmable motors without needing Pi. The existing LLM server harness carries over.

Pi 5 happens then. LEGO Technic too. When the budget is there.

If you've read from EP 0 to here: I wanted it to feel like watching someone figure it out in real time, not a polished retrospective. Not a finished project showcase. A record of conversations that built something.

My son knows this series has been going up. At first he was a little self-conscious about his story being on the internet. Now he's the one who says "post what we did today."

That's probably the best signal.

댓글

댓글 쓰기

이 블로그의 인기 게시물

개발자는 코드를 쓰는 사람이 아니다 — AI 시대에 남는 자리는 '책임'에 있다

-
개발자는 코드를 쓰는 사람이 아니다 — AI 시대에 남는 자리는 '책임'에 있다 AI 때문에 개발자가 없어질 거라는 말, 요즘 진짜 많이 듣는다. 특히 AI를 매일 쓰는 개발자들이 더 불안해하는 것 같다. 매일 AI가 자기 대신 코드를 짜는 걸 보다 보면, 이 일이 얼마나 더 이어질까 싶은 거다. 예전엔 주니어만 대체된다는 이야기였는데, 이제는 시니어까지 같이 묶여서 이야기가 나온다. 얼마 전엔 지인 한 명이 커리어를 바꿀까 고민 중이라고 했다. 30년 넘게 코딩했는데, 요즘 자기가 일주일 걸릴 작업을 AI가 몇 시간 만에 해치우는 걸 보면서 "내가 지금 쌓고 있는 스킬이 3년 뒤에도 의미가 있을까"라는 생각이 든다고 했다. 다른 한 명은 반대로, AI를 거의 안 쓰는 동료들을 보면서 "저 사람들 지금 뭐하고 있는 건가" 싶다고 했다. 같은 시기에 완전히 다른 종류의 불안을 겪고 있는 거다. 둘 다 이해는 간다. 근데 이 불안의 전제 자체가 틀렸다고 본다. 정확히는 반은 맞고 반은 틀렸다. 개발자가 없어지는 게 아니라, "개발자"라는 단어가 가리키는 대상이 바뀌고 있을 뿐이다. 근데 그 변화가 혼란스러워 보이니까 "개발자가 사라진다"는 쉬운 말로 퍼지는 것 같다. 작년에 사내에서 한 발표를 준비하면서 이 주제를 나름대로 정리해본 적이 있다. 주제는 복잡한 크로스플랫폼 데스크톱 앱을 리팩토링하면서 겪은 경험이었는데, 진짜 하고 싶었던 이야기는 프로젝트 경험담이 아니라 한 발짝 더 떨어진 곳에 있었다. "AI를 잘 쓰는 법"이 아니라, AI를 쓰다 보니 내가 하는 일의 무게중심이 어디로 옮겨가고 있는가. 그 이야기를 하고 싶었다. 그리고 1년이 지난 지금, 그때 적었던 결론은 오히려 더 또렷해졌다. 구현자에서 "다시 설계자로" 발표에서 썼던 문장 하나가 지금 다시 봐도 정확한 것 같다. 개발자가 사라지는 흐름이라기보다는,...
자세한 내용 보기

Harness Engineering in Practice — How Anthropic Designs AI Agents

-
Harness Engineering in Practice — How Anthropic Designs AI Agents The previous post covered the concept and components of harness engineering. This time, it's the real thing. Drawing on the concrete architecture patterns Anthropic published in their official engineering blog — along with experimental results from the OpenAI Codex team — let's look at how harnesses are actually applied in practice. The Basic Structure of an Agent Loop: The Inner Loop At the heart of every AI agent sits an agent loop . In Claude Code, it's called queryLoop . At its core, it's a while(true) loop. while (true) { 1. Prepare context (plan-mode attachments, task reminders) 2. Call the model (streaming API call) 3. Execute tools (detect tool call → validate schema → check permissions → execute) 4. Decide whether to continue (does the model have more to do?) } Each iteration is one "think, act, observe" cycle. The model thinks, invokes a tool, observes the resul...
자세한 내용 보기

What Is Harness Engineering — Designing the Reins for AI Agents

-
What Is Harness Engineering — Designing the Reins for AI Agents In Part 1 of this series, I talked about the decline of prompt engineering. With CLI-based tools on the scene, the value of manually crafting elaborate prompts was fading. But as 2026 unfolded, I realized that what replaced prompt engineering wasn't simply "better tools." Prompt engineering gave way to context engineering, and now context engineering is giving way to an entirely new paradigm: harness engineering . In this post, I'll break down what harness engineering is, why it matters right now, and what its key components look like. A Harness for a Horse, a Harness for an Agent A harness originally refers to the tack fitted onto a horse. Bridle, saddle, stirrups — equipment designed not to suppress the horse's power, but to channel it in the right direction. In AI, the term means exactly the same thing. A harness is the entire external system that controls and directs an AI agent's power...
자세한 내용 보기