goldtag

Building an LLM Robot with My Son — EP 4. Choosing the Right Local LLM for Robot Control We needed to pick a model. Connecting a local LLM to the robot means committing to a specific open-source model. If we were using a cloud API, this decision would be trivial — just call GPT-4o or Claude. But our architecture runs a local LLM server on the home LAN. We had to test and decide ourselves. I set three evaluation criteria. Tool use — to send structured commands like "forward" or "stop," the model needs to reliably call JSON functions. If it sometimes returns proper JSON and sometimes writes prose explanations, parsing fails. Consistency matters more than peak performance. Korean language — my son gives instructions in Korean, and I want to read debug output in Korean. A model that drifts into English mid-response is just harder to use. Vision — we don't need it now, but we'll need camera frame input later. If the model has a vision variant in the same...

Building an LLM Robot with My Son — EP 3. Local LLM Speed Compared Across Mac M1, M4, and M4 Pro The first time I ran a local LLM on the Mac mini M1, I watched Qwen2.5-7B output tokens one character at a time and paused for a second. About 8 tokens per second. Not slow, exactly. But whether that's fast enough for real-time robot control is a different question — how long does it take from the robot sending a camera frame to receiving a command back? That needed a measurement, not a guess. I had three Macs already: Mac mini M1 16GB, Mac mini M4 24GB, MacBook Pro M4 Pro 14" 24GB. Same prompt, same model, three machines. The comparison made itself. Test Setup Model: Qwen2.5-7B-Instruct, Q4_K_M quantization. Both mlx-lm and llama.cpp Metal backend, measured separately. Metrics: - tok/s : tokens generated per second - TTFT : Time to First Token - Memory usage : at 32K and 128K context - Thermals : CPU/GPU temperature after 5 minutes of sustained load Three prompt types:...

Building an LLM Robot with My Son — EP 2. First AI Coding Experiment — Context Strategy Changed Everything My son gave his first AI coding command that evening. He sat down in front of the laptop, thought for a moment about what to type, and entered this: "stop if there's something in front of it" Claude Code produced code. He skimmed it — couldn't tell what most of it meant. "Can I upload it?" We uploaded from Arduino IDE. He put his hand in front of the robot. It didn't stop. "Why isn't it working?" I looked at the code. Claude had assigned the HC-SR04 Trig pin to pin 2. Our wiring uses pin 4. The CLAUDE.md context hadn't loaded in this session — he'd opened a new Claude Code session, and in that session there was no project context. That one moment is the whole point of this episode. What Happens When AI Writes Embedded Code Without Context HC-SR04 is a well-known ultrasonic sensor. Claude knows how it works — pulse ...

Building an LLM Robot with My Son — EP 1. Designing the AI Coding Agent Harness There was a day when I first asked Claude Code to write robot code. "Write Arduino code to control two DC motors with PWM using an L298N motor driver. IN1 on pin 8, IN2 on pin 9, ENA on pin 10. IN3 on pin 11, IN4 on pin 12, ENB on pin 13." The code came back clean. But when I tested it, the motors behaved wrong. ENA was assigned to pin 10, and internally the code was touching Timer1 — which was conflicting with the Servo library I had running. The error message was vague. It took two hours to find the cause. Claude didn't know I was also using the Servo library. It didn't consider the relationship between pin 10 and Timer1 in the context of my setup. As generic code, it wasn't bad. It just didn't know my project. That one bug was enough to reach the conclusion: you can't just use a generic AI coding tool as-is. What Breaks When You Use Generic AI Tools for Embedded Proje...

Building an LLM Robot with My Son — EP 0. Why We Started an AI Robot Project Series note : This is a retrospective — four months of work organized into ten episodes (EP 0 through EP 9) and published together. Each episode follows the actual sequence of events; the publication date is the same for all. This series isn't a finished project showcase. As I write EP 0, the starting point is exactly that — the beginning. There's a half-assembled acrylic chassis on my son's desk with one wheel still spinning in reverse. That's where we're starting. If you're in a similar situation — you want to do something technical with your kid but don't know where to begin — read this like you're watching someone else figure it out in real time. We Both Like Building Things My son loves LEGO. He's twelve, and he has the patience to sit with a Technic set spread across the floor and build it piece by piece from the manual. When it finally works, he gets that quie...

What I Thought Was Solid Ground: An Ordinary Developer's Notes on the AI Era I went up a mountain with a tangled mind. There was no destination. I just felt I had to walk. Anyone who has stared at a screen long enough knows the feeling — when something won't unknot, you have to look away to finally see it. That day, the screen wasn't the problem, but the instinct to leave my seat was the same. I put on my shoes, and my feet pointed me toward the mountain. I'm not sure if it was a conscious choice. Maybe I just wanted to be somewhere high. Maybe from up there, the place I was standing now would look small. If I'm being honest, nothing got sorted. If anything, things got more tangled. But one thing came into view. The path. There are several kinds of trails on a mountain. There are paths so trampled that the ground is fully packed. Every step lands on something firm. No matter how hard you stomp, no footprint remains. That path has been hardened by thousands, t...

아들과 함께 만들어보는 인공지능(LLM) 로봇 만들기 프로젝트 — EP 9. 4개월 회고, 그리고 다음은 레고 EP 0를 쓰던 날, 책상 위에 반쯤 조립된 아크릴 샤시가 있었다. 한쪽 바퀴가 반대로 돌고 있었다. 지금은 그 로봇이 주방까지 혼자 걸어가서 물컵을 찾는다. 가져오지는 못하지만. 4개월이 걸렸다. 잘 된 것들 에이전트 하네스 방식이 효과 있었다 CLAUDE.md 기반으로 도메인 지식을 주입하는 방식이 생각보다 효과적이었다. 매번 컨텍스트를 설명하는 피로가 없어졌고, AI가 프로젝트 규칙을 지키면서 코드를 짰다. 처음에 10줄로 시작한 파일이 120줄이 됐는데, 그게 프로젝트 자체의 성장 기록이기도 했다. 아들이 8번의 프롬프트로 동작 하나를 완성했다 EP 5에서 쓴 그 장면이 이 프로젝트 전체에서 가장 기억에 남는다. 아들이 혼자 앉아서 여덟 번 프롬프트를 다듬어서 장애물 회피 동작을 완성한 날. 코드를 한 줄도 안 쳤다. 근데 "내가 짠 거야"가 틀리지 않았다. Apple Silicon 로컬 LLM 조합이 맞았다 M4 Pro에서 112 tok/s. 클라우드 API 지연 없이 LAN 안에서 0.5초 이내 응답. 이게 실제로 작동한다는 걸 확인했다. 집에서 로컬 LLM 서버를 운영하는 게 생각보다 현실적이었다. 예상과 달랐던 것들 아두이노 → Pi 전환이 생각보다 빨리 왔다 EP 6에서 노트북 브릿지를 쓰는 순간, "이건 임시방편"이라는 게 너무 명확했다. 처음 설계에서는 EP 7이 Pi 전환이지만, 솔직히 EP 4~5 시점부터 이미 전환하고 싶었다. 한 편에 하나씩 넣는 시리즈 구조 때문에 참았다. 비전 LLM이 생각보다 잘 됐다 EP 8에서 "주방 가서 물컵 찾아줘"가 세 번째 시도에 됐을 때 솔직히 놀랐다. 하드코딩 없이, 지도 없이, 카메라 하나로 공간을 인식하고 목표를 찾아가는 것—이게 7B 파라미터 모델에서 된다는 게 2년 전이었으면 믿기 어려운 일이었을 거...