go
-
-
-
The other day, I found myself wondering how big 52! (52 factorial) is, and that led me to ponder how these could be estimated without a calculator or a computer. It turns out there’s some fairly interesting math behind being able to estimate the size (number of digits) of …
Eli Bendersky's website Aug 28, 2026
-
Go 1.27 adds generic methods—a highly desired language feature.
The Go Blog Aug 26, 2026 -
This is part 8 in a series of posts on writing concurrent network servers. In this part, we'll switch to Go and see how it tackles the challenges described earlier in the series. All posts in the series: Part 1 - Introduction Part 2 - Threads Part 3 - Event-driven Part 4 - libuv …
Eli Bendersky's website Aug 22, 2026
-
I used an AI agent to make a big update to my GiftyWeddings.com website, taking it from a gift registry to a full website builder in 275 commits.
Ben Hoyt's technical writing Aug 20, 2026 -
Porting Go's standard library to platform-agnostic C.
Anton Zhiyanov Aug 20, 2026 -
Whatever you think of AI coding, the way we build software is changing. Everyone wonders, “What will stay relevant?” All we have is opinions, so here’s mine: most ideas behind Domain-Driven Design are now more relevant than ever, as DDD has never been strictly about code. As we do more coding with agents, we still need to understand the domain, model it well, and work as a team. Let’s see what DDD can still teach us and what AI can’t replace. The usual disclaimer: I write about complex projects maintained over months or years, where teams tend to struggle and where DDD shines. There’s no reason to use advanced patterns in pet projects or CRUDs. What hasn’t changed since 2003 Eric Evans published Domain-Driven Design more than two decades ago, but it’s still surprisingly fresh. The book covers many topics, but the main message is that the hard part of software engineering is understanding the problem domain and modeling it well in code. It calls on engineers not to get lost in the technical details and to focus on what they’re solving, because it’s more difficult to get that part right. Instead, the technical talent goes to work on elaborate frameworks, trying to solve domain problems with technology. Learning about and modeling the domain is left to others. Complexity in the heart of software has to be tackled head-on. To do otherwise is to risk irrelevance. Eric Evans Domain-Driven Design We now see that the implementation details become less relevant thanks to AI tools. To a degree, you can be productive without a deep knowledge of a programming language or a framework, and it’s easier to move to another tech stack. If you’ve tried using the top coding agents, you’ve probably seen them quickly generate decent code. But knowing high-level software concepts is still essential to guide agents and to decide whether the output makes sense. Most of all, the domain complexity (the business problem you’re solving) is still the hard part. The good news is that coding is easier than ever, so we can focus on these complex problems instead of getting lost in the implementation details. Except, instead of obsessing over building the frameworks (or adding more microservices, or whatever else is a cool trend), we now follow the model benchmarks and optimize our agentic setup to generate better code with less effort. As Evans noted back in 2003, we still try to solve the domain problems with technology. Funnily enough, this time the CEOs also believe it and push us toward it. There’s just one small thing left to figure out: how the thing should work. If someone tells us, we can throw tokens at the implementation and call it a day. And if no one knows, well, maybe an agent could write the plan for us? The domain model isn’t an artifact The foundation of DDD is model-driven design: distilling how the domain works into a model that’s expressed in code. The domain model is an abstract concept, and there’s no one way to represent it. You can use documents, diagrams, and code, but they are all simplifications of how the domain works. An accurate model comes from switching between design and implementation, and applying what you’ve learned over and over. DDD calls this knowledge crunching. You need domain experts (the people who know how the business works), and engineers who are experts at building software. They work together to understand what the software should do and how to implement it. (For example, in an Event Storming session, a workshop where developers and stakeholders map out how the business works with sticky notes.) Figuring out the domain is a team effort and hard work, so it’s tempting to have someone else do it. For example, AI agents are brilliant at research and analysis, and can access your documents, chats, and code. It seems they could create the domain model for you, and then also implement it. But this naive approach misses the point. The value of working on the model is that you (and your team) understand how the domain works. An AI-generated wall of text doesn’t help you figure out the business problem. The agent will just create an impressive artifact no one reads. A domain model isn't an artifact. The challenge has never been to write a long document on how something could possibly work. More often, software projects fail because of other issues: Engineers work on the wrong thing. No one knows what needs to be done in the first place. The team tries to build everything at once (scope creep). Here’s one trap I’ve fallen into many times. In engineering teams, it’s easy to feel like we’ve got it all figured out: we’re programming experts, know our patterns, and follow best practices. We just need someone (a business stakeholder or a product manager) to tell us what to do. But often, the domain experts don’t have a clear plan either. They know the problem they have, but not what the solution is yet. There’s no complete document with all requirements. A developer’s first instinct may be to think, “Let me know once you come up with something, I’ll be around.” In other words, we’re here for the fun part of coding. (Note this is the part AI agents automate.) At the other extreme, non-technical managers try to come up with the entire solution and hand it to the team to implement. Or, the most recent trend, someone uses AI to generate a full feature or product spec, doesn’t even read it, and passes it along. So the plan sort of exists, but it’s nowhere near something ready for production. (If someone claims otherwise, check how complex the thing they’re shipping really is.) In all of these scenarios, no one has a clear idea what they’re doing. No matter who writes the code or how fast, it has little value at this point. In contrast, when you work on the model together, your team understands the problem domain and agrees on a solution. Developers implement it, and domain experts can jump in once something unexpected comes up or another iteration starts. Even if you do most coding with AI, you need to get deeply familiar with the domain to guide the agent, just as you would a teammate. How you configure the agents and what LLM you use don’t matter as much as a clear mental model of the solution. If I had to choose, I’d rather never write any code myself than skip thinking about what needs to be done. Design before writing generating code Writing code has always been easier than reading it. It’s just more extreme now that an agent can single-shot a big feature for you. Even if it works, you have no idea what’s going on under the hood. We now get stuck on code review more often, but it’s also not a new problem. It even feels familiar, like working with a lone-wolf developer who prepares complete features and drops a massive PR on you. You know nothing about the design and there are often gaps that no one has considered. Reviewing the PR is painful. There’s a way to avoid situations like this, and I’ve seen it work in many projects. You design and discuss the solution as a team before sitting down to code. First, the domain parts: what you’re solving and how it’s supposed to work (see knowledge crunching above). Then, the technical side. After you kick off the implementation, there’s little chance of anything unexpected coming up. You don’t need to agree on every tiny detail, and don’t need a complete written spec. It’s enough to sketch a high-level plan that everyone is aware of. Some sticky notes and a whiteboard are good enough. The best moment to change the design is before anyone writes code. After sessions like this, reviewing PRs is smooth. It’s easy to split the work into small chunks if you don’t discuss the entire design in PR comments. And there are fewer gaps once the feature lands in production, because you’ve already agreed on the details and everyone understands what they’re building. Skipping the design may seem like saving time. No one likes more meetings. But if your team first learns about the feature from the PR, it takes much longer to review, discuss, and fix all the issues. Since reviews are usually asynchronous, you end up with many rounds of comments. Code review should be a double-check that the implementation is correct, not the start of a discussion about whether the approach makes sense at all. Starting with a brief plan saves you time on review later on. Working on design before code will also help you work better with AI agents. If you give the agent more high-quality context, its output will be more accurate and the coding part will be shorter. And the more your team knows about the solution, the easier the review, even if you generate all the code. Ubiquitous Language: Speaking the same language as your agents Recently, I’ve been trying to cut our cloud costs. It’s the perfect task for AI, since there’s an obvious metric to measure. I told the agent to find where our Go service uses the most memory and to cut it. After a couple of runs, I saved us many megabytes of RAM. But then I realized memory is pretty cheap, and the updates reduced our costs by $1/month. Not surprising, right? I didn’t explain to the agent that I was trying to cut costs. I just gave it the memory usage as the target. It should sound familiar, because it’s exactly what happens between humans in teams. Good practices that help you communicate in a team are also relevant when you work with AI. (Who would have thought soft skills may help you deal with computers?) DDD addresses this with a few ideas around the language you use. The first is Ubiquitous Language: speaking the same language across teams and in code, so everyone knows what you’re talking about. In many teams, messy names are the default. In code, engineers use technical terms that drift away from the names that the rest of the company uses. Even if you feed AI documents and decision records, the agent needs to make sense of whatever language you use there. During knowledge crunching, consider what you call the concepts in the domain. Initially, it’s common to have more than one name for each. Choose a name you already use, or come up with a more accurate one. Then, use it in the documents, spoken language, and code. An Event Storming session is a great place for choosing the names. You can’t choose the names just once, because the language will keep evolving with the project. It’s something you need to care about during the design sessions and discussions. Another idea directly connected to the Ubiquitous Language is that in bigger systems, the names you choose won’t be universal in all areas. DDD calls each of these areas a Bounded Context. You can think of your customer as a profile in the support context, and as a user in the e-commerce context. You want to use consistent names within each context, not force them across the entire project. This is especially important with bigger products, when you need to split the software between teams. If you focus too much on creating one model for all kinds of tasks, it becomes difficult to draw boundaries. With bounded contexts in mind, it’s easier to end up with a decoupled system. Agents have access to your whole repository, and they’ll look for the names you give them. They may naively try to unify similar entities, so make it clear they are separate for a reason. The same domain concept can have different names depending on the context. If you use well-known, precise concepts, it’s easier to express what you mean. It’ll work both with your teammates and in your prompts. A vague prompt can make the agent spend a lot of effort on the wrong solution: Add user to CRM and support after it’s created You’ll make the goal more obvious by sticking to the precise names: Once the user signs up on the website, asynchronously create: 1) a customer entry in the CRM, 2) a profile in the support system You want the details about your domain language in your agents’ context. It’s yet another reason to keep the documentation up to date. Markdown files close to the code are a good starting point, since you don’t need a custom integration to fetch the documents. Developers become domain experts The models and agents keep getting better, but you can’t fully trust them. They still make mistakes, and you need someone accountable in the end. How can you know if the outcome is correct? Working in the domain is how you build trust that you know what you’re doing. People who design the system and understand how it works can tell you whether something is likely to go wrong. Developers become experts in the subject they’re working on. They learn skills beyond writing code, and for a curious engineer, even a complex business domain becomes intuitive with time. If you need help in that area, you know who to ask. You save yourself massive headaches by working with someone you can trust to get the job done without extremely detailed instructions. There are two recent AI narratives. One is that anyone can now generate a clone of a well-known app. This was shocking at first, but isn’t that interesting in the long run. We already have too many half-baked apps out there even without AI. The more interesting story is how experienced engineers can build more complex applications now. For example, a team can replace expensive third-party software with an in-house solution. But how can they trust their version works well? Because of the knowledge they already have thanks to working in that specific domain. They combine what they know with AI and build their own version so fast, it’s cheaper than paying for the license. On top of that, they can also add any custom features they need. For employers, having product-minded engineers around who know the domain well is huge. They can figure out the obscure details and guide the discussions. It’s always been naive to treat developers as people who code what you tell them, and now it’s even more so. For engineers, learning to work with an unknown domain pays off more than focusing on technical skills alone. It’s easier to work in a profit center if you’re good at solving business problems, rather than being an expert in one framework. Don’t delegate thinking DDD shows how there’s much more to software than writing code. If you’ve worked with complex systems, I’m sure you’ve seen it many times. This topic brings back memories from my first software job. I once worked on a tough problem, and spent hours doodling with pen and paper. I finally figured it out, even though I barely touched the keyboard. I had only recently started, so I was speechless: “Wow, today I got paid to think.” This felt wild at the time, but today it feels even wilder. Back then, my manager wouldn’t blink an eye if I said I spent a day thinking about a problem. Today, we focus more on the output. Why not have the AI do the thinking instead? Thinking, delegated. For one, because the only reason I can judge whether the agent’s output makes sense is that I’ve spent long hours thinking about such problems in the past. There’s the argument that you no longer need this experience, since AI will handle everything for you anyway. Relying on AI could eventually become the new normal, but there’s one issue I can’t ignore. You need to exercise your brain to work on complex problems, and you can’t build new mental models without thinking. The best analogy I have is reading a book versus a summary of it. Reading forces you to think about the topic for hours. You slowly build a complex model in your mind, and you connect it to other things you know. You understand the topic, instead of reading a few sentences that you forget ten seconds later when you close the browser tab. No one knows the best practices yet, so it’s a good time to experiment and see what works. For now, I’ll mix old-school problem-solving with trying out new ways of working with code. In this post I focused more on the philosophy behind DDD than individual patterns. I plan to write more, including about the tactical patterns closer to the code. I still find them useful, even if I don’t type them by hand. Sign up below so you don’t miss the next posts.
Learn Building Modern Go applications Aug 19, 2026 -
Go 1.27 adds generic methods, encoding/json/v2 package, uuid package, faster memory allocation, goroutine leak profiles, and more.
The Go Blog Aug 19, 2026 -
This is part 7 in a series of posts on writing concurrent network servers. In this part, we discuss how the challenges described in earlier parts are tackled in the Rust programming language. All posts in the series: Part 1 - Introduction Part 2 - Threads Part 3 - Event-driven Part 4 - libuv …
Eli Bendersky's website Aug 15, 2026
-
Reusing Go's tooling and standard library for a systems language.
Anton Zhiyanov Aug 9, 2026 -
This week I was working on one non-critical feature that seemed simple at first sight. 50k lines of code later, it seemed to work as expected. In a side project, or a service where “nobody will be upset if it’s dead and kills everything around”, it would be ready to ship. But for a production-grade project, the risk of an outage is too big to skip reading the code. Unfortunately, reviewing 50k lines of AI-generated code is much harder than reviewing 50k lines written by a human. We’re now obsessed with how much code we can create fast. But is it the right thing to optimize for? The biggest bottleneck of implementation quietly shifted from producing code to reading code. Fortunately, we don’t need to invent how to make code more readable: software engineering has had solutions for that for decades. The next bottleneck: creating code that we can be accountable for Many people no longer read the code that they ship. So why should we care about good coding practices and code quality if agents don’t? Keeping our code readable is still critical for one reason: it helps us overcome our biggest bottleneck, the human. When you write code by hand, you have the entire context of what you implemented. With agent-generated code, you have to rebuild that context, and it’s easy to get overwhelmed. You miss bugs that nobody would ever write by hand. You also lose the edge cases you would have found while writing the code yourself. Coding got faster. Review didn’t. Coding got faster. Review didn’t. We can review less code than before. It may be tempting to skip review entirely, but that’s throwing the baby out with the bathwater. The last block in the diagram is “successful delivery” for a good reason: our goal is not just to ship, but to ship working things that don’t break other things and won’t slow us down in the long term. Judging by how many outages big companies had recently, this goal is not obvious to everyone. It’s a well-known pattern: new techniques and tools make top-performing teams even faster, while low performers get slower. It happened with the cloud and with microservices. The pragmatic teams shipped faster, while low performers got stuck in never-ending migrations. The same will happen with AI. Teams with high friction between idea and implementation won’t see the same performance gain. And if you invest in AI in the wrong places, you may even end up slower. You can generate 100k lines of code and burn thousands of dollars on tokens in one day. That means nothing until you’re sure it works, doesn’t break other things, and solves the problem it should. In other words, it doesn’t matter how much code you can generate. What matters is how much of it you can take accountability for and whether it solves a real problem. You probably see PRs like this too often now You generated a new feature, and after a couple of iterations you believe it’s ready. What now? You have a couple of options for how to find out if it works as expected: Just create a PR without reading the code and let the reviewer check the feature… and wait for your team to hate you for submitting slop-PRs. Just read the tests and pray that AI agents didn’t forget any edge cases, that they covered all side effects, and that previous features are also protected against regressions. Read and understand the damn code. The third option is the safest, but unfortunately the hardest one. Maybe we saved a lot of time we would spend on writing code, but now we have a harder task: reviewing it. The volume of code generated by agents is much bigger than what people used to write by hand. What’s worse, reviewing code written by an agent is much harder than reviewing code that you have written by hand. After spending hours reading and fixing the code, you start thinking “Maybe writing it by hand would be faster…”. Writing by hand sounds like a comfortable solution, but even counting the hours you spend polishing the agent’s code, you would not be faster. What helps here is making the agents’ code more readable. The good news is that we don’t need to reinvent the wheel. Creating readable code is something that software engineers have been trying to figure out for several decades. Good practices exist to help us extend code faster. But we often miss the one part that matters most for review: readability. Not all software is the same We have always been big advocates of Domain-Driven Design and Clean Architecture in the Go community. We have seen how they helped us to rescue many projects. To our surprise, promoting them used to trigger pushback from many people saying “it’s overengineering and Go code should be simple”. And they have been right: Using DDD and Clean Architecture leads to overengineering… and so does skipping them. It all depends on the kind of project you apply them in and on how well you understand them. When we talked to skeptics who advocate for “Simple Go”, we quickly figured out that their opinion often came from a single poorly implemented project. Usually, the people who built that project had no idea how to use those tools. Or this project was just a simple CRUD that didn’t need any sophisticated techniques. Any tool, when wrongly used, does more harm than good. What works for prototypes, CRUDs, and side projects doesn’t work for production-grade complex projects. On the other hand, prototypes and simple code will suffer from many ceremonies that production-grade critical software needs. The difference matters more than ever with AI-assisted coding. Not reading generated code is also fine I don’t read every piece of AI-generated code. One type of code I often don’t review at all is the presentation layer (frontend, for example). As long as it looks good and works as expected, I don’t care about the implementation. There is one important precondition to make it happen: the frontend should be a thin presentation layer without overcomplicated logic, with a sane API behind it. When you have complex frontend state with GraphQL under the hood, such a strategy is much riskier. This is where any kind of layered architecture helps us: we separate the boring code from the critical code, so we don’t need to spend the same mental effort on all of it. Next time you hear someone say they never read generated code, check what kind of code they write. Most of the time it may be presentation-heavy non-critical software. It can be a sane strategy, as long as you apply it deliberately to one part of the project. The more things change, the more they stay the same Many software engineers now face an existential crisis: did everything we learned over the years become obsolete? If someone spent years learning languages, frameworks, and vim motions, then it’s true: those skills are much less relevant now. Even if we see more and more examples of agents doing impressive work, it’s mostly in scenarios with a clear feedback loop that agents can “brute-force”. The challenge starts when success criteria are unclear. That describes most of the business software we create. I also see arguments like “software is dead, my girlfriend just vibe coded in a weekend what I used to write in half a year”. Even if it may be true, we come back to the same point: not all software is the same. Shipping a PoC in a weekend is light years from building production-grade software with many users. If that PoC became popular, it would take a team of engineers to develop it and keep it working (and making money). An expert product engineer decides what to build and why, not only how. If that’s you, your knowledge of what code to write is still valid and in demand. Part of this job was always taking accountability for the changes we ship. It doesn’t matter if the code was written by hand or generated by an agent: you ship it, you own it. Not much has changed. Taking accountability was always the area where the best job offers were sitting. Good coding practices were never for the machines. Compilers never cared about them, and agents may work slightly better with clean code, but they are fast enough without it. Good practices were always for you, the person accountable for what you ship. The one thing that changed is that we, as software engineers, now have to figure out how to collaborate with agents to create code efficiently. In the next posts, we’ll show you how we do it.
Learn Building Modern Go applications Aug 5, 2026 -
In Physics simulations or game engines it’s sometimes useful to determine the speed with which two objects are approaching each other. This post will discuss the concept of closing speed, which is the normal component of the relative velocity of two objects. Relative velocity and its components Suppose we …
Eli Bendersky's website Aug 4, 2026
-
Building a generic iterator wheel from scratch.
Anton Zhiyanov Aug 3, 2026 -
A strict subset of Go that translates to regular C.
Anton Zhiyanov Jul 25, 2026 -
The developer job as we knew it is ending. Handoff chains collapse into teams of generalists who own requests end to end, directing AI agents to design, code, and test. What that means for the role.
alexop.dev Jul 18, 2026 -
The Fourier series is a great tool for analyzing periodic functions. But what about functions that don’t repeat? We’ve seen that we can compute Fourier series for a non-periodic function defined on a finite interval, as long as we don’t care about its behavior beyond that interval …
Eli Bendersky's website Jul 15, 2026
-
When I want to add sprinkles of interactivity to a web application, I'm a big fan of using HTMX. I like that it makes it easy to give interactions a smooth app-like feel, I like that it minimizes the amount of JavaScript that I have to write, and I like that it allows me to keep the consistency and safety of server-side HTML rendering with Go's html/template package. In this post I'm going to run through how I typically use HTMX in conjunction with Go. Although I'm going to talk a bit about how HTMX works, the main focus is going to be on the Go side of things. Specifically: Structuring and rendering HTML templates The patterns I use for sending back partial and full-page HTML responses to HTMX Managing redirects and errors when using HTMX The standard HTMX configuration settings that I use, and why To illustrate these things, we'll run through the build of a small application that ultimately implements a filter on a list of users like this: Note: If you're not already familiar with the basics of using HTMX, I recommend skimming through the HTMX docs before continuing. Also note: A lot of the patterns for working with HTML templates should also be a good fit for other HTML-over-the-wire tools like Unpoly and Hotwire too, if you prefer to use those. Project setup If you'd like to follow along, go ahead and run the following commands to create a skeleton structure for the project: $ go mod init example.com/htmx $ mkdir -p assets/static/css assets/static/img assets/static/js assets/html/partials assets/html/pages cmd/web $ touch assets/efs.go assets/html/base.tmpl assets/html/partials/images.tmpl assets/html/pages/home.tmpl cmd/web/main.go cmd/web/handlers.go cmd/web/html.go That should give you a file tree which looks like this: . ├── assets │ ├── efs.go │ ├── html │ │ ├── base.tmpl │ │ ├── pages │ │ │ └── home.tmpl │ │ └── partials │ │ └── images.tmpl │ └── static │ ├── css │ ├── img │ └── js ├── cmd │ └── web │ ├── handlers.go │ ├── html.go │ └── main.go └── go.mod Installing HTMX There are a few different ways to install HTMX, and you could load it from a CDN or install it using NPM, but I almost always download a copy and serve it as a static file from my web application. It's simple and avoids the downsides of using a CDN. For the purpose of this demo project, we'll also download Bamboo (a classless CSS framework) and an image of a gopher from github.com/egonelbre/gophers. Go ahead and run the following commands to download all three things into the assets/static folder: $ wget -P assets/static/js https://cdn.jsdelivr.net/npm/htmx.org@2.0.10/dist/htmx.min.js $ wget -P assets/static/css https://cdn.jsdelivr.net/npm/bamboo.css@1.4.0/dist/bamboo.min.css $ wget -O assets/static/img/gopher.png https://raw.githubusercontent.com/egonelbre/gophers/refs/heads/master/sketch/misc/standing-left.png The contents of assets/static should now look like this: assets/static ├── css │ └── bamboo.min.css ├── img │ └── gopher.png └── js └── htmx.min.js The HTML templates OK, now that the project skeleton and our static assets are in place, let's get to the main thrust of this post and talk about HTML templates. My starting point in almost all projects is an assets/html directory which has a folder structure like this: assets/html ├── base.tmpl ├── pages │ └── home.tmpl └── partials └── images.tmpl Under this structure: The assets/html/base.tmpl file contains the common HTML 'layout' markup for all web pages. The files in the assets/html/pages directory contain the page-specific content for individual web pages. The files in the assets/html/partials directory contain reusable chunks of HTML markup that can be used in different places. If you're following along, go ahead and add the following markup to the base.tmpl file: File: assets/html/base.tmpl {{define "base"}} <!doctype html> <html lang='en'> <head> <meta charset='utf-8'> <title>{{template "page:title" .}}</title> <meta name="viewport" content="width=device-width, initial-scale=1"> <link rel="stylesheet" href="/static/css/bamboo.min.css"> <script defer src="/static/js/htmx.min.js"></script> </head> <body> <h1><a href="/">Example website</a></h1> <main> {{template "page:content" .}} </main> </body> </html> {{end}} There are a few things to point out about this: In the <head> section we import the Bamboo CSS file and the HTMX JavaScript file. Note that when importing HTMX we use the defer attribute. This means that HTMX will be fetched by the browser in parallel as it is parsing the web page HTML, but the script won't be executed until the HTML is fully parsed and the DOM is built. There's an excellent blog post which describes how defer works and why it's the right choice here. When writing HTML templates, I like to give all of my templates explicit names by surrounding the markup in {{define}}...{{end}} actions — even if (like in this case) a file only contains one template and it's not strictly necessary. YMMV, but I prefer the consistency and clarity of being able to always refer to templates by defined names from my Go code, rather than using a mixture of defined names and filenames. Within the template, we use actions like {{template "page:title" .}} to inject the appropriate page-specific content in the right place. Talking of which, let's now add the page-specific content for the homepage to the assets/html/pages/home.tmpl file: File: assets/html/pages/home.tmpl {{define "page:title"}}Home{{end}} {{define "page:content"}} <button hx-get="/gopher" hx-swap="outerHTML"> Wanna see a cute gopher? </button> {{end}} In this page we have a <button> with two HTMX attributes: hx-get="/gopher" and hx-swap="outerHTML". These mean that when this button is clicked, HTMX will intercept the click, send a GET /gopher request to our application, and then replace the button in the DOM with whatever HTML our application sends back. Note: The colon character in the template name like {{define "page:title"}} is just an arbitrary separator and you could name it something else, like page_title, page-title, pageTitle or even just title if you prefer. But I like using : because it feels like a natural and clear way to 'namespace' template names. Lastly, let's add a template to the assets/html/partials/images.tmpl containing some HTML for displaying our downloaded gopher image, like so: File: assets/html/partials/images.tmpl {{define "partial:image:gopher"}} <img alt="Gopher" src="/static/img/gopher.png" width="{{.}}"> {{end}} Note that we're using width="{{.}}" in this markup, so that we can pass a dynamic value for the image width to the template. Embedding the assets Since file embedding was introduced in Go 1.16, I normally embed HTML files and static assets into a Go binary rather than reading them from disk at runtime. Let's update the assets/efs.go file to embed the contents of the assets/html and assets/static directories, and make them available in two global variables called HTMLFiles and StaticFiles respectively. Like so: File: assets/efs.go package assets import ( "embed" "io/fs" ) //go:embed "html" "static" var files embed.FS var ( HTMLFiles = sub(files, "html") StaticFiles = sub(files, "static") ) func sub(f embed.FS, dir string) fs.FS { sub, err := fs.Sub(f, dir) if err != nil { panic(err) } return sub } In this code, the //go:embed "html" "static" directive embeds the contents of the assets/html and assets/static directories into the files variable, which is an embed.FS rooted in the assets directory. I've then used a small sub() function to create two sub-filesystems with their roots in the html and static directories, and assigned them to the HTMLFiles and StaticFiles variables respectively. Doing this has two benefits: It provides a clear separation between the static and HTML files when we are using them from our Go code. Code that is intended to only work with our static files won't have unnecessary access to our HTML files, and vice-versa. Code using the HTMLFiles and StaticFiles filesystems doesn't need to include the html/ or static/ path prefix when opening files. Note: If you don't want to call panic() from the sub() function, you could restructure this to return an error instead, and initialize the HTMLFiles and StaticFiles variables from within your main() function. But the risk of a runtime panic here is extremely low — the fs.Sub() function will only return an error if the dir value is not a valid path, and the static strings "html" and "static" always pass this check. In practice, I've never had any problems using this approach. HTML template rendering For rendering the HTML templates in an HTTP response, I've found that a nice pattern is to create a htmlRenderer type which a) parses a set of shared templates at startup; b) has a render() method that clones and extends the shared template set, before executing a specific named template and sending it as an HTTP response. Go ahead and create the htmlRenderer type in the cmd/web/html.go file like so: File: cmd/web/html.go package main import ( "bytes" "html/template" "io/fs" "net/http" "time" ) type htmlRenderer struct { templateFS fs.FS sharedTemplates *template.Template } // The newHTMLRenderer function creates a new htmlRenderer containing a shared // set of parsed templates with support for any custom template functions. func newHTMLRenderer(templateFS fs.FS, sharedTemplateFiles ...string) (*htmlRenderer, error) { funcs := template.FuncMap{ "now": time.Now, // Other custom template functions go here... } sharedTemplates, err := template.New("").Funcs(funcs).ParseFS(templateFS, sharedTemplateFiles...) if err != nil { return nil, err } r := &htmlRenderer{ templateFS: templateFS, sharedTemplates: sharedTemplates, } return r, nil } // The render method clones the shared template set, optionally parses additional // templates, executes the named template with the supplied data, and writes the // response. func (h *htmlRenderer) render(w http.ResponseWriter, status int, data any, templateName string, additionalTemplateFiles ...string) error { ts, err := h.sharedTemplates.Clone() if err != nil { return err } if len(additionalTemplateFiles) > 0 { ts, err = ts.ParseFS(h.templateFS, additionalTemplateFiles...) if err != nil { return err } } buf := new(bytes.Buffer) err = ts.ExecuteTemplate(buf, templateName, data) if err != nil { return err } w.WriteHeader(status) buf.WriteTo(w) return nil } And then in the cmd/web/main.go file, let's create a basic web application like so: File: cmd/web/main.go package main import ( "log/slog" "net/http" "os" "example.com/htmx/assets" ) // The application struct holds the dependencies needed for our handlers, // including a htmlRenderer type. type application struct { logger *slog.Logger html *htmlRenderer } func main() { logger := slog.New(slog.NewTextHandler(os.Stdout, nil)) // Initialize a new htmlRenderer, parsing the base template and all partial // templates from assets/html into the shared template set. htmlRenderer, err := newHTMLRenderer(assets.HTMLFiles, "base.tmpl", "partials/*.tmpl") if err != nil { logger.Error(err.Error()) os.Exit(1) } // Include the htmlRenderer in the application struct. app := &application{ logger: logger, html: htmlRenderer, } // Create a file server that serves the files from assets/static. fileserver := http.FileServerFS(assets.StaticFiles) // Register the application routes. mux := http.NewServeMux() mux.Handle("GET /static/", http.StripPrefix("/static", fileserver)) mux.HandleFunc("GET /{$}", app.home) // Start the HTTP server. logger.Info("starting server", "port", 5051) err = http.ListenAndServe(":5051", mux) if err != nil { logger.Error(err.Error()) os.Exit(1) } } The important and relevant thing for this post is the initialization call to newHTMLRenderer(). In this call we pass in the glob paths "base.tmpl" and "partials/*.tmpl", which means that the base template and all templates in the partials directory will be available in the shared template set. And with that in place, we can then write the code for the home handler in cmd/web/handlers.go like so: File: cmd/web/handlers.go package main import ( "net/http" ) func (app *application) home(w http.ResponseWriter, r *http.Request) { err := app.html.render(w, 200, nil, "base", "pages/home.tmpl") if err != nil { app.logger.Error(err.Error()) http.Error(w, http.StatusText(500), 500) } } When we call render() in the code above, we are effectively saying append the templates in pages/home.tmpl to the shared template set, and then render the base template along with a 200 OK status. At this point, you should be able to successfully run the application: $ go run ./... time=2026-06-27T21:05:01.668+02:00 level=INFO msg="starting server" port=5051 And if you visit http://localhost:5051 in your browser, you should see the homepage displayed like so: Rendering partials While you're on this homepage, if you open developer tools and then click the "Wanna see a cute gopher?" button, you'll see that it sends a GET /gopher request that 404s. Let's fix this so that our application includes a GET /gopher route, which returns the contents of the partial:image:gopher template. First add the new route like so: File: cmd/web/main.go package main .. func main() { ... mux := http.NewServeMux() mux.Handle("GET /static/", http.StripPrefix("/static", fileserver)) mux.HandleFunc("GET /{$}", app.home) mux.HandleFunc("GET /gopher", app.gopher) ... } And then in cmd/web/handlers.go create a new gopher() handler, which renders the partial:image:gopher template with a width of 100px. File: cmd/web/handlers.go package main ... func (app *application) gopher(w http.ResponseWriter, r *http.Request) { width := 100 err := app.html.render(w, http.StatusOK, width, "partial:image:gopher") if err != nil { app.logger.Error(err.Error()) http.Error(w, http.StatusText(500), 500) } } Because we've set up our htmlRenderer type so that the shared template set already includes all partials, it's sufficient for us to call render() like this without passing in any additional file paths. If you re-run the application now and click the button, you should see that it gets swapped out for a gopher image like so: So, it's taken a while to get here, but the pattern that we now have in place is neat and has some nice benefits. Our templates (and static assets) are embedded into the Go binary, which makes for easy distribution and deployment. We can use the same htmlRenderer.render() function to send either complete HTML pages or specific partials to the client, which makes it easy to send back partial responses when they are needed by HTMX. We can keep the HTML markup nice and DRY by using the base template and partials. The partials can be inserted in the base template, page-specific content, or even in other partials. A more complex example That was very basic in terms of interactivity, so let's do something a bit more realistic and create a 'user search' page that mimics the active search example from the HTMX website. To make this work, we'll create two new routes in our application: A GET /users route which returns a full HTML page containing a table of all user details. A GET /users/search route which returns an HTML partial containing table rows only for users whose names or emails match a specific search value. Now that we've got all the groundwork in place, it should be pretty quick to do. Let's first add an assets/html/pages/users.tmpl file with the page-specific HTML content: $ touch assets/html/pages/users.tmpl File: assets/html/pages/users.tmpl {{define "page:title"}}Users{{end}} {{define "page:content"}} <input type="search" name="query" placeholder="Begin Typing To Search Users..." hx-get="/users/search" hx-trigger="input changed delay:500ms, keyup[key=='Enter']" hx-target="#search-results" hx-push-url="true"> <table> <thead> <tr> <th>Name</th> <th>Email</th> <th> </th> </tr> </thead> <tbody id="search-results"> {{template "users:rows" .}} </tbody> </table> {{end}} <!-- Fragments for the users page --> {{define "users:rows"}} {{range .}} <tr> <td>{{ .Name }}</td> <td>{{ .Email }}</td> <td> {{if .IsGopher}} {{template "partial:image:gopher" 24}} {{end}} </td> </tr> {{end}} {{end}} There are a couple of interesting things here. The first is the HTMX attributes on the <input> control. We've configured this so that when a user types into the input, after a delay of 500ms (or immediately if they press Enter), HTMX will send a request containing the search term as a query string like GET /users/search?query=foo. When a response is received, HTMX will then swap the response into the inner HTML of the <tbody id="search-results"> element. For demonstration purposes in this project, we're also using the hx-push-url="true" attribute, which will result in the browser URL bar being updated and a new entry added to the browser history each time HTMX makes a request. I've also structured the file so that the table rows are rendered in their own users:rows template, rather than as part of the page:content template. We'll use this in the GET /users/search to render just the matching user table rows for HTMX to swap in. Note: In theory, we could define the users:rows template inside the partials directory instead, and that wouldn't be an unreasonable thing to do. But if I have a HTML fragment that is only used on one specific page, I think it's clearer and neater to define that fragment inside the page file alongside the other content for the page. YMMV though, and that's OK. Then let's set up the two new routes in main.go: File: cmd/web/main.go package main .. func main() { ... mux := http.NewServeMux() mux.Handle("GET /static/", http.StripPrefix("/static", fileserver)) mux.HandleFunc("GET /{$}", app.home) mux.HandleFunc("GET /gopher", app.gopher) mux.HandleFunc("GET /users", app.listUsers) mux.HandleFunc("GET /users/search", app.searchUsers) ... } And lastly let's go to the handlers.go file and create a hardcoded list of user details, along with the two new handlers listUsers and searchUsers, like so: File: cmd/web/handlers.go package main import ( "net/http" "strings" ) ... // Define a user type. The fields need to be exported so that we can reference // them in our HTML templates. type user struct { Name string Email string IsGopher bool } // Create a hardcoded list of users. var users = []user{ {"Alice Madsen", "alice.madsen@example.com", true}, {"Theo Thatcher", "theo.thatcher@example.com", true}, {"Maxwell Albright", "maxwell.albright@example.com", false}, {"Ruby Thompson", "ruby.thompson@example.com", false}, {"Leona Rowan", "leona.rowan@example.com", false}, {"Alicia Lennox", "alicia.lennox@example.com", true}, {"Ruben Mason", "ruben.mason@example.com", false}, {"Leo Reynolds", "leo.reynolds@example.com", false}, {"Max Lester", "max.lester@example.com", true}, {"Theodore Allister", "theodore.allister@example.com", false}, } func (app *application) listUsers(w http.ResponseWriter, r *http.Request) { // Render a full HTML page containing the content from "pages/users.tmpl" // and all user details. err := app.html.render(w, 200, users, "base", "pages/users.tmpl") if err != nil { app.logger.Error(err.Error()) http.Error(w, http.StatusText(500), 500) } } func (app *application) searchUsers(w http.ResponseWriter, r *http.Request) { // Filter down the list of users to find ones that match the query. query := r.FormValue("query") var matches []user if query == "" { matches = users } else { for _, u := range users { if strings.Contains(u.Name, query) || strings.Contains(u.Email, query) { matches = append(matches, u) } } } // Render just the "users:rows" template from the "pages/users.tmpl" file // with the matching user details. err := app.html.render(w, 200, matches, "users:rows", "pages/users.tmpl") if err != nil { app.logger.Error(err.Error()) http.Error(w, http.StatusText(500), 500) } } When it comes to template rendering, in both of these new handlers we are adding the templates from the pages/users.tmpl file to the shared template set, but in listUsers we execute the base template and in searchUsers we execute just the users:rows template. So with just a little bit of thought to how we structured the markup and defined the templates in the pages/users.tmpl file, it's straightforward for us to send back either a complete HTML document or the appropriate partial HTML fragment for HTMX to do its thing. If you want, try this out by visiting http://localhost:5051/users and you should see the list being filtered as you type. Note: I've been deliberately keeping this web application simple so that the focus is on templates and templating. In a real application, you might want to merge listUsers and searchUsers into a single handler, create some centralized helpers for error handling and logging, use middleware to add Content Security Policy headers and recover panics, set appropriate server timeouts, etc. Checking if a request is coming from HTMX This all works well, but what if someone visits a link like http://localhost:5051/users/search?query=leo directly? Or shares a link to it? Anyone visiting this directly would only see the partial HTML response in their browser, similar to this: This obviously isn't ideal. A much better approach would be to change the response that our searchUsers handler sends, depending on whether the request is coming from HTMX or not. Specifically: If the request is coming from HTMX, we should return an HTML partial that it can swap into the table, just like we already are. If the request is not coming from HTMX, we should return a full HTML page that contains the matching user details. As you may already know if you've used HTMX before, requests that come from HTMX always include an HX-Request: true header. So all we need to do is check for the presence of that in the request, and send back the appropriate response. To help with this, I normally create a little isHTMXRequest() function and use it like so: File: cmd/web/handlers.go package main ... func isHTMXRequest(r *http.Request) bool { return r.Header.Get("HX-Request") == "true" } func (app *application) searchUsers(w http.ResponseWriter, r *http.Request) { query := r.FormValue("query") var matches []user if query == "" { matches = users } else { for _, u := range users { if strings.Contains(u.Name, query) || strings.Contains(u.Email, query) { matches = append(matches, u) } } } // Render the base template by default. template := "base" // But if the request is coming from HTMX, render the users:rows template instead. if isHTMXRequest(r) { template = "users:rows" } err := app.html.render(w, 200, matches, template, "pages/users.tmpl") if err != nil { app.logger.Error(err.Error()) http.Error(w, http.StatusText(500), 500) } } If you restart the application and visit http://localhost:5051/users/search?query=leo again now, you should see a full HTML page containing only the matching user records. But there are a couple more things we need to do to finish this up. Because we're sending back different responses from searchUsers based on the value of the HX-Request header, we should also set a Vary: HX-Request on the response to tell any caches between our server and the client that responses may be different based on the value of this header. We could set the Vary: HX-Request header in searchUsers, but I think it's easier to just always set it on all responses in the render() function. It does mean that we'll be setting the Vary header on all responses — including those from our home handler and listUsers — which isn't strictly necessary and a little bit wasteful. But I think it's worth it to avoid having to remember setting the Vary header correctly in individual handlers, and the risk of bugs that forgetting it may cause. File: cmd/web/html.go func (h *htmlRenderer) render(w http.ResponseWriter, status int, data any, templateName string, additionalFiles ...string) error { ... w.Header().Add("Vary", "HX-Request") w.WriteHeader(status) buf.WriteTo(w) return nil } Lastly, we need to consider back-button behavior. Whenever HTMX adds an entry to the browser history (which it will do when you use the hx-push-url or hx-boost attributes), it caches the HTML for the complete page in the browser's local storage. When the user clicks the back button, this complete cached HTML page will be reshown to them. By default the HTMX cache stores up to 10 pages. If there is a cache miss (i.e. the user navigates back far enough that there is no longer a matching page in the cache), HTMX will resend the request to the server to refetch the content for that URL. The problem is that this request will include the HX-Request: true header, and our application will send back a partial HTML response rather than the complete HTML page that it needs to redisplay to the user. To deal with this scenario, there is an historyRestoreAsHxRequest setting which controls whether HTMX will include the HX-Request: true header when it's sending a request because of a cache miss. The documentation advises: This should always be disabled when using HX-Request header to optionally return partial responses. So let's go ahead and configure HTMX so that the historyRestoreAsHxRequest setting is false. There are a couple of ways you can configure HTMX settings, but I generally like to set them in a meta tag in the base.tmpl file like so: File: assets/html/base.tmpl {{define "base"}} <!doctype html> <html lang='en'> <head> <meta charset='utf-8'> <title>{{template "page:title" .}}</title> <meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="htmx-config" content='{ "historyRestoreAsHxRequest": false }' > <link rel="stylesheet" href="/static/css/bamboo.min.css"> <script defer src="/static/js/htmx.min.js"></script> </head> <body> <h1><a href="/">Example website</a></h1> <main> {{template "page:content" .}} </main> </body> </html> {{end}} Now that this is set, if there is a cache miss when using the back button, HTMX will send a request to our application without the HX-Request: true header, and our application will send back the complete HTML page to reshow to the user. Managing redirects Using HTMX in your application normally reduces the need for 3xx redirects. For example, when submitting a form you can often send back an HTML partial with a success message that can be swapped into the page, rather than using the standard Post/Redirect/Get pattern and redirecting to a confirmation page. But still, there may be times that you want to redirect to a completely new page after a form submission coming from HTMX. A common example would be redirecting to a profile page after a successful login. Unfortunately, to achieve this you can't just send a regular 3xx response. The crux of the problem is that that browsers will automatically intercept and follow 3xx responses before HTMX has access to them — so HTMX never gets to see the 3xx response, only the final response after any redirects. It doesn't know that a redirect happened behind the scenes, and will just swap in the returned content like normal. Instead, if you want something that behaves more like a regular redirect, you need to send a 2xx response along with the HX-Redirect header. For example, when you include the response header HX-Redirect: /foo/bar, it will make HTMX tell the browser to navigate to /foo/bar, triggering a full-page reload. Importantly the HX-Request: true header will not be included in the request to /foo/bar. But you also need to handle the situation where the original request might not be coming from HTMX — especially if you are using progressive enhancement so that your application still works if JavaScript is disabled or HTMX doesn't load correctly. In that case, it's important to fallback to sending a regular 3xx response from your Go handler, rather than the 2xx response and HX-Redirect header. Putting this together, I normally create a redirect() helper which leverages the isHTMXRequest() function we made earlier and looks like this: func redirect(w http.ResponseWriter, r *http.Request, url string, code int) { if isHTMXRequest(r) { w.Header().Set("HX-Redirect", url) w.WriteHeader(http.StatusNoContent) return } http.Redirect(w, r, url, code) } And, for example, in the scenario of wanting to redirect to a /profile page after a successful login, I use it in my Go handlers like this: redirect(w, r, "/profile", http.StatusSeeOther) As I mentioned above, using HX-Redirect will trigger a full-page reload. But there is another option — the HX-Location header — which makes HTMX mimic the behavior of a redirect without a full-page reload. It essentially makes HTMX fetch the HTML for the provided URL, swap it into the HTML body, and add a new entry to the browser history. Importantly, when fetching the HTML the HX-Request: true header is used. At first glance, using HX-Location might seem preferable because it doesn't make a full-page reload, which gives your application a smoother more SPA-like experience. But it's a problem if the route that you are redirecting to uses the HX-Request: true header to conditionally send HTML partials. The handler has no way of telling whether the request is coming from HTMX following a HX-Location redirect (in which case it should send a full-page response) or from a 'normal' HTMX request (in which case it should return a partial). Unfortunately, unlike history restore requests, HTMX doesn't provide a setting to disable the HX-Request: true header when redirecting. So, most of the time I think it's easier and safer to use HX-Redirect and accept the downside of a full-page reload. But... if you are careful and structure your application so that the routes you redirect to only ever return full HTML pages, you may want to change the redirect() helper to use HX-Location instead like so: func redirect(w http.ResponseWriter, r *http.Request, url string, code int) { if isHTMXRequest(r) { w.Header().Set("HX-Location", url) w.WriteHeader(http.StatusNoContent) return } http.Redirect(w, r, url, code) } Managing errors If our demo application returns a 4xx or 5xx response, by default HTMX will not swap in the response. Instead it leaves the DOM as-is, and logs an error message in the console. If you'd like to see this in action, go ahead and change the gopher handler to render a "partial:image:missing" template (which doesn't exist). This should cause our application to error and send a 500 status code and a plaintext "Internal Server Error" response to the client. File: cmd/web/handlers.go func (app *application) gopher(w http.ResponseWriter, r *http.Request) { err := app.html.render(w, http.StatusOK, 100, "partial:image:missing") if err != nil { app.logger.Error(err.Error()) http.Error(w, http.StatusText(500), 500) } } If you run the application and click the "Wanna see a cute gopher?" button, now nothing on the screen will change, but in your developer tools network tab you'll see the 500 response and a record of the problem in the console, like so: In most cases, this isn't ideal. If an application is sending back an error message, I normally want the user to actually see this message rather than having the operation fail silently for them (or at least, silently unless they have developer tools open 😉). And also in most cases, I want to display any error message from a 4xx or 5xx response as full-page HTML (in the same way that it would be shown if we weren't using HTMX) by swapping it into the <body> element rather than swapping it into the regular HTMX target. The only exception to this is the 422 Unprocessable Content status, which I typically use when sending back a form with validation errors in it. In this case, I want HTMX to swap the returned content into the target element as normal. Luckily, you can use the HTMX responseHandling setting to configure different behavior for different responses codes. I normally configure this so that: For 204 No Content responses, no action is taken and no changes are made to the DOM. For 422 Unprocessable Content responses, HTMX swaps the response content into the target as normal. For all other 4xx and 5xx responses, HTMX swaps the returned content into the <body> element. For any other response, HTMX swaps the response content into the target as normal. And just like before, I normally configure this via the HTMX configuration meta tag like so: File: assets/html/base.tmpl {{define "base"}} <!doctype html> <html lang='en'> <head> <meta charset='utf-8'> <title>{{template "page:title" .}}</title> <meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="htmx-config" content='{ "historyRestoreAsHxRequest": false, "responseHandling":[ {"code":"204", "swap": false}, {"code":"422", "swap": true}, {"code":"[45]..", "swap": true, "target": "body"}, {"code":"...", "swap": true} ] }' > <link rel="stylesheet" href="/static/css/bamboo.min.css"> <script defer src="/static/js/htmx.min.js"></script> </head> <body> <h1><a href="/">Example website</a></h1> <main> {{template "page:content" .}} </main> </body> </html> {{end}} With that change made, if you restart the application and click the button again, you should now see the "Internal Server Error" message shown as a full-page response, like so: Note: If I ever want to swap an error message into a target that isn't the <body> element, then I use the response target extension to override the above settings for a specific interaction and swap into a specific target. But... I find that the above settings are a good starting default. Current browser URL Because HTMX makes AJAX requests and swaps in responses without changing the browser URL (unless you use the hx-boost, hx-push-url or hx-replace-url attributes), the request URL that we see in our Go handlers when accessing r.URL may be different to the one that the user is seeing in their browser. Occasionally, there are times when I want to know in my Go handler exactly what URL the user is currently seeing in their browser. Fortunately, HTMX sends this information with each request in the HX-Current-URL header. Usually I make another little function to help with this, which parses the HX-Current-URL value and returns it as a url.URL, falling back to returning r.URL if no HX-Current-URL header is present. Like so: func browserURL(r *http.Request) (*url.URL, error) { cu := r.Header.Get("HX-Current-URL") if cu != "" { return url.Parse(cu) } return r.URL, nil } Additional HTMX configuration Lastly, there are a few other HTMX configuration settings that I normally change from the default values. I tend to disable the HTMX cache completely by setting historyCacheSize to 0. Caching pages in local storage is a source of bugs and security issues, so I think it's simpler and better to just disable it completely. If you do this, no pages will be cached in local storage, and when the user clicks the back button HTMX will send a request to the server to refetch the HTML. It's worth noting that caching in local storage will also be disabled by default in future versions of HTMX for the same reasons. I prefer to disable HTMX attribute inheritance by setting disableInheritance to true. I think it's clearer and lowers the risk of bugs or unintended behavior when HTMX attributes are always declared explicitly. Again, it's worth noting that attribute inheritance will also be disabled by default in future versions of HTMX. I also disable HTMX indicator styles by setting includeIndicatorStyles to false. For consistency, I prefer not to have HTMX injecting styles, and would rather define any indicator styles alongside my other CSS rules. By default there is no timeout on HTMX requests, and they will wait as long as necessary for the server to respond. It's project-specific, and depends on how I'm handling timeouts and deadlines in my Go application, but sometimes I may also use the timeout setting to set a default timeout (in milliseconds) on the HTMX end. All in all, the starting point for my HTMX configuration settings normally looks like this: <meta name="htmx-config" content='{ "includeIndicatorStyles": false, "historyCacheSize": 0, "historyRestoreAsHxRequest": false, "responseHandling":[ {"code":"204", "swap": false}, {"code":"422", "swap": true}, {"code":"[45]..", "swap": true, "target": "body"}, {"code":"...", "swap": true} ], "timeout": 5000 }' > Page-specific layouts Let's finish up this post with a final note about HTML templates in larger applications. For some applications, having a base template along with page-specific templates might not be enough. You might also want 'layout' templates that sit between the base template and your page-specific content — for example, you might want to use a layout template for your admin-area pages that is different to the rest of your regular application pages. The patterns that we've talked about in this post can be extended fairly easily to accommodate this. For example, you can change the base template to insert a "layout" template in the body element instead of the page-specific content directly: {{define "base"}} <!doctype html> <html lang='en'> <head> <meta charset='utf-8'> <title>{{template "page:title" .}}</title> ... </head> <body> {{template "layout" .}} </body> </html> {{end}} Then you could create an assets/html/layouts/admin.tmpl file containing the common 'layout' markup for the admin pages: {{define "layout"}} <h1><a href="/">Admin area</a></h1> <nav> <a href="/admin/users">Users</a> <a href="/admin/orders">Orders</a> </nav> <main> {{template "page:content" .}} </main> {{end}} And then you can specify which layout template you want to use in your Go handlers as part of the call to render(). Like so: func (app *application) adminOrders(w http.ResponseWriter, r *http.Request) { err := app.html.render(w, 200, nil, "base", "layouts/admin.tmpl", "pages/admin-orders.tmpl") if err != nil { app.logger.Error(err.Error()) http.Error(w, http.StatusText(500), 500) } }
Alex Edwards Jul 14, 2026 -
Worker pools, channels, and mutexes - backed by pthreads.
Anton Zhiyanov Jul 10, 2026 -
Google Cloud just shipped the Open Knowledge Format: a directory of Markdown files with YAML frontmatter as portable knowledge for AI agents. I already run this in every repo. Here is why having an actual format is the good part.
alexop.dev Jun 27, 2026 -
A reference on conversation compaction in VS Code Copilot's agent (now in microsoft/vscode under extensions/copilot): when it triggers, the budget math, what survives, the 8-section summary format, and the config keys. Traced from the source, with interactive diagrams.
alexop.dev Jun 27, 2026 -
My Vue MAD 2026 talk, written up. What an agent actually is, and the three things every Vue project needs to get more out of AI: context, feedback loops, and discoverability. The twist is that none of it is new.
alexop.dev Jun 27, 2026 -
A strict subset of Go that translates to regular C.
Anton Zhiyanov Jun 26, 2026 -
Working, semantic, procedural, episodic. The CoALA framework splits agent memory into four kinds. Here is what each one is, and how Claude Code actually implements them on disk.
alexop.dev Jun 21, 2026 -
A hands-on tutorial: build a real, working eval harness for an AI agent in a single Bun file. Run the agent in a sandbox, grade it deterministically and with an LLM judge, vote across trials, and gate CI on the result.
alexop.dev Jun 14, 2026 -
Cloudflare runs a fleet of specialized AI reviewers across tens of thousands of merge requests. I didn't have GitLab CI or OpenCode, but I had Claude Code workflows. Here's how I rebuilt the idea as one script, and the one thing I changed.
alexop.dev Jun 8, 2026 -
Go often ships with experimental features as part of a release. These experimental features can take different forms: sometimes they're completely new packages in the standard library, sometimes they're changes to the compiler or runtime, or – very occasionally – they can be breaking changes to Go's behavior. Most of the time, the purpose of experimental features is to get real-world feedback from users before something graduates to general availability and becomes a permanent part of Go. If the feature causes regressions, or gets negative feedback from the community, it can be changed before it is finalized – or even abandoned entirely. Some examples Let's look at a few recent examples to illustrate the type of things that Go experiments can cover. Go 1.24 shipped with experimental support for a new testing/synctest package (which provides support for testing concurrent code). After feedback, the package API was adjusted slightly and it graduated to general availability in Go 1.25. Go 1.25 shipped with experimental support for a new garbage collector design with better performance. After incorporating feedback, the new garbage collector became the default in Go 1.26. Go 1.21 shipped with an experimental behavioral change to loop variable semantics. This change closed off a previously common bug with Go code, but was technically a breaking change to the language. Shipping the change as an experiment gave people a chance to test their code before the new behavior became the default in Go 1.22. Experiment lifecycle There isn’t a single fixed lifecycle for experiments, but there are some common patterns. Most experiments initially ship as off-by-default. You explicitly opt-in to try out the feature, usually by setting the GOEXPERIMENT environment value (which we'll talk about more in a moment). If things go well, one or two releases later the experimental feature is finalized, graduates to general availability, and becomes on-by-default. If an experiment affects the behavior of something, then after it graduates to general availability there is sometimes – but not always – a transitional grace period where it's possible to temporarily disable it and use the old behavior. For example, in Go 1.26 the new garbage collector design (which we briefly mentioned above) graduated to general availability and is on-by-default, but it's still possible to disable it and use the old garbage collector if you need to. So that's the most common pattern, but sometimes things take longer or work out differently. For example: Go 1.22 shipped with an experimental implementation of the compiler's inlining logic, which is still off-by-default and under evaluation more than two years later. The same release also shipped with a memory arenas experiment. After negative feedback and concerns from users, it remains off-by-default, is on indefinite hold, and may eventually be removed completely. Or finally, when the Go team is confident in a change, they might skip the feedback stage and go straight to general availability... but there may still be a transitional grace period where it's possible to disable it. A good example of this is when Go 1.24 changed its map implementation to use Swiss tables. The Go team was confident enough in the implementation and its performance benefits for this to go straight to general availability and become on-by-default, but – at least for now – it's still possible to opt out and use the old map implementation if you want to. So in practice there are really three broad experiment states: Off-by-default and under evaluation Off-by-default and on hold/dormant On-by-default with a temporary opt-out Permanent experiments Go also has a handful of experimental features that aren't really “experiments” in the normal sense. These are features that are off-by-default, but they're not under evaluation, not seeking feedback, and there's no expectation that they will ever graduate to general availability and become on-by-default. Although they are controlled by the GOEXPERIMENT environment setting in the same way as other experiments, really they are more like optional Go features that you might want to use in specialist situations. I'll refer to these as "permanent experiments" in the rest of this post. For example there is a field tracking diagnostic feature that tracks which struct fields are accessed. It's been available for a decade, and there's no intention for it to ever graduate to general availability. Or there is a static lock ranking feature, which is a diagnostic for finding potential deadlocks in the Go runtime. What experiments are available right now? It's surprisingly difficult to find out what experimental features are currently available and what their status is. Unfortunately, there isn't a page in the official Go documentation or Go Wiki that tracks experiment status, and for this post I've had to piece together the information from various places. If you want to do the same: You can get a list of all available experiments by running $ go doc goexperiment.Flags. You can figure out which experiments are on-by-default by reading the source code of src/internal/buildcfg/exp.go – specifically looking at the baseline variable declaration in the ParseGOEXPERIMENT() function. You can cross-reference the experiment names with the Go release notes and search through GitHub issues to try to figure out the current status. As far as I can tell, as of Go 1.26 here are the available permanent experiments: Experiment name Description Status FieldTrack Diagnostic to track which struct fields are accessed Off-by-default and permanent fixture StaticLockRanking Diagnostic to validate lock acquisition order to catch deadlocks Off-by-default and permanent fixture CgoCheck2 Diagnostic to check cgo pointer passing rules; too expensive to run by default Off-by-default and permanent fixture BoringCrypto Replaces Go's crypto with FIPS-validated BoringSSL; no longer relevant since Go 1.24 Off-by-default and permanent fixture but will be removed soon PreemptibleLoops Allows scheduler to preempt goroutines at loop back-edges; generally not relevant since Go 1.14, but still may be useful on platforms where preemption is otherwise unsupported Off-by-default and permanent fixture Here are the current off-by-default experiments and their status: Experiment name Description Status HeapMinimum512KiB Reduces minimum heap size from 4MB to 512KiB; may be useful for constrained environments Off-by-default and likely dormant Arenas Memory arena implementation Off-by-default and on hold following negative feedback NewInliner Rewritten compiler inliner with better call-site heuristics Off-by-default and under evaluation (available since Go 1.22) JSONv2 New encoding/json/v2 package with improved JSON encoding/decoding functions Off-by-default and under evaluation (available since Go 1.25) RuntimeSecret New runtime/secret package with functions for zeroing out memory; available on Linux amd64/arm64 only Off-by-default and under evaluation (available since Go 1.26) GoroutineLeakProfile Adds a goroutineleak pprof profile type Off-by-default and under evaluation (available since Go 1.26) SIMD New simd/archsimd package providing access to architecture-specific SIMD operations; only available on amd64 Off-by-default and under evaluation (available since Go 1.26) RuntimeFreegc Allows immediate reuse of memory without waiting for a GC cycle when safe to do so Off-by-default and under evaluation (available since Go 1.26, but see #74299 for status information) SizeSpecializedMalloc Enables malloc implementations that are specialized per size class Off-by-default and under evaluation (available since Go 1.26, but see #74299 for status information) And here are the currently on-by-default experiments: Experiment name Description Status LoopVar Per-iteration loop variable scoping On-by-default since Go 1.22, but opt-out kept for edge cases Dwarf5 DWARF 5 debug info generation; reduces binary size On-by-default with a temporary opt-out (opt-out may be removed in a future release) RandomizedHeapBase64 Randomizes the heap base address at startup as a security measure On-by-default with a temporary opt-out (opt-out expected to be removed in a future release) GreenTeaGC New garbage collector with improved performance; unavailable on darwin/ios/aix On-by-default with a temporary opt-out (opt-out expected to be removed in Go 1.27) RegabiWrappers ABI wrappers for calling between ABI0 and ABIInternal functions; only available on 64-bit architectures On-by-default with a temporary opt-out, but opt-out is effective for s390x only, and will be removed in Go 1.27 RegabiArgs Enables register arguments/results in all compiled Go functions; only available on 64-bit architectures On-by-default with a temporary opt-out, but opt-out is effective for s390x only, and will be removed in Go 1.27 How do you enable and disable experiments? Experiments are controlled using the GOEXPERIMENT environment setting. If there are some off-by-default experiments you want to try, you should include the experiment names as comma-separated lowercase values in GOEXPERIMENT. For example, if you wanted to build your application with the JSONv2 and GoroutineLeakProfile experiments enabled, you would do so like this: $ GOEXPERIMENT=jsonv2,goroutineleakprofile go build ./... If there is an on-by-default experiment that you want to turn off, you do so by prefixing the lowercase experiment name with no. For example, if you want to build your application with the GreenTeaGC and RandomizedHeapBase64 experiments turned off, you would do so like this: $ GOEXPERIMENT=nogreenteagc,norandomizedheapbase64 go build ./... It's totally fine to mix enabled and disabled experiments: $ GOEXPERIMENT=jsonv2,nogreenteagc go build ./... Note that if you build the same package with different GOEXPERIMENT values, Go treats them as different builds and stores separate entries in the build cache. I've used go build in the examples above, but you can use exactly the same pattern when using go run or go test too. If you want to try it yourself, try creating the following program which uses the experimental encoding/json/v2 package: package main import ( "encoding/json/v2" "fmt" ) type Person struct { Name string `json:"name"` Age int `json:"age"` City string `json:"city"` } func main() { p := Person{Name: "Ada", Age: 36, City: "Vienna"} data, _ := json.Marshal(p, json.StringifyNumbers(true)) fmt.Println(string(data)) } If you run this normally, the program won't compile and you'll get an error message similar to this: $ go run main.go package command-line-arguments imports encoding/json/v2: build constraints exclude all Go files in /usr/local/go/src/encoding/json/v2 But if you enable the JSONv2 experiment, the program will run as expected: $ GOEXPERIMENT=jsonv2 go run main.go {"name":"Ada","age":"36","city":"Vienna"} Which experiments should you actually care about? If you're a run-of-the-mill Gopher like me, who mainly uses Go to write programs rather than working on Go itself, most of the available experiments probably won't be very relevant to you. The most interesting and relevant ones probably are: GreenTeaGC – If you're using Go 1.26, you're already using this by default. But if you notice any performance or behavior problems, it's worth being aware that you can still disable it (and you should also file an issue). Dwarf5 – Again, if you're using Go 1.25 or later then you're already using this by default. But if you run into any problems, it's useful to know that you can still disable it. JSONv2 – I don't recommend switching to this until it graduates to general availability, but if you write a lot of code that deals with JSON, it's worth experimenting with the new encoding/json/v2 package, familiarizing yourself with what's coming, and giving feedback if you notice any problems. GoroutineLeakProfile – This one is immediately useful and worth enabling if you suspect you have a goroutine leak and need to debug it. RuntimeSecret – Worth experimenting with and giving feedback on if you write cryptographic code or need to handle sensitive data. RuntimeFreegc – If you have an application that leans heavily on the garbage collector, it may be worth benchmarking your code with this enabled to see if it improves performance, and giving feedback if you notice any issues. Finally, it's worth emphasizing that experimental features are not covered by the Go compatibility promise. Their APIs, behavior, and performance characteristics may all change, so it's generally a good idea to avoid adopting too early and depending on experimental features before they are finalized. But experimental features often act as a preview to some of the biggest changes in Go. If you know that an experiment is likely to affect you or your code once it eventually becomes generally available and on-by-default, it's a good idea to try it out, run benchmarks where appropriate, and give feedback if you find issues. If you want to keep track of what experiments are available and their status, the Go release notes have recently started doing a much better job of documenting experimental features and how to use them. Between this blog post and browsing the release notes when there's a new Go release, you should have a decent idea of what's going on.
Alex Edwards Jun 1, 2026 -
I was too scared to publish my first blog post and ended up on stage at Vue Amsterdam. Here's the honest path from zero to international speaker, visibility first.
alexop.dev May 30, 2026 -
Workflows let you script how Claude Code fans out across dozens of subagents, then synthesizes the results. Here's how they work, the primitives, and the small example I built to understand them.
alexop.dev May 28, 2026 -
Introducing the new programmatic API for pkg.go.dev, allowing developers to fetch package and module data directly.
The Go Blog May 21, 2026 -
A pipeline for shipping a 5-point ticket while you're away from the keyboard: spec with the human in the loop, slice into vertical tickets, run a Ralph loop with TDD per slice, refactor, then let an agent-browser do QA.
alexop.dev May 10, 2026 -
A strict subset of Go that translates to regular C.
Anton Zhiyanov May 6, 2026 -
Most components break under their second or third variant — props pile up, shells get copy-pasted. Here's how compound components (the pattern behind Reka UI and shadcn-vue) keep a Dialog flexible without flag soup.
alexop.dev Apr 29, 2026 -
A short, framework-agnostic concept of what a modern quality pipeline and testing strategy look like for any JavaScript or TypeScript frontend project.
alexop.dev Apr 25, 2026 -
A thin Bun runner that hands a coding agent one charter and lets it drive a real browser through an exploratory QA session. Works with Claude, Codex, or Copilot, and any browser CLI.
alexop.dev Apr 19, 2026 -
In which I have some fun with Claude Opus 4.6, a seemingly indecisive AI coding agent.
Ben Hoyt's technical writing Apr 12, 2026 -
Claude Code and Agent Browser let you test your web app in a real browser without hardcoded selectors. Manual browser control, AI-driven exploration, and structured JSON output.
alexop.dev Apr 11, 2026 -
A Playwright fixture that listens to the browser console for hydration mismatch warnings and fails your E2E tests when server and client HTML disagree.
alexop.dev Apr 6, 2026 -
With allocators, benchmarks, and some optimizations.
Anton Zhiyanov Apr 3, 2026 -
Dependencies are a huge supply chain security risk; the more of them you have, and the more often you update, the bigger the attack surface.
Ben Hoyt's technical writing Apr 2, 2026 -
Automate annotated screenshot documentation for any web app or live website using an AI coding agent skill. Works with Claude Code, Cursor, Windsurf, or any agent that supports custom skills.
alexop.dev Mar 27, 2026 -
Interfaces, slices, multi-returns and alloca.
Anton Zhiyanov Mar 25, 2026 -
Choosing the right names in your codebase is an important (and sometimes difficult!) part of programming in Go. It's a small thing that makes a big difference — good names make your code clearer, more predictable, and easier to navigate; bad names do the opposite. Go has fairly strong conventions — and a few hard rules — for naming things. In this post we're going to explain these rules and conventions, provide some practical tips, and demonstrate some examples of good and bad names in Go. If you're new to the language, all this information might feel like a lot to take in, but it'll quickly become second nature with a bit of practice 😊 Identifiers Let's start with the hard rules for identifiers. By identifiers, I mean the names that you use for the variables, constants, types, functions, parameters, struct fields, methods and receivers in your code. Identifiers can contain unicode letters, digits, and underscores only. Identifiers cannot begin with a digit. You cannot use any of the following Go keywords as identifiers: break default func interface select case defer go map struct chan else goto package switch const fallthrough if range type continue for import return var So long as you stick to those three rules, any identifier name is technically valid and your code will compile all OK. But there are a bunch of other guidelines that it's good practice to follow: You should use camelCase for unexported identifiers, or PascalCase for exported identifiers. Don't use alternative casing variants like snake_case, Pascal_Snake_Case, SCREAMING_SNAKE_CASE or ALLUPPERCASE. Words that are acronyms or initialisms (like API, URL or HTTP) should use a consistent case within the identifier. So, for example, apiKey or APIKey is good, but ApiKey is not. This rule also applies to ID when it is used as shorthand for the words "identity" or "identifier" — so that means write userID rather than userId. Although all unicode letters are allowed, using non-ASCII letters can often make your code harder to read and more awkward to write, and it's rare to see them used. Unless you have a really appropriate use-case, you should stick to using ASCII letters in identifiers. For example, use pi instead of π, use beta instead of β, use naiveBayes instead of naïveBayes. To prevent confusion for readers and potential bugs, avoid choosing identifiers that clash with Go's builtin types. So, for example, don't create variables with names like int, bool or any. Similarly, avoid creating functions with names that clash with Go's builtin functions. So, for example, don't create functions with names like min, max, len or clear. Generally, avoid including the type in identifiers — for example, don't use names like fullNameString, scoreInt or float64Amount. The main exception to this is when you have to convert a variable to a different type, and you want to distinguish between the original variable and the one containing the converted value. In this situation, including the type in the identifier is a common and acceptable way to distinguish between the two. For example, code like this is OK: userID := 42 userIDStr := strconv.Itoa(userID) Where possible, try to avoid choosing identifiers that clash with the standard library package names. This is a 'softer' convention than the others because the standard library steals a lot of good identifier names — such as json, js, mail, user, csv, path, filepath, log, regexp, time and url — and sometimes it can be hard to come up with decent alternatives. However, you definitely should avoid creating identifiers that clash with the package names that your code is actually importing and using. So, for example, if you are writing code that imports the url and net/mail packages, then don't use the words url and mail as identifiers in that code. Here are a few examples of good and bad identifier names: Bad Reason Better order.total := 99.99func load-user() Punctuation not allowed orderTotal := 99.99func loadUser() const 3rdParty = "x"func 2FactorAuth() Cannot start with a digit const thirdParty = "x"func twoFactorAuth() max_value := 10func Fetch_user() Non-standard casing maxValue := 10func FetchUser() type HttpClient struct{}func parseXml() Inconsistent acronym casing type HTTPClient struct{}func parseXML() func GetSessionId()type OrderId string ID should be all caps func GetSessionID()type OrderID string résuméCount := 2const Σ = 100 Non-ASCII letters resumeCount := 2const sum = 100 func clear()int := cache.Internal() Clashes with builtin types or functions func clearQueue()data := cache.Internal() intCount := 42resultSlice := []int{} Type included in name count := 42results := []int{} type json struct{}var log = newLogger() Clashes with stdlib package names type payload struct{}var logger = newLogger() Exported and unexported identifiers Identifiers in Go are case-sensitive. For example, the identifiers apiKey, apikey and APIKey are all different. As you probably already know, when an identifier starts with a capital letter it is exported — that is, it's visible to code outside of the package it's declared in. This means that the casing of the first letter is significant. It impacts the behavior of your codebase. In turn, this means that you shouldn't start identifiers with a capital letter just because they look nice — you should only start them with a capital letter if you want them to be exported and accessible to code outside the package they are declared in. As a tip, try to write packages using unexported identifiers by default. Only export them when you actually have a need to. Typically, the less you export, the easier it is to refactor code within a package without affecting other parts of your codebase. There's a nice quote from The Pragmatic Programmer, which I'll adapt slightly for the Go nomenclature: Write shy code - packages that don't reveal anything unnecessary to other packages and don't rely on other packages' implementations. As a second tip, it's very rare for a main package to be imported by anything, so the identifiers in it should normally all be unexported and start with a lowercase letter. The most frequent exception to this is when you need to export a struct field so that it's visible to packages that use reflection to work, like encoding/json, encoding/gob or github.com/jmoiron/sqlx. Identifier length and descriptiveness In general, the further away that an identifier is used from where it is declared, the more descriptive the name should be. If you have an identifier which is narrow in scope and only used close to where it is declared, it's generally OK to use a short and not-very-descriptive name. For example, if you're naming something that is only used in a small for loop, range block, or very short function, then using short or even single letter names is very common in Go. But it you're naming something that has a larger scope, or is used far away from where it is declared, you should use a name that clearly describes what the thing represents. Here is a nice example that Dave Cheney gave as part of his Practical Go presentation: type Person struct { Name string Age int } func AverageAge(people []Person) int { if len(people) == 0 { return 0 } var count, sum int for _, p := range people { sum += p.Age count += 1 } return sum / count } In this code, within the short range block we use the identifier p to represent a value in the people slice — the range block is so small and tight that using a single letter name is clear enough. In contrast, the count and sum variables are declared, then used inside the range, then again in the return statement. Giving them more descriptive names makes it immediately clearer what the code is doing and what they represent, compared to single letter names like c and s. But these variables are only ever used inside the AverageAge function, so giving them even-more-descriptive names like peopleCount and agesSum would be unnecessarily verbose. It's not an exact science, but when writing Go code you are encouraged to use the right length identifier — sometimes that might be long and descriptive, sometimes it might be short and terse. Naming packages The hard rules for package names are the same as for identifiers: they can contain unicode letters, numbers and underscores, must not begin with a number, and must not be a Go keyword. But in practice, the conventions for naming a package are much tighter. Conventionally: Package names should contain lower case ASCII letters and numbers only. Because package names will need to be typed out a lot when writing code, the name should ideally be short, easy to type, and reflect the contents of the package. Often simple one-word nouns (like orders, customer and slug) work well. If you want to use more than one word in the package name, you should concatenate the words all in lowercase with no separator. So, for example, ordermanager is a good package name — orderManager or order_manager are not. If a package name feels too long, it can be OK to use abbreviations in the name. You can see this in some of the standard library package names, like expvar (instead of exportedvariables) and strconv (instead of stringconversion). To prevent conflicts and confusion, try to avoid using the same name as commonly-used standard library packages. Package names with the prefix . or _ are 'invisible' to Go and completely ignored when you run go build, go run, go test etc. So don't start your package name with these characters, unless you specifically want them to be ignored. Directories with the names vendor, testdata and internal have a special meaning in Go, so to avoid any confusion or bugs, don't use these words as package names. Avoid using 'catch all' package names like common, util, helpers, types or interfaces, which don't really give any clue to what the package contains. For example, does a package called helpers contain validation helpers, formatting helpers, SQL helpers? A mix of all the above? You can't guess from just the name alone. As well as not being clear, these kind of 'catch all' names provide little natural boundary or scope, which can lead to the package becoming a dumping ground for lots of different things. In turn the package may become imported and used throughout your codebase — which increases the risk of import cycles and means that changes to the package potentially affect the whole codebase, rather than just a specific part of it. In other words, catch all package names encourage creating packages which have a large 'blast radius'. If you find yourself wanting to create a utils or helpers package, ask yourself if you can break up the contents into smaller packages with a specific focus and clearer names instead. Bad Reason Better package 3rdpartypackage 2fa Cannot start with a digit package thirdpartypackage twofa package OrderManagerpackage order_manager Non-standard casing / separators package ordermanager package opackage stuff Too vague and not descriptive package orderspackage slug package ordermanagementsystem Too long / hard to type package orderspackage ordermgr package urlpackage mail Clashes with stdlib package names package linkspackage mailer package _cachepackage .hidden Ignored by Go tooling package cachepackage hidden package internalpackage vendorpackage testdata Special directory names in Go package internalauthpackage supplier package utilspackage helpers Catch-all names with unclear scope package validationpackage formatting Naming files In an ideal world, a .go filename should summarize what the file contains, be one word long, and all in lowercase. Some examples of good filenames from the standard library net/http package are cookie.go, server.go and status.go. If you can't think of a good one-word name, and want to use two or more words, there is no clear convention for how those words should be separated. Even in the Go standard library itself there isn't consistency. Sometimes underscores are used to separate the words in filenames (like routing_index.go and routing_tree.go), and other times they are concatenated with nothing between them (like batchcursor.go, textreader.go and reverseproxy.go). Because there isn't a strong convention around this, I recommend just picking one of these two approaches and sticking to it consistently within a codebase. Personally, I think it's better to concatenate words with nothing between them (like routingindex.go), and reserve the underscore character for only when you want to use a special filename suffix. Talking of which, there are some filename prefixes and suffixes that have a special meaning in Go. You should avoid using these in your filenames unless you want to trigger the special behavior. Specifically: Like packages, filenames with the prefix . or _ are 'invisible' to the Go tooling and completely ignored when you run go build, go run, go test etc. Files with the suffix _test.go are only run by the go test tool. They are ignored when using go run or go build. Files with any of the following suffixes will only be included when compiling for that specific operating system: _aix.go, _android.go, _darwin.go, _dragonfly.go, _freebsd.go, _illumos.go, _ios.go, _js.go, _linux.go, _netbsd.go, _openbsd.go, _plan9.go, _solaris.go, _wasip1.go, _windows.go. Similarly, files with any of the following suffixes will only be included when compiling for that specific architecture: _386.go, _amd64.go, _arm.go, _arm64.go, _loong64.go, _mips.go, _mips64.go, _mips64le.go, _mipsle.go, _ppc64.go, _ppc64le.go, _riscv64.go, _s390x.go, _wasm.go. Avoiding chatter When you are naming exported functions, try to avoid repeating the name of the package they are declared in. For example, if you have a package called customer, then function names like NewCustomer() or CustomerOrders() would be 'chattery' and unnecessarily repeat the word 'customer' when you call them from outside the package — like customer.NewCustomer() and customer.CustomerOrders(). Calling the functions New() and Orders() is sufficient and reads better at the call site — like customer.New() and customer.Orders(). The same advice also applies to exported types. For example, if you want to represent an address or phone number in a customer package, it's sufficient and less chattery to name the types Address and PhoneNumber rather than CustomerAddress and CustomerPhoneNumber. Bad Reason Better customer.NewCustomer()customer.CustomerOrders() Chattery function call customer.New()customer.Orders() customer.CustomerAddresscustomer.CustomerPhoneNumber Chattery type reference customer.Addresscustomer.PhoneNumber Note: It's common to want to declare an exported type that shares the same name as the package. For example, a customer package might export a Customer type that represents an individual customer. We would then reference this type in other packages by writing customers.Customer. This is obviously chattery, but it's hard to avoid this repetition without giving either the package or type a name that makes it less clear. So in practice, this is something that you'll see a lot. For example, in the standard library the time package has a Time type, which you reference by typing time.Time, the context package contains a Context type which you reference by typing context.Context, and the regexp package contains a Regexp type which you reference by typing regexp.Regexp. Similar to function and type names, method names should ideally not 'chatter' too much when calling them. For example, if you are writing methods on a Token type, for example, it's probably OK to call a method Validate() rather than ValidateToken(), or IsExpired() rather than IsTokenExpired(). Method receivers When you are creating methods, it is conventional for the method receiver to have a short name, normally between 1 and 3 characters long and often an abbreviation of the type that the method is implemented on. For example, if you are implementing a method on a Customer type, an idiomatic receiver name would be something like c or cus. Or if you were implementing a method on a HighScore type, a good receiver name would be hs. The Go code review comments advise against using generic names like this, self or me for the receiver. Also, you should be consistent with the receiver name. All methods on the same type should use the same receiver name — don't use c for one method and cus for another. type Order struct { Items int } // Good: uses a short receiver func (o *Order) Validate() bool { return o.Items > 0 } // Bad: uses a longer receiver name func (order *Order) Validate() bool { return order.Items > 0 } // Bad: uses a generic receiver name func (self *Order) Validate() bool { return self.Items > 0 } Getter and setter methods on structs Typically, it is not necessary to create 'getter' and 'setter' methods on struct types in Go. Instead, you just access the struct field directly to read or change the data. The major exception to this is when you have a struct with an unexported field, but want to provide a way to get or set the field value from outside the package. To do this, you need to create exported 'getter' and 'setter' methods that read and write to the unexported field. When doing this, it is conventional to prefix the setter method name with Set, but not prefix the getter method name with Get. Like so: type Customer struct { address string } func (c *Customer) Address() string { return c.address } func (c *Customer) SetAddress(addr string) { c.address = addr } Interfaces By convention, interfaces that only contain one method should be named by the method name plus an '-er' suffix or similar. For example: type Speaker interface { Speak() string } type Authorizer interface { Authorize(ctx context.Context, action string) error } type Authenticator interface { Authenticate(ctx context.Context) (User, error) } The Go standard library has quite a few examples of interfaces that follow this convention, such as io.Reader, io.Writer and fmt.Stringer. Also note that the guidance to avoid including the type in the name still applies to interfaces. Don't give your interfaces names like UserInterface or OrderInterface unless you really can't think of a decent alternative. Breaking from conventions There are rare occasions when breaking a convention can actually make your code clearer — and in my view, it can be OK to do that... especially if it's in a private codebase worked on by a small team. For example, a couple of years ago I was working on a Go program that synchronizes data between some other external systems. In this project, I ended up breaking some of the Go naming conventions around casing and separators — instead using exactly the same identifiers that the external systems used. This actually made the intent of the program clearer, and more immediately obvious what was being synchronized with what. But the vast majority of the time, you should endeavour to follow the naming rules and conventions we've discussed in this post. They exist for good reasons: they make your code more predictable and consistent, easier for other Gophers to quickly understand, and reduce the risk of certain bugs.
Alex Edwards Mar 24, 2026 -
Go 1.26 simplifies type construction and enhances cycle detection for certain kinds of recursive types.
The Go Blog Mar 24, 2026 -
We already have everything we need to build software factories. Teams will change. The only variable is speed.
alexop.dev Mar 22, 2026 -
How Go 1.26's source-level inliner works, and how it can help you with self-service API migrations.
The Go Blog Mar 10, 2026 -
Build a real-time, offline-capable todo app with Jazz and Vue 3. Learn CoValues, drag-and-drop with fractional indexing, and shareable URLs. No backend needed.
alexop.dev Mar 7, 2026 -
A description of some of the recent changes to do allocations on the stack instead of the heap.
The Go Blog Feb 27, 2026 -
Go 1.26 includes a new implementation of go fix that can help you use more modern features of Go.
The Go Blog Feb 17, 2026 -
Replace Claude Code's default spinner verbs with custom One Piece references using the spinnerVerbs config in settings.json.
alexop.dev Feb 12, 2026 -
Claude Code hooks let you run shell commands on lifecycle events. I wired up Age of Empires sound effects to session start, prompt submission, task completion, and context compaction.
alexop.dev Feb 11, 2026 -
Go 1.26 adds a new garbage collector, cgo overhead reduction, experimental simd/archsimd package, experimental runtime/secret package, and more.
The Go Blog Feb 10, 2026 -
Agent teams let multiple Claude Code sessions coordinate, communicate, and self-organize. Here's how they work, when to use them, and what they cost.
alexop.dev Feb 8, 2026 -
How I built a skill that generates interactive codebase walkthroughs with clickable Mermaid diagrams—works with Claude Code, Amp, and any agent that supports the skills standard.
alexop.dev Feb 8, 2026 -
Learn how to add animated hand-drawn annotations to your presentations using VMark. A complete tutorial covering all annotation types, colors, and timing controls.
alexop.dev Feb 3, 2026 -
A practical workflow for tackling large refactors with Claude Code using parallel research subagents, written specs, and the new task system for context-efficient implementation.
alexop.dev Feb 1, 2026 -
A battle-tested linting configuration that catches real bugs, enforces clean architecture, and runs fast using Oxlint and ESLint together.
alexop.dev Jan 31, 2026 -
Quick takeaways Start with a monolith - don’t create microservices from day one; the overhead isn’t worth it until you have real pain points Microservices solve human problems, not just technical ones - they help teams work independently, not just scale systems Tight coupling over HTTP is still tight coupling - separating services doesn’t automatically give you isolation Watch for signals - lots of calls between services or frequent changes across multiple services suggest wrong boundaries Sometimes joining services is the answer - merging tightly coupled microservices back together can be the right move Introduction We talk about splitting software: when to do it, how to avoid common pitfalls, and why microservices aren’t always the answer. We share stories from past projects where we jumped on the microservices hype train without thinking through the real problems we were trying to solve. The key insight is that splitting software is more about organizing teams and maintaining good boundaries than about technology. Notes The Domain Engineer training - includes a complex modular monolith example Modular monolith approach - starting with a well-structured monolith that can be split later if needed Event Storming - a technique for discovering system boundaries by mapping domain events Conway’s Law - the idea that systems mirror organizational structure Team Topologies book by Matthew Skelton and Manuel Pais - recommended for understanding how team structure affects software design Reverse Conway Maneuver - designing organizational structure first, then letting software follow Service mesh tooling (like Istio) - for observability between microservices Quotes This aspect of microservices I think can be also misleading. Because it seems like you have these separate things. You see the deployments, you see the different repositories. For an outsider, you can think they are separate beings that can be changed independently of each other. It can be a trap. Miłosz You can say that microservices are maybe solving a bit of technical problems, but it’s mostly solving the human problem. Robert So if you consider splitting your project, I would start with this question: if you even need to do it. Because you often see people struggling or hear them asking, I’m starting a new project, kicking off some application, and what services do I need to create? So probably at this stage, you don’t need to consider this at all. Miłosz I think it’s also nicely visible if you take your service mesh tooling, and probably will see that between some microservices will have a lot of calls going back and forth. And it’s usually the sign that maybe those two services shouldn’t be separate. Robert The more developers you add to the project, the more complicated it gets. Because this is also important factor. If you are a single person working on software like this, it will be fine probably because you can keep the context in your head. You will refactor to whatever you want. But if you have like 20 people working on it, it starts getting more complicated. Miłosz But there’s one good thing about microservices is that it’s forcing the boundaries really hard. So it’s hard to go over network if it’s not possible. Robert Timestamps 00:00:00 - Introduction 00:01:37 - AI and splitting software 00:02:42 - When splitting matters 00:04:39 - Monolith first approach 00:06:08 - Microservices hype era 00:09:34 - Microservices force boundaries 00:13:16 - The isolation illusion 00:18:31 - Merging services back 00:23:09 - Event storming for boundaries 00:27:41 - Core vs generic domains 00:32:05 - Bounded contexts and shared kernel 00:35:30 - Conway's Law and Team Topologies Transcript Miłosz [00:00:00]: So, if you consider splitting your project, I would start with this question: if you even need to do it, because you often see people struggling or hear them say, ‘How do I start?’ I’m starting a new project, kicking off some application. What services do I need to create? This aspect of microservices I think can be also misleading. Because it seems like you have these separate things. You see the deployments, you see the different repositories. For an outsider, you can think they are separate. Entities separate beings that can be changed independently of each other. It can be a trap. You have to think: what problem do you want to solve? This way, rather than just ‘Oh yeah industrial best practice. Robert [00:00:41]: Probably, can say that microservices are maybe solving a bit of technical problems, but it’s mostly solving the human problem. Probably, you’re mentioning the monolith first approach that we are big advocates. We’ve tried it multiple times for products that become bigger later. If done properly, it’s working. It’s accelerating you a lot at the beginning and allows you to change it later. Miłosz [00:01:05]: And what’s surprising to me, it’s not as popular. Miłosz [00:01:11]: So, welcome after break. Let’s talk about splitting software, different ways of splitting software, and why it’s important in the first place. Robert [00:01:20]: Yeah, I think it’s a good topic because, probably, it’s the topic that everybody writing software in this or another way needs to know how to do that. And if it’s not done properly, it may create a lot of issues. Miłosz [00:01:37]: Probably even if we end up doing coding with AI in the next couple of years, it will be an important topic because it’s like one of the key architectural decisions you can make in your project. Robert [00:01:50]: Especially, from our experience, even if people were doing that, usually they were missing that. And I would not expect that some cloud code or something like that. Miłosz [00:02:00]: Handle it better than probably the opposite, actually, because there are so many anti-patterns on the internet. You know, like people tend to make mistakes when splitting software or do it in a very naive way. So yeah, I would expect coding agents to probably make similar mistakes. Robert [00:02:20]: I think it’s you know not only about some inexperienced people—I think also sometimes very experienced people, without experience, may miss some signals from the systems. So when you’re building some system, it’s sometimes hard to miss some signal when something is going in the wrong direction. And it happens even for those experienced people. Miłosz [00:02:42]: So it’s often the issue with big codebases, right? When we work with a big project, over time tends to be more difficult to work with, especially with bigger teams or multiple teams. And then the topic of splitting comes up. And I was thinking about, you know, our past projects on different companies, how important the problem is. And one thing I realized is that when we were working on our own projects before our full-time jobs, the stopping wasn’t really that important. Important so it tells me something right, that’s even if those applications were sometimes fairly complex, wasn’t. If it’s just you and maybe another person working on some project, it might still be fine sticking to one repository or codebase. Robert [00:03:45]: Yeah, so probably can explain why, for example, people like Peter levels can have one PHP file that has 10,000 lines of code and he can work. Robert [00:03:55]: But if you are two, also probably. But if you have people of five, ten, twenty, fifty people, it no longer works. Robert [00:04:05]: When some application is done as proof of concept, later it evolves to some product that is making money. Sometimes it’s missing this. Transition phase from the application that was built as proof of concept by a team of five— even if okay, if you have team of five, it’s still not a big issue to handle, even very bad code, and be able to extend that. But when the team is growing, the product is growing, the number of features is growing, well, we start to have a problem. And the question is, what you do about that. Miłosz [00:04:39]: Yeah, so if you consider splitting your project, I would start with this question: if you even need to do it. Because you often see people struggling or hear them asking, ‘How do I start I’m starting a new project, kicking off some application, and what services do I need to create? So we are probably at this stage, you don’t need to consider this at all. Right, because it’s not not the pain point, and the overhead may be big enough that it will not pay you back. Robert [00:05:14]: So you probably are mentioning the monolith first approach that we are big advocates of. We tried it multiple times for products that become bigger later, and if done properly, it’s accelerating you a lot at the beginning and allows you to change it later. Miłosz [00:05:32]: And what’s surprising to me it’s not that popular, or maybe now it’s probably a bit more popular than it was. I i see it mentioned here and there, but back back when we first tried it, I think no one talked about it that much. Robert [00:05:45]: Yeah, but I think it’s probably still some small percent. And it’s maybe also a bit because of our bubbles. Yeah, maybe. But it’s still not. Miłosz [00:05:55]: It’s not like a crazy different idea from creating microservices. Robert [00:06:00]: But if you ask a random developer, what do you hear about? Kubernetes or modular monolith? I’m pretty sure I know the statistics. Miłosz [00:06:08]: Yeah, the hype is a funny factor here. So I remember in one project—when I joined— we had one huge codebase. Miłosz [00:06:20]: Typically, that’s the software no one wants to work with, and everyone would like to see refactored. And it was also around the time of the microservices hype era. So somehow we made a decision together as a team that we will split the monolith, as probably everyone back then did. Robert [00:06:40]: Yeah, I remember how everybody was optimistic because it was like, ‘whoa,’ microservices. Finally, a solution for all of our problems. Miłosz [00:06:49]: And when I think about it now, it’s funny how we had all these issues with this project and we assumed, ‘Okay, microservices will solve this for us I mean, the project was better in the end for sure, and I was responsible for the design of the framework and all the networking and stuff. So, as a programmer, it was very fun. It was great. Probably we followed the Robert [00:07:16]: ‘fun’ principle too often. Miłosz [00:07:18]: Yeah. But looking back at it, did it need all this complexity and distributed systems stuff? Probably not. Probably some parts could be simplified. Miłosz [00:07:31]: But I think we just jumped on the hype train back then. Robert [00:07:37]: But I think a good lesson from that was that you, as a team, saw some signal that ‘okay, there is something wrong with this project,’ probably it was like, ‘okay, we cannot maintain it, we cannot add new features easily.’ Robert [00:07:51]: So the problem was visible there. Miłosz [00:07:54]: Right. So the question is whether splitting software is a good approach to fix these kind of issues. Robert [00:08:02]: But do you think that maybe what would work better there was maybe a bit more balanced approach— so maybe a bit less microservices or staying as a monolith— was better. Miłosz [00:08:16]: Maybe less microservices or modular monoliths for the start. Because the issue for sure was that the boundaries were wrong in the initial software, so there was, you know, some function calls all over the place from one place to another with no control. The more developers you add to the project, the more complicated it gets. Because this is also important factor. If you are a single person working on software like this, it will be fine probably because you can keep the context in your head. You will refactor to whatever you want. But if you have like 20 people working on it, it starts getting more complicated. Robert [00:08:57]: But still, I think it’s still manageable with modular monoliths. Miłosz [00:09:01]: Yeah, for sure. Yeah, but the issue is if you don’t have these good boundaries, you don’t know how to progress. Everyone tries to do something and it ends up as this big ball of mud. Robert [00:09:14]: So in other words, you are modifying something in one place and everything around it goes up and you have no idea why. It should have zero dependency, but it seems that it does. Miłosz [00:09:27]: Yeah, it’s risky and you don’t like making changes there. Robert [00:09:30]: So the big bow of math or spaghetti code by the right, right. Miłosz [00:09:34]: So if you have these good boundaries, as microservices, promised everything will be fine because you make these changes in one small Please. Miłosz [00:09:45]: There’s low impact, you can deploy it independently from the other services, everyone’s happy, which makes sense, but you can’t do the same thing without microservices. Robert [00:09:56]: But there’s one good thing about microservices is that it’s forcing the boundaries really hard. So it’s hard to go over network if it’s not possible. I mean, when you have a big ball of mud, you can call everything if you don’t have proper encapsulation. Okay, you can do proper encapsulation, but it requires some kind of regime of hardware. Or tooling, at least. Yes, yes. And with microservices, as long as you will not get exposed public endpoint, it’s not. So it’s probably showing why sometimes microservices may help. And have the same outcome with more cost, obviously. But again, if you go too far, it’s much harder to undo that because, for example, if you were wrong with modules in modular monolith, you can copy from one catalog to another, join everything. We have one module with microservices. You need to change your Kubernetes configs. You need to change all dependencies in other team services. Robert [00:11:02]: It’s a mess, right? Miłosz [00:11:03]: So maybe it’s easier to see the microservices as well because you can show someone, ‘Yeah, this is our Kubernetes deployments. They can see, okay, yeah.’ So this is your, those are your modules or microservices. The monolith is more complex because you need some kind of structure. It’s still possible, and we did that, but maybe not as easy to see. Robert [00:11:28]: And I think there is also one thing that microservices may make easier out of the box. So this is all sort of observability, because you can put some Istio or whatever between those services and you have it kind of out of the box again, not for free. Yeah, with much less flexibility, because Again, it’s just put in the place by some cloud ops, DevOps team, whatever. But probably the flexibility is a lot limited. Right. From other side. You can achieve the same thing with your module monolith, but it requires some effort because you need to write it in code and you need to maintain that. other side, for example, I like this approach because it’s giving you a lot of flexibility. And it’s also removing some external dependency on the team that, in my opinion, should be more about running some transparent infrastructure rather than adding something like that. But from other side, it also depends because, for example, for some big enterprises, super big products, it kind of makes sense because you can have it out of the box and have it unified everywhere. Robert [00:12:36]: So again, it depends on what kind of company we’re talking about. Miłosz [00:12:40]: Right. So microservices are kind of more tangible. You have better observability. You can see them. And modules are more abstract. You have to more understand the project to see them. Yeah. Robert [00:12:54]: But yeah, trade-offs. We are mentioning that to show that there is not only one way, because I think now the way with doing it with Kubernetes, with some service mesh, it can be a way. Definitely for some bigger companies. But again, if you’re working on a startup, maybe not— maybe you can do it by hand and it will be much more flexible, much more expensive. Miłosz [00:13:16]: But this aspect of microservices, I think, can be also misleading because it seems like you have these separate things, right? You see the deployments, you see the— you know. Miłosz [00:13:30]: different repositories. From an outsider, you can think they are separate entities, separate beings that can be changed independently of each other, but it can be a trap. I remember one project where, when we joined, they were described as independent and isolated from each other. So you can change one thing, it doesn’t impact the others. But it turned out the kind of isolation layer was HTTP API between them. So, on the outside, it looks like nice, clean separation. But in reality, they are still very tightly coupled. They just communicate over the network instead of function calls. I remember this confusion we had and trying to explain it to the management as well. Why this nicely separated system is suddenly a problem? Miłosz [00:14:34]: Because from the outside, it looks like you have those tangible, separate microservices. The reality is much different. Miłosz [00:14:43]: That’s also one of the hard parts: how to know. If the isolation is good enough. Robert [00:14:51]: Yeah, and I think it’s often outcome of, let’s say, simplistic approach to splitting those services. So I would probably see two cases. One. We’re getting database tables and we have separate service for each database table. Hooray, microservices. Miłosz [00:15:07]: I often see this when someone asks us, on Discord or whatever, for some feedback on a system they did. And very often it’s something like, ‘So I have this application. And I have users microservice and orders microservice. And at this point, I know this won’t be easy.’ Miłosz [00:15:31]: For most e-commerce systems, you can just separate users out of the orders, or it will be, you know, there will be some tight coupling probably between them anyway, as long as you’re not building some extremely big yeah, exactly. So it depends, but you know, if you take a fairly simple system, and you have those three main entities, and you think each entity should have their own microservice. Then you will probably end up with many problems because just separating those services won’t give you isolation. It will just give you this illusion of isolation. Robert [00:16:13]: Yeah, and I think it’s also nicely visible if you take your service mesh tooling, and probably will see that between some microservices will have a lot of calls going back and forth. Robert [00:16:26]: And it’s usually the sign that maybe those two services shouldn’t be separate. For example, those users and orders, those orders. So you can see that. Robert [00:16:40]: To do an update to order or to do some shipment stuff or whatever, you need to call users many, many, many times. Robert [00:16:50]: Can see that if, for example, you would need to add some field to, probably it can be customers, maybe not users in this case. So let’s say orders and customers. Commerce. And let’s imagine that orders is handing the shipment or something like that. And you can see the Skype link nicely when you have some new. For example, you need to add to user something that needs to be added to shipment. And in worst case, if it’s handled by two teams, a project that could be done in one day with modular monolith is starting to take one month because you need to go back and forth with two teams and agree on contracts and on everything. Miłosz [00:17:30]: You have to wait for the other team to pick it up for the sprint or whatever they work with. So it takes twice as long at least. Robert [00:17:38]: Yeah, yeah. And, you know, It’s often killing these small features that could be developed within one day or even if I code it now. Sometimes those features are that simple that it can be easily vibecoded. But with microservices, you no longer can because it needs to be orchestrated on a higher level. Maybe, you know, era of vibecoding will be era of modular monoliths. Hopefully modular. For sure, easier to deploy. But. And I think it’s also one important technique to mention when we are talking about these patterns, when we see that. So I would say that dark two signals when you should consider if something is wrong with us. One signal is when there’s a lot of calls between those two services to do some scenario. And the second thing is when you are modifying some functionality. Robert [00:18:31]: It requires changes in two or many services multiple times. So you see this pattern over and over. And it may be a sign that it may be wrong separation of services. Robert [00:18:41]: To solve that, there’s one technique that I think it’s very rarely mentioned because maybe it’s, I don’t know, some heresy for many people, but it’s the microservice station. So basically you’re getting those two services and joining them. Robert [00:18:56]: And I know that for many people it sounds like a heresy. I hope it will change over the time. Miłosz [00:19:02]: And yeah, it’s not never not easy, depending on how they are built. Robert [00:19:06]: But Yes, but if you are getting into problem of changing those two services over and over and two teams that needs to do that, or even within one team, you need to always change two services and kind of synchronize how you’re deploying them. It may be worth considering if it’s worth investing in joining them. Because yeah, probably to introduce some costs, they need to do some kind of cost and income. Miłosz [00:19:32]: Yeah, it’s a good thing you mentioned Teams because it’s also something I think you can’t skip when designing. Miłosz [00:19:40]: Creating software, especially if you have teams working on the same project, maybe not the same code base, but microservices that work with each other. Miłosz [00:19:53]: And if you have many of the services. I think once we worked for a company that had probably more services than developers or it was something like that, which sounds funny now, but I think it’s the reality for many companies actually. Because when it’s easy to spin up a new service, maybe you have great tooling and some platform team that handles it for you. Robert [00:20:16]: It’s interesting risk, actually, of having great tooling for easily creating microservices. I mean, if it wasn’t easy, you would think twice before creating a new microservice. Miłosz [00:20:28]: If creating that takes one minute, you end up with teams who own 10 or more. Microservices for very simple tasks sometimes I don’t know— maybe it’s fine. Sometimes, for things like keeping avatars, for example, so it’s like—overhead— but I think the most problematic story I remember from from this was where there were some services that had no clear owner, and we both our teams and other teams’ responsibilities kind of crossed in one area. That’s not really a technical issue—it’s funny, that’s like a team organizational issue. Mm-hmm. Miłosz [00:21:19]: And it’s very difficult to figure out if you didn’t start with a good design of the system first, but you just, you know, added the microservices for this entity here. This entity here— this entity here— maybe this database table sounds like a separate service, and then you have to kind of— you know— create some teams and you assign them the services. Artificially. Yeah. So that’s not a good idea. Robert [00:21:47]: And two years later, if somebody asks why those services are like that, it’s: ‘It was always like that.’ Miłosz [00:21:54]: Why you ask? Yeah. So we finally figured it out somehow, but it was very painful. Miłosz [00:22:02]: Not only because of the technical challenges, but also all the discussions between teams, company politics, stuff like that. Robert [00:22:13]: What I remember was useful there. So in this scenario, we used event storming for that to discover that. But you don’t need to use event storming. You can use any technique to see how the flow of the system is going. And I remember that it was quite funny to see that entire critical flow of the application was. So, for example, the beginning of the flow was owned by one team, later it was some small element owned by another team, it was going over again to the previous team. And it was a lot of some small pieces. Crossing the boundary. Yes, yes. And those were usually the biggest problems because if something didn’t work, it was sometimes not clear who is responsible. And it was also harder to change this law because if team didn’t have entire ownership of this law, they were just not able to do it without some cross-team coordination. Robert [00:23:09]: Well, what we did there, the solution was quite simple. So it was looking on high level on this and trying to understand how we can change ownership. Robert [00:23:19]: Historical ownership, and it helped a lot. And we added also to that a bit of the microservice station, and it helped even more. Miłosz [00:23:26]: I thought about even storming when you mentioned service mesh visualization. Miłosz [00:23:31]: So maybe instead of using this visualization after the fact, after you have the microservices in place, you could just have this high-level plan with even storming or not. Robert [00:23:45]: Seeing how system works before it works? How? Robert [00:23:49]: How is it possible? Miłosz [00:23:51]: What kind of magic is that? Miłosz [00:23:54]: Try to design it first and then see. But you know, it’s also not that easy, right? Because it’s not just a UML. Diagram where you have users, orders, and you see—okay, this is separate, but more like ‘yeah,’ maybe that’s why I would still recommend event storming because it focuses on behavior, not on the entities. So, if you map the events that happen, then the behaviors you can see there are some— those clusters of things that happen together— and it probably means they are a good place for some kind of boundary. And if you have, for example, two clusters that are joined by two events or something like that, you can clearly see, ‘okay,’ those things can be mostly separate and they won’t interfere with each other. They won’t be tied up. Robert [00:24:53]: And I think what’s also pretty good there, compared to some UML diagrams, is the context of time. And because, when you are looking at UML diagrams, you cannot see the order of flow. Even storming is more so for people that don’t know how to it looks like. So TLDR is that it’s basically some brainstorming session when you’re putting sticky notes for domain events or whatever events that are happening in the system. And you don’t need to do event-driven architecture for that later. So it’s more for the high-level modeling and how you implement that later, whatever. And what’s nice, it’s putting the in time order. Robert [00:25:32]: So basically, when you have flow, you can see when the flow is starting, when it’s ending, and it’s kind of in a magical way helping you to see how to do the split there. Anyway, I think it’s a big advantage compared to those UML diagrams or in general some diagrams. Robert [00:25:52]: Flow diagrams probably are a bit better because they also go from left to right, but I think they are not readable for very complex systems. You cannot make a flow diagram for entire system. Don’t try. But with event storming, basically you can do super big systems on higher level, but you can see the big picture of it. Miłosz [00:26:16]: So it’s a great technique if you’re just looking for the boundaries, or even if you have an existing system and you want to see how it could be done in a different way. Robert [00:26:25]: If you would like to hear a bit more about event storming, we’ve been discussing it one episode earlier, two episodes earlier, something like that. That so check it out but okay let’s come back to splitting. Miłosz [00:26:38]: So the nice benefit is of having everything mapped out on some whiteboard or you know some software. Is that you can look at it from different perspectives. For example, one thing you can do is try to differentiate core domains from the generic domains. So for example, you can see some cluster where there is more of your domain language of what your software is very specific. About, different from other software. This is probably your core part and it might be one cluster. And maybe there are some parts that are more supporting, more generic and they don’t need to be involved as much. So they don’t need to know all the details of your domain. Very often this helps you to simplify this part. For example, it can be some e-commerce software or some account. Miłosz [00:27:41]: Mounting software you can just buy, or you know, wipe code, or whatever— you don’t need to build it yourself from scratch if you don’t want to. Robert [00:27:51]: So, in other words, not all software in your product is equal. So some parts of software probably require a bit more of love. Miłosz [00:28:01]: And the less they know of each other, the cleaner your boundaries and the simpler the software as well. Miłosz [00:28:08]: So even if you build yourself this generic part, I would do it in a way that it could be reused in a different company. Miłosz [00:28:18]: It doesn’t mean you need to do it in the end, but it often helps to keep these boundaries clean and just keep some translation between them. Hmm-hmm. Of course, it can also be overdone if you try to make it super generic or use cases. So you have to be careful about that. But in general, I would try to not let the domain terms spill out of the core parts. Robert [00:28:48]: Yeah, I think it’s also some other direction that sometimes you can also watch out for with buying things because, it sometimes works that there are some services that are generic enough that can buy it, like sending emails, sometimes accounting software, but it depends. So sometimes when you are handling very specific things like marketplaces, you might not find accounting software that can handle it for you. I mean, for example, you are building some food delivery application. I guess you won’t find accounting software that will handle it for you. So it’s kind of interesting, because if you’re building this kind of application, it’s not your core domain, probably, but for other sites, it’s a very important domain. Unit also yeah— probably building a house, but parts of it will be generic, right? Miłosz [00:29:42]: Like printing invoices, that’s probably something someone solved some time ago and you don’t need to write it from scratch. Or if you do, we can keep it very generic. Robert [00:29:52]: Or maybe, for example, trying to understand what’s art. tax rate for each country is a good type. I mean, you can probably hire an entire team that is spending their entire time researching what the rates for every type of food or whatever that you can have in any countries. But probably you can just buy it from external company. And other techniques for event storming. So maybe not forever event storming, but I have also in mind one thing that you mentioned: sometimes you have parts of the system, some maybe services, maybe some libraries that multiple teams are kind of owning. And sometimes it’s on the pattern, but I think sometimes it’s also not, because sometimes it can happen. For example, from domain-driven design, we have concept of shared kernel, and it’s there for some reason. So sometimes you have some parts of code that two teams need to share, need to own. M. You can use it, but I would say that it’s rather an exception than something that you need to use by default, because if you try to use it by default, probably you’ll overuse it. Robert [00:31:02]: But I think it’s also important to keep it in mind that sometimes you have some things like that and it’s also fine. Miłosz [00:31:09]: Yeah, it’s a very important decision when you have two teams working with some similar entity or the same entity in different contexts. So first, there’s also the bounded context concept from DDD. What I think about it the most is exactly this situation when, for example, you have this user entity which can be also a customer entity in another context and, in other contexts, be something else. And you have multiple teams working with it. One naive approach would be to deduplicate and create one main user that decides about everything and has all the fields and behaviors and so on. But it that means all teams share this part and you know they need to agree on how to develop it or, from other side, if you’re one team, don’t that because it will just create too much overhead. Robert [00:32:05]: And this was something that we see often: there was one team and went too far with all those patterns. SPEAKER_2 [00:32:12]: And later, it’s like, oh, no, Domain Driven Design is making our software so complicated. Miłosz [00:32:19]: Yeah, but if you have multiple teams, you can use the bounded context concept. So you try to figure out how this user entity works. Different contexts may be separate; you know, maybe both teams can have their own user entity and they are completely separated and that’s fine. And sometimes you have this shared kernel, as you mentioned. So some parts that are so common and probably specialized. Miłosz [00:32:49]: Like you mentioned, maybe the tax rate calculation, something like that. But it doesn’t really make sense to separate that. You just agree, okay, on this small common part. We will work together and you know both teams— for example— need to agree on reviews and stuff like that on the design of it instead of the entire big system. So when I think about what’s difficult about this problem of splitting software well, I think it’s a combination of organizational and technical complexity. It’s not just one thing. Robert [00:33:23]: So sometimes I remember that there are some companies where the teams and services split and organization is somehow going from the top. Maybe it’s not from the top, but level or two levels higher than you are. Miłosz [00:33:39]: Or another way is that it became kind of the default to create microservices, and very often in a naive way. Miłosz [00:33:50]: So, for example, when I joined some project and there was some kind of migration happening from an old system to a new one. Miłosz [00:33:59]: A few people in the team assumed, let’s use microservices from the start. It’s kind of like a default approach for some people, but it can be a trap. Because you have to think, what problem do you want to solve this way, rather than just, ‘Oh yeah, it’s the industry best practice.’ Robert [00:34:19]: Worse if you have problem that you’re trying to solve, because sometimes people may think, ‘Yeah, we have problem with I don’t know scalability or ‘We had overcomplicated POC earlier and microservices will solve our problem.’ Yes. Miłosz [00:34:36]: So it’s helpful to think of microservices. Deployment patterns, something we discussed some time ago, I think. Miłosz [00:34:45]: It helps when you want independent deployments, for example, for different teams. That’s a good excuse to split software. Robert [00:34:56]: Or for example, you have that many deployments of your module or monolith that the queue is longer than you. I mean, queue is long enough that not every person is able to merge within one day. Probably it’s already designed that you can split it. Miłosz [00:35:13]: So it’s connected to teams and how you organize developers. Robert [00:35:20]: So in other words, probably we can say that microservices are maybe solving a bit of technical problems, but it’s mostly solving the human problem. Miłosz [00:35:30]: Humans working together problem. Miłosz [00:35:33]: There’s Tim Topology’s book. I’ve read some time ago. It’s great. I can recommend it. It’s specifically about this problem with the reverse Conway maneuver, I think it’s called. So basically, instead of creating a software first and then trying to assign people who work on what, you do it the opposite way. So you first design, as we mentioned, event storming, a bit similar idea. You first design the organizational structure you want to have in place, and then the software system follows that. Robert [00:36:08]: I love when such old patterns, like always low, are still working those days. Robert [00:36:14]: Even if people don’t realize, but, but I think. It’s good to know about some that something like that exists, because you can be a bit maybe more mindful when you’re doing such decisions. And it’s not like, oh, just everybody’s doing that or we’ve been doing it like that for ages, why we should change it. Yeah. Miłosz [00:36:36]: It’s also great that we are past hype. Era now and many people talk mainly about how complex microservices can be. So now it’s probably we are more balanced. Robert [00:36:47]: Probably won’t you won’t be put on the stake after saying that microservices is not the only way. Robert [00:36:55]: Maybe in 10 ways we’ll be again in the place when everybody is using monoliths. Probably not modular, something tells me. Probably it will be still a problem. Miłosz [00:37:06]: I think, as an ending note, we can recommend starting with monoliths if you are just getting started, especially if you are just a single developer or a small team. Yeah, I hope we should. And do it the modular monolith way. Robert [00:37:20]: Yeah, I hope that we’ve shown that there are alternatives and it’s really not that hard to do things in those alternative ways. Robert [00:37:30]: Definitely will not finish with this episode about modularization because it’s a topic that requires a lot of attention, in my opinion, and it can give a lot back— i mean, when you are doing it properly, it can really accelerate your work a lot, yeah, especially with modular monolith. Miłosz [00:37:49]: Because I think what we’ve figured out works pretty well. Robert [00:37:54]: And we’ve seen it work with teams. And we also see that it’s not that popular approach. So I think it may be also some unique skill to have. Miłosz [00:38:04]: Yeah. And if you’d like to see a real example of this, we have a domain engineer training coming up and we will have an example of a pretty complex application with different modules built as a modular monolith. And we mentioned parts of it in this episode as well. Yeah, I think it will be cool. So we’ll leave a link in the description if you want to learn more. So we shared our stories with Spitting Software. Let us know about yours so we can also learn something and maybe help you figure out something. Robert [00:38:40]: Yeah. So we’re here to help you. So this podcast is for you. And yeah, if you will be happy to answer in comments and maybe include something in the next episode as well. Miłosz [00:38:53]: Yeah, if you have any topics you would like us to cover, let us know. Miłosz [00:38:58]: See you in the next episode. See you next time. Bye. Bye-bye.
Learn Building Modern Go applications Jan 28, 2026 -
Major updates to VS Code Copilot including parallel subagent execution, a new skills system, deeper Claude integration with extended thinking, terminal improvements with kitty keyboard protocol, and instruction files that now work everywhere.
alexop.dev Jan 24, 2026 -
Workshop covering the transformation from LLM to Agent, context engineering, AGENTS.md, subagents, and skills in VS Code Copilot.
alexop.dev Jan 24, 2026 -
A complete demo of presentation mode with v-click animations and drawing annotations. Press P to see keyboard navigation, incremental reveals, and press D to draw on slides!
alexop.dev Jan 23, 2026 -
The 2025 Go Developer Survey results, focused on developer sentiment towards Go, use cases, challenges, and developer environments.
The Go Blog Jan 21, 2026 -
Proof that the fast unrounded scaling implementation is correct. (Floating Point Formatting, Part 4)
research!rsc Jan 19, 2026 -
Fast and simple conversion between floating-point and decimal. (Floating Point Formatting, Part 3)
research!rsc Jan 19, 2026 -
AI coding tools are stateless—every session starts fresh. The solution isn't cramming everything into CLAUDE.md, but building a layered context system where learnings accumulate in docs and specialized agents load on-demand.
alexop.dev Jan 18, 2026 -
How I built a skill that lets Claude search its own conversation history, turning it into a persistent coding partner that remembers past solutions.
alexop.dev Jan 17, 2026 -
Software development as translation work is dying. Software engineering—the strategic, architectural discipline—is more valuable than ever. The shift is already here.
alexop.dev Jan 14, 2026 -
When Stryker doesn't support your test stack, AI agents can execute mutation testing manually. A practical approach for Vitest browser mode and Playwright.
alexop.dev Jan 13, 2026 -
A birthday proof for Don Knuth (Floating Point Formatting, Part 2)
research!rsc Jan 10, 2026 -
Why I use custom research agents instead of MCP servers for AI-assisted development. Learn how llms.txt enables context-efficient documentation fetching with a practical Nuxt Content agent example.
alexop.dev Dec 31, 2025 -
When should you use CLAUDE.md, a slash command, a skill, or a subagent in Claude Code? A decision guide with real examples for each, so you stop guessing which one fits the job.
alexop.dev Dec 21, 2025 -
Quick takeaways Complexity comes from two extremes - projects fail both when they’re overcomplicated with unnecessary patterns and when they’re oversimplified for a complex domain Essential vs accidental complexity - essential complexity comes from the domain itself and can’t be removed, accidental complexity is created by poor implementation choices “Keep it simple” is lazy advice - achieving simplicity takes effort; closing your eyes to complexity just pushes it elsewhere Match patterns to the problem - using the same approach everywhere is a red flag; mix simple solutions for simple parts and sophisticated patterns for complex domains Ship fast and iterate - if you can’t deploy daily and fear making changes, something is wrong regardless of whether it’s over or under-engineered Introduction We discuss where complexity in software projects comes from and how to deal with it. There are two common extremes: projects that are overcomplicated because someone applied patterns they saw at a conference without understanding them deeply, and projects that started simple but became a tangled mess as they grew. Both can be equally difficult to work with. The key insight is understanding the difference between essential complexity (inherent to your domain) and accidental complexity (created by your implementation choices). We share diagnostic signals to identify unhealthy projects and practical advice on matching your tools to the actual problem you’re solving. Notes No Silver Bullet paper by Fred Brooks - discusses essential vs accidental complexity Domain-Driven Design (DDD) - a pattern for tackling complex domains Clean Architecture - another approach for managing complexity YAGNI (You Aren’t Gonna Need It) - principle against premature generalization Defensive programming - validating inputs early to prevent errors propagating Canary releases and rollbacks - deployment strategies that can add overhead Three Dots Labs blog - articles on Go patterns and architecture Go with the Domain ebook - free ebook about DDD in Go with 60,000+ downloads Wild Workouts example DDD project - complex Go project demonstrating real-world patterns Microservices architecture - discussed as sometimes adding unnecessary complexity CRUD applications - simple approach that works for some domains but not others The Domain Engineer training - upcoming training mentioned for early next year Quotes On one hand, you have someone who says we need to support 10 million users a day—this is one extreme— but on the other hand, you also have people who say, ‘just keep it simple and it will be fine’ and it sounds good. But to me, it’s one of the lazy advice. It’s easy to say to tell someone, ‘keep it simple but it’s not that easy in the end. Miłosz The complexity doesn’t go anywhere, no matter how you implement it. Miłosz If your team is unhappy working with the project. If the team brings up plan for migration or rewrite every other day, that’s a solid sign something went very wrong. You’re probably on one extreme already and it’ll be hard to get out of it. Miłosz If just keeping projects simple would be that easy, all projects would be great. But unfortunately, it’s the opposite. Robert So you cannot close your eyes and say that this complexity doesn’t exist. It will pop up somewhere else and it will be much more expensive to handle at the end. Robert Don’t be dogmatic about using any tools because it’s never ending up properly. Robert Timestamps 00:00:00 - Introduction 00:01:21 - Two sources of complexity 00:05:14 - Complex vs complicated 00:09:20 - Essential vs accidental complexity 00:14:00 - The trap of generic solutions 00:19:15 - Complexity doesn't disappear 00:23:05 - Patterns and defensive programming 00:26:58 - Red flags in projects 00:32:05 - Avoiding premature optimization 00:35:12 - Don't be dogmatic 00:37:40 - Practice and resources Transcript Miłosz [00:00:00]: On one hand, you have someone who says we need to support 10 million users a day—this is one extreme— but on the other hand, you also have people who say, ‘just keep it simple and it will be fine’ and it sounds good. But to me, it’s one of the lazy advice. It’s easy to say to tell someone, ‘keep it simple but it’s not that easy in the end. Robert [00:00:18]: If just keeping projects simple would be that easy, all projects would be great. But unfortunately, it’s the opposite. Miłosz [00:00:25]: It takes a lot of effort to have simplicity in the project. Robert [00:00:28]: Your eyes and say that this complexity doesn’t exist, it will pop up somewhere else and it will be much more expensive to handle at the end. It’s a very important concept to understand the problem. Accidental complexity and essential complexity. Accidental complexity is created by poor implementation. If you approach the implementation of your application in an overly complicated way, so we’ll put just some patterns that are unnecessary, you will have accidental complexity. On the other hand, if your project will be, for example, this complex financial product and you will try to build as a CRUD, you’ll end up with a lot of accidental complexity there. Miłosz [00:01:05]: Yeah, but you can reduce it if you use some patterns that play well with the domain. So you can reflect it in the code in a more natural way. Then it’s easier to work with. So the accidental complexity goes down. Robert [00:01:21]: I’ m Robert. Robert [00:01:21]: And I’m Miłosz. Robert [00:01:22]: And today we’ll talk about complexity in projects and from where this complexity is coming. And from my experience, there are probably two types or two sources from where it can come. So the first one. Probably. Many of you can relate to that. It’s the situation when you are joining some project and there is some team leader or some very senior engineer. And this engineer is dictating, let’s say, some good practices and some patterns in this project. Robert [00:01:55]: You can clearly see that those patterns are not adding a lot to this project. Actually, it’s adding a lot of unnecessary complexity. Often, this person is getting those techniques from some conference, maybe, or from some video, or maybe this person read some article. Later, this person is forcing everybody to use those patterns. Miłosz [00:02:15]: So it’s this ivory tower architect who dictates from the top how to do things and the solutions are already invented. Robert [00:02:26]: Yeah, so it’s often the problem. It doesn’t need to be architect because it’s sometimes maybe, again, some more senior person. But from my perspective, very often the problem in this situation is that this person doesn’t really understand those patterns very deeply. Robert [00:02:45]: As I said earlier. This person has seen those patterns in one video or maybe one talk and knows how it looks like. So you are going to conference, there is some idea, it looks great on the paper. In practice, Well, there are a lot of complexities and it doesn’t work as simple as it was thought on the conference. Miłosz [00:03:04]: And being an engineer on this team, maybe peer of this person, you will probably feel like this project is overengineered, right? There is too much patterns and it seems overcomplicated. It’s very difficult to get something done. Miłosz [00:03:22]: Because there’s so much work involved. Robert [00:03:24]: Definitely. And I think for many people, it will be the main source of complexity in projects, but maybe you have some other idea what can be another issue that can happen in projects. Miłosz [00:03:35]: So on the other hand, I can remember other types of projects where everything seems very simple, but at some point it’s very difficult to continue development. Miłosz [00:03:50]: Project lifetime, it’s sufficient to have some very basic patterns in place. Maybe no patterns at all. Robert [00:03:58]: If you have small enough application, maybe you can just do only one file and sometimes it’s starting as some proof of concept or maybe side projects and because of some lack reason this is becoming pretty successful project at the end. And it’s becoming also the problem because it started to be it started as a proof of concept. And at the end, it it became a successful product. Miłosz [00:04:22]: These are fun. You can just add new endpoints, add new database tables. Miłosz [00:04:30]: Works fine, but then that team grows and at some point it starts to be very difficult to extend this project because you know everything is coupled together when you change something you don’t know what else is going to change and usually at this time you you might be long for some patterns actually to be used in place and I started researching some architecture, how to approach this thing. It has two extremes, right? One is overcomplicated, the other is no patterns at all and both can be equally annoying to work with, difficult to develop. Miłosz [00:05:14]: Thank you. Robert [00:05:14]: In this episode, we’ll cover a bit more. How to understand if your project is in the bucket of overcomplicated by using too many patterns or maybe oversimplified. So the problem that you are solving or the product that you are working with is already pretty complex and building is a crud with just frameworks. It may not work anymore. And I think it’s a very important skill that many people is missing. So I think many people are stopping on the side that, yeah, this project is very overcomplicated because we use too many patterns but it’s not only the case yeah so it’s a similar issue but Okay. Miłosz [00:06:01]: two extremes, right? But the root cause is kind of the same. It’s using wrong tools to the wrong problem. Or you don’t know what the problem is yet and you use only what you know. Robert [00:06:14]: Bye. It’s not only about the complexity of the project but sometimes it’s also the scalability. So we’ll also cover a bit about this today, because also you probably heard many stories like: ‘This application will have millions of users.’ At the end, we have 10 users. And it’s hard to add anything more because you prepare your application for big traffic. Miłosz [00:06:35]: And because of that, it’s very, very complex. On one hand, you have someone who says, ‘we need to support 10 million users a day,’ which is one extreme. other hand, you also have people who say, ‘keep it simple and it will be fine and it sounds good but to me it’s one of the lazy advice. Miłosz [00:06:55]: It’s easy to say, to tell someone, ‘keep it simple.’ But it’s not that easy in the end. Robert [00:07:03]: If just keeping projects simple would be that easy, all projects would be great. But unfortunately, it’s the opposite. It takes a lot of effort to achieve simplicity in the project. If you’re joining a new project and you notice that this project is complex. So in this case, do you wonder if it’s complex because of some patterns that make it overly complicated? Or maybe this project has some kind of baseline complexity that made this project complex. Miłosz [00:07:33]: I think it’s very different. Tell the difference when you just joined the project— actually. You might know that it is difficult to work with, but you might not yet know what it solves, what’s the business domain. Miłosz [00:07:47]: So it might be not that easy to tell if it’s over-engineered or it’s just inherent domain complexity that’s not going anywhere. Robert [00:07:58]: And I think what may be pretty useful here is distinction between complex and complicated, because I think those words are nicely encapsulating the different characteristics of this. So when we have a project that is complex, so it’s a project that has, for example, some complex domain like accounting, complex e-commerce, or is pretty old product supporting many use cases. So in this case, it’s complex, but it doesn’t need to be complicated. So, if somebody will not approach it properly, it may be complicated, but I think what’s also pretty interesting is that some projects are not complex, but they are just complicated. And in this case, it’s the scenario when, for example, this team leader went from a conference and just read some blog article, started to copy some patterns, and just unnecessarily complicated that. For the sake of it, instead of solving something. Miłosz [00:08:53]: Exactly. Yeah, but it might be hard to tell the difference if you are not very deep into the project. I remember working with some MVPs that looked complicated on the surface. And then you start rebuilding them or trying to understand what’s going on. And you see that, okay, this is not actually technical complexity. This is how the business works. So that’s why someone did this this way. Robert [00:09:20]: So some examples of projects that are complex, but maybe not complicated, are for some financial-related applications. So for example, some accounting systems that are supporting multiple countries and even hundreds of countries, different tax rules that can be different for multiple countries and some domain. May have some special requirements, like for example, taxi or something like that. And this kind of projects are very big. Baseline complexity. And by the way, we’re currently working actually on a training that we’ll be releasing at the beginning of the next year when we’ll be showing how to build such very similar domain actually. So you can see the description. Yeah, so check the domain engineer in Google. So you will find it. But what I think it’s pretty interesting in this kind of projects is that if you, at the first site, look on all these accounting rules and support of multiple countries, you might have a feeling that it’s super complicated, but actually depends how you approach implementation of that. Robert [00:10:37]: So no matter what you do, in the implementation part, it will be complex because it’s just a complicated domain. Miłosz [00:10:46]: The complexity doesn’t go anywhere, no matter how you implement it. Robert [00:10:49]: And often this is named as essential complexity. This is the complexity that is kind of created by the domain. And the opposite of that is. Accidental complexity. So accidental complexity is complexity that is created by engineers, basically. So when you are, for example, preparing your application for global scalability, millions of users. So this is where you are creating the accidental complexity. Miłosz [00:11:16]: Yeah. I think it comes from the No Silver Bullet paper from Fred Brooks, which shares the name of our podcast. It’s actually a good read. Robert [00:11:24]: What a cool incident. Miłosz [00:11:26]: Check it out. It goes into detail on essential versus accidental. But maybe we can just quickly go over it. Hmm, hmm. Miłosz [00:11:37]: Hmm. Robert [00:11:37]: Also, because I think it’s very important to understand that concept to understand the entire problem of accidental complexity and essential complexity. So basically, the accidental complexity is created by poor implementation. Because if you approach the implementation of your application in an overly complicated way, so you’ll put just some patterns that are unnecessary, you will have accidental complexity. But on the other hand, if your project will be, for example, this complex financial product and you will try to build as a CRUD, you’ll also end up with a lot of accidental complexity there. Miłosz [00:12:16]: Yeah, but you can reduce it if you use some patterns that play well with the domain. So you can reflect it in the code in a more natural way. Then it’s easier to work with. So the accidental complexity goes down. Robert [00:12:31]: Unfortunately, if you do the opposite— so if you have a complex problem to solve and you try to approach it in an oversimplified way— it will add. So we will have the base complexity, so the essential complexity, and you will have also accidental complexity. So this product probably will be a great mess that nobody would like to touch. Miłosz [00:12:51]: And suddenly, we developers often bring this on ourselves, focusing on the technical details, trying to build a framework that will capture cases in the future, stuff like that, which all can add up to extensive complexity before you need all the features. Robert [00:13:11]: It’s starting with a solution that is generic from day one, and it’s basically creating a solution that is generic. Always much more complicated than creating a solution for one use case. I have some examples when we’ve been adding some functionality and some teammates were proposing like, ‘Oh, let’s maybe create a library to do that.’ You will not need to do it in the future. But after trying to design that, we found that actually making it in a generic way is a couple times more complicated than just doing it in one way for one scenario. And sometimes it’s even easier to just copy some logic and have a bit different implementations if it doesn’t change that often in two places and independently. Miłosz [00:14:00]: Because again, having this generic solution be just very expensive, yeah, so you need to be very conscious of where to use the generic solution, where to when to abstract away the details, and for for what system you need, what will approach. Robert [00:14:18]: Let’s go back again to the problem of complex projects, because I think it’s also interesting that if you are working with these more complex projects, sometimes some people from other teams or some new joiners may look at it and think, ‘Oh, guys, you’re overcomplicated that so much. You’re using so many patterns and it’s hard to go over this code and understand how it’s working. It may look like this at the beginning when you are working with some more complicated domains. But people may sometimes miss the context here. And sometimes you may be in a rescue project, basically, that earlier failed. Because it was created as a CRUD, you are trying to save this project and implement it in a way that you can tackle this. Robert [00:15:10]: Essential complexity. And some people looking at that later may be missing this context here. Miłosz [00:15:22]: Overengineering. Yes. Yeah, but on the other hand, it’s also the opposite issue, right? If you oversimplify things from the beginning, it may be too late to change the architecture. Robert [00:15:38]: And I think it’s also sometimes interesting when, for example, the team was changed and most people left this team. Robert [00:15:47]: If it was a complex project, not overcomplicated, but the person that was trying to change it later. May have in mind that, oh, okay, it’s so overcomplicated, let’s try to rewrite it. So this person or this thing is spending like their half year or year to rewrite that to just realize at the end that basically, all the complicated or the complex, not complicated stuff is again there. Because when you start to support more and more use cases and edge cases, you’ll see that actually all those checks and all those complex parts were necessary there so yeah and they don’t go away when you change the approach. Miłosz [00:16:31]: Even if you change the technical approach, the underlying domain complexity doesn’t disappear. All right. And. If you’re like a developer focused on simple code, if you like simple code, you might be annoyed that there’s new requirements coming in. Miłosz [00:16:49]: I know you need to change the factor to capture it, but it’s just the reality. You need to support the business this way. And however you model it, the complexity is there. And if you try to avoid patterns at all costs and just do it simple, you may end up down the road with some weird solutions, like for example, instead of having a good transaction boundary where you need it, you start running cron . checks for consistency. In your system and report some errors, and for some teams this is like the business as usual approach. Let’s just fix it afterwards, but that’s so much— you know— with operating efforts. You are bombarded with alerts all the time. Miłosz [00:17:52]: You need to react and fix the system. Miłosz [00:17:57]: After the fact, instead of just designing in the first place in a way that it’s resilient and I think it doesn’t need to be. Robert [00:18:04]: We don’t need to reuse any advanced patterns to avoid that. So sometimes it’s basically just some defensive programming. So let’s take an example when we’re building this accounting system, for example, and you are not validating if, for example, the amounts are positive or non-zero. And you can accept any value and issue thousands of invoices that have a total value of zero. And. Okay, you can later clean it up. You maybe have some chrome that will void them or do whatever with that. But basically, it will be much cheaper to do this defensive programming approach earlier and ensure that it will not propagate further. It’s not only basically about uh operating overhead from side of your team, but it may be also some financial team seeing later that, oh, no, we have thousands of invoices that we need to do something about that. And those developers again. Miłosz [00:19:09]: Right. So you only push around the complexity this way instead of solving it. Robert [00:19:15]: So you cannot close your eyes and say that this complexity doesn’t exist. It will pop up somewhere else and it will be much more expensive to handle at the end. Miłosz [00:19:24]: Some analogy that comes to my mind is like if you decided you won’t have files bigger than 100 lines in your project. And you will have simple files, right? Small files. Sure, it’s great, but then you end up with thousands of them. And you push the complexity to the number of files instead of the length of the file. Miłosz [00:19:47]: Back in the day, there was some big question: how big should be a microservice, and some people suggested metrics like this. This comes to my mind. Miłosz [00:20:00]: Let’s build our huge application into really tiny services. So you solve this complexity of big code base, but now you deal with thousands of code bases. Robert [00:20:15]: So basically, divide and conquer doesn’t always work with software. It can work, but you need to know basically how to split them. I think it may be also an interesting topic for some other episode because, Well, not a surprise. There are some patterns and ways how to detect that. And spoiler alert, it’s not as small service as you can because, at the end, you can just end up with big mess and some sagas that you need to orchestrate some really small part of the flow. But I think it’s enough for today. Miłosz [00:20:51]: The problem here is you think the size of the code base is the issue, right? It’s similar to what we discussed before. The complexity is in the domain, so you can’t just change this. If you have a complex business and it needs a big application to work, this complexity will be there, however you model it. So you just need to make effort to model it in a way that’s best to reflect how it works and for teams to work with it. Robert [00:21:20]: And I think it may be misleading sometimes, because you may feel that splitting this big problem into smaller microservices can actually make this problem simpler, easier to tackle. On the other side, sometimes you may wonder. How? Robert [00:21:38]: Some application can have, I don’t know, 5000 of developers for running and developing such simple application. I know that it’s sometimes hard to judge an application from outside because you may not know all the complexity inside. But come on, thousands of developers— it’s a ton of people. Splitting it into hundreds of microservices can create an illusion that you just cut this problem into smaller and it’s easier. But there are cases when, basically, this application could be one service with 50 times less developers developing it. Miłosz [00:22:17]: Maybe you won’t need a whole platform team then to manage all this complexity and communication between them. Robert [00:22:25]: And if you add the extreme scalability requirements that are, I think, also often out of nowhere, because really, you can probably handle Twitter-like scale on a single VM. Just a reminder that you can have VMs that have one terabyte of RAM, 96 cores. On GCP, it’s even 150 cores per one machine. So come on, you can really handle Twitter scale on this one VM. Miłosz [00:22:54]: RAM is expensive now, so I don’t want to do that. Miłosz [00:22:58]: So coming back to the essential versus accidental complexity, what is the solution here? Robert [00:23:05]: So I think for the case with cron. Again, some simple solutions like defensive programming and very very very very very very defensive programming. So you have any input that this not as expected, don’t accept that. Don’t allow to propagate it further. And the other thing, when you are splitting problems into smaller, also watch out to not split it too granularly. Other thing that is coming into my mind is trying to tackle this essential complexity instead of always splitting that. Because there are some patterns that are helping to tackle that. In, for example, this one service, it can be clean architecture, it can be domain-driven design. Thank you. Very important disclaimer. I’m not saying about some dogmatic clean architecture or domain-driven design because it’s very easy to go into caricature of those techniques instead of some pragmatic approach. So don’t be this person that you see in one video or one presentation or one article. Robert [00:24:08]: Dig a bit deeper, try to understand what problems those techniques are solving, what problems they are not solving, when to not use them, and apply them in places where you have the essential complexity to not add more accidental complexity. Robert [00:24:27]: Sounds like a plan. Miłosz [00:24:28]: Yeah, that’s something I would recommend to consider where in the project you use what approach. If there’s one approach you use in your entire project everywhere, that’s probably an amber flag. Probably something is weird because The most projects don’t have only you know, only complex or only simple domains. There will be a mix of some very generic parts that, you know, we can use something simpler for it. And the core domain where you need your most sophisticated patterns. Robert [00:25:06]: Definitely. And also when you are using some patterns that you read about, you’re trying to apply that. Also, if you feel that it doesn’t fit, don’t try to push it by force because probably it may not be fitting there or maybe you don’t understand something about this pattern. Miłosz [00:25:21]: Sometimes you can look back after implementation and I remember we did it many times. Uh-huh. Uh, which created some domain layer and try to model something in it. And then we look at it and, hey, it’s like mostly boilerplate. Maybe we can just simplify it because there is nothing it gives us, no benefit. Robert [00:25:44]: It’s basically a crud. So we just can have one struct getting it from API, put to the database and it’s fine. Miłosz [00:25:51]: We have a mix of these parts of complex domains, complex patterns and some very simple ones. I think it’s a healthy sign of a codebase in general. Robert [00:26:01]: So watch out for being dogmatic. So it’s. Robert [00:26:05]: It’s coding, it’s not religion. Robert [00:26:09]: Many people will probably disagree, but it’s reality. Robert [00:26:13]: And I think it would be useful to maybe give you some advice from our experience. Robert [00:26:20]: What are some, let’s name it, diagnostic signals to understand if your project may be over complicated or oversimplified? Because I think we can probably find a couple of signals that may suggest that there is something wrong with this project. Well, I think it’s pretty interesting is the fact that those signals may be the same for projects that are oversimplified and overcomplicated. So you may remember that at the beginning we said that it’s the same problem, but from another side, basically. Robert [00:26:58]: It’s about half of the codebase in general. So what would be, from maybe your perspective, some red flags that the project is unhealthy? So it may be overcomplicated or oversimplified. Miłosz [00:27:11]: Probably the king of all would be if you can ship the project fast and do it over time, iterate often, and maintain it— that’s a good sign. If you can’t do it, something is wrong. So it’s like a big number one rule. Robert [00:27:33]: Some anti-examples here, maybe some old enterprises that are deploying. Every half year, for example. Miłosz [00:27:40]: Yeah, like the other extreme. Yes. For example, you need to hand over your deployment to someone who does it during the night because it’s, I don’t know, risky. Robert [00:27:56]: And you can also do it daily because it’s the scenario when you have an essential with accidental complexity. So it’s a very old project that has already a lot of edge cases to handle. But it’s also overcomplicated because maybe in some places it’s not planned strategically properly. Miłosz [00:28:15]: Yeah. Miłosz [00:28:17]: So if you can deploy daily, that’s probably one red flag. If you fear merging changes or just changing some areas of the project, because you don’t know what can happen. That’s another red flag, I would say. Whatever the reason, you should be comfortable deploying and managing stuff. Robert [00:28:40]: So in other words, if your project is crude and you are able to merge it daily or a couple of times a day, and you are not afraid to introduce new changes, it’s totally fine. It’s probably enough. But for example, when you have some complex domain and you are not able to do that. Well, you may have some excuse that, oh, it’s so complicated. We are working with money. We cannot merge every day. But yeah, there it is. The reality is that if you tackle this accidental complexity, if you handle this essential complexity properly, you can even deploy multiple times a day application. Pretty complex and is doing some risky operations. Miłosz [00:29:21]: Yeah, if you are working with money, I hope you just don’t just you know cross your fingers and deploy. But you are confident this will work. If you, if you’re not, then probably something is wrong. Maybe the project is oversimplified. Robert [00:29:37]: Other red flag that I have in my mind is when you have some new joiner in your team and it’s taking weeks to onboard this person. So it’s kind of connected. Previous points because, if you aren’t able to deploy daily, you have fear of touching anything in your project. The same will also apply to new joiners and even more, because you’re a new joiner. You see that this project is complex, and/ or people warn you or don’t touch this part. Miłosz [00:30:09]: You don’t want to see this. Miłosz [00:30:14]: I’ve seen projects like this. Robert [00:30:16]: Exactly, exactly. And in a healthy project, I would say that it should be quite safe to fail, basically. And it’s connected to defensive programming. Basically, and also having some good infrastructure in place that is maybe doing some Canary releases and rollbacks, but it can be also the opposite. For example, if you don’t have application with complex domain, and making mistake is not that big problem, but you have very complicated infrastructure for rollbacks, for Canary releases. It can add a lot of overhead for your deploying pipeline and, in this case, it even cannot deploy the simple application daily because the process of deployment is too heavy compared to what’s required for this application. Miłosz [00:31:05]: Also, if your team is unhappy working with the project. If the team brings up plan for migration or rewrite every other day, that’s a solid sign something went very wrong. You’re probably on one extreme already and it’ll be hard to get out of it. Robert [00:31:28]: I think many people may say, like, ‘projects like that don’t exist.’ I think it’s a nice excuse, but it’s not the case. Such projects exist that you can really deploy daily, even if it’s not very simple projects. Robert [00:31:46]: So, to maybe recap. Robert [00:31:48]: Those indicators a bit. So I think there are some questions that you may ask yourself. So the first one is asking yourself, Can you remove some stuff without fear that you’ll break everything, basically? Robert [00:32:05]: And when you’re thinking about those red flags, remember, it’s not just because some patterns that some people use sometimes, it may be because premature optimizations, like preparing for scale that will never happen. Miłosz [00:32:21]: Yeah, that’s a lot of accidental complexity there. Miłosz [00:32:25]: Or focusing on performance before you need it. So instead of choosing something more explicit and easier to grasp, you choose something very optimized, but not really needed at the time. Robert [00:32:41]: And sometimes it may be also preparation for the future—like, maybe at some point we’ll need X. Let’s do it now, so it will be easier. Unfortunately, usually it’s a trap. Usually it will never happen, or if this functionality will be needed, if we will need to add this functionality, we’ll find out that actually what we prepared earlier no longer works in this way. And we spent a lot of time on writing that and maintaining it. So, jagni. Miłosz [00:33:10]: Yes, it’s easier to get things that are easy to remove or replace instead of looking for all the possible edge cases. Robert [00:33:19]: Always have in mind that most of the techniques, tools, or whatever you are using, you’re using it to make your development lifecycle as efficient as you can. So, if you see that but something is making your development lifecycle worse, slower, and it’s slowing you down, you should maybe rethink if it’s needed or you’re applying it in the right way, or maybe it’s not needed at all. And at the end, you should aim to making some small incremental changes. And I know that it sounds easier than it is, it’s easier to say than do. But it’s a matter of experience and learning, learning and learning, and trying to be as close to this go to deploy multiple times a day. And also, watch out to not go into the trap that we already invested couple of months into this feature to make it super scalable. It doesn’t matter that we need to spend now weeks on maintaining that. Robert [00:34:21]: Watch out for this trap. Don’t be afraid to remove some stuff that is slowing you down. Robert [00:34:29]: Yes. So it’s important to be aware of that. So if you already invested a lot of time into something, it’s harder to work some framework. Some generic library trying to do everything. Some stocks. But it’s showing it similarly. Like you invested in something. Company and you see that it went down, and you are waiting years to recover. And probably just saying that and buying some better stock can help more. By the way, not investing advice. Robert [00:35:03]: I was about to say that, yeah. Robert [00:35:07]: And at the end, I think it’s important to emphasize one thing. Robert [00:35:12]: Don’t be dogmatic about using any tools because it’s never ending up properly. Robert [00:35:22]: Have an analogy of some worker that is, I don’t know, big fan of pneumatic hammer, and this person is just using pneumatic hammer everywhere. And it may work if this person is demolishing buildings, but if you need to nail nails, well, probably using nomadic armor for that. At best, you’ll hurt yourself. Robert [00:35:44]: But in this case, it would be just more practical to use the normal hammer. But from the opposite side, maybe there is some fun in using a normal hammer. Saying that a normal hammer is great and it’s simple— it doesn’t need any other energy source. You can just use it everywhere. But it’s true. However, if you need to demolish a wall, it will take ages. Miłosz [00:36:07]: For example, something that we mentioned in a previous episode about having a toolbox. You can choose your tools basically for the job. Robert [00:36:16]: And even if it sounds, maybe, very obvious. From other side, we’ve met a lot of people that were like these people with simple hammer or pneumatic hammer used for everything, and even if it sounds that obvious, they were doing it in this way. Miłosz [00:36:38]: So it helps if you ask yourself, why are you doing this? Why use this tool in the first place? If you want to solve something specific, for example, if you have a complex domain and that you want to model better, that’s probably a good reason to use something. Or if you kick off a project that you know is supposed to be a big one, you probably want to have some idea of the architecture of the design beforehand. But on the other hand, if you just kick off an MVP, you don’t need all this stuff. You can just start simple and grow. Miłosz [00:37:23]: and yeah watch out for extremes, what we mentioned before. So if you are far on the over complicated or over simplified spectrum, the project probably will be difficult to work with. Robert [00:37:40]: And the thing that we can probably recommend most, and we are saying it over and over, but play with technics sometimes in work, sometimes after work, also. So if you would like to be good at what you are doing, it’s it’s also good to practice this outside of work, because you’re also sometimes limited by the projects or maybe sometimes you’re limited by some dogmatic people that, for example, like oversimplified code and you cannot do much about that, but you can always experiment a bit in in the home. So one thing that we can probably recommend is checking our blog tree . tech and where we have a lot of articles that are totally free. We also created one pretty big Go project that you can play with and try to apply. Look how we applied some patterns there. And you can also try to play with this project and add some of your features there. And you can gain this muscle memory on how to use those patterns and later, when you will encounter some problem, you’ll have some tools in your toolbox and you’ll be able to experiment with that. Miłosz [00:38:51]: So we tried to create more complex examples, so you can see the ‘why’ behind the patterns, not just some toy projects where they don’t make sense. Robert [00:39:01]: Yeah, because if you have sometimes those dogmatic people about using some patterns, you can notice that very often they have very shallow knowledge because they’ve been reading some article. That again, are very shallow. So it’s like five minutes read. And they are thinking that they know everything about some technique after doing that, or doing it differently. So our articles are not simplifying things, but there are—pretty long, but they are long enough to prepare you for applying some patterns in the wild. There are some good articles out there. It’s not like only our articles are good, but very often there are some articles that are very, very shallow. And when you look on them, you can just see that applying what’s suggested there will introduce more troubles and this accidental complexity without solving any problem actually. So we would have your base complexity with accidental complexity, and at the end it will be how to work with. Miłosz [00:40:01]: So we also have the Go with the Domain ebook about working with domain-driven design in Go. And it shows refactoring of a bigger codebase. Miłosz [00:40:14]: This is not a toy project, but more like a real-world scenario. Robert [00:40:19]: And it’s, I think, already pretty popular. So last time when I was checking it, it had over 60,000 downloads. And we also received a lot of emails from people that it helped them a lot in the project. So for some people, it’s quite useful. Robert [00:40:34]: Check it out. It’s also free by the way, so just google go with the domain and I’m sure you will find that Cool. So I think that will be all for today. So thank you Miłosz. Thank you Robert. See you next time. That’s it.
Learn Building Modern Go applications Dec 19, 2025 -
Here’s a small quiz derived from some incorrect advice from an AI coding assistant. This program prints two timestamps; will they be a. Roughly the same time (ie, the same second)b. Roughly 10 seconds apartc. Something else Answer after the fold
Dave Cheney Dec 18, 2025
-
Learn a practical testing strategy for Vue 3 applications using composable unit tests, Vitest browser mode integration tests, and visual regression testing.
alexop.dev Dec 14, 2025 -
Learn how to display model name, context usage, and cost directly in your terminal while using Claude Code. A step-by-step guide to creating custom status line scripts.
alexop.dev Dec 14, 2025 -
Discusses some design choices used in Jubilant: Python subprocess.run to wrap a CLI tool, a code generator to convert Go structs to Python dataclasses, and the use of uv with a simple Makefile to run commands.
Ben Hoyt's technical writing Dec 13, 2025 -
A practical guide for writing production-quality Vue 3 composables, distilled from studying VueUse's patterns for SSR safety, cleanup, and TypeScript.
alexop.dev Dec 13, 2025 -
Learn how to build an automated QA engineer using Claude Code and Playwright MCP that tests your web app like a real user, runs on every pull request, and writes detailed bug reports.
alexop.dev Dec 13, 2025 -
Quick takeaways Domain complexity matters more than technical complexity - Most projects fail not because of technical challenges, but because they don’t handle the business domain well. DDD is a toolbox, not a religion - You don’t need to use every pattern from Domain-Driven Design. Pick what solves your actual problems. Start with the domain model - Understanding how the business works is more important than designing the perfect schema. Avoid solving imaginary problems - Spending months on frameworks or platforms before building actual features often leads to wasted effort. Strategic patterns are essential for everyone - Even if you don’t use tactical DDD patterns, thinking about core domains and module boundaries matters in every project. Introduction In this episode, we discuss why software projects become legacy code that nobody wants to touch. We talk about how Domain-Driven Design can help, but also why it’s often misunderstood or overused. Instead of treating DDD as an all-or-nothing approach, we suggest to use the ideas pragmatically - picking the patterns that solve real problems in your project. We share stories to show how focusing on domain complexity rather than technical complexity leads to better software. Notes Go With the Domain ebook - our guide to applying DDD ideas in Go The Domain Engineer training - launching early 2026 Domain-Driven Design (The Blue Book) by Eric Evans - the original book that introduced DDD almost 25 years ago Design Patterns book by Gang of Four - classic software design patterns book Event Storming - collaborative workshop technique for understanding domains (covered in the previous episode Quotes The bold assumption for today’s episode is that you can go deep into many of the ideas and they shine in complex projects, but very often you can use some more basic version of the key ideas and apply it to most projects actually. Miłosz If you are hiring specialists in many fields, the sentence ‘if you have hammer in your hand, you will see nails everywhere’ may happen. They will spend a ton of time on solving problems that may be not the problem in this company yet. Robert If you consider most of applications, like SaaS apps, they are usually very similar. If you change jobs, you can end up at another completely different company and use the same tech stack. It’s not much different to use the same programming language. But on the other hand, what the company does can be completely different, and this is the differentiating factor between them. Miłosz The worst thing about that is sunk cost fallacy. So you’re spending more and more time on maintaining that, losing a lot of resources, time, and nerves on maintaining that. But it’s no longer needed. Robert We spent many hours creating this messaging framework using generics. And in the end, we had this good interface that you can very easily publish an event, very easily subscribe to it. And the result was that it was probably the only game jam we never finished. Miłosz You should use proper technique for a proper problem. That’s it. Robert If you boil DDD down to one primary theme, I would say something like it’s about understanding the domain you work with and then modeling it well in the code. Whether you call it DDD or not, it’s like a mindset. Miłosz It’s kind of forcing you to think, ‘Okay, is this code really important to my business, or is it less important?’ And it can kind of affect how we are thinking about this code. Robert Timestamps 00:00:37 - Introduction 00:04:18 - Technical vs domain complexity 00:12:13 - Technology choices 00:16:43 - Framework trap 00:21:12 - What is DDD? 00:31:36 - Core theme of DDD 00:38:22 - Context matters 00:43:01 - Domain model 00:52:00 - Wrap up Transcript Miłosz [00:00:37]: Hello, I am Miłosz. Robert [00:00:38]: And I’m Robert. And this is No Silver Bullet Podcast, where we discuss mindful backend engineering. In this show, we share takes on software engineering that will help you to grow into principal engineer level. Miłosz [00:00:51]: Let’s talk about the usual arc of a software project. Miłosz [00:00:55]: So first we have the kickoff, and now we have high hopes for good design and this product will be great and everything is fine. This time we’ll do it. Right. Yeah. And then we work on it for some time. Then we deploy it to production. Everything is great. And suddenly it becomes a legacy project no one wants to touch. Did you ever see this happen? Robert [00:01:20]: Every time. Okay, not every time, but yeah, it’s very often the case. Miłosz [00:01:27]: So it’s interesting, right? Why does it happen? Why every project ends up as this big ball of mud no one wants to work with. Miłosz [00:01:40]: And the first version works quite well. We come up with something, it’s pretty much what we wanted, that’s fine. Miłosz [00:01:48]: And then new features come in and very often they are quite different than what the software did in the first place. Right? Miłosz [00:01:59]: So we need to change the design. But very often there is not enough time or developers are lazy or they don’t feel like it, whatever. But they don’t remodel everything to fit the new design. Instead, we add some workarounds or hacks and it works pretty well. Miłosz [00:02:20]: For a while, except that maybe some bugs, maybe some unusual behaviors. And we repeat it over and over, over months or years. And then we arrive at the state that we are afraid to touch the codebase. Is there a way to fix it? Robert [00:02:39]: Oh. It. Wouldn’t be surprising to say that it’s a problem that software always had. So probably there are some magic techniques that are trying to solve this problem. One of the most popular, let’s say, tools or techniques that are trying to approach that is domain-driven design. What? Robert [00:03:00]: If you heard about Domain Driven Design, you probably also noticed that for some reason it’s not the default way of many teams to work. From one side we have some solution, from other side probably didn’t solve all the issues that we have. Miłosz [00:03:16]: And it’s also sometimes confusing; sometimes it seems like something you would use only for enterprise projects, but actually it worked for us before. So we like talking about it and sharing the ideas. Robert [00:03:29]: And look how people are surprised sometimes to hear that it worked for us because many people tried to use domain-driven design and it introduced more mass for them. Or maybe they worked in a project when somebody tried to use domain-driven design and it’s the worst experience that they have. Interesting. Miłosz [00:03:45]: Mm. Miłosz [00:03:47]: So the bold assumptions for today’s episode are that you can go deep into many of the ideas and they shine in complex projects, but very often you can use some more basic version of the key ideas of it and apply it to most projects actually. Robert [00:04:06]: So— In other words, in this episode, we will cover How to not use domain-driven design, but some part of domain-driven design and how to make them useful in your projects. Miłosz [00:04:18]: First, let’s think about what is difficult about building software. Miłosz [00:04:24]: So let’s consider someone wants to start. and they want to learn some programming language right, what would you recommend them to learn? Robert [00:04:34]: Maybe not, why, what I would recommend, but what people are using. Miłosz [00:04:37]: Maybe it was a default, like us were here. Robert [00:04:40]: So. They probably need some programming language, right? Yeah. Kubernetes, OOP design patterns. Miłosz [00:04:48]: There are some frameworks, some good practices, some algorithms, maybe data structures, stuff like that. Robert [00:04:55]: Useful stuff that you are using on interviews, in other words. Exactly. And it’s actually, I think, pretty interesting that a lot of learnings nowadays is focused around things that are useful for interviews, but on the other side they are not that relevant when you are working in a job. And I would say that it’s even worse because you are solving problems that normally you should not solve in the problems that you are working like. Robert [00:05:23]: You should probably not implement your own databases and you implement binary trees because most graphics have that out of the box. Miłosz [00:05:31]: But at the same time, if this person wants to learn, wants to join a company, they probably need to learn some computer science theory, like what is big O notation. How to write optimized algorithms for sorting, how to create data structures, stuff like that. Robert [00:05:52]: It’s useful. Miłosz [00:05:54]: Yeah, it is. And it’s often what you need to know to join a company. But my question is, is it enough to ship good software? Robert [00:06:03]: Well, I would say that it’s not enough, and probably it’s even not needed, because well, we are people who are self-taught, so we’ve been learning programming ourselves, and we’ve been shipping multiple projects without that, basically, and many people were using that. They were maybe not perfect, but for the other side, they worked. Miłosz [00:06:27]: Yeah, so I would say it’s like half of the skills you need. Miłosz [00:06:32]: And it’s useful, it’s needed. It’s often what we focus on. But there’s also the other half of stuff you need— to ship good software. So what we discussed so far is the technical complexity, right? So making software fast and scalable, and optimizing memory usage. Miłosz [00:06:54]: Stuff like this. Robert [00:06:55]: Probably we can put it into the bucket of software engineering. So this is this hard part. Maybe not hard, but this. Yeah, this low-level part maybe. Yeah. Miłosz [00:07:06]: The coding part. Miłosz [00:07:09]: And the other half, how would you call it? Robert [00:07:14]: So if you are working with product, probably product engineering. Miłosz [00:07:18]: Yeah. So the product or domain challenges. So basically figuring out how the thing should work and what should it do. Robert [00:07:29]: So this product engineering or those domain challenges, it’s less about those technical parts, but rather thinking. Robert [00:07:42]: How to tackle complexity, how this feature should work, maybe even thinking if it will be useful. Because if you know if something is not useful, you can remove some functionality and decrease complexity with that. Robert [00:07:58]: It’s also this: less priced part, let’s say, and that it’s connected to communicating to people. Robert [00:08:06]: Yeah, figuring out together what you want to build. Miłosz [00:08:11]: Maybe also talking to your users? Robert [00:08:13]: I think it’s also partly related to some long-term maintenance. So it’s, I would say, partially part of the software engineering, but I think it’s often lost somewhere because, when you have software that is not that maintainable from other side, it’s also challenging. And I would say that a lot of people that have this enduring mindset, they like challenges. So if software is complicated, it’s great because we have a lot of challenges there to solve. Miłosz [00:08:45]: Yeah, the technical parts are very interesting to solve. Miłosz [00:08:49]: We like puzzles. At least I know I like them too. And the technical parts often have a very specific answer to them. Mm-hmm. Miłosz [00:09:01]: For example, we can have an SQL query, you can benchmark it, and then you can come up with a faster query. And it’s very easy to compare if it takes 2 seconds or 10 milliseconds and it feels great to have this measurable progress. But for many businesses, it’s not really critical at all. If your website handles a request in 50 or 70 milliseconds— or whatever it is— usually not a huge difference, except for the first page load, which is critical. But for many web applications, saving some data takes one second longer or shorter; it’s not a huge deal and probably not worth spending days on. Miłosz [00:09:52]: coding on. Robert [00:09:53]: As long as cost is not the important part, but again, it’s also some subset of products. For many products, it’s not that important at the end. Miłosz [00:10:03]: Exactly. Miłosz [00:10:05]: But for most businesses, the hard part is taking the problem domain they need to solve and representing this in code. Miłosz [00:10:17]: Especially if the problem domain itself is complex. Miłosz [00:10:23]: And usually the real world is messy in itself. Right. So we have no easy way to compare if something works or not, like with the technical parts. We often need to experiment, see if something sticks, see how users use it. Robert [00:10:47]: Basically, the feedback loop is much, much longer and it’s not that obvious. Like you said with SQL query, it’s easy to measure if SQL query is faster or not. But obviously you can measure some customer satisfaction or some feature, but it’s more indirect and it’s usually taking a bit longer to get back to you. And it’s also a bit harder to understand sometimes how it’s happening because you can change some functionality. And notice that, okay, some behavior changed, but sometimes you may have that many changes around it—so it’s hard to judge if it’s because of it, actually. Miłosz [00:11:28]: Yeah, so it’s much more vague. And if you consider most of applications, like SaaS apps, they are usually very similar. If you change jobs, let’s say you used to work on web application using Go and TypeScript. Miłosz [00:11:50]: You can end up at another completely different company and use the same tech stack. It’s not much different to use the same programming language. Miłosz [00:12:00]: Same framework, basically, protocols, databases are very similar. They don’t change much, but on the other hand, what the company does can be completely different, and this is the No. Miłosz [00:12:13]: the differentiating factor between them. It’s sometimes funny how companies try to make their technical choices something that makes them distinct. So I don’t know if you remember— About 10 years ago, there was this hype on MongoDB, for example. Oh yeah. Miłosz [00:12:32]: And some companies were very open about it. Robert [00:12:39]: Proud. So when you’ve been sending CVs and it was the job posting, it was ‘Yes, we are using MongoDB. Yes, we are web-scale, but of course, it did not really matter in the end. It’s so absurd. But it’s also interesting because during that time, some people were saying that it’s absurd, but from other sites, many people were following that and saying, ‘Yeah, it’s so great.’ You’re using MongoDB, your web scales. Robert [00:13:08]: Let’s see what we’ll see 10 years. Miłosz [00:13:12]: You can just as well use JSON columns in Postgres and achieve pretty much the same thing, except for some differences in very specific cases. Robert [00:13:22]: Or losing changes because of inconsistencies between nodes. Miłosz [00:13:28]: Yeah, but basically, if you take two companies that do the same thing and one uses one database and the other MongoDB, whatever. Miłosz [00:13:39]: It probably won’t impact that much the end result. How well they do in business, how many users they have, how good the product is. Miłosz [00:13:49]: There’s a lot more factors in it. Robert [00:13:52]: Okay. In other words, we can say that many companies, many people maybe solve on the wrong thing because when starting your project, they’re thinking, ‘Okay, this time we’ll use this framework, this database.’ Miłosz [00:14:06]: Yeah, very different this time. Robert [00:14:08]: Yeah, yeah. Robert [00:14:10]: This time everything will be great. And again, we are in the cycle of starting in the greenfield and ending up with legacy, like: It was done five years ago, 10 years ago. So probably. Robert [00:14:28]: The technology that we’re using— the framework, database, maybe not the most important indicator if project is successful or not, and by ‘successful’ I mean multiple things. So, if it’s maintainable, if it has users, if it makes money— probably from perspective of developers, the maintenance is the most important part. But again, it’s not the only thing. Miłosz [00:14:54]: Yeah, but we treated that as something that’s a core thing we need to solve. Sometimes. Right. Miłosz [00:15:03]: I remember some companies with the idea that we will dedicate top talent to this core platform framework— or something— and once we solve it, then adding new features and apps to it will be trivial. Just hire anyone who’s able to code in this language, and they will just use this great framework to implement a new feature in one day. Miłosz [00:15:34]: I don’t know where this idea comes from, but I think it’s quite common. Robert [00:15:40]: And I think one part of the problem may be that if you are hiring specialists in many fields, the sentence ‘if you have hammer in your hand, you will see nails everywhere’ may happen. I mean, if you find hiring specialists from multiple areas, they will spend a ton of time on solving solving problems that may be not the problem in this company yet. So, for example, this startup and it’s starting. They don’t have scale yet. And probably it could be run on a single VM. And later, you have, for example, some Kubernetes specialists. So what they can do, they can create multiple Kubernetes clusters for multiple environments and later serve. Requests per second and even 10 requests per second. It’s like nothing. You can do it with a single VM, but it’s not for free. Obviously, you are using your top talents, but for solving problems that are not important. Miłosz [00:16:43]: And I remember myself falling into this trap, probably multiple times. But one story that I remember very well is how I used to run some game jams with friends, just making game in two days, right? And one time we wanted to create a story-driven game with a lot of dialogues, so you can choose dialogue options and it leads you to a different part of the game. Somehow we came up with the idea that we will do it event-driven. So each option will be an event that sends and triggers something. We had this idea that, now, this would be a lot of events, which will be hard to manage. So we will need this even during the framework. So, if only we get this framework right, all that’s left is to implement the dialogues. Miłosz [00:17:30]: That would be it. So we spent many hours creating this messaging framework using generics. Miłosz [00:17:40]: C# and I didn’t work very well. So it was super fun to come up with it. And in the end, we had this good interface that, you know, we just, you can very easily publish an event. Very easily subscribe to it. And the result was that it was probably the only game jam we never finished. We had this technical solution, but we didn’t progress with the game at all. Robert [00:18:05]: So in other words, you are not the best specialist to be hired to build event-driven games. Exactly. Exactly. Robert [00:18:12]: I can create a framework for you. Miłosz [00:18:14]: I think it shows. It’s like a very condensed, you know. Scenario that I think happens on bigger scale as well, and I don’t even know where it comes from, but somehow we tend to think, if only we can create this framework first, and everything will be easy. Robert [00:18:36]: And sometimes, if you succeed in creating this framework or whatever it is, this prerequisite, and you’re starting to rework on the things that matter. Robert [00:18:47]: You’re starting to notice that, okay, actually, it’s totally not needed. Robert [00:18:52]: We just spent two months on that. And you know what’s the worst thing? The worst thing about that is sunk cost fallacy. So you’re spending more and more time on maintaining that, losing a lot of resources, time, and nerves on maintaining that. But it’s no longer needed. Miłosz [00:19:12]: Yeah, and you don’t want to get rid of it because you invested so much time. Yes, I think we’ve seen many examples like that, right? I think it’s very common when you have a platform team in a company that is supposed to make all teams work together. Very similar ways this is exactly the idea of having one great system that you just add tiny pieces to, and of course it makes sense for a bigger company. But sometimes you can get go too far. I remember one time we had the platform team create service mesh for developers. To use it, you have service discovery so you can communicate between services using just service names. Sounds great, and they spent half a year working on it. But first of all, no one asked for it. Miłosz [00:20:07]: And we really didn’t have any use for it later. It just looked great on paper. Robert [00:20:13]: I think we even mentioned that. Yeah, but maybe we don’t need that because we already are doing it in Middleware. Miłosz [00:20:21]: Maybe if you have five services, maybe you can just call it with the URL of Kubernetes and wow, it’s enough. Robert [00:20:29]: Yeah, and all the cross-cutting concerns could just handle it with common library and middlewares. That’s it. It’s probably again, it depends on context. If you have a super complex project with multiple programming languages, maybe. But in this case, we just have one technology there and it solved the problem already. Miłosz [00:20:52]: So this is all the first part, the technical part you mentioned. Sometimes it’s easier than the domain part, the product part. Robert [00:21:00]: And it’s very often. You can move it between projects as you said earlier. So it can be very similar across and you don’t need to reinvent the wheel basically. So we already know that this technical part is very often similar between the projects and the thing that is different is the domain. Robert [00:21:24]: part of the project. And we already mentioned once domain-driven design, that it’s supposed to be helping us with this. Robert [00:21:33]: Think that is unique to our company. and many of you, maybe heard about this earlier, but it’s not. Robert [00:21:43]: wide load up. do you have some ideas why it’s like that? and actually, what the point of indesign at the first point? Miłosz [00:21:51]: maybe it’s even difficult to define what the approach really is in a very short description. and i remember before i knew about the dd. i think my first contact was when my manager wanted to have the entire team join the training for off-domain design. Miłosz [00:22:17]: and they sent me over like a pdf description of it with lots of details what the training is about and i remember reading through the document. Miłosz [00:22:29]: and at the end i was like, what is this? i have no idea. How is this supposed to help me with my day-to-day work? I just completely had no idea what. Robert [00:22:41]: What’s the idea? So it was more covering some non-technical things and at this point you were thinking, ‘Okay, it doesn’t matter’ or ‘it was not sold properly,’ let’s say. Miłosz [00:22:51]: Yeah, I think it was missing the ‘why.’ And I think also DDD has a lot of unique names or terminology that you didn’t hear before. So if someone tells you, you will learn about bounded contexts. You’re like, ‘What?’ Robert [00:23:10]: I know about design patterns, but what is this? Or maybe aggregate and, oh, okay, so I can aggregate some data. Robert [00:23:20]: Okay, another reason why it’s not that widely adopted, and maybe now it’s very often missing interpret. It’s also the reason that point-driven design is out there already for a pretty long time. So the blue book, so the book that was introduced it. Introducing Domain Driven Design was released almost 25 years ago and the programming world 25 years ago was totally, totally different. Miłosz [00:23:54]: And it was based on years of experience from Eric Evans, so I guess you can say it was about software written in the 90s. So it sounds like a different era today, kind of. Miłosz [00:24:09]: But on the other hand, you can still hear about the DD a lot. Maybe not every company practices it, but there are conferences and books dedicated to it. Miłosz [00:24:23]: So, probably suggest to you that there are many good ideas. That’s how evolution works. Good ideas tend to survive for longer. Robert [00:24:32]: Unfortunately, there’s also a side that many good ideas are misunderstood with time. So some good ideas are useful at the beginning. More people are trying to adopt them. But unfortunately, if those good ideas are not straightforward, but require bigger investment to understand them. Some people tend to get shortcuts and not use them as intended or use them everywhere and it’s quick ending up with some kind of caricature of some techniques, tools. So in other words, many techniques or tools at the beginning were good ideas and were solving some problems, but with time, when people were adopting that, they were starting to adopt some caricature of this technique and it’s often started to do more harm than good. It was visible in discussions that we have in many teams that some people have seen domain-driven design in some projects, but it wasn’t really domain-driven design what they see. Robert [00:25:38]: They’ve seen some overengineering, maybe, just doing it for the sake of doing it. Yeah, so using it in places when it wasn’t needed. So it was just too simple application maybe to use that. And to make it worse, it wasn’t used in there. Right way, because the initial investment was just a big end. People tend to not spend enough time to learn something and, Well, it’s easy to imagine that after 25 years, many, many of projects like that was created and it created a lot of bad PR. Miłosz [00:26:17]: And you can hear people saying, ‘I don’t want to work like this ever again,’ because it was a disaster. It was a super complex project that was made overcomplicated. Robert [00:26:28]: The similar situation is also, for example, with clean architecture, when we sometimes hear that, ‘ah, no, clean architecture doesn’t work,’ and later we hear about seven layers and. Miłosz [00:26:39]: Yes, but on the other hand, if you read the introduction to the Global. I think the core ideas are still very relevant. The introduction describes the state of most software projects. How they get stuck, what are the issues. And now it’s almost 25 years later, everything pretty much in the same spot. Miłosz [00:27:08]: Sense. We got better with frameworks and some technical stuff, but the core idea of working with people on the product. Miłosz [00:27:21]: Not that much different. Robert [00:27:22]: Are you suggesting that we are reinventing the wheel in software every 10 years? No, it’s not possible. And it’s 25 years during. since releasing the book. I can imagine that we reinvented the wheel at least twice since that time. Miłosz [00:27:41]: microservices and then ditched them. Twice. It was a full cycle. For example, in the Eric Evans book, he says that XML would be a promising technology to communicate between services. Yeah, it’s something fresh to send XML over RPC. Yeah, so since then we probably reinvented this idea a few times. We’ve. Miłosz [00:28:10]: microservices and protocol buffers over gRPC. It’s still the same. Coming back to the question why it’s difficult to understand what DDD is, it’s very broad. It’s like an entire book of many, many ideas. and from design patterns to communication between teams. So it’s very difficult to say what doing GDD means. It’s just very broad method. Robert [00:28:43]: We can also say that it may be philosophical in some way. Hmm, hmm. Miłosz [00:28:48]: Yeah, and you can interpret the patterns in many ways. Robert [00:28:54]: The domain part of projects can differ between projects. So it’s hard to give one solution that fits all. So it’s probably because of that, but it’s also creating a lot of space to interpret things. Hmm. It remains one thing. If you are with us for longer, you maybe already know that some time ago we decided to leave our daily jobs. And now we are doing our own business. But it’s not the first time when we’ve been trying to do that. And we’ve been trying to do it multiple times earlier. But I think. One thing that probably didn’t allow us to succeed earlier was that we didn’t spend enough time to understand actually how to run business. Robert [00:29:43]: How it actually worked that you’re running a successful business. So maybe you had some idea that if you would like to have a successful business, you need to sell things. Uh, do some marketing, do some sales, and earn more than you are spending. Miłosz [00:29:58]: Easy, but just— you just need to read one business book, yeah, which is called ‘Business’ and explains how to do business. Robert [00:30:07]: Yeah, but in practice, it’s not that simple and it’s more sophisticated, let’s say. And if you will not spend enough time to understand how it works deeper. Robert [00:30:20]: The chance that your business will fail is much, much higher. I think the statistics are like in five years, 50% of businesses are failing, and in terms of startups, it’s probably 80% in Twitter or something like that. So it’s a crazy number actually. Robert [00:30:38]: And on the other side, it’s something that you can learn, but you need to spend a bit more time to learn those principles and to not do. Caricature of your business. Miłosz [00:30:50]: It’s similar a bit to DDD. It’s a very broad term and a general approach to solving and you need to learn many patterns. Robert [00:31:00]: And it’s hard to give one advice that will fit all cases. But it’s doable, but you need to spend some time and keep in mind that many, many, many people are not doing this exercise and are later surprised that it doesn’t work. Robert [00:31:18]: Like with DDD, you can also see that many people are saying that running business is super hard. Robert [00:31:23]: So I would say that it’s hard, but I think it’s also not. That hard that people think. It’s just about knowing the principles and spending enough time on it. Miłosz [00:31:36]: Right, so if you were to boil down DDD to one primary theme, I would say something like it’s about understanding the domain you work with and then modeling it well in the code. Whether you call it DDD or not, it’s like a mindset. More than patterns, maybe? Robert [00:31:57]: In other words, you should always do that. I mean, if somebody will tell you that, okay, you should not think about the domain, what problems you are solving in your software and how you are writing in an account like it sounds crazy. Miłosz [00:32:12]: Yeah, it’s like you’re deciding. Yeah, I will create a website. I just don’t know which one. So it sounds like you do something for the sake of learning the technology, which is fine. Sure. But if you want to create a product, it’s not really helpful. Robert [00:32:32]: My observation is actually that dark. Robert [00:32:37]: Two kinds of people that are often visible in the area of techniques like domain-driven design, architecture, take anything. Robert [00:32:46]: There are two comps that are totally opposites. One comp is saying that you should always use that. Because it’s great, etc. And the second is you should never use it. That is the worst idea in the world. And I think it’s pretty interesting that even if they are opposite, they have many in common because they are extreme in some way. Miłosz [00:33:06]: So they treat it more like kind of religion rather than something you want to use to deliver a product. Robert [00:33:17]: And in practice, probably you should be somewhere in the middle. Robert [00:33:23]: It may sound obvious, but you should use proper technique for a proper problem. Robert [00:33:29]: That’s it. Robert [00:33:31]: It’s important when you see some religious people about techniques or tools that are against them or love to use it always that it’s the same coin, but just other sides of it. Robert [00:33:46]: I think it— gets into one thing that you should watch out to not use any techniques or tools like DDD just for sake of using it. So you should identify the problems that you have and use the right tools for that. And the Winterwind design have couple tools that you can take out of that like a tool from a toolbox and just use what you need from that. Miłosz [00:34:12]: There’s probably no project that will use every pattern described in the book. Robert [00:34:16]: Yes, but probably you agree that every project should use some parts of domain-driven design. So it may sound controversial, but it’s getting into a point that you may be using some parts of domain-driven design. Even without realizing that. It may be because of reinventing the wheel. Again, it’s probably easier to just see that, okay, you have some tools like that, but many of them, some people just can find them, yeah, probably except for very trivial projects, but we don’t care about them here. Miłosz [00:34:51]: Like, if you create something that’s very trivial and doesn’t need any kind of design and architecture, and you probably don’t need to figure out how to do it better. Miłosz [00:35:05]: So we don’t worry about this here. Miłosz [00:35:09]: Maybe also something very technical like Linux kernel. I don’t know. But probably also there should be at least one idea from DDD that applies there. Maybe value object. Miłosz [00:35:21]: Yeah, but in general. Miłosz [00:35:24]: If you boil DDD down to some very key ideas, I think you should be able to use them if you know what you’re looking for. Robert [00:35:34]: I think it’s a similar situation like with the book Design Patterns. So pretty old one. I don’t remember actually how many years. It’s older than DDD. But I think 96 or 97. So still, pretty old book, but still pretty relevant and many of us may be using patterns from this book. Even without knowing that those are patterns from this book, because it was used by somebody else and we learn those patterns by replicating them. But still. We may be missing some of the context maybe of them. And if we read that, maybe we could understand them a bit more and apply them in the better situations also. Miłosz [00:36:15]: I remember there was a joke or people making fun of people in interviews. When asked about design patterns, they will tell something like, ‘Oh, yeah.’ There’s the singleton, and that’s it. But that’s also kind of the point. You know, there’s an entire book, and doesn’t mean you need to know all the patterns and use them all in a project— even if you learn about some, like maybe you can use a strategy here or decorator in another place. I think it’s still a good result. You don’t need to force them and rewrite your entire project to use, I don’t know, abstract factory method or something. Robert [00:37:00]: It’s always about identifying some problem that is really a problem, and not solving. Robert [00:37:08]: Some imaginary problems. Yeah. Miłosz [00:37:10]: Especially if you force the patterns. I think that’s the worst thing that can happen because it just leads to overengineering. You just read the book and you like the ideas, so you use them everywhere in your next projects and of course they don’t apply to all of them. Robert [00:37:27]: And I think it. It plays nicely with a couple of things that we covered in the previous episode that we’ve mentioned: that being a principal engineer doesn’t mean that you’re forcing to use some patterns. You should instead be able to convince people what value those patterns give and and convincing them that it’s worth using some patterns there. Hmm. And at the end of the day, it’s also a matter of solving your problems, not imaginary problems. Miłosz [00:37:59]: There is no one solution to suggest. Because there are so many factors in play, like the size of your team. If you are a 20-person team in a corporation, a developer working on open source software, or maybe you are a five-person startup, you will probably use a very different approach to software. Robert [00:38:22]: And also, what’s the timeline for the project? Because if you are in a startup that you’re starving for, if you don’t know if you will survive half a year, and you’re doing refactoring, it’s probably not the best idea, but I mean, it depends, obviously, but it’s again going back to solving real problems that you have. So, if you are a startup that has a half year of runaway and you’re fighting for survival, you should choose your fight wisely, basically, and not put maybe refactor some code. That you didn’t touch for a year and put design patterns there. But maybe you have some part of the application that, for example, you are doing a pivot now and you need to— it quickly, but you’re wasting a ton of time to modify this code, maybe doing this refactoring and putting those design patterns may be good at the other, even if you’re fighting for survival. Robert [00:39:26]: Because maybe it may allow you to build this new functionality or do this pivot within this time that you have left. Miłosz [00:39:33]: Have some idea that this project will be for you longer. You probably need some better maintenance here. But but maybe some some areas are more less important. Maybe you will get rid of them in two months if it doesn’t work out. But because it’s sometimes difficult to decide. Miłosz [00:39:57]: This might be a bit controversial, maybe, but I think you don’t need to go all in. Most of the patterns from DDD you could decide to use them in a more, let’s say, shallow version, other than deep version. So you basically think of the the first principles, the big idea behind the pattern. You don’t need to implement all of it and still get some good results. Because you often don’t know if the project you work on will be with you for longer, or maybe it’s just a prototype. Miłosz [00:40:33]: But I think there’s a balance between having some very scrappy code and something that’s very enterprising. While design software. Robert [00:40:44]: It’s actually an interesting thing because it’s one of the things that Domain Driven Design covers with core and supporting domains. Robert [00:40:56]: It’s kind of forcing you to think, ‘Okay, is this code really important to my business, or is it less important?’ And it can kind of affect how we are thinking about this code. Robert [00:41:09]: I know that many people like consistency in the code, like, ‘Okay, let’s do.’ Something in the consistent way everywhere. End. It may sound like a decent idea, let’s say, but it may be also dangerous because you at some point start to over-engineer some parts because some parts of the application are more important, more often touched, more complicated, and some are simpler. You may just lose a lot of time for maintaining this consistency for sake of nothing. Basically. Miłosz [00:41:44]: So this episode is more like an introduction, but in the next episodes we want to cover more patterns from both shallow and deep end, so you can better decide what to use. Robert [00:41:58]: But I think we should put big emphasis on the thing that people are very often missing. So the strategic part, because Tactical patterns— those strategic domain-driven design— are the patterns that are not that much about how you write the code, but it’s more about strategic thinking, like how the modules are interacting, how modules are split. Very often cross-team collaboration. Robert [00:42:28]: How important are multiple parts of the code? And what’s interesting, everything what I just said, I think you should probably think about that in any project that you are doing. You’re using domain-driven design or not. It’s cool to have some tools that are helping you to do that, and some frameworks— let’s say it’s not a framework, but some way of thinking how to approach this. And from other side, you have technical patterns that you’re using in the code, and those ones are not always needed. And you don’t need to always apply them. Miłosz [00:43:01]: So we won’t be diving deep into patterns today, but let’s look at the core idea of DDD. Miłosz [00:43:12]: The domain model or model-driven design or knowledge crunching, as it’s specified by the book. What is it about? Robert [00:43:23]: There’s one thing in multiple projects that is a bit like running business, like we said earlier. So it’s obvious that you should earn more money than you are spending. so you have a successful business. In software, it’s a matter of understanding what kind of problem you are solving and how it should work. Robert [00:43:45]: It makes all of us like, yeah, we are starting a project. We should understand how it works and how we should implement that. From other sites It’s not happening. Miłosz [00:43:59]: yeah because sometimes the business is complex by itself and even for simpler ones like consider e-commerce seems like a simple domain it’s just people People buy products and that’s it. But if you dive deeper, there’s capturing payments and delivering the products and getting the inventory somehow. So it’s more complex than it seems. Robert [00:44:24]: How it looks in practice? People are thinking, ‘Oh, okay, it’s simple.’ We know what to do. Let’s go to implement that. Exactly. Two years later. Miłosz [00:44:33]: Consider, if you wanted to to open an online shop. Miłosz [00:44:38]: Oh. If you ask a software developer, they will probably tell you, ‘Yeah, you just need to host this WordPress website. With an e-commerce plugin and you’re done.’ Robert [00:44:52]: I think many software developers will say, ‘No, no, we need to implement it from scratch.’ Miłosz [00:44:57]: Oh yeah, that’s even better. But my point is, they probably won’t think about that— that you need to. Order the inventory from somewhere, store it somewhere, and then somehow send it out to people, which is the hard part of the business and your software probably needs to work with it somehow. Robert [00:45:16]: It actually works nicely with this part of strategic patterns of domain-driven design when we are considering what’s our core domain and what’s the value for our company. Miłosz [00:45:27]: And many projects are much, much more complex than e-commerce. If you consider accounting, for example, you can get into some very very specific edge cases depending on the country and if you do do it globally, then it’s yeah, it’s crazy amount of knowledge you need to understand and to implement. Miłosz [00:45:48]: Complexity in those projects doesn’t really come from the technical challenges, but from the business itself. It’s just complex to run this kind of business, so the software becomes also complex. Robert [00:46:03]: So, in other words, you have multiple dimensions, complexity dimensions, let’s say, to this product, and you can add more and more and more. So, for example, you’re starting with one country. Right. We need to scale globally. Okay. We need to support multiple countries, multiple continents. Great. Later, we need to support multiple types of product. So it can have multiple tax rates and multiple rules. It’s getting more and more and more complex. Miłosz [00:46:28]: And it’s always complex even before you introduce any kind of technical challenges. Just not considering software at all. This is already complicated. Miłosz [00:46:39]: So you want to model it well in the code, so at least you don’t need to deal with the additional complexity. Robert [00:46:51]: And to make it worse, often the initial idea is also changing with the time. So I have two good examples of the companies that were like that. So Netflix. Robert [00:47:01]: Maybe many of you know that Netflix at the beginning was an extreme platform. At the beginning they were just renting DVDs. So it’s interesting how it changed over time, how they were making business, and probably how much it changed how they’re operating. The other interesting business that changed over time is Instagram. Instagram wasn’t this kind of social media platform that you know now; it was more checking location applications. So I guess you were able to send people information like, ‘Hey, I’m here’ or something like that, not share pictures. Miłosz [00:47:40]: Yeah. And so imagine in both cases. Miłosz [00:47:45]: At some point, developers complained that the requirements weren’t clear. We didn’t know this component would become this thing it is now. Of course, it wasn’t clear because no one knew about it back then and they had to adapt somehow. Robert [00:48:00]: Those examples were more extreme, let’s say, but most companies are changing their initial assumptions pretty much, so it’s also adding this complexity there. Robert [00:48:14]: I also like example of the cyberpunk game that the cars are horses from Witcher 3 because they’re using the same game engine. Miłosz [00:48:25]: Yeah, so that goes back to the discussion of evolving software. The first version is usually simple, right? Everything works fine. But then someone tells you, actually, we’re not renting DVDs. We’re going to be streaming videos. Miłosz [00:48:42]: And at this point, you have to change your software to a completely different model sometimes. Miłosz [00:48:52]: It’s great if you could do it easily, if you don’t need to spend one year to rebuild everything. What helps is also what we mentioned in the last episode about developers working directly with domain experts. Robert [00:49:06]: So even if it’s a bit controversial, so we’ve seen how it can change things and again affect how you are implementing things. Miłosz [00:49:17]: So we don’t need a product owner or product manager acting as a proxy? Robert [00:49:22]: Always, because sometimes it makes sense, but sometimes it makes sense to just go direct. Miłosz [00:49:27]: But the idea of the domain model is to create a model that represents the problem domain as well as you can at the moment. Miłosz [00:49:37]: Developers can understand how it works, the more information they get from the people who work directly with the business. The better. The model doesn’t need to be physical, so it’s not a document or diagrams or anything else, although it helps, but it’s more abstract. It’s about understanding how it should work. And. Miłosz [00:50:03]: The ultimate artifact is the code. Robert [00:50:06]: And it also doesn’t need to cover everything. So I always like to compare this model part to a map. So map is not a perfect representation of real world, but it’s a useful representation of the real world. So you can look on the map and see the details that you care about and do the job that you need. Miłosz [00:50:28]: And it’s also not a data model, which is a common misconception. Miłosz [00:50:35]: Very often in software you can see models used for both the domain logic and some storage, for example, or transport. Miłosz [00:50:49]: For developers this is unnatural. Miłosz [00:50:52]: No. thing to do treat them as the same so we often start implementation with database schema, for example, because it feels natural to think of storage. Miłosz [00:51:03]: But here we think about the more abstract domain model, how the thing works, what behaviors it has. Robert [00:51:11]: So this is pretty big part. Paradigm shift and it’s also not consistent with how we’ve been learning earlier. So this is also the theme that was pretty common in the last two episodes. But how we are learning to code is very far away from the real world. And we are not prepared actually to work in the companies. And later it’s ending up with the. Robert [00:51:37]: golden cage concept from the previous episode. Getting out of that is not simple, but I think it’s mandatory if you would like to be successful and survive on the tough market and work in the better places. Miłosz [00:51:52]: Another tool from the previous episode is event storming, which is also closely related to figuring out the domain. Robert [00:52:01]: So today we’re more focusing focusing on giving you some introduction. Robert [00:52:09]: To— Maybe not Domain Driven Design really, but some parts of Domain Driven Design that we’ve been using in the project. Sometimes we’ll name it Domain Driven Design, sometimes not, because again, its patterns are secondary. I think it’s more important to focus on problems, not on the patterns, because when you are focusing on patterns, you are just with this hammer in your hand and looking for nails everywhere. So I think it’s more important to focus on problems that we’re solving and how we can do that. And yeah, this is something that we’ll be doing in the next couple episodes and it’s connected to the fact that now we are working on our new training the domain engineer and that will be available at the beginning of next year in January. Miłosz [00:52:56]: Yeah, so in the next episodes we will look at more use cases for the strategic patterns. Robert [00:53:02]: Yeah, so we’ll give you some more practical tools also, how to approach that. Miłosz [00:53:08]: So let us know in the comments if you have any topics you would like to hear about. You can also check our ebook, ‘Go With the Domain,’ which is about working with domain-driven ideas. Miłosz [00:53:22]: In Go specifically. Robert [00:53:24]: Yeah, so we will be preparing agenda for next episode soon. So let us know on YouTube comments or Spotify comments. So read all of them and it will be. Cool to help you with some projects or problems that you have. We have many, many, many stories from projects that we’ve been working in, but we’d also love to introduce a bit more there. Miłosz [00:53:50]: So that’s it for today. Miłosz [00:53:53]: Subscribe to our channel so you can see the new episodes as they come. Miłosz [00:53:59]: And hit that like button. Robert [00:54:01]: And if you don’t want to miss when the domain engineer will be available, just Google the domain engineer and join our newsletter if you are not there. Yet. So we’ll send you an info when it will be ready. And you will also notified when new episodes will be out, because we know that sometimes notifications from multiple platforms are not sent. So when you’re in our newsletter, you are sure that you will always get the notification. Robert [00:54:29]: Okay, so I think that’s all for today. So thank you, Miłosz, for your time. Miłosz [00:54:33]: Thank you, Robert. See you next time. Robert [00:54:35]: See you in two weeks. Bye-bye.
Learn Building Modern Go applications Dec 10, 2025 -
Build a custom TDD workflow with Claude Code using skills and subagents that enforce Red-Green-Refactor discipline for your Vue projects.
alexop.dev Nov 30, 2025 -
Here’s a silly example extracted from real code. Does this program print true or false?
Dave Cheney Nov 27, 2025
-
Quick takeaways Developers are often kept in “golden cages” - treated as coding monkeys receiving tasks without understanding the broader context or product vision Requirements are usually incomplete or wrong - understanding the business context helps developers spot issues early and deliver what’s actually needed Collaboration with stakeholders is crucial - developers have valuable product insights and should be involved in planning, not just implementation Event Storming is a powerful collaboration tool - a lightweight workshop technique that brings developers, product managers, and stakeholders together to align on complex topics Change takes time and trust - transforming company culture requires starting small, showing results, and convincing people through solving their actual problems Introduction In this episode, we talk about why software projects in regular jobs are delivered much slower compared to side projects, and what you can do about it. We share our journey from building hobby projects as teenagers to working in professional environments, and the differences we encountered. Developers are often kept isolated from product decisions and treated as “coding monkeys in golden cages” - just receiving tasks without understanding the why behind them. We discuss techniques like Event Storming that can help break down these barriers, and improve collaboration between developers and product managers. Notes Event Storming - a workshop technique created by Alberto Brandolini for collaborative domain modeling Quotes Five days of coding can save one day of planning. Robert If you want to be promoted, for example, delivering is the very best way to do it. So instead of just waiting for requirements and complaining that it doesn’t work, you can try to help whoever needs what they need. Miłosz How it often looks like: it’s like companies are trying to make the programmers coding monkeys in some golden cages, basically. Robert Instead of complaining that we never get anything done, you tell them, yeah, this feature could have been one week, but we spent two months. Miłosz The worst thing that we noticed is that the performance of how fast we’ve been able to deliver new stuff was much worse. Robert But what if instead, you have this domain expert, this stakeholder meet with developers directly and tell them what they need. And then usually developers will tell them, oh my god, this is so complicated, it will take two months. But if we drop this one button from the dashboard, it will take one week. Miłosz Timestamps 00:00:00 - Introduction 00:01:11 - Our story 00:05:06 - Slow projects 00:07:10 - Golden cage problem 00:09:30 - Incomplete requirements 00:13:14 - Developer time optimization trap 00:15:15 - Level 2: Collaboration 00:19:09 - MVP and simplification 00:23:33 - Talking to stakeholders 00:28:40 - Levels of engagement 00:30:29 - Why you should care 00:34:30 - Event Storming 00:44:37 - Getting started 00:50:17 - Practical advice 01:02:22 - Changing culture Transcript Robert [00:00:59]: I’m Robert. Miłosz [00:01:00]: And I’m Miłosz. And this is No Silver Bullet Podcast, where we discuss mindful backend engineering. In this show, we share takes on software engineering that will help you grow into a principal engineer. Robert [00:01:11]: In this episode, maybe let’s start with a bit of our story, because I think it’s pretty important to understand that, how we started to build some production systems and from where our experience came. So probably some of you know that we met with Miłosz in the high school, so a pretty long time ago. We were just talking before recording. That it was, I think, 17 years ago, so a pretty long time ago. Actually, I know Miłosz two months longer than he knows his wife. Miłosz [00:01:42]: That’s pretty long, and the reason I know you longer is because we are already working on projects before everybody started. So we started with some toy websites and then grew into more complex projects. Robert [00:01:59]: And many of those projects actually became pretty successful, so many people were using them. The downside was that, well, there wasn’t great business sense at the time, and well, a lot of people used that, but it wasn’t enough for making for a living of that. So we decided, okay, let’s go to university. Maybe we’ll learn something useful. Long story short, we did it. So we left the university at some point. So we quickly dropped out. Yeah, because, well, it was probably a bit boring since we’ve been already coding for multiple years and we’ve seen that, OK, maybe we’ll learn physics and mathematics, but… That’s not probably relevant in the software that we are writing. Miłosz [00:02:41]: We still work on our projects in the meantime, but there was almost no money coming from it. We had to just start regular jobs, which was fun at the time, because we didn’t have this professional experience, let’s say that’s our own. Robert [00:02:55]: I just remember that. During the time, the market also wasn’t that easy. I mean, we’ve been over decoding for a couple of years already, and I was already afraid that I will need to go back to my parents’ home because I I struggled a bit to find a job, despite having a couple years of experience already. Miłosz [00:03:11]: Even the basic wage we made back then was great for our student life, compared to hobby projects. Robert [00:03:21]: That’s true. But I think it wasn’t worth thinking about that because, since we’ve been working already on multiple projects, we’ve been working, so we have two people— team basically, so me and Miłosz. It was a bit shocking to start to work in the normal job, let’s say, let’s name it in this way. So, you know, in the way that you are no longer your boss, but instead you are getting requirements from somebody else. You are not inventing the project and product that you are building. It’s totally different dynamics. And for some people, I know it’s fine because they don’t need to touch this ugly business and product they can just code. But we’ve been different. We’ve always wanted to create something and invent something. And it was a bit shocking. Miłosz [00:04:08]: The upside is you don’t need to worry about… If the mana is coming through. Someone else handles that for you. But alongside this, someone tells you, here’s this system that has been around for 10 years and you need to add some feature to it. And usually it’s not an easy task. Robert [00:04:27]: Yeah, yeah, exactly. And you know, you have other challenges, like you need to collaborate with more people. So it’s no longer two of us, but the team is bigger, you need to take maintenance into account more because there’s 50% of a chance that it will be the person that will be maintaining it later. In this case, it’s totally different because you’re working with someone’s legacy code that some person who worked here 10 years ago wrote. This person no longer works there. Code that we have written 10 years ago is still running out of there. Miłosz [00:04:58]: You have to agree with other people in the team how to do it, probably, because you are not the expert anymore. Robert [00:05:04]: So it’s creating a lot of challenges. Robert [00:05:06]: Those challenges are pretty important because the worst thing that we noticed is that the performance of how fast we’ve been able to deliver new stuff was much worse. And in this episode, we will discuss some tools and techniques that should help with that. Robert [00:05:27]: Can give you some tools to handle situations when you are not working in a solo project, but in a team and you need to collaborate in some way and create software that is maintainable. In longer term than one month. So can you keep the same pace as when working with two people or just a solo developer? So if those problems sound familiar for you, please stay with us. So we will give you some hints how to overcome those problems when you are working in a daily job. So at the beginning, let’s think why projects in your daily job are delivered slower. So in other words, why, when you are working with your friends after work or on your pet project, you are much more productive? What do you think it is? So we already mentioned one thing: when you’re working on the project alone, you are basically your own boss and you’re working on your ideas, which is great. But the downside is that it’s also much harder to make business out of that. I mean, we are software engineers, we are good problem solvers, but from our experience, very often we are not best businessmen. Miłosz [00:06:32]: Yeah and we often start side projects and you know, some websites maybe web apps, but often it’s just something for fun because it’s for learning. And this is how we often are learning, and it’s great. Yeah, but making it into a business is a much different story. Miłosz [00:06:52]: Which might be also why developers usually in companies don’t have that much impact on the product side. So very often they are kept in a silo of software development. Robert [00:07:10]: Yeah, I actually like to name it a bit more brutal— and they need a golden cage. So and how it often looks like: it’s like we are companies are trying to make our programmers coding monkeys in some golden cages, basically. I know it sounds brutal, but this is how it looks in multiple components, basically. So what do you mean by coding monkeys? So there are a couple elements of this problem, let’s say, and also a couple of levels of how companies are approaching this. So the most extreme scenario is when you’re this coding monkey in your golden cage and you’re just receiving tasks to code. I even heard about worse cases when some developers even received empty classes and they were diagrams, empty classes and some UNLs and just please feel the methods and don’t care about anything else. It’s similar to LLMs today, just like a human instead. Yeah, yeah. Robert [00:08:13]: And funny thing, if many people were working like that entire life, well, I can imagine that they may struggle now a bit because… So this is actually, it’s not very performant at the end because the software that is created in this way, obviously the chance of working is very, very low and making some… value. But still, it can be replaced in some way. Miłosz [00:08:36]: So engineers in this scenario are just kind of implementators of some idea. From the outside. I think especially in product companies it might be an issue because developers usually have great insight into the product area as well. Miłosz [00:08:53]: It can work better in some very technical domains where you need developers just to work on some specialized algorithms or something like that. I guess even waterfall can work in such a scenario. Miłosz [00:09:09]: But if it’s a product company, it’s much different. Robert [00:09:12]: Some of you may wonder, ‘Okay, why is that?’ I can just sit in my nice cage and code. It’s great. But… Unfortunately, it’s pretty unproductive. If you’re working with software a bit longer, you probably already know that requirements that you receive, they’re in most cases… Maybe not wrong, but… Miłosz [00:09:30]: Incomplete. It’s probably the biggest issue developers have in their day-to-day work. I can’t work on this. This is not clear. This is missing some requirements. And it’s discouraging because you can’t do anything with it. You can just, you know… tell your product owner or manager or whoever that, sorry, this is not complete. I can do anything with it. And then the product owner needs to go back to whoever knows what to do and figure it out and come back to you with clear requirements, which is never the end of it. Robert [00:10:08]: And needless to say, it’s super unproductive. When you’re working alone, you can ask yourself, ‘Ah, okay, it doesn’t make sense.’ Maybe let’s do it in this way. When you have proxy… Project manager well, it’s very ineffective because probably this project manager also doesn’t know. So this project manager needs to go to another person, that person ask another person, and it can also probably explain why often those requirements are not perfect. Miłosz [00:10:36]: And worst case scenario is probably when the product owner also decides who gets to do what. Miłosz [00:10:45]: Split the tasks between developers so the workload is even across the team. So that’s probably the biggest anti-pattern you can see in project management. Robert [00:10:57]: Yeah, exactly. Because, well, if the project manager is assigning tasks to each developer, probably he may not or she may have not enough context to know. Which person has most knowledge there, or maybe some other things that if team will be responsible for assigning those tasks. It will be much more effective because they have more context to do this decision better. Miłosz [00:11:23]: Yeah, that’s a better approach because the team knows what they specialize in, so they can share the knowledge as well between the team members. Everyone knows a bit about what’s going on in the project. It’s not like there’s one person responsible for everything. Robert [00:11:44]: And I think it’s quite interesting because it’s based on crazy and pointless ‘why’ to work in this way. But there are multiple reasons why, in many companies, it’s the default mode. I remember that. Robert [00:11:58]: In many components, when I was starting at the beginning, it was unfortunately the default mode of working, and it was often some work needed to change that. I mean, it was possible and it wasn’t that hard at the end when you’ve shown what problems it’s generating, but… Yeah, unfortunately, it’s often the default way of working. Do you have some ideas maybe why companies are doing it in this, well, probably crazy way of assigning tasks to people and closing them into the cages? Miłosz [00:12:32]: I guess maybe it’s just the way things always used to be. Miłosz [00:12:38]: Just the default mode of, you know, we have those developers who are implementers and product people who come up with ideas. So the development team becomes the downstream of those ideas. Miłosz [00:12:56]: Ping pong between each other. Robert [00:12:59]: Yeah, I also found one interesting reason for that. And the reason was that developers are expensive. So their time is expensive. So it means that we should use as small amount of their time as we can. Miłosz [00:13:14]: Yeah, basically use as much as we can for all the tasks we need. Robert [00:13:20]: So basically, let’s optimize developers’ time so they can code as much as they can because, well, we’re paying them for coding. This is why we’re paying them that much. Miłosz [00:13:28]: It’s basically like in a factory. You want to utilize all the devices to do the work for you. Robert [00:13:37]: And you’re trapped, because, okay, from one side you are… ensuring that those developers are using 100% of their time on coding. But if you think about that a bit for a bit longer, you will notice that, okay, maybe they’re coding more. But useless things, because even if we will force them to code some stuff, and they will write them after it will be shipped to production after in this kind of companies, half year let’s say, you’ll see that, okay, it’s actually not what we needed. The worst part is it all feels like a good job. Yeah, and for the developers. You may also feel very productive, like I’ve been coding for 12 hours and it’s so great. Robert [00:14:18]: Yes, but there is nothing to show for it. You more like wasted the time. Probably one sentence here will be very accurate. And I really like to remind it over and over. So five days of coding can save one day of planning. And this is basically that. Yes. So how… Miłosz [00:14:35]: talking about or coming up with ideas how something should work feels much less productive than just typing away and creating this. stuff. So we are very eager to jump to implementation, maybe build some prototype. Sometimes it’s useful, but very often it would be better to spend some time. Trying to understand what you really need to build, what will be useful for the users, for example. Miłosz [00:15:06]: But it doesn’t feel as productive as this starting coding. So developers don’t really have an issue with this most of the time. Robert [00:15:15]: So let’s all go maybe one level up in our ladder of how good or bad putting developers into cages can be. So the second level. That I noticed in some companies is some a bit better situation. So in this situation, product owner is going to the team. With some requirements gathered from stakeholders. And in this case, the product manager or product owner is not the person who is creating the tasks, but it’s the person who discusses basically what we need with the developers and he’s working or she’s working with developers to learn how to implement that. Miłosz [00:15:57]: So they present some high-level idea and then developers split it into smaller tasks or come up with implementation, maybe propose some solutions. Robert [00:16:06]: So we no longer receive tasks. It’s no longer assigned to people. We can talk to our closest stakeholder and discuss what we would like to implement. Miłosz [00:16:17]: You get even better knowledge sharing this way. So everyone is on the same page. Also easier to work with code later. Miłosz [00:16:24]: Because everyone has insight how this will impact the end product. You don’t just work on some obscure subtasks on the other end of it. You see the full picture. But there’s also a challenge here, right? It’s not really common knowledge how to do it. Miłosz [00:16:44]: It can take long hours to come up with the final solution. Robert [00:16:49]: And can go into never-ending meetings. Robert [00:16:55]: My observation, obviously, it’s always about balance. But my observation is that often it’s when you have hours of discussion. It’s basically alternative because sometimes even better alternative. I mean, the worst alternative can be that, okay, you are going to saying that, okay, ‘We are discussing already for too long.’ Let’s go. Let’s do what we think should be done. Yes. And later, you’re not spending hours on planning, but you’re spending weeks on implementing something just later to see that why we did it in this way. It’s crazy. It doesn’t make any sense. And nobody’s using that. Robert [00:17:33]: So I think it’s one popular trap that we’re trying to optimize this discussion part. Sometimes, as you said, we don’t have good tools how to discuss requirements with product owner and/ or product manager. We’ll get into that soon. So we’ll give you some tools. Robert [00:17:50]: Yeah, it can lead to that, but it’s, I think, important to ask ourselves the question, what’s the alternative here? And often, the alternative is not the best. Miłosz [00:18:00]: outcome of this can be simplification. Miłosz [00:18:03]: I think you need skilled developers that can have this. Product vision a bit and can talk with product owner, for example, and suggest some solutions. But because often the reason the meetings drags on so long is that we talk about two big problems at once. So we want to create this full system or big feature up front. And there are many edge cases. Developers love poking those edge cases and they will come up with many many issues. So then you have like 20 things you need to tackle before you start the development because there are so many edge cases to cover. And very often, if you start with a simpler version of the same feature, it may be much, much easier to discuss and start. And also very often you… you later realize and actually, you know this is like 80% good, maybe we don’t need most of what we thought we needed in the beginning. Robert [00:19:09]: I think it’s some kind of meta advice that it’s already coming in multiple episodes that we had earlier. So some golden standards that we have tried to have in teams that we’ve been working in. To be able to deliver each task within one day and to be able to deliver some, let’s name it, story within one week. Because, if you’re implementing, you’re creating some functionality into something existing. But it helps us to… Simplify those discussions because, if you are discussing some feature for multiple hours, and you would like to implement within one week, you already see that okay it cannot happen. Because, if you are discussing some feature for multiple hours, the implementation of that will just take much longer. So it’s good because you can start a discussion like, ‘Okay, it’s already scope creep. Let’s try to figure out what we can do. Some kind of how we can make some kind of MVP out of that, and start with that, and iterate on that. I said that it’s working for features that you’re already building on already existing functionality. Robert [00:20:17]: Because, in this case, you can do it faster. It’s a bit different when you are doing kickoff of new project. Because, if you are doing kickoff of some project, sometimes you need a bit more time to have something that is functional. So, but I would say that you should also watch out to not exceed, let’s say, one month. For example, and have in one month some MVP that already delivers something, and somebody can already use that, and you can already receive some feedback on that. And again, I think it may sound very obvious, but… Robert [00:20:48]: It just works, basically. So it’s one of those advices that are too simple to be true, but this is how it works. Miłosz [00:20:57]: Splitting work is not easy, for sure. We’ll probably need another episode on this. But coming back to this… with product owner. This is probably an upgrade on the previous approach. Miłosz [00:21:09]: Product owner deciding on everything themselves, but it’s still… not perfect, right? Because this is a single point of failure. This one person, they understand the domain, they talk with stakeholders or users. Miłosz [00:21:24]: But they still don’t know everything. So they are kind of a proxy between the development team and whoever needs the product built. Robert [00:21:34]: And if you are working this way, probably more than once you have situation like you are asking some question to your product owner, he or she is seeing you for a while and I don’t know, I need to ask this stakeholder. I will go back to you in a week. Robert [00:21:52]: And it’s good because if this question is there, because maybe it will save you one week of implementation that it will be useless at the end. So that’s good. But from other side, you need to often wait for some time for having those. Miłosz [00:22:06]: Yeah, I remember those planning sessions where we were deciding what to… Take on for the sprint for the next two weeks, and of course, we poke holes in some plant features. We see some magic cases that are not solved, and the product owner says, ‘Yeah, you’re right.’ So yeah, I need to go figure out what to do. Meanwhile, here are some well-defined but very unimportant tasks you can do in your sprint. So you postpone the important work this way, and because you don’t have anything better to do, you start working on something smaller but probably something that’s not really game-changing for the company. Robert [00:22:50]: But sometimes it’s good, because very often you have this backlog of things that we should do later, a . k . a. Miłosz [00:22:57]: never. Sure, but this way you never do these game-changing features. Probably the product needs the most. That’s true. Miłosz [00:23:08]: Because they are so important and risky, no one wants to just… start doing them with some simpler approach. It feels like we need this perfect implementation or specification for it. So you wait until you have it planned perfectly. Robert [00:23:28]: So what’s the crazy alternative for that? Miłosz [00:23:30]: Because I know— because we tried this crazy alternative. Miłosz [00:23:33]: Yeah, so if you go one step further, I guess you could have… developers talk with stakeholders directly? Oh, no, it sounds so expensive. Miłosz [00:23:44]: Or, you know, just domain experts in general, whoever this is, I don’t know, maybe your users or your CEO or whoever. Robert [00:23:54]: That are often the source of those requirements. So there are people with knowledge, there are people with some interest, let’s say, but basically… those are people who give you the direction of work that you are working on. And those are the people to whom your product owner is going to ask how to do something. Exactly. Miłosz [00:24:15]: So it might be your VP of sales or your operations specialist in another team. Robert [00:24:23]: Or even your CEO. Depends on the company. Robert [00:24:27]: Um, Or it may be also an operation theme. So it really depends on the functionality that you’re implementing. Robert [00:24:34]: Usually, if you’re implementing some functionality, there is some person that is mostly interested in that and the idea maker. Miłosz [00:24:44]: go to your product owner and they will tell them I need this new dashboard built and they will reply okay I will talk with my developers in two weeks later. Miłosz [00:24:58]: I will let you know two weeks later what we think. But what if instead, no, you have this… This domain expert, this stakeholder meet with developers directly and tell them what they need. And then usually developers… will tell them, oh my god, this is so complicated, it will take two months. But if we drop this one button from the dashboard, it will… take one week. And then the stakeholder says, oh, really? I don’t need this button at all. Miłosz [00:25:31]: Wow, impossible. Now we save two months of development and also two weeks of planning to figure this out. Robert [00:25:39]: And this is not a story that we invented. It’s actually what happened multiple times for us when, basically, we could spend ages to implement. Something that the stakeholders would say that it’s not that important. But if we have proxy as product owner, it will never happen because the communication between developers and stakeholders will never happen. And to be exact, so it’s not about gathering all stakeholders in the conference room and discussing. Everything because it will be super inefficient. We’ll go into tools very soon. It’s more about having them meet with developers in some way. At some point. And also not doing it for every feature. So to be exact here. So we’ve been doing it for some bigger project kickoffs. So we’ve been not doing it for… every story that we’re implementing because it would be super expensive, especially that our stakeholders were living in a different country. So it meant that gathering, for example, five stakeholders, and discussing everything with them, would cost flights, hotels, etc. Robert [00:26:46]: So it was pretty expensive, but still it will work for us and we’ve been doing it multiple times because, if it wouldn’t, it will just… Miłosz [00:26:55]: Yeah, but this is a bit extreme scenario, right? When we fly over people to meet in the conference room. But this same idea can also be applied on a more simple level. Robert [00:27:04]: It’s easier if you have stakeholders nearby, let’s say. Miłosz [00:27:09]: Yeah, but even if you have someone in another team that uses your software and they have some issue, you can just have a call with them and have them show you directly what’s the problem they have. It’s the same idea applied to a more shallow level, so you don’t need your… Thank you. product owner to be a proxy between you two. You just talk directly to them and you understand, okay— this is the issue they have. So now I will figure out how to fix it instead of waiting for perfect requirements to figure out what to do. But I think the basic idea here is that it’s difficult to come over, having developers think about the solution and offer some simplification. And I remember some product owners trying to do it themselves. They somehow… In a way they knew the software developers were working on and they tried to figure out themselves how complex it would be to add a feature here or there. It’s a disaster always. I guess today they will ask AI which feature is the most complex and how I can simplify it. Miłosz [00:28:27]: Because it won’t work, because you need this expertise to understand what’s going on really. That’s why you need to close the gap, have developers in a discussion. Robert [00:28:40]: Yeah. So to maybe recap this. So I would say that there are three or four levels of how you can basically be engaged in developing the feature. Robert [00:28:56]: The top level is pretty close to how you are working on this feature alone. In this situation you are together working with stakeholders— people that basically can make decisions if the feature can look in this way or not, because this is very important. So to have people that you can ask, okay, can we… skip this and they can say yes, no, and you can go further and not wait one week for decision. It doesn’t need to be like that for every story. It’s probably better for kicking off a big feature. So when we’ve been working on some features like some bigger epics, let’s say, that we’ve been working for Hazier, for example, or some new functionality. This is what we’ve been using. Robert [00:29:41]: One lever lower, you have situation when your product owner, product manager, is gathering requirements and you are working on them together. Robert [00:29:51]: In case of doubt, the product owner can go and ask some stakeholders if it’s problematic or not. It works nicely for some stories because usually you can wait for a while and you’re planning it in advance, so it’s not an issue that it can be a bit delayed. Robert [00:30:09]: I would probably not recommend the ‘golden cages’ approach when basically you are receiving some specification or some task from somebody else and you are just just writing this without really affecting how it’s implemented because it’s very ineffective. Miłosz [00:30:24]: So we said that it’s better from product perspective. Miłosz [00:30:29]: Work directly with domain experts and so on. But why should developers care? Because, for many developers, they are fine being just an engineer and waiting for requirements. Is there anything for them in this scenario? Miłosz [00:30:48]: What do you think? Robert [00:30:50]: So, I think that there are two things that you should keep in mind. And, as you said, so… Many developers, most, hard to say, but many developers don’t care that much about, let’s say, component success. They are just here working to code and they don’t care that much. But I think it’s not that simple because… You could be such this kind of developer in the market when everybody could get a job easily. But as we already said, it’s not like that anymore. And you need to be a bit more outstanding to get job easily. And to understand job market now, it’s probably important to understand about what companies… care when they’re hiring developers. And well, they care about hiring people that will be useful for them. Let’s put it in this way. Miłosz [00:31:44]: So basically, if you are able to deliver stuff that they need, they’re kind of happy and you need to make help them make more money, basically, so they can share some of it with you. Now you know there’s less funding than 10 years ago, it was very easy to Oh. Miłosz [00:32:03]: get VCs to invest in whatever idea you have. So you need to actually deliver something more often. And it’s quite a rare skill to work directly with product. And I think for me the funny thing is: We somehow don’t test this on interviews very often. Also, no one teaches this. Miłosz [00:32:27]: Universities or even your job, you’re mostly taught to write code. I would even say that we are probably more prepared to be those coding monkeys. Yeah, exactly. If you go for a tech interview, you will encounter the slit code exercises or you will be asked to write some algorithm. Miłosz [00:32:51]: On a piece of paper or on a whiteboard. Perfect for coding monkeys. Robert [00:32:54]: Those are requirements. Don’t ask questions, just implement them. Miłosz [00:32:59]: Yeah, the technical details. But then you start… Working and turns out those things are really not that much used. They are not that important. You will spend more time figuring out what to build and what not to build. Robert [00:33:15]: And often you’re hiring this kind of people that are great in algorithms. And later you are finding out that they are expecting them to give them tasks, requirements, and they can just code that. And obviously there are companies where it works, but in the product companies… We already covered why it’s not working. Robert [00:33:36]: It’s also pretty hard to change your mindset if you’ve been working for entire career in this way. So this is obviously the important thing to think about. I mean, if you’re working this way. Robert [00:33:50]: And you will be working this way for your entire… Okay, maybe not your entire life, but for longer time, it’s harder later to switch. And companies are also— now better in picking, if people actually have some skills to be a bit more engaged in the in the way how product is developed. Miłosz [00:34:13]: It’s harder to hide your incompetence behind some algorithm. and technical stuff. For many product companies, it’s not that important. For the technical skills you have, I ignore it. Miłosz [00:34:30]: You know, data structures you can implement on a whiteboard. They are not that much relevant. They need to deliver something quickly to the market, and iterate over it, and do it over and over. So, if you can communicate well, you can talk with domain experts, you can figure out what they need to build, and you can deliver it. And that’s someone companies probably want to hire. Robert [00:34:54]: And even if somebody will manage to be hired, many companies are also pretty open to let some people go if they will not meet some requirements of how they expect that some people will be operating. Miłosz [00:35:10]: And on the other hand, if you want to be promoted, for example, delivering is the very best. Probably why to do it. So instead of just waiting for requirements and complaining that it doesn’t work, you can try to help. Whoever needs what they need. And in many companies, probably in most companies, this will help you get promoted because they will tell your manager or whoever that you do a great job. It’s really quite simple. Miłosz [00:35:45]: I think we can say it works. Robert [00:35:49]: And we mentioned earlier that nobody is actually teaching us how to do that. So how to work with requirements of product owner. Robert [00:36:01]: Nobody’s giving us some tools how to do it efficiently. I mean, There are a lot of resources how to build classes, how to do architecture. Robert [00:36:12]: How to, I don’t know, use vMotion to be super fast in your IDE. And it’s great, but from other sides, I’m not sure if it’s giving that much return on investment. It’s easier to learn, for sure. Robert [00:36:26]: And you know, it’s coding. It’s—uh— more cool definitely than working with product. But from other sides, I know that many people are also complaining how much time they were spending on meetings. Robert [00:36:41]: Actually, okay, maybe if you’re spending a lot of time on meetings, maybe it’s worth to spend some time to understand what you can do to do it more efficiently because it’s also a skill and often you may say, ‘Okay, it’s the responsibility of the product owner, I don’t care.’ Yes, but unfortunately it often doesn’t work this way. And in many companies that we worked earlier, if we wouldn’t initialize to do some changes if it would work like it was working earlier, like people will be coding monkeys in case. The default mode. Yeah, yeah. So I think it’s important to get some initiative and to go outside of this comfort cage or comfort zone. Oh, it sounded so coaching. Robert [00:37:30]: Thank you. Yeah, it is what it is. I mean, sometimes you need to do some things that you are not that comfortable with, and it’s fine. Miłosz [00:37:37]: Yeah, but I think it’s also important to mention it’s not for… for company’s sake or just because it’s some moral thing to do. It’s basically how you can grow your career this way. Because you become someone who can deliver stuff, and this is what companies want to pay for. Exactly. So I would say do it for your own sake, for your career’s sake. And it will affect your productivity much, much more. Robert [00:38:08]: I can attest that you can have a bit less of bad meetings that you have. And if you have some tools how to do that. Okay, so we’ve already mentioned a couple of times about that we have some tools, techniques earlier, and probably you are waiting to learn what those tools or techniques are. are. So you can try to use them, experiment with them, and maybe improve a bit how you can do that. Miłosz [00:38:35]: So, is there any tool you recommend for this? Robert [00:38:39]: I think I’m not sure if you agree, but I’m quite sure that one of the tools that created most impact in the teams that we’ve been using was brainstorming. When we’ve been working some things that at the beginning was working this default mode close to golden cages, it helped us to… to trigger some discussion between developers and non-developers and to think more how to implement those features. Miłosz [00:39:11]: Yeah, so maybe if someone doesn’t know event storming, we can… Uh, give a quick TLDR: this is a brainstorming session where you gather developers and domain experts in one room and have them use sticky notes. I know how it sounds—yeah, but basically, the idea is to have this lightweight conversation about complex topics so everyone is on the same page. Robert [00:39:42]: Even if it sounds, again, maybe I like, again, some coachy, agile, Scrum Master-ish thing. Robert [00:39:52]: It’s actually a very good technique created by Alberto Brandolini, who is a software engineer. He’s not some agile coach or whatever. Robert [00:40:02]: software engineer, so it’s very close to implementation. So the artwork that you’re creating is very close to implementation and it can be mapped to the code. Nicely and it gives us some nice framework to discuss some functionalities that we’d like to implement. And we’ve been using it in multiple contexts. So we’ve been using it for this smaller planning for these weekly stories, but we’ve been using it also for some bigger strategic features that we’ve been spending, for example, here. And it’s not about creating a big upfront design. It was rather to have some high-level idea on how everything interacts, how we would like to interact with other teams, to see complexity where it’s there. Because very often… There is one problem with stakeholders. They are going to you with some functionality and they are saying that ‘Why so long? Miłosz [00:40:57]: Yeah.’ Miłosz [00:40:58]: At least you have a way to have everyone start to discuss a topic in a different way than just sitting and looking at a presentation. And at first everyone is a bit confused, but then very often you have these lively discussions about the product. Robert [00:41:20]: I can also recommend you to maybe Google event storming so you can see how the artifact at the end looks like. But the idea is that basically you are using notation of domain events. Robert [00:41:32]: In the simplest kind of session and trying to model how our system works. And it’s nice because it’s creating some visible artifact that’s showing the interactions in the system, but it’s also showing the complexity there. I mean, if you’re working on this artifact together, with product owner or stakeholders, it’s kind of closing all those questions: ‘why is it that complicated because if system is complicated, it’s usually visible in the board— that’s a very visual representation. Miłosz [00:42:04]: And this is not a technical representation, right? So it’s not about diagrams or database schemas. Robert [00:42:09]: It’s something that stakeholders can also understand, but it’s also cool because you can also translate it later to your code very directly. Miłosz [00:42:17]: So very often, if you do it for an existing system that has been around for a while, you will see how developers use a bit different names than other people in the company and start to close this gap. Miłosz [00:42:33]: It’s very interesting when you ask your colleagues from other teams to map out how it works from their perspective. Hmm-mmm. It’s like you see a bit different system than you are used to. It’s the same thing, but they use a bit different names. They don’t know all the details. And if you do it together, you get a lot of insight from this. Robert [00:43:01]: And as I said, so for us, it was game changer in multiple things that we’ve been in. And from my perspective, it’s really a shame that it’s not adopted much wider. So, the domain-driven design community is probably best in adaptation of that, so a lot of domain-driven design practitioners are using that by default. I would say that outside of this cycle, it’s not that widely adopted. Again, I think it’s a shame. I’m not sure actually why, but maybe it’s the thing of the forum that for a lot of people it’s like, ‘ah, some agile or coaching stuff, sticky notes again.’ Maybe it’s… because of some cynism. I don’t know. Miłosz [00:43:43]: You also need someone who facilitates this session so it doesn’t end up with… People discussing stuff in smaller groups and not participating together or just going to some off-topic. So it’s not that easy. Always to run it you have to figure out how to do it a bit, but I think it’s worth it. In a way, it’s probably better than having one person speak and everyone sleep on the chairs in the conference room. After the meeting, you decide to run another meeting because you don’t know what to do next. Robert [00:44:20]: I think you need to also have some more open-minded people for that. It’s great if you would like to try that, but you’re not always working in teams when you have good climate for running it. I know how hard it sometimes can be. Robert [00:44:37]: It can be the reason. And in this episode, we’ll not cover into details how to run this kind of sessions, because probably it will be… pretty long. There’s also one very good resource from which we’ve been learning how to do that. This is the ebook from the creator of the technique. It’s named Introducing Event Storming by Alberto Brandolini. We’ll link this in the episode notes. Robert [00:45:07]: This is what I recommend you to check. But we have some tips how to try to convince your teammates, your product manager and people around to try that. Probably the first tip will be to try it maybe on a smaller scale. So don’t start with this big session of planning for a half year, getting your stakeholders from abroad. And don’t start with booking. Miłosz [00:45:38]: plane tickets? Robert [00:45:39]: Yeah, so I would recommend to start with maybe those planning sessions that you are doing with the rest of your team, maybe with product manager. And try to learn how to run those sessions. Because we’ve been running sometimes some bigger session with 20, 30 people. It’s much, much harder than with just your team. Miłosz [00:45:57]: That’s for sure. Especially if you have someone who’s skeptical of this idea, they will just try to discourage everyone to continue because, oh, we know how everything works already. Why do you do it again? Miłosz [00:46:14]: I remember sessions like this. Robert [00:46:16]: But also don’t try to look for excuses like, ‘I’m introvert, I don’t like to do that.’ I’m also, and I did that, and you can just do it. No worries. Miłosz [00:46:25]: It can be tiring, but there is some end goal in all this. You want to figure out the domain. It’s not just a meeting for the meeting’s sake. Robert [00:46:36]: It’s tiring, but from other side, I think it’s also very satisfying when it was successful and later you are sitting to implement that and you can later feel how this implementation is just flowing. And it’s also causing a lot of questions and solving a lot of problems that we’ve been mentioning at the beginning of the episode. Robert [00:46:57]: So, if you would like to start with brainstorming, the first good place to try to adopt that is within your team. So, run some smaller sessions. We recommend in-person sessions since the dynamics are a bit better, but I know that also a lot of people doesn’t have this privilege, let’s say, to work in the office. So, if you’re working in a hybrid way, we would recommend to gather and do this in person because it’s working just much better. Because if you’re doing it on Miro or any other online tool, it’s just harder to maintain everybody’s attention because everybody will try to open some other tab. You can do that, but it’s super hard. Miłosz [00:47:40]: It also happens in in-person meetings when people come to the session with their laptops. Yes, yes, and they are always very confused when you ask them to close them and stand up. Miłosz [00:47:53]: The first sessions are always fun. Robert [00:47:54]: But this is mentioned in the ebook, so we’ll not go super deep into that. And it makes sense. So the first thing you should also discuss it with your team. So you should have some better understanding of what problems you may have with delivering and planning. Robert [00:48:13]: You can use them to try to convince people that. Let’s try to maybe do it in a different way and try to get buy-in from your team to try that. Miłosz [00:48:23]: I know it’s often difficult, and we often get asked how to convince my team to start using and driven design. This is a similar idea here. Miłosz [00:48:35]: I think maybe you can find, if it’s not your entire team, a people like likely-minded as you who want to do something better and try to figure out together how you can improve how you deliver software, basically, and like we’ve been mentioning at the beginning—so, in our podcast, we would like to give you some tools that can help you to be promoted to principal engineer, for example. Robert [00:49:04]: So getting buy-in of people is a very important skill to be principal engineer. Robert [00:49:11]: We believe that if you’re a principal engineer, you’re not the person that is forcing people to do something. It’s rather showing people some benefits of your ideas. They will just commit to your ideas and they will be happy because it’s giving some solution for the problems that you have. Miłosz [00:49:28]: Yeah, so if you want to start, you can just take your team, invite them to a session, try to figure it out together, or you can lead if you feel like it. Miłosz [00:49:44]: Hopefully, you come at something that’s better than your current way of working and this way they see the value. And similar with your… or product person, product owner, or product manager. You can invite them to the session. They should be probably also eager to find better ways to work for all of you because it’s also there in their interest, unless they are, you know, kind of micromanagement freak. Robert [00:50:17]: But from other side, this micromanagement is often coming out of something, and at the end of that, if you… go deeper, it’s ending up that they would like to deliver stuff faster. Miłosz [00:50:28]: If you use this card, it’s nice to yeah and it’s it’s usually quite easy to point out the issues you already have. So you can look back at what happened in your recent plannings, let’s say. You can point that out. But this feature took us two months because of all the issues we didn’t know, of all the gaps in the requirements and so on. Maybe we could do it faster the next time. Maybe we can learn from it. Robert [00:51:07]: It’s also one important argument that is pretty close to the time. So it’s money. I mean, you can say, ‘Okay, last time we one month, which is this amount of money, because this is the average salary of the developer in our teams, if we could spend maybe this next time on planning it better, we could just save this money at the end.’ Miłosz [00:51:34]: Probably the best way to introduce new concepts is to build. Trust first. So, if you just just join the team and you see how that everything is not done the way you would do it— Probably telling the team on the third day that they should reorganize is not a good idea because you don’t know anything about the team and the company. But if you deliver consistently, you build trust, and then people will be more eager to listen to your ideas. You can gradually introduce new ways of working. Miłosz [00:52:16]: So I would start with this. One common issue with meetings or sessions is the never-ending discussions. Miłosz [00:52:26]: They tend to make the meetings hours long. Robert [00:52:30]: And there are some tips about that in the ebook, but I think it’s also worth mentioning because, even if you will read this ebook, it’s easy to get into some never-ending discussions anyway. Miłosz [00:52:44]: Especially if you have… If you run this big picture strategic sessions with 10 or more people in the room and you start mapping the entire flow on the board, people tend to run into discussions because you know, for every system you have in your company, there will be someone who says, ‘Oh yeah, this is the issue I need this fixed’ and Miłosz [00:53:08]: It can take a lot of time and it will derail you from the main topic. So it’s useful to have someone act as facilitator and cut that. the discussions early. Robert [00:53:20]: I’d probably use better work to maybe park it because, if you cut it, somebody may feel like, ‘why are we cutting that?’ It’s probably better. So, what we’ve been doing usually during the session was using the red sticky note. So, we’ve been writing down the problems that we’d like to park, maybe later during the session, maybe after the session, and put it to the board. So, the person that has this problem will feel that, ‘okay, it’s also something that it’s not disappearing, it’s there.’ We won’t forget about it. Yes, yes. But important. Let’s go back to that. Because we’ve already mentioned the trust earlier. So, if you will be parking things and not going back to that later, you may also have problems with this trust part. Miłosz [00:54:07]: Yeah, so later we had separate discussions, maybe in smaller teams. Maybe not everyone needs to be involved in all of the red cards. Maybe some of them we will… decide just are not important and forget about them or ignore them. But for most, we need to revisit and discuss this again. But this way, we will focus on one thing at a time. Robert [00:54:31]: Another thing that I think is also in the ebook, but worth mentioning. So this is the time boxing problem. We’ve already mentioned it earlier, but I think it’s one of the biggest issues. So it’s important to emphasize it and mention it again. Robert [00:54:48]: Getting some balance between cutting discussions but also not skipping discussions that are important. And it’s hard to give some good algorithm how to park some discussion and when to follow it. It’s probably some skill that you need to develop with time, but, as we said earlier, sometimes cutting some discussion may mean that it will go back to you during the implementation and making, changing some. During implementation, it’s much, much more expensive than during planning because you need to go back to planning. Maybe something that you already implemented is not compatible with that and you need to undo that. It’s important to keep it in mind. Experiment. So maybe sometimes try to discuss a bit longer. Robert [00:55:39]: Maybe sometimes try to cut, and also set expectations that, usually, when we’ve been doing some planning sessions, some vendor storming sessions, we’ve been like, ’excuse.’ After that, because it’s it’s been sometimes hours to do that, but at the end it was worth— I mean, very rarely— we had like, ‘okay,’ we’ve been spending on even storming too much, and we designed it too well. Miłosz [00:56:04]: We could better deliver something this time. Miłosz [00:56:08]: This comes a bit back to the topic of cutting scope. If you plan huge features, it will be very difficult to discuss all of it. Yes, but again, cutting scope is not always easy, depending on the project. Robert [00:56:24]: And again, it’s mentioned in the e-book, but in the… book, you can read that event storming has multiple phases. So at the beginning, you’re generating a ton of ideas. And with the time you’re removing more and more of the things that are not relevant. And usually, at the end, maybe not at the end, but close to the end of the session, you should do some reality check and to think: ‘Okay, is what we plant here deliverable? In depends if you’re adding some feature or doing something big, maybe a week, in worst case one month.’ Robert [00:57:04]: If it’s one month, probably it will be in practice a bit longer. But if it’s longer, probably you should look at it and see what we can cut out of that. Robert [00:57:15]: To make it functional, usable, let’s say, and work on this basically. It’s not obvious, but we’re often forgetting about that and we are trying to implement a feature that It should take half a year, and it’s going to be two years later. Miłosz [00:57:34]: Yeah, because no one challenges it. You start planning. Some feature that comes from someone from another team or your CEO or whoever and no one challenges you. Miłosz [00:57:47]: You need the entire thing. Maybe you need just 80% of it or 50%, and it will be much simpler to deliver. Robert [00:57:55]: Okay, so we’ve given you some, let’s say, brief introduction to event storming. Again, it’s not something for episode because I think even one episode will be not enough for that. But if it sounds interesting for you, please leave us a comment. Maybe we’ll do some episodes just about event storming. But. Yeah, I hope that it will give you some overview why it makes sense to try to think out of this thing out of the cage. Miłosz [00:58:29]: Maybe let’s reiterate some ideas. What can someone do if they see this is an issue, if they want to work better with the product? Robert [00:58:40]: Yeah, so I think at the beginning, you should assess in what level of cage you are, let’s name it. So, if you are receiving tasks and it’s assigned to you, or you are maybe closer to the way when you’re planning tasks with your product owner but you’re never discussing with stakeholders. Robert [00:59:05]: Tchum! Think on which level you are. Mm-hmm. Uh, later. Discuss with your teammates if they also see those problems. You should be able to identify the problems with feature planning already. On this episode, so it should give you some weapons. Miłosz [00:59:24]: Yeah, try to figure it out together. Just not try to force. Your teammates to do some of your ideas right away. Try to think together what are the issues you want to improve. Miłosz [00:59:41]: Can write down specific problems you have or even situations. So I remember we had a very issues like this— one task has been on hold for one month because of something, I don’t know, some miscommunication or whatever. Robert [01:00:00]: Or maybe you lost a lot of time on implementing something that used to be useless at the end and your stakeholders said… This is not what we asked about. Miłosz [01:00:10]: Yeah, we delivered something and then it was scraped because no one needed it. And… probably the more specific it is the better because you can then rise it with your manager or product owner or whoever is in charge Instead of complaining that we never get anything done, you tell them, yeah, this feature could have been one week. but we spent two months. Robert [01:00:38]: In other words, use the language and arguments that other side care about. You know about what product managers will care. your teammates probably they care about their car so you can also say that okay maybe would like to if would like to get better job later or be promoted, maybe we should try to think about that. And again, if this convincing people to change approach to use your approach. It’s a very important skill of principal engineers. Robert [01:01:10]: Being principal engineer doesn’t mean that you are forcing people to do something. It means that you are convincing people that your solution will help them with their issues. And that’s it, basically. Miłosz [01:01:21]: And if it really does, then you get trust. Miłosz [01:01:26]: You need to accept that some people just won’t care, and it’s also fine. Don’t try to force them to do anything, whether they are developers or not. And I would instead try to find people who care and try to work with them. So if your product owner doesn’t care, doesn’t want to change how they work, maybe to go to their manager and tell them there’s an issue. Basically, find someone who cares about work getting done in this company. Robert [01:01:58]: If it’s not possible due to political reasons, just maybe do it within your team. Robert [01:02:05]: We don’t recommend that, but it’s better to do event storming session within your technical team without product owner than not doing it at all. Miłosz [01:02:16]: There’s also the last advice we didn’t mention in this episode yet, which is change your job. Miłosz [01:02:22]: It’s always the solution. Robert [01:02:26]: But again, please don’t use it also as an excuse to not include your product manager, because you would be surprised. I mean, how often they care more than you. You think. They just have a bit different goals, but… Miłosz [01:02:39]: And they also might have been burned working with developers in the past. Exactly. If they worked with someone who’s just like, yeah, I don’t care. I will code and give me what to do. They might not be eager to work with you directly. So this goes both ways. Robert [01:02:54]: Yeah, but from our experience, it’s not the case in 96% of cases. It’s very, very rarely a case that it’s… those are people that I don’t care. So please try and you will be really, really, really surprised. Also, I think it’s important to set expectations. Robert [01:03:16]: There are no magic techniques that will make all the planning accurate because ideas will always change during implementation. We are going closer to planning better. Robert [01:03:31]: It will also always take some time to do planning. It’s not magic tool. There is no magic tool that will totally remove meetings. It’s just about making it more efficient. Also be prepared that, okay, if you, for example, decide to use event storming. It will be not super successful from the first session, but it’s also a pretty simple technique, so you’ll be surprised that the result will be pretty good anyway. But also set some expectations. Be prepared to experiment and also improve after each session. Maybe do some evaluation after every session to ask people, okay, did you like that? What you didn’t like? And improve, improve, and improve. Miłosz [01:04:13]: Especially if you change company culture, which is very, very hard to change. And especially if it’s been like this for a long time. Robert [01:04:22]: But it’s possible. We did it more than once and later people were eager to work in this way. So it was cool. And it was also cool later to transform it to other companies and also have some success stories that helped us too. Robert [01:04:39]: Be in different places later. Okay, so I think it will be all for today. So if you have any questions about windstorming, about getting out of the cage or any similar topics, let us know in comments on Spotify. Miłosz [01:04:54]: Yeah, also share your stories maybe. I’m sure there are a lot of stories how projects are run. Let us know in the comments how it works. Works for you. Robert [01:05:05]: If you have any questions, just ask us. We are answering all the comments, so we’ll be also happy to help you. Robert [01:05:14]: Would be interested in listening about event storming, for example. Let us know so we can maybe also record entire episode just about event storming. Miłosz [01:05:23]: Hit that like button and subscribe to the channel so you don’t miss the next episodes. We plan to release a new episode every two weeks now. Robert [01:05:32]: Yep. So, thank you, Miłosz, for your time. And see you in two weeks. Miłosz [01:05:35]: Thank you, Robert. See you. Robert [01:05:36]: Bye.
Learn Building Modern Go applications Nov 26, 2025 -
How to set up Claude Code notifications using hooks. Get desktop alerts when Claude finishes a task, needs your input, or requests permission, instead of watching the terminal.
alexop.dev Nov 23, 2025 -
Learn how to transform Claude Code from a chatbot into a deterministic engine using Slash Commands. This guide covers the technical setup and a complete 'Full Circle' workflow that automates your entire feature lifecycle.
alexop.dev Nov 22, 2025 -
A practical guide to Claude Code's features: MCP, CLAUDE.md, slash commands, subagents, hooks, plugins, skills, and scheduled tasks. Updated April 2026 with deferred tool loading, worktree isolation, agent teams, and more.
alexop.dev Nov 9, 2025 -
How I built a Claude Code plugin to generate skills, agents, commands, and more—and stopped copy-pasting boilerplate.
alexop.dev Nov 8, 2025 -
Learn how to build scalable applications using Nuxt Layers to enforce clean architecture boundaries without the complexity of microservices.
alexop.dev Nov 2, 2025 -
A DNS outage is like a flash flood. It hits you and disappears as if it were never there. “It was probably the DNS” became a meme. But, half the time, we do not know the exact cause. The outage is seldom long enough to diagnose, sequester, replicate, simulate, and test the failure conditions. How can we be certain that this type of failure will not compromise our systems again in the future? The DNS can fail spectacularly in a variety of ways. A better way to address intermittent failures, such as DNS outages, is to accept them as unavoidable occasional execution disruptions. We might not be able to address each of them exhaustively, but we can design our system to handle almost any interruption of its internal logic. We can harden the system against the disruptions to attain durable execution. What is Durable Execution? Durable execution is a guarantee that a program output is correct regardless of failures anywhere in the execution process. Competitive business applications require durable execution. When it is absent, an employee will spend an hour or two fixing each incorrect output when it is detected. In large business applications, incorrect output often goes undetected for weeks, resulting in expensive chaos around holidays. Are we talking about bugs? No. Bugs are the program. Some intermittent bugs erode durability, but the durability loss is most often a result of an interruption of the execution process. The interruption can be caused by a network outage, service failure, security breach, or a delinquent update. All of those reasons are common. Durable execution does not care what caused it. It is a promise to recover from almost any failure or combination of failures and deliver the correct output. Event-Driven Architecture (EDA) is the industry best practice for supporting durable execution. However, most EDA production systems are not provably durable. What happened? Note In the second guest post on our blog, we welcome Dima Kotik, who contributed the SQLite Pub/Sub backend to Watermill. Please give Dima a warm welcome! 🎉 Miłosz & Robert Durable Execution Requires Immediate Input Storage There is a difference between a guarantee and a wish. A guarantee has some proof, a history of recovery from a variety of system crashes. It begins with emulating crash conditions in integration tests. After writing many such tests, you will realize a simple but profound principle: durable execution begins with writing execution state into storage. There is no other way to proceed, because anything present in random-access memory or ephemeral storage is liable to disappear if the system crashes. The first step of a durable system is to stash its input before any processing takes place after parsing and validation. Security is more important than durability, so input validation takes precedence. Stash the Input After Parsing and Validation Wait, what? I have to commit all the system input immediately into storage? Yes. Is that even possible? Not all at once, but this is not as daunting a task as one might imagine. If you are using event-driven architecture with an publisher/subscriber management library like Watermill, you are already halfway there. Watermill is one of the most popular event buses in the Go ecosystem. Watermill is an unusually flexible library in that it can bridge events from a large number of different backends. Your core business logic may stream events to a PostgreSQL database, and your input may initially be stored in a SQLite databases of your edge nodes to achieve durability without sacrificing performance. Watermill can link the two to tango together seamlessly. Unfortunately, many business applications that utilize Watermill continue to use the Go channel ephemeral backend here and there despite the documentation urging developers not to do it. This backend is a basic in-memory implementation based on Go channels intended for testing purposes. The events are queued into an asynchronous channel. They will disappear when the system crashes. They will clog up the server with a long queue if the execution process slows down for any reason. They will compete for resources with a critical server process. In other words, do not write input into random-access memory. You are asking for trouble. There is an easy persistent drop-in replacement for an ephemeral backend. It is not a silver bullet, but it is an easy first step towards durable computing. Try the new Watermill SQLite backend. It does not require setting up any additional infrastructure, and it is more performant and reliable than any other storage method because it does not need a network connection or a custom transport protocol. It writes to a file on a disk. It does not require CGO. Suppose we are processing book orders, and we would like to store the orders as events in an SQLite database: db, err := sql.Open("sqlite", "orders.sqlite3?journal_mode=WAL&busy_timeout=1000&cache=shared") if err != nil { return err } defer db.Close() db.SetMaxOpenConns(1) // driver limitation publisher, err := wmsqlitemodernc.NewPublisher( db, wmsqlitemodernc.PublisherOptions{ InitializeSchema: true, }, ) msg := message.NewMessage( watermill.NewUUID(), // serialized order event []byte(`{"order_id": "1", "title": "1984", "customer": "Queen Elizabeth II", "address": "Buckingham Palace"}`), ) if err := publisher.Publish("book_orders", msg); err != nil { return err } For cloud instances, ensure that you are writing the SQLite file to a persistent volume, which is mounted explicitly. The root file system is typically ephemeral. You may want to eventually migrate the collected input into your main database after thinking about performance implications. The matching subscriber is not any different from normal Watermill usage other than the backend initialization: subscriber, err := wmsqlitemodernc.NewSubscriber( db, wmsqlitemodernc.SubscriberOptions{ InitializeSchema: true, }, ) if err != nil { return err } messages, err := subscriber.Subscribe(context.Background(), "book_orders") if err != nil { return err } for msg := range messages { // consume messages from the channel processBookOrder(db, msg) } The SQLite publisher and subscriber are drop-in replacements. They can be also used without Watermill router. In other words, you can just use low-level Publish and Subscribe methods. Full examples are provided in documentation. Nothing else needs to change in your code to achieve this first step towards durable execution: write the input events to persistent storage. For complete durability, however, the event handlers for each step of processing the book orders must have two provable qualities: Idempotency: the handler can consume the same event multiple times without generating more than one downstream event. Atomicity: the handler either completes successfully or fails completely. It must never succeed partially. The combination of these properties with persistent storage ensures durable execution. When the a step in business logic fails for any reason, it is attempted again from the last safe system state. Prove Idempotency with Event Duplication Idempotent event handlers must not change system state when processing identical events. In other words, any repetition of input any number of times makes no difference to the system. We can achieve this effect by de-duplicating events, but it is best that the program logic itself tolerates repetition. First, ensure that the event message is acknowledged only after the all the business logic has been executed successfully: func processBookOrder(db *sql.DB, msg *watermill.Message) error { // use only one database transaction per event handler tx, err := db.BeginTx(context.Background(), &sql.TxOptions{ Isolation: sql.LevelReadCommitted, }) if err != nil { return err } defer func() { if err != nil { _ = tx.Rollback() msg.Nack() } else { err = tx.Commit() // acknowledge the message ONLY AFTER successful transaction commit // what happens if the the execution fails here? if err != nil { msg.Ack() } else { msg.Nack() } } }() // process input using the business domain logic // ... return nil } What happens if the transaction succeeds but the message acknowledgment fails on line 16? If the handler is idempotent, the event will be reprocessed, but the system state will not change. An easy way to achieve this is to constrain the database table field order_id to be unique. Then, drop the error if it is caused by a unique constraint violation: // Example for SQLite3 "modernc.org/sqlite/lib" if err != nil { if liteErr, ok := err.(*sqlite.Error); ok { code := liteErr.Code() if code == lib.SQLITE_CONSTRAINT_PRIMARYKEY || code == lib.SQLITE_CONSTRAINT_UNIQUE { // duplicate order id is normal behavior // it means that the event was already processed // ignore this error return nil } } // ... } // Example for PostgreSQL's "github.com/lib/pq" driver if err != nil { const UniqueViolationErr = pq.ErrorCode("23505") if errors.Is(err, UniqueViolationErr) { // duplicate order id is normal behavior // it means that the event was already processed // ignore this error return nil } // ... } What if there are multiple SQL table key constraints? The best solution is to have an idempotency test in place. Then, methodically groom the business logic until it tolerates repetition and passes the test. Use duplicator handler middleware to clone messages and check that the final program state is the same as running the test without the duplicator. Prove Atomicity with Chaos Engineering Once we achieve idempotency, how can we prove atomicity for every step of the business logic process? At-least-once atomic execution is theoretically facilitated by Watermill, but there are subtle ways of subverting it. If you rely on low-level primitives, always acknowledge the message after all the business logic finishes. And, ensure that business logic is granular enough that all of it runs anew in case of an interruption. A simple rule to follow is that all state changes in one event handler must apply within the same database transaction. If you have two database transactions, you will never achieve durable execution. If you are publishing another event inside the event handler, that event publisher must operate within the same transaction as the event handler. The outbox pattern is essential when making any additional changes to the database together with publishing the event. How do you prove atomicity with a test? Write a chaos middleware to pair it with the retry middleware for your testing setup: func ApplyChaos(h message.HandlerFunc) message.HandlerFunc { failNextOperation := true return func(msg *message.Message) ([]*message.Message, error) { result, err := h(msg) if err != nil { return nil, err } failNextOperation = !failNextOperation if failNextOperation { return nil, errors.New("world chaos cancelled the event handler") } return result, nil } } This middleware will break the event handlers half the time. The retry middleware should ensure that the operations complete even with half of the system experiencing outages. How do you prove that an event handler is atomic? Write another middleware that pollutes the message context: func PolluteContextWithChaos(h message.HandlerFunc) message.HandlerFunc { return func(msg *message.Message) ([]*message.Message, error) { polluted, cancel := context.WithCancel(msg.Context()) defer cancel() go func() { <-time.After(time.Millisecond) cancel() // simulate premature cancellation }() msg.SetContext() return h(msg) } } If all your input and output operations are context-aware (they should be - it is best practice), the premature context cancellation will simulate an execution interruption inside the event handler. This simple example cancels after one millisecond. Simulate a variety of pseudo-random cancellation patterns. Never use true random, because it will make the tests brittle. Check that the output is the same, and you are well on your way to a durable system. Conclusion Business applications need durable execution. Programs that have an innate ability to recover from almost any error or crash are a joy to work with. Durability can be achieved with proper system design and a rigorous testing discipline. You can see how durable execution can get complicated quickly, but the first step is the easiest. Write the input event to a persistent volume. It will give you some durability right away. You can work out the rest of it later. SQLite Pub/Sub backend for Watermill is a great choice to try this. This new driver works without CGO and, like Watermill, is free and open source. Cloud providers have begun to offer distributed SQLite clusters for use cases that offer some durability and the fastest possible read speeds. Those projects are a strong indicator that SQLite-powered storage is a viable and increasingly desirable option for durable execution: LiteFS keeps all the classic SQLite conventions like the single-writer constraint. This gives good reasons to anticipate that in the future durable execution could also become fast even with global transactions. Turso comfortably runs on edge or mobile devices with poor connectivity. Cloudflare durable objects are an SQLite persistence companion to Cloudflare Workers. There are two disadvantages to those commerical offerings compared to Watermill over SQLite Pub/Sub: (1) the vendor lock-in and (2) some difficulties in binding database transactions to event handlers reliably. This means that Watermill might get you farther past persistence towards idempotency and atomicity on the journey to provable durable execution. Give it a try!
Learn Building Modern Go applications Oct 19, 2025 -
Learn how to handle API calls in Pinia using the Elm pattern for predictable, testable side effects. Includes complete examples with the Pokemon API.
alexop.dev Oct 17, 2025 -
Go 1.25 introduced a new http.CrossOriginProtection middleware to the standard library — and it got me wondering: Have we finally reached the point where CSRF attacks can be prevented without relying on a token-based check (like double-submit cookies)? Can we build secure web applications without bringing in third-party packages like justinas/nosurf or gorilla/csrf? And I think the answer now may be a cautious “yes” — so long as a few important conditions are met. If you want to skip the explanations and just want to see what those conditions are, you can click here. The http.CrossOriginProtection middleware The new http.CrossOriginProtection middleware works by checking the values in a request's Sec-Fetch-Site and Origin headers to determine where the request is coming from. It will automatically reject any non-safe requests that are not from the same origin, and will send the client a 403 Forbidden response. The http.CrossOriginProtection middleware has some limitations, which we'll discuss in a moment, but it is robust and simple to use, and a great addition to the standard library. How it works Modern browsers automatically include the Sec-Fetch-Site header in requests. This header indicates the relationship between the origin of the page making the request, and the origin of the page being requested. Two pages are considered to have the same origin if their scheme, hostname and port (if present) exactly match, in which case the browser will include a Sec-Fetch-Site: same-origin header in the request. If the two pages don't have the same origin, the Sec-Fetch-Site header will be set to a different value to indicate this, and http.CrossOriginProtection will reject the request. If no Sec-Fetch-Site header is present, http.CrossOriginProtection will fall back to checking the Origin header. Specifically, it will compare the request's Origin header and Host header to see if they match. If they don't match, then it considers the request to not be from the same origin and it will reject it. If neither the Sec-Fetch-Site nor Origin headers are present, then it assumes the request is not coming from web browser and will always allow the request to proceed. The checks described above only take place on requests with non-safe methods (POST, PUT, etc.). Requests with safe HTTP methods (GET, OPTIONS, etc.) are always allowed to proceed. If you're interested in learning more about the design and decision making behind http.CrossOriginProtection, the original proposal by Filippo Valsorda is an excellent read. At its simplest, you can use it like this: File: main.go package main import ( "fmt" "log/slog" "net/http" "os" ) func main() { mux := http.NewServeMux() mux.HandleFunc("/", home) slog.Info("starting server on :4000") // Wrap the mux with the http.NewCrossOriginProtection middleware. err := http.ListenAndServe(":4000", http.NewCrossOriginProtection().Handler(mux)) if err != nil { slog.Error(err.Error()) os.Exit(1) } } func home(w http.ResponseWriter, r *http.Request) { fmt.Fprint(w, "Hello!") } If you want, it's also possible to configure the behavior of http.CrossOriginProtection. Configuration options include being able to add trusted origins (from which cross-origin requests are allowed), and the ability to use a custom handler for rejected requests instead of the default 403 Forbidden response. When I've wanted to customize the behavior, I've been using a pattern like this: File: main.go package main import ( "fmt" "log/slog" "net/http" "os" ) func main() { mux := http.NewServeMux() mux.HandleFunc("/", home) slog.Info("starting server on :4000") err := http.ListenAndServe(":4000", preventCSRF(mux)) if err != nil { slog.Error(err.Error()) os.Exit(1) } } func preventCSRF(next http.Handler) http.Handler { cop := http.NewCrossOriginProtection() cop.AddTrustedOrigin("https://foo.example.com") cop.SetDenyHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusBadRequest) w.Write([]byte("CSRF check failed")) })) return cop.Handler(next) } func home(w http.ResponseWriter, r *http.Request) { fmt.Fprint(w, "Hello!") } Limitations The big limitation of http.CrossOriginProtection is that it is only effective at blocking requests from modern browsers. Your application will still be vulnerable to CSRF attacks coming from older (generally pre-2020) browsers which do not include at least one of the Sec-Fetch-Site or Origin headers in requests. Right now, browser support for the Sec-Fetch-Site header is at 92%, and for Origin it is 95%. So — in general — relying on http.CrossOriginProtection is not sufficient as your only protection against CSRF. It's also important to note that the Sec-Fetch-Site header is only sent when your application has a "trustworthy origin" — which basically means that your application needs to be using HTTPS in production (or localhost in development) for http.CrossOriginProtection to work to its full potential. And you should also be aware that when no Sec-Fetch-Site header is present in a request, and it falls back to comparing the Origin and Host headers, the Host header does not include the scheme. This limitation means that http.CrossOriginProtection will wrongly allow cross-origin requests from http://{host} to https://{host} when there is no Sec-Fetch-Site header present but there is an Origin header. To mitigate this risk, you should ideally configure your application to use HTTP Strict Transport Security (HSTS). Enforcing TLS 1.3 Looking into this got me wondering... What if you're already planning to use HTTPS and enforce TLS 1.3 as the minimum supported TLS version? Could you be confident that all web browsers which support TLS 1.3 also support either the Sec-Fetch-Site or Origin headers? As far as I can tell from the MDN compatibility data and tables from Can I Use, the answer is "yes" for (almost) all major browsers. If you enforce TLS 1.3 as the minimum version: Older browsers which don't support TLS 1.3 simply won't be able to connect to your application. For the modern major browsers that do support TLS 1.3 and can connect, you can be confident that at least one of the Sec-Fetch-Site or Origin headers are supported — and therefore http.CrossOriginProtection will work effectively. The only exception to this I can see is Firefox v60-69 (2018-2019), which did not support the Sec-Fetch-Site header and did not send the Origin header for POST requests. This means that http.CrossOriginProtection will not work effectively to block requests originating from that browser. Can I Use puts usage of Firefox v60-69 at 0%, so the risk here appears very low — but there are probably some computers somewhere in the world still running it. Also, we only have this information for the major browsers — Chrome/Chromium, Firefox, Edge, Safari, Opera and Internet Explorer. But of course, other browsers exist. Most of them are forks of Chromium or Firefox and therefore will likely be OK, but there's no guarantee here and it is hard to quantify the risk. So if you use HTTPS and enforce TLS 1.3, it's a huge step forward in making sure that http.CrossOriginProtection can work effectively. However, there remains a non-zero risk that comes from Firefox v60-69 and non-major browsers, so you may want to add some defense-in-depth and utilize SameSite cookies too. We'll talk more about SameSite cookies in a moment, but first we need to take a quick detour and discuss the difference between the terms origin and site. Cross-site vs cross-origin In the world of web specifications and web browsers, cross-site and cross-origin are subtly different things, and in a security context like this it's important to understand the difference and be exact about what we mean. I'll quickly explain. Two websites have the same origin if they share the exact same scheme, hostname, and port (if present). So https://example.com and https://www.example.com are not the same origin because the hostnames (example.com and www.example.com) are different. A request between them would be cross-origin. Two websites are 'same site' if they share the same scheme and registerable domain. Note: The registerable domain is the part of the hostname just before (and including) the effective TLD. Here are a few examples: For https://www.google.com/ the TLD is com and the registerable domain is google.com. For https://login.mail.ucla.edu the TLD is edu and the registerable domain is ucla.edu. For https://www.gov.uk, the TLD is gov.uk and the registerable domain is www.gov.uk. You can find the complete list of effective TLDs here. So https://example.com, https://www.example.com and https://login.admin.example.com are all considered to be the same site because the scheme (https) and registerable domain (example.com) are the same. A request between these would not be considered to be cross-site, but it would be cross-origin. Note: Some browser versions use a different definition of same-site which doesn't require the same scheme, only the same registrable domain. For these browser versions, https://admin.example.com and http://blog.example.com would also be considered same-site. Nowadays, this is typically referred to as schemaless same-site, but in historical versions or documentation it may have just been called same-site. So what are the points that I'm building up to here? Go's http.CrossOriginProtection middleware is accurately and appropriately named. It blocks cross-origin requests. It's more strict than it would be if it only blocked cross-site requests, because it also blocks requests from other origins under the same site (i.e. registrable domain). This is useful because it helps to prevent a situation where your janky-not-been-updated-in-the-last-decade WordPress blog at https://blog.example.com is compromised and used to launch a request forgery attack at your important https://admin.example.com website. When most people — myself included — casually talk about "CSRF attacks", what we are referring to most of the time is actually cross-origin request forgery, not just cross-site request forgery. It's a shame that CSRF is the commonly used and known acronym to describe this family of attacks, because most of the time CORF would be more accurate and appropriate. But hey! That's the messy world we live in. For the rest of this post though, I'll use the term CORF instead of CSRF when that is exactly what I mean. SameSite cookies The SameSite cookie attribute has generally been supported by web browsers since 2017, and by Go since v1.11. If you set the SameSite=Lax or SameSite=Strict attributes on a cookie, that cookie will only be included in requests to the same site that set it. In turn, that prevents cross-site request forgery attacks (but not cross-origin attacks from within the same site). There is some good news here — all major browsers that support TLS 1.3 also fully support SameSite cookies, with no exceptions that I can see. So if you enforce TLS 1.3, you can be confident that all the major browsers using your application will respect the SameSite attribute. This means that by using SameSite=Lax or SameSite=Strict on your cookies, you cover off the risk of cross-site request forgeries from Firefox v60-69 that we talked about earlier. Putting it all together If you combine using HTTPS, enforcing TLS 1.3 as the minimum version, using SameSite=Lax or SameSite=Strict cookies appropriately, and using the http.CrossOriginProtection middleware in your application, as far as I can see there are only two unmitigated CSRF/CORF risks from major browsers: CORF attacks from within the same site (i.e. from another subdomain under your registrable domain) in Firefox v60-69. CORF attacks from a HTTP version of your origin, from browsers that do not support the Sec-Fetch-Site header. For the first of these risks, if you don't have any other websites under your registrable domain, or you're confident that the websites are secure and uncompromised, then this might be a risk that you're willing to accept given the extremely low usage of Firefox v60-69. For the second, if you don't support HTTP on your origin at all (including redirects) then this isn't something you need to worry about. Otherwise, you can mitigate the risk by including a HSTS header on your HTTPS responses. At the start of this article, I said that not using a token-based CSRF check might be OK under certain conditions. So let's run through what those are: Your application uses HTTPS and enforces TLS 1.3 as the minimum version. You accept that users with older browsers will not be able to connect to your application at all. You follow good-practice and never change important application state in response to requests with the safe methods GET, HEAD, OPTIONS or TRACE. You use both the http.CrossOriginProtection middleware and SameSite=Lax or SameSite=Strict cookies. It's important to still use SameSite cookies for general defense in depth, but more specifically to mitigate CSRF attacks from Firefox v60-69. Because of the unprotected risk of a same-site CORF attack from Firefox v60-69, you either don't have any other websites under your registrable domain, or you're confident that they're secure and uncompromised. There is either no HTTP version of your application origin at all, or you include a HSTS header on your HTTPS responses. Finally, you are willing to accept the difficult-to-quantify risk of CSRF/CORF attacks from non-major browsers that support TLS 1.3 but don't support the Origin header, Sec-Fetch-Site header or SameSite cookies. Does any such browser exist? I don't know, and I'm not sure there's a way to answer that question with 100% confidence. So you'll need to do your own risk assessment here, and it's a risk that you probably only want to accept if your application is a low-value target and the impact of a successful CSRF/CORF attack is both isolated and minor.
Alex Edwards Oct 14, 2025 -
Learn how to combine The Elm Architecture (TEA) principles with Pinia's private store pattern for testable, framework-agnostic state management in Vue applications.
alexop.dev Oct 3, 2025 -
It’s been almost a year since the last Watermill release post, and we have a bunch of news to share! In case you’re new here, Watermill is a Go library for building event-driven applications the easy way. It’s like an HTTP router but for messages. Watermill is a library, not a framework, so there’s no vendor lock-in. Watermill is now close to 9k GitHub stars and 79 contributors, just on the main repository. It supports 13 Pub/Subs including Kafka, Redis Streams, NATS, Google Cloud Pub/Sub, Amazon SQS/SNS, SQL, and RabbitMQ. See the documentation for more details. Let’s look at what’s new. SQLite Pub/Sub Watermill now officially supports the SQLite Pub/Sub. The messages are moved between publishers and subscribers using tables in an SQLite database. A huge thank you to Dima Kotik who contributed the new package: github.com/ThreeDotsLabs/watermill-sqlite. See the documentation. SQLite Pub/Sub is simple to set up, as you don’t need to run any external services: everything is just a file on disk. It supports persistence, so you don’t lose messages when your application restarts. So far, we recommended PostgreSQL or MySQL Pub/Sub for when you don’t want to set up a separate message broker. Now it’s even easier to start with SQLite. It can also be a great alternative to using the GoChannel Pub/Sub in smaller projects. We’ll soon publish Dima’s article about when it shines. Stay tuned! The implementation has two packages: modernc.org/sqlite and zombiezen.com/go/sqlite. You can choose either one when creating the Pub/Sub. Here’s an example of creating a Subscriber with the modernc driver: import "github.com/ThreeDotsLabs/watermill-sqlite/wmsqlitemodernc" // ... subscriber, err := wmsqlitemodernc.NewSubscriber( db, wmsqlitemodernc.SubscriberOptions{ InitializeSchema: true, Logger: logger, }, ) messages, err := subscriber.Subscribe(ctx, "messages") for msg := range messages { fmt.Printf("Received message: %s, payload: %s\n", msg.UUID, string(msg.Payload)) msg.Ack() } And then publishing messages: import "github.com/ThreeDotsLabs/watermill-sqlite/wmsqlitemodernc" // ... publisher, err := wmsqlitemodernc.NewPublisher( db, wmsqlitemodernc.PublisherOptions{ InitializeSchema: true, Logger: logger, }, ) msg := message.NewMessage(watermill.NewUUID(), []byte(`{"message": "Hello, world!"}`)) err := publisher.Publish("messages", msg) We consider this a beta release: the full test suite passes (and it’s pretty extensive!), but we want to gather more feedback from real-world usage. If you try it, please share your experience. You can use the main repository’s issues or join our Discord. Watermill Quickstart We recently launched Watermill Quickstart, a free hands-on training that teaches you how to use Watermill in your project. We keep looking for the most effective way to learn programming topics. It’s clear that reading docs and books is not enough: you learn best when you actually practice by writing code, failing, and fixing it. The common advice for newcomers in any technology is to “create a pet project.” But coming up with an idea and sticking with it long enough to learn something isn’t trivial. The Quickstart guides you to build this kind of project. You code in your own IDE, so the next time you want to use what you learned, you already know the setup and have the experience. If you try it, let us know what you think! Watermill Quickstart Watermill v1.5 While there are no major new features in the main Watermill package, we reviewed a ton of PRs and merged many improvements and fixes. You can see the releases page and the full changeset on GitHub. Thank you to everyone who contributed, and please keep the PRs coming! Let’s look at some API changes and deprecations. CQRS Marshaler updates cqrs.ProtobufMarshaler is now deprecated and replaced with cqrs.ProtoMarshaler. Ideally, we’d remove it completely to get rid of the github.com/gogo/protobuf dependency, but that would be a breaking change. We recommend using ProtoMarshaler, which uses google.golang.org/protobuf instead. There is now an easier way to decorate marshallers. If you want to add some extra logic when marshaling or unmarshaling commands or events, you usually wrap the marshaler in a custom struct. Now you can use cqrs.CommandEventMarshalerDecorator to do that with less boilerplate. cqrsMarshaler := cqrs.CommandEventMarshalerDecorator{ CommandEventMarshaler: cqrs.JSONMarshaler{}, DecorateFunc: func(v any, msg *message.Message) error { event, ok := v.(Event) if !ok { return fmt.Errorf("%T does not implement Event and can't be marshaled", v) } partitionKey := event.PartitionKey() if partitionKey == "" { return fmt.Errorf("PartitionKey is empty") } msg.Metadata.Set("partition_key", partitionKey) return nil }, } Small API changes You can now create messages with context using message.NewMessageWithContext. msg := message.NewMessageWithContext(ctx, watermill.NewUUID(), payload) This is a trivial addition, but we hope it’ll help make it clear that it’s the message that carries the context. For newcomers, it can be confusing that Publish() takes no context argument. This API is similar to net/http with NewRequestWithContext. A tiny change that should improve the developer experience. Similarly, we deprecated the Router’s AddNoPublisherHandler in favor of AddConsumerHandler. When we initially designed the Router, we assumed the default behavior of handlers would be to always publish messages. In practice, it’s the opposite: most handlers consume messages and don’t publish anything, or publish the message in other ways. AddNoPublisherHandler sounds a bit awkward, so we decided to rename it. Context in GoChannel Pub/Sub There is now an option to propagate the message context to the handler when using the GoChannel Pub/Sub. This has been something many people have asked for, especially for passing some arbitrary values through the context. You can enable it by setting the PreserveContext field to true. pubSub := gochannel.NewGoChannel( gochannel.Config{ PreserveContext: true, }, logger, ) Since GoChannel works in-memory, you can access the context you set when creating the message in the handler. However, keep in mind this is not how most Pub/Subs work. When using Kafka, Redis, or any other “proper” Pub/Sub, the context is lost when the message is sent over the network. Use it only if you know what you’re doing! If you switch to another Pub/Sub later, this will break. Watermill SQL v4 We finally released watermill-sql v4.0.0 after a long time as a release candidate. You can now use it with packages other than database/sql, like pgx (Postgres driver) or ORMs. This includes support for transactions, which is key for the Outbox pattern. Shoutout to @julesjcraske for contributing these changes! Subscriber and Publisher constructors now take a Beginner interface argument instead of *sql.DB. subscriber, err := sql.NewSubscriber( sql.BeginnerFromStdSQL(db), sql.SubscriberConfig{ SchemaAdapter: sql.DefaultMySQLSchema{}, OffsetsAdapter: sql.DefaultMySQLOffsetsAdapter{}, InitializeSchema: true, }, logger, ) Migration: Wrap your *sql.DB in sql.BeginnerFromStdSQL(db) when passing it to the watermill-sql API. Pgx has its own BeginnerFromPgx(db) constructor. When working with transactions, use TxFromStdSQL(tx) or TxFromPgx(tx) instead. If you use another driver or ORM, you can implement your own adapter that implements the Beginner or Tx interface. See the implementation. The other major change is that we decided to make SchemaAdapter and OffsetsAdapter interfaces more flexible. Instead of raw arguments, they now take a params struct and always return an error. We will stick to this approach for most APIs going forward. While more verbose, it’s easier to extend without breaking changes. type SchemaAdapter interface { InsertQuery(params InsertQueryParams) (Query, error) SelectQuery(params SelectQueryParams) (Query, error) UnmarshalMessage(params UnmarshalMessageParams) (Row, error) SchemaInitializingQueries(params SchemaInitializingQueriesParams) ([]Query, error) SubscribeIsolationLevel() sql.IsolationLevel } type OffsetsAdapter interface { AckMessageQuery(params AckMessageQueryParams) (Query, error) ConsumedMessageQuery(params ConsumedMessageQueryParams) (Query, error) NextOffsetQuery(params NextOffsetQueryParams) (Query, error) SchemaInitializingQueries(params OffsetsSchemaInitializingQueriesParams) ([]Query, error) BeforeSubscribingQueries(params BeforeSubscribingQueriesParams) ([]Query, error) } Migration: If you use the default schema or offsets adapters, you don’t need to change anything. If you have custom implementations, update the method signatures to match the new ones. Watermill GoogleCloud v2 The Go Pub/Sub client library v1 will lose support next year. We updated watermill-googlecloud to use the new v2 library. There’s only one breaking change: SubscriberConfig no longer exposes the SubscriptionConfig field. Instead, it now has the GenerateSubscription function. The function returns *pubsubpb.Subscription, which mostly maps 1:1 with the old SubscriptionConfig. Migration: Replace SubscriptionConfig with GenerateSubscription. googlecloud.SubscriberConfig{ ProjectID: "tests", GenerateSubscriptionName: subscriptionName, - SubscriptionConfig: pubsub.SubscriptionConfig{ - RetainAckedMessages: false, - EnableMessageOrdering: enableMessageOrdering, - }, + GenerateSubscription: func(params googlecloud.GenerateSubscriptionParams) *pubsubpb.Subscription { + return &pubsubpb.Subscription{ + RetainAckedMessages: false, + EnableMessageOrdering: enableMessageOrdering, + } + }, Unmarshaler: unmarshaler, }, We considered mapping the config to make this change non-breaking. While it’s possible, we wouldn’t get rid of the old dependency, and we’ve seen reports that the old and new clients don’t work well together. So this means a major version bump, but the migration should be straightforward. Hall of Fame 🏆 We had 43 unique contributors to all Watermill repositories since the last release post! 💪 Thank you all for your help! @Abdullah-AlAttar @abramovd @AceFire6 @addshore @alexandregv @Amr-Shams @avlajcic-axilis @chaindead @chengehe @co60ca @dkotik @edebernis @ekazakas @EvanMad @HansvdLaanNedap @HolyKingCrusader @jgersdorf @jjcollinge @jlevesy @jpradass @julesjcraske @marczahnfc @matdurand @minxinyi @NathanBaulch @nkonev @Orbsynated @petergere @pxwanglu @rhhammond @samartha-kar23 @seamusv @stephanvanzwienen @sysradium @thpk @tjnet @unaik @weiye-Lee @whosehang @xico42 @yashb042 @ygaros @ytnsym Docs refresh We updated watermill.io a bit, so it’s easier to tell what the library is about. It should also be easier to choose the proper learning path. Once again, give the Quickstart a try and let us know how it went. We consider making free training like this for other open-source libraries as well. If you have any suggestions, please share!
Learn Building Modern Go applications Sep 23, 2025 -
The biggest paradox in life: the hardest things are usually the ones that help you grow. Exploring why challenge and difficulty are essential for meaning and personal development.
alexop.dev Sep 10, 2025 -
Build a Vue 3 microfrontend app with Module Federation. Clear decisions, working code, and a small reference project.
alexop.dev Sep 10, 2025 -
A few weeks ago Anton Zhiyanov published the blog post Expressive tests without testify/assert. It's a good and well thought-out post, and I recommend giving it a read if you haven't already. In the post, Anton makes the argument for not using packages like testify/assert for your test assertions, and instead creating your own minimal set of assertion helpers to use in your tests. In fact, so minimal that there are only 3 helpers he uses: AssertEqual, AssertErr and AssertTrue. There are some people who would argue that even this is too much, and that you shouldn't use assertion functions in your tests at all. In fact, the Go Code Review Comments for Tests states that using assert packages should be avoided, which we'll talk about in more detail at the end of this post. But I agree with the general direction of Anton's thinking. I do use assertion functions — and I've always preferred to write my own rather than using a third-party package. Over time I've whittled them down to a standard collection of nine basic functions that I use: Assertion What it checks Equal(got, want) Checks that got and want are equal NotEqual(got, want) Checks that got and want are not equal True(got) Checks that got is true False(got) Checks that got is false Nil(got) Checks that got is nil NotNil(got) Checks that got is not nil ErrorIs(got, want) Checks that got is an error that wraps or equals want ErrorAs(got, target) Checks that got is an error that can be assigned to target via errors.As MatchesRegexp(got, pattern) Checks that got matches the regex pattern Between these nine functions, I'm able to easily do the vast majority of the checks that I want in my tests. Here are some examples from a web application that I'm currently working on: assert.Equal(t, w.StatusCode, http.StatusTeapot) assert.Equal(t, w.Header().Get("X-Custom-Header"), "custom-value") assert.NotEqual(t, updatedSession.token, originalSession.token) assert.True(t, defaultShutdownPeriod > defaultWriteTimeout) assert.True(t, strings.Contains(buf.String(), "level=ERROR")) assert.False(t, strings.Contains(string(decodedCookieValue), "this is a test value")) assert.Nil(t, err) assert.ErrorIs(t, err, sql.ErrNoRows) assert.MatchesRegexp(t, user.HashedPassword, `^\$2a\$12\$[./0-9A-Za-z]{53}$`) From the perspective of someone reading the code, I think it's quite easy to understand what these assertions are checking — even if you've never seen them before. And this might be personal preference, but when writing tests I actually prefer having only a small number of basic assertion functions to remember and pick from, rather than lots of very specific ones. If there is a complex check, which can't be done in a single line as part of the function call, I normally create an additional function and use it in conjunction with the True or False assertions. For example, when testing a web application, I will sometimes want to check if an HTML response body contains a specific HTML node (based on a CSS selector), so I will make a containsHTMLNode() function and then use it in my tests like this: assert.True(t, containsHTMLNode(t, res.Body, `meta[name="page"][content="home"]`)) assert.True(t, containsHTMLNode(t, res.Body, `form[method="POST"][action="/login"]`)) In theory, these assertion helpers could be reduced further. For example, the Nil(got) and NotNil(got) functions could be dropped in favour of using Equal(got, nil) and NotEqual(got, nil). Or MatchesRegexp(got, pattern) could be dropped in favour of using True() to check that a got value matches a specific regexp pattern. But these are checks I use often enough that I like having a specific assertion function for them. Go back to Anton's post for a moment, he effectively combines the assert.Nil(), assert.NotNil(), assert.ErrorIs() and assert.ErrorAs() functions that I have into a single AssertErr() function. The exact kind of check that is carried out by AssertErr() depends on what arguments you pass, or don't pass, to it. However, I prefer assertion functions to be responsible for checking one specific thing. I think it's less prone to mistakes, as well as clearer for a reader exactly what is being checked. Overall, I'm happy to have a few more assertion functions in exchange for some extra convenience, clarity and precision. Here's the complete code that I'm currently using for those functions: package assert import ( "errors" "reflect" "regexp" "testing" ) func Equal[T any](t *testing.T, got, want T) { t.Helper() if !isEqual(got, want) { t.Errorf("got: %v; want: %v", got, want) } } func NotEqual[T any](t *testing.T, got, want T) { t.Helper() if isEqual(got, want) { t.Errorf("got: %v; expected values to be different", got) } } func True(t *testing.T, got bool) { t.Helper() if !got { t.Errorf("got: false; want: true") } } func False(t *testing.T, got bool) { t.Helper() if got { t.Errorf("got: true; want: false") } } func Nil(t *testing.T, got any) { t.Helper() if !isNil(got) { t.Errorf("got: %v; want: nil", got) } } func NotNil(t *testing.T, got any) { t.Helper() if isNil(got) { t.Errorf("got: nil; want: non-nil") } } func ErrorIs(t *testing.T, got, want error) { t.Helper() if !errors.Is(got, want) { t.Errorf("got: %v; want: %v", got, want) } } func ErrorAs(t *testing.T, got error, target any) { t.Helper() if got == nil { t.Errorf("got: nil; want assignable to: %T", target) return } if !errors.As(got, target) { t.Errorf("got: %v; want assignable to: %T", got, target) } } func MatchesRegexp(t *testing.T, got, pattern string) { t.Helper() matched, err := regexp.MatchString(pattern, got) if err != nil { t.Fatalf("unable to parse regexp pattern %s: %s", pattern, err.Error()) return } if !matched { t.Errorf("got: %q; want to match %q", got, pattern) } } func isEqual[T any](got, want T) bool { if isNil(got) && isNil(want) { return true } if equalable, ok := any(got).(interface{ Equal(T) bool }); ok { return equalable.Equal(want) } return reflect.DeepEqual(got, want) } func isNil(v any) bool { if v == nil { return true } rv := reflect.ValueOf(v) switch rv.Kind() { case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice: return rv.IsNil() } return false } Are assertion functions an anti-pattern? As I mentioned at the start of this post, the Go Wiki says that using assert packages should be avoided. It starts with this example of some 'bad' test code: assert.IsNotNil(t, "obj", obj) assert.StringEq(t, "obj.Type", obj.Type, "blogPost") assert.IntEq(t, "obj.Comments", obj.Comments, 2) assert.StringNotEq(t, "obj.Body", obj.Body, "") And suggests this as a 'good' alternative: if obj == nil || obj.Type != "blogPost" || obj.Comments != 2 || obj.Body == "" { t.Errorf("AddPost() = %+v", obj) } Let's quickly run through the arguments in the Go Wiki for these approaches being good and bad. [The bad code] either stops the test early (if assert calls t.Fatalf or panic) or omits interesting information about what the test got right There are packages out there (such as testify/require) that will stop a test early on the first assertion failure, and when they do you lose information about what subsequent checks would have passed. But not all of them do this, and if you make your own helpers for test assertions, you control what they do. You can call t.Errorf() to record the failure and continue the test if you want to. [The bad code] also forces the assert package to create a whole new sub-language instead of reusing the existing programming language (Go itself) I think this is a valid point and worth keeping in mind. Sometimes it can be frustrating to have to learn how a third-party package works, and to remember its API and syntax. And if the package is used in a project that lots of people work on, you're forcing all of them to spend the time to learn it. Sometimes it's easier to just read and write Go code that uses the standard library — even if it means you end up with more lines of code. But that said, I'm not sure that having a small number of basic assertion functions adds that much overhead... even for new people working on a codebase. Does having 3 assertion helpers like Anton, or 9 like me, really count as creating a whole new sub-language? Even if you argue that it does, it's a very small sub-language. Assert libraries make it too easy to write imprecise tests I think this is a good point in some — but not all — cases. If you're using a package that does different kinds of assertion checks in the same function (e.g. depending on the type of the argument passed to it, or the presence or not of a variadic argument) then yes, it's possible to see how it potentially increases the risk of bugs or a loss of precision in your tests. But if the assertion function checks one thing and one thing only, I don't see how it would be less precise. The assert.Equal() function that I use is a good example of this. It's imprecise because it checks whether both values are nil or whether both are the same based on an Equals() method or they are equal according to reflect.DeepEqual(). The ors introduce a subtle loss of precision that wouldn't exist if we were only checking one of those things. However, the go-cmp/cmp.Equal function, which the Go Wiki goes on to recommend using for equality checks, is imprecise in a similar way. I'm not sure that the assert.Equal() code above is really any worse in this sense. [Assert libraries] inevitably end up duplicating features already in the language, like expression evaluation, comparisons, sometimes even more. Yes. And I think this is why my preference is to use a small set of very basic assertions, like assert.Equal() and assert.True(). It means that I can write assertions like assert.True(t, len(mySlice) > 3) or assert.False(t, strings.Contains(name, "admin")) using the normal Go functions and operators. I don't get stuck down a rabbit hole implementing helpers like assert.SliceLengthGreaterThan() or assert.StringDoesNotContain() for every kind of check I need to do. On the flip side, the Go Wiki doesn't provide balance and mention the upsides of using assertion helpers, which is a shame. In terms of developer experience, I suspect that even the most hardened Gopher would agree that writing three lines of code like this: assert.Equal(t, w.StatusCode, http.StatusTeapot) assert.ErrorIs(t, err, sql.ErrNoRows) assert.True(t, defaultShutdownPeriod > defaultWriteTimeout) Is a faster and more enjoyable experience than writing the equivalent code like this: if w.StatusCode != http.StatusTeapot { t.Errorf("got %d; want %d", w.StatusCode, http.StatusTeapot) } if !errors.Is(err, sql.ErrNoRows) { t.Errorf("got error %q; want error to be or wrap %q", err.String(), sql.ErrNoRows) } if defaultShutdownPeriod <= defaultWriteTimeout { t.Errorf("default shutdown period %s must be greater than default write timeout %s", defaultShutdownPeriod, defaultWriteTimeout) } Not only is the code shorter, but it takes away the cognitive overhead of having to write a failure message for each check. Which is both good and bad. I find it good because it frees up my brain to focus on arguably the most important thing — which is the logic of the test and what is being tested. When I'm thinking about test logic, I don’t want to get distracted trying to craft a perfect failure message, or having to look up for the 100th time whether it is got before want or want before got. Being able to type out assertions quickly, without losing my train of thought, is something that I really value and appreciate. And it's bad, because having useful and thoughtful failure messages can make debugging a problem easier. Getting a failure message that reads like this: --- FAIL: TestServerConfiguration (0.00s) — FAIL: TestServerConfiguration/Default_timeouts_are_reasonable (0.00s) server_test.go:24: default shutdown period 5s must be greater than default write timeout 10s Is much better than using an assert.True() helper and getting a failure message like this: --- FAIL: TestServerConfiguration (0.00s) — FAIL: TestServerConfiguration/Default_timeouts_are_reasonable (0.00s) server_test.go:22: got: false; want: true In this second example, all you have to go on to start debugging the failure is the file name and line number of the check — it doesn't even include the value that caused the check to fail. I do think this, in particular, is a genuine downside of the assert.True() and assert.False() helpers that I use. Summary I've found that the nine assertion helpers I shared above have worked well for me in a variety of projects — and they might work well for you too. But ultimately whether they are the right fit depends on your preferences, your team members, and the specific project. If you use a small collection of basic assertion functions like this, rather than a large third-party package, then I think that most of the criticisms that the Go Wiki makes of assert packages don't really apply. But you still need to accept that the failure messages printed by assertion functions may not be as helpful as a tailored, specific, failure message would be. On the plus side, they make for a good developer experience when writing tests. I particularly appreciate that they are quick to write and allow my mind to stay focused on the logic of what I'm testing. And on balance, anything that encourages me to write more tests is probably a good thing : )
Alex Edwards Aug 22, 2025 -
Learn how MCP (Model Context Protocol) standardizes AI tool integration, enabling LLMs to interact with external services, databases, and APIs through a universal protocol similar to USB-C for AI applications.
alexop.dev Aug 10, 2025 -
The peak of hype isn’t the best moment to reflect on AI. Will it take your job, or is it the next fad like NFTs? Are AI startups ridiculously overvalued, or are the companies that sleep on AI doomed? Time will tell. LLMs are far from perfect, but I’m excited they’re here anyway. Not because of a silly promise to make me 10x more productive, but because they can solve some problems that were previously unsolvable. If you enjoy building software like me and Robert, it’s a new tool worth trying. And there’s a chance AI experience helps you create a better product or land a good job. Did you ever get stuck debugging your project at 2 AM, and went to sleep frustrated? We spent a few weeks adding an AI Mentor to our learning platform to help our students avoid this. As always, it wasn’t as easy as most tutorials show. Trivial use cases, like a weather-checking assistant, are fun to make, but aren’t much more useful than the default chat interface of any model. I’ve heard before that implementing an AI feature is easy, but making it work correctly and reliably is the hard part. You can quickly build an impressive demo, but it’ll be far from production grade. We can now confirm this is true. Plus, we’ve seen something not talked about that much: how an AI feature can fail in production and mislead users. I want to share what we learned in detail, hopefully saving you time on research. Let’s set aside the world-changing stories for a moment and see what we can build with this stuff. Note If you’re just starting out I know many developers stay away from AI because they consider it overhyped, unethical, or useless. They also feel massive FOMO because there’s currently no other topic in tech. If you feel this way, I’m not trying to change your mind. I encourage you to approach it like an engineer trying out a new technology. Look for what problems it can solve, while most people are busy applying AI to everything. Once you use it in practice, you’ll have a good idea where it’s useful and where it falls short. You’ll make up your mind much better this way than reading hot takes from CEOs and tech bros. Why AI Mentor? We’re not VC-backed and don’t seek to sell our company, so there’s little value in slapping “AI-First” on our homepage. We wanted to use LLMs to solve a real problem our users had. Our Academy platform is all about learning by building real-life projects. One of our challenges is helping students finish the training. If you have ever tried an online course, you may know this problem — you put it off after a few days and never return, distracted by something else. One reason to put learning off is that you get stuck. While working on complete projects, it can be difficult to spot a bug. Example solutions don’t always help, since each project is unique. We help students move forward by reviewing their solutions. Often, it’s a small typo or mistake that’s easy to catch with fresh eyes. There’s no learning value in fighting with such bugs for hours. Frustrated and stuck, some people give up. Last year, we thought LLMs could help. You can feed them entire codebases and ask them to point out or even fix errors. And they are always available, in contrast to our limited time for customer support. Back then, the models were expensive, far less impressive (GPT-4 had just been released), and there were few sources to learn from. But most of all, we lacked time and postponed the project. We came back to it earlier this year. (If you want to see it live, the first few modules of Go in One Evening require no login.) Milestones We decided to start with an MVP and focus on the fundamentals, so we’re not distracted by nice-to-haves. We picked a few milestones and planned to implement each one in production before moving on to the next one. A single “Help me!” button on the website for Go in One Evening. Users would receive a single message with a hint on what’s wrong with their code. Conversations. While a chatbot wasn’t our main goal, asking a follow-up question about the solution could be helpful. Enable the Mentor for Go Event-Driven. Go in One Evening is a simpler training with shorter exercises. We assumed solving the issues found there would be easier compared to Go Event-Driven, where students build more complex projects. (The final project is about 4k lines of Go code.) The mockup of our initial idea. The end result. The “Help Me!” Button We created a proof of concept that should be easy to extend. We started with two HTTP endpoints: a POST for requesting help (for now with a constant “Help me” message) a GET streaming all messages over Server-Sent Events (SSE). In theory, a single endpoint could do both, but we wanted it to be asynchronous by default. Users can click the button and refresh the page without losing the response. Separating writes from reads is a good practice, especially for long-running tasks. The downside is a bit more complex setup, but we use proven tools that we know well. We prefer event-driven patterns for asynchronous architecture. We use Watermill and Google Cloud Pub/Sub to trigger the Mentor and send updates back to the user. All of our events end up in BigQuery, so we get storage for conversations out of the box. This is important because we need the history of messages to include in the next prompts. High-level overview. This seemingly simple setup already has some challenges, even without considering the AI part: How to handle race conditions between the frontend app, the Pub/Sub, and the SSE handler? How to handle errors and retries? How to deal with BigQuery’s insert and select latency? How to approach message ordering? Events Event-driven patterns turned out to be a good fit for working with LLMs. Behind the scenes, there are many moving parts, and most need no input from the user. Failures and retries often happen, but they are all hidden behind a single response. It’s why we don’t recommend using a single HTTP handler for the whole process. There are delays and interruptions, and you need to handle them, so your users don’t see the mess. Most LLM APIs support streaming chunks of messages. You can show them on the UI before the model has finished generating the response. SSE is a perfect fit for this. The first response of the SSE handler is a list of all messages (for the given user and exercise). Then, we stream all chunks over the Pub/Sub to the SSE handler and then to the browser. We had a great experience working with SSE before, and watermill-http supports it out of the box, so it integrates nicely with our architecture. We’ve improved the implementation by adding support for generics (although it’s not public yet, we may release it at some point). It’s all hidden behind a high-level interface. The HTTP handler receives a channel of incoming events and a channel to send responses back to the client. func (h *Handler) HandleEventStream( r *http.Request, events <-chan *events.MentorMessageSent, responses chan<- any, ) { // Send the initial response with all messages messages, _ := h.getAllMessages(r) for _, msg := range messages { responses <- msg } // Process incoming events for event := range events { // ... responses <- response{} } } Note If you want to learn more about Server-Sent Events, we have a dedicated post about implementing it in depth: Live website updates with Go, SSE, and htmx. Calling LLMs We initially used langchaingo for calling the LLMs, but the API felt a bit bloated for our needs. We created a small wrapper on top of the official SDKs to be able to switch the models quickly. The design of most LLM APIs is very similar. You can describe a prompt call like this: type Model interface { Prompt(history []Message) (Message, error) } It’s a function that accepts a chat history and returns the next message in the chat. In practice, we ended up with a few more details: func (l *LLM) Prompt( ctx context.Context, history []Message, user User, opts PromptOpts, ) (*Output, error) { // ... } type PromptOpts struct { MaxOutputTokens int Model ModelName Tools []ToolSchema } func (o *Output) Listen() (<-chan Chunk, error) { // ... } func (o *Output) WaitAndGetAll(ctx context.Context) (string, error) { // ... } This is a tiny abstraction over the LLM API, just enough to make it easy to switch between models. Similarly to SSE, Go channels are the perfect API to stream chunks. It works well with the whole asynchronous model, and it’s great to have support for it on the language level. Prompts & Context Engineering We spent a lot of time iterating over the prompts and the context we fed to the model. For example, we wanted the Mentor to hint at the right direction, but not simply give the complete answer. We ended up with quite a long prompt for something as simple as “Help with this exercise”. We have many files related to each exercise (the user’s solution, an example solution, and many others). Even though models now easily handle long contexts, we still need to be careful about how much we feed them. Deciding what context to provide needs careful balance. If you dump too much and the context window grows, the prompts get more expensive, and the models are less likely to follow the instructions. If you provide too little, the model may not understand the problem and give a useless answer. There are really no rules that work for every case. One thing we found helpful is strong language in the prompt. The models seem to follow wording like “ALWAYS” or “NEVER” better than “should” or “shouldn’t”. Free-form chat After the initial response, users can ask about the solution or other things related to the exercise. This part was quite straightforward since we had already made the MVP ready to work with threads of messages. The key here is getting the historical messages to include in the future prompts. In our case, we fetch the chat history from BigQuery. We retrieve what we can concurrently, so the time to the first response chunk is short. Solving Complex Projects This approach worked pretty well, even though we used a single prompt. For shorter code snippets, it could guide students on what’s wrong with the solution. But for complex exercises, it failed miserably. With big projects, the model often made mistakes. It tried to fix issues that didn’t exist or completely missed what was wrong. We improved it a bit with better prompts and more context about common concepts. But for the most complex examples, it wasn’t enough. So we experimented with a more advanced approach, which you could consider an agentic system. Fixing the Solution The tricky part of generating hints is that we can’t be sure whether the model is right. It’s how your AI feature can (and likely will) fail in production. To counter this, you need to be able to verify the model’s output. In our case, we built a system similar to coding agents like Claude Code. We use it to fix the user’s solution before it gives any hints. If it succeeds, we have some confidence that the model is on the right track. This approach solved the issue of the model making wrong assumptions. If it tried one approach, and it didn’t work, it would realize the issue is elsewhere. Among other things, the agent can edit the user’s submitted files. These changes happen only in the Mentor’s context — the user doesn’t see them. It also plans what to do, can summarize the changes, and a few other tricks. Different models excel at different tasks, so we can use the best one for each step. In the code, an agent is pretty much a for loop plus tool uses. It’s a bit like a regular chatbot, except the model “talks” to itself. You keep the history and add messages with feedback on the environment. The context grows with each iteration, so it makes sense to limit the attempts. Parallel Runs The new approach sometimes brilliantly succeeded. But other times it was stuck, failing to edit a file, or just not getting the original issue. The good part was that it could now tell the user it didn’t know the solution, rather than making things up. That’s a huge improvement — if someone is already stuck, we don’t want to mislead them further. But it wasn’t able to fix the solution on each attempt. You had to retry a few times if it failed. We noticed that if the model didn’t succeed in the first few attempts, it would only get worse. It made more sense to start over. So we started running a few agents in parallel, each trying to fix the solution. Once one succeeds, we stop the rest. Managing concurrent code like this is a pretty common pattern in Go. We feel it’s a great fit for AI engineering thanks to its concurrency model (compared to other languages, like Python). wg := sync.WaitGroup{} wg.Add(workers) for i := 0; i < workers; i++ { go func() { defer wg.Done() for cfg := range configsChan { ctx, cancel := context.WithTimeout(ctx, workerTimeout) defer cancel() select { case <-ctx.Done(): return default: } result <- Solve(ctx, mentorContext) } }() } go func() { wg.Wait() close(result) }() for { select { case <-ctx.Done(): return nil, ctx.Err() case r, ok := <-result: if !ok { return nil, ErrCantHelp } if r != nil { return r, nil } } } We use many combinations of models to increase the chances, hence the configsChan. Different models excel at different tasks, so we can use the best one for each step. Also, if a cheaper model manages to finish the task fast, we can save some money on the expensive ones. Often, a single call couldn’t reliably figure out the solution. This shotgun approach improved the chances. The tradeoff is cost: we pay for all started parallel runs, even if we finish them early. Don't miss new posts.Join over 20k subscribers of our newsletter and get a free e-book! Go With The Domain Three Dots Labs Failed to subscribe, please try again later. Subscribe Loading... 🔒 We do not send spam. You can unsubscribe at any time! QA: Tests & Evals In many projects, testing is an afterthought. It turns out it’s crucial when working with LLMs. And testing AI products can be a nightmare. It doesn’t seem that different at first. Calling a model is similar to using any other API, so you need to prepare test cases with inputs along with the expected outputs. But there are some caveats. Challenge #1 is judging the responses. An optimistic scenario is when you make the model return a structured output, with a well-known list of valid values. Good old unit tests can work well if you have enough test cases. But if you let the model respond in free-form text, like in most chatbots, it quickly gets complex. You need an evals system where you judge if the output is correct. You could either use a human to do this or another LLM call. Whatever you choose, expect to read lots of data. You need a realistic set of inputs and outputs, and you need to review and label them. Once we kicked off the project, we spent a few days manually going through possible test cases based on production data. It seemed unproductive at first, but if we skipped this part, we would have no idea if what we built works. It also helped us understand the problems people actually face. Challenge #2 is time. If you want to use the smartest or thinking models, a request can easily take tens of seconds. But even with faster models, running a few prompts in a sequence is far from unit test speed. This slow feedback loop is annoying to work with, especially if you tweak the prompt slightly and want to see the impact right away. Challenge #3 is cost. The best models are also the most expensive. If your context grows (some of our evals have context with 100k+ tokens), each test run costs real money. Because of #2 and #3, forget about running a big set of evals in CI on every commit (unless you have too much VC money to burn). You can run some of them, or do it only once in a while. In our case, the trivial test cases (fast and cheap) are not that interesting, because even the less capable models solve them easily. The longest and the most expensive ones are what we care about the most. In regular tests, you can mock the expensive or slow resources. It makes little sense here, as you need to validate the same models you’ll run in production. Since we run parallel workers for each mentor call, it also impacts the evals. While not a big difference in the production environment, it makes evals much more expensive. It was also easy to hit the API limits. So, in the evals, we run the workers sequentially, starting with the cheapest ones. It slows down the tests, but saves us some money (this makes a difference, especially for bigger contexts). On top of all this, LLMs are unpredictable, so you may want to run a few iterations of each eval for more realistic results. This multiplies both time and cost. You need to balance many factors. We didn’t use any framework, but a script written in Go that runs the evals and collects the results. Because evals take a long time to run and could be interrupted by some error, we implemented a cache and retry mechanism, similar to what go test does. It’s frustrating to spend five minutes waiting for tests, only to realize you forgot a simple change and have to start over. A few of the evals we run for the Mentor. Failing in production After rolling out the first version of the Mentor, we monitored the responses to see how it did. Shortly after, came this: Mentor hallucinating. This is wrong! I started sweating when I saw this. People pay us to teach them, so it’s a huge risk if they learn obviously wrong things. In this case, the student tried to understand if assigning a slice to another slice in Go results in a deep copy. The model happily agreed, even though it’s not true. We take teaching people seriously. I almost wanted to scrape the entire feature when this happened. We contacted the student right away to clarify the issue. Thankfully, they appreciated the detailed explanation and weren’t angry. To counter this, we switched to a better model for conversations and added more context to the prompts (details below). The worst part about hallucinations is that the model is very confident about being right. When you just start working with LLMs, you’re surprised how often it happens. You need to be ready for it, because it WILL happen in production. This is why I’m not comfortable learning something in-depth just from an LLM. If it’s something you can quickly check, it’s not a big deal. But learning fundamentals the wrong way is another story. RAG and Sources Initially, we assumed we didn’t need to use RAG (Retrieval-Augmented Generation), and we simply always added some documents to the context. But after the first issues, we decided to implement a simple RAG with things like Go documentation, our blog posts, and other resources. We have a simple database of content in Markdown format. The RAG hype presented it as some mindblowing technology, but it was rather simple to implement. It took us just two days to have an MVP in production. It helped that we could split the work and implement two parts in parallel: building the database and querying it in the mentor code. As a bonus, we also link to the sources in the response, so students can read more about the topic. The list of useful links. Predictability If you expect a structured output from the model, like “true” or “false”, or a set of answers, testing is much easier. But dealing with free-form answers, like with most chatbots, is entirely different. One day, I started tweaking the temperature setting, assuming lower values should work well for our needs. Then, I realized there’s no way to tell unless I compare multiple responses manually or write an eval that focuses on this difference. And even then, I’m not sure if the reason was my change. It’s very difficult to get predictable results from LLMs. Small changes in the prompt have effects on the whole pipeline. The more complex the system, the less predictable the outcome, and the harder it is to test. After you change the prompt, you need to run the evals. But probably not all, since they are slow and expensive. But then, if you test only a few, you don’t know if it covers all cases. If the response changes, you can’t be sure it’s due to your changes. Maybe it’s just a different random response. Or maybe the model has been updated overnight (you have no visibility into it). To have a better idea, you could run more iterations of each eval, but this increases both time and costs. The developer experience of changing the prompt and waiting for results is quite poor. It’s nothing new: it’s like working with a slow or flaky CI/CD pipeline, where you commit changes and can’t spend the next five minutes on anything productive. The context switch is painful. You can counter this with parallel runs, but you can hit usage limits this way, and it’s then easy to burn money within a few seconds. How you model your evals is key. We ended up with two kinds of evals: Strict evals, where we expect a given percentage of scenarios to pass. We can use them reliably in the CI pipelines. Graded evals, where we run a set of scenarios and manually compare how the results changed according to grading. Shifting the Mental Model How do you cope with software that doesn’t follow instructions? Often, you like the LLM’s output 90% of the time, but the remaining answers are confusing or outright wrong. This can be frustrating if you’re used to working with code. There are many ways to reduce hallucinations, but there’s no way to avoid them completely. Realizing this helped me change how I think of LLMs. I see people go to two extremes: Treating LLMs as sentient beings, trusting the model to do exactly what they ask, if the prompt is good enough: “Bro, I found a way to make GPT never lie to me, just use this prompt.” Skeptical about using LLMs entirely, making fun of prompt engineers having to beg the model to do what they want. A middle ground I found is to treat an LLM as a function that is excellent at parsing vague inputs, and the output is often brilliant, but sometimes nonsense. I treat prompts as code, but with less strict rules than I was used to. With this mindset, it’s easier to be pragmatic and remember not to blindly trust the output. But also not getting discouraged after something breaks. ✅ Tactic: Expect LLMs to fail LLMs are impressive but unpredictable. Don’t treat them like the code you’re used to. Be ready for failures and hallucinations. If you can, verify the output. Agentic systems or autonomous agents? The top 2025 trend is running autonomous agents that make decisions independently. But thinking of an agent as a sentient being that can figure out what to do in every scenario is a trap. Code always works in contrast to an unpredictable LLM. We didn’t need to go all in on the agent running all workflows by itself. For example, we can just decide (in code) what steps the coding agent should take next. When writing regular code, you need to control the agent loop more directly. It’s tempting to think you can just include in the prompt what the agent should do in every scenario. But the reality is, you have no guarantee it will follow your instructions exactly. Our agent isn’t super complex, but it doesn’t need to be. If you can predict when to hand over to regular code or another LLM call, just do it in the code. No need for an agent framework where agents know about each other and can decide who to “talk” to. Especially if you can’t verify the output, how can you trust the agent to make the right decisions? This agentic hype feels like the early days of microservices when we told ourselves the story of independent entities talking to each other. Then everyone jumped to implement the solution, without considering whether it actually solved anything. Many teams learned the hard way that distributed systems are difficult to get right. Even with classic, perfectly predictable code. I’m not eager to add unpredictable models to the mix. ✅ Tactic: Use LLMs where they shine If you can handle the problem with regular code, there’s no need to use LLMs. Don’t think of agents as sentient beings that can figure everything out by themselves. Where is the complexity? The length of this post shows how much goes into one AI feature, and not all of this effort is related to LLMs. Most of it is good old software engineering and product design. I will risk saying that building the Mentor was 80% regular software development and 20% AI-specific parts. And 80% of the latter was creating and running evals. Most of the complexity is in the orchestration. All well-known patterns and good practices still apply. This agent-orchestrating code is your domain logic. Once again, Go shines for such use cases. It’s trivial to run concurrent workflows, and channels are a great way to communicate between different parts of the system. The logic is easy to follow. (And Go is easy to learn!) Models We accidentally picked a hot moment in model releases. We started with Gemini 2.0 Flash — it had a 1 million token context window, a generous free tier, and was relatively cheap overall. Then, Gemini 2.5 Pro came and made Flash look silly. It had a much better understanding of code. Then GPT-4.1-mini came and made Pro look expensive and slow. In retrospect, it’s good we didn’t implement this feature one year ago, since the models got so much better over time. Picking the right model is a balance between speed, cost, and quality. You need to be able to switch models easily for tests, so make sure your system is not tightly coupled to a specific model. We eventually landed on OpenRouter. You can use the Go OpenAI SDK to call any provider’s model, which is great by itself. It also has no strict rate limits (we hit limits running evals on OpenAI). Finally, it centralizes billing and lets you set limits and budgets (something unheard of for Gemini — crazy!). The downside is some extra cost and latency. Limits & Costs One feature of our internal LLM library is limits. We designed it so that calling an LLM is not possible without using the limits. Even tests track limits in-memory. It’s too easy to lose track of how many tokens you use. One day, we spent $100 running evals. After that, we made the cost clearly visible in all of our tooling, including tests running in the CI. A cost spike because of running evals. Gemini is an especially awful experience. There’s no way to set budget limits, and the usage shows up with a delay. Thankfully, other providers offer basic services like budgets or prepaid payments. As mentioned above, OpenRouter also works great for this. We can set limits on a few different levels. The primary one is per user. It’s not a perfect solution, since more complex projects can quickly exceed the limit. It’s something we will try to improve in the future. The costs of running in production so far are just a tiny fraction of our infrastructure costs. The surprisingly expensive part has been running the evals. The faster you want them to run, the easier it is to burn money quickly. Thankfully, this happens only once in a while when we change the prompts or add new evals. Observability & Tooling As in any complex system, debugging AI features is difficult. Good tooling helps. Tracing is a great way to understand how the system works. We didn’t use anything AI-specific for tracing, as we like simple tech stacks and proven technology. We already use tracing in our project, so it was just a matter of adding spans and propagating them. A cool thing about tracing is how it shows which parts run concurrently. We also created a back-office dashboard to track the cost and usage, and a set of CLI tools to easily manage evals. Moderator We wanted the Mentor to avoid unrelated topics, as it was more likely to give incorrect answers then. We set some guardrails in the prompts, but it was easy to work around them. Our second approach was a separate moderator prompt. It has two functions: filter out irrelevant topics and decide if the user needs help with the exercise or just asks a question. If it’s just a question, we don’t need to fix their solution, so it’s a much simpler and faster prompt. It’s a classic example of how asking the model to do too much at once is problematic. Similar to classic software, the separation of concerns is a good idea. One prompt should focus on one task. Moderator stopping the conversation early. Encouraging students to ask for help Implementing the Mentor was one thing, but it wouldn’t be effective if students didn’t know it existed. People are already fed up with companies shoving AI features their way, and since most chatbots are dumb, it’s not tempting to click the button. We track the failures and show small hints in the CLI. Plus, we use the same SSE endpoint to stream hints and show an indicator on the website. Sometimes, students still ask us directly for help. We don’t want to point them straight to the Mentor, as our support is part of the product. Now, we can use the Mentor ourselves to find what the issue is, and then we just need to explain it to the student. Finding a small typo is much easier this way. The UI We’re not frontend experts, so I will shortly mention the UI parts for completeness. We use React and have vibe-coded most of the Mentor’s frontend code. We’ve been working with frontend for many years, but it’s not something we enjoy. Generating the code was a great way to start: we understand enough to know what we want, but we don’t have to write it ourselves. We find AI-assisted coding works well for software that has no “strict behavior”, and you can quickly tell if it works, like UI and video games. For example, if the generated CSS is silly, but looks fine, it’s not a big deal. Especially if it’s just a single component. Of course, if you go too far with it, it can turn your project into a mess. We use a polyfill for SSE. It supports features not in the official spec, like passing custom headers in the API (Authentication), and it works out of the box. One thing to consider with a chat interface is that you need to animate the chunks when streaming (delay the letters slightly). If you make them appear as they come, it doesn’t look as good. At some point, we added status messages to show that the Mentor is working on the response. Sometimes it takes a few minutes, so it’s easy to lose patience and think it stopped working. These messages are not fake — we stream them from the backend via the same SSE endpoint. Outcomes We ended up with something good enough to make it public and observe what happens. We see people use it, and most of the time, the response does the job. What we haven’t tried yet: Running an open-source model on our own. Fine-tuning models. Judging the responses by another LLM on the fly. Pre-computing hints before someone asks for help. Partially reading files instead of dumping the whole codebase to the context. Even without these, we feel the result is pretty impressive compared to most chatbots. The key lessons learned for us (hand-crafted by a human so you don’t need to paste this post into an LLM): LLMs are not magic, but they can solve some problems that were previously unsolvable. Building an AI feature is easy, but making it work reliably is a whole different story. The complexity is in the orchestration, and the old software engineering practices still apply. You don’t need a framework to get started. A tiny wrapper around the LLM API is enough. You need to watch out for: The hype stories don’t mention the full picture. Try things yourself! LLMs WILL hallucinate in production, be ready to handle it. Testing AI systems (evals) is difficult to get right and can be expensive. It’s easy to go over budget. Implement limits early. If you want to see the Mentor live, you can try the free part of Go in One Evening (no login required).
Learn Building Modern Go applications Aug 6, 2025 -
Discover how VueUse solves SSR issues with browser APIs and keeps your Vue composables safe from 'window is not defined' errors.
alexop.dev Jul 14, 2025 - Loading…
Select an article to read.
Keyboard shortcuts
- j / k
- Next / previous article (also n / p)
- g / G
- First / last article
- Enter / o / v
- Open in the reader
- Middle / ⌘-click
- Open the article source in a background tab
- Space
- Page down (Shift = up)
- s
- Star / unstar
- m
- Toggle read
- A
- Mark all read
- r
- Refresh feeds
- u
- Go to Unread
- t
- Add a tag
- /
- Search
- ?
- This help
- Esc
- Close / back to list