On Inspectability
by Laurence Chen
At the beginning of 2025, I took over a client project at Gaiwan. It was a legacy code maintenance project that used re-frame. While familiarizing myself with the codebase, I also began fixing the bugs the client had asked me to address. This process was anything but easy, which led me to reflect on what makes an effective approach to taking over legacy code.
I have yet to reach a conclusion on the general approach to inheriting legacy code. However, when it comes to taking over a front-end project built with re-frame, I do have a clear answer: install re-frame-10x, because it significantly improves the system’s inspectability.
To illustrate, let’s examine my debugging process for one particular frontend bug in the project:
I knew that pressing a certain button would trigger an event, which would then be handled by a specific handler. However, due to the high level of abstraction in the code, I couldn’t directly determine which event the button was sending just by reading the source code. The event was determined by a section of code injected via dependency injection. Since I was reading through a large amount of code without being able to answer a simple question, the total volume of code I had to track quickly exceeded my working memory limit, making it increasingly difficult to proceed. As a side note, the project’s UI was in Dutch, which consumed even more of my working memory.
After installing re-frame-10x, the frontend events became inspectable. I could now easily view, via the browser, which event was sent when I clicked the button. As a result, I was able to answer the same question while saving a significant amount of time that would have otherwise been spent reading code. I also no longer needed to insert a ton of
prn
statements in the code. My mental load dropped significantly, and my development efficiency improved dramatically.
Have you ever had a similar experience? This made me rediscover the importance of inspectability.
Here is a proposed definition of inspectability:
A system provides certain mechanisms that allow developers to observe its internal state, events, behaviors, and errors during runtime or development. These mechanisms should also enable developers to correlate observed errors with the relevant sections of the running code, with minimal modification to the source code or configuration, facilitating understanding and diagnosis.
Based on this definition, the benefits of inspectability include:
- Reducing modification costs. When developers realize they need to inspect the system’s internal state to proceed, a lack of inspection mechanisms forces them to manually insert
prn
,tap>
, or similar debugging statements and advance through trial and error. - Lowering mental load. Quickly viewing the system’s internal state, events, and behaviors reduces cognitive overhead. Developers no longer need to retain large chunks of code in memory to infer how the system operates.
- Reducing the “maze effect.” In legacy codebases without inspection mechanisms, developers often rely on fragmentary knowledge to understand the system. However, this knowledge is typically unidirectional—developers work forward from the code to infer system behavior. The problem is that similar-looking code scattered across different files may trigger the same behavior, leading to a maze effect where developers face multiple branching paths, making it difficult to determine which code branch is correct.
One paradoxical yet interesting fact: The importance of inspectability is often felt most strongly by those unfamiliar with a system. People deeply familiar with a system tend to naturally follow the happy path while developing, and even if they make occasional mistakes, they are rarely far from it. When debugging, they can quickly infer the approximate cause based on their holistic understanding of the system. In contrast, for someone taking over maintenance, inspectability becomes critically important.
With this in mind, when developing a system, consider proactively improving its inspectability—even if you currently feel no pressing need for it due to your familiarity with the system.
Below, we’ll explore several approaches to improving inspectability.
Supporting Tools
In our definition of inspectability, we mentioned that “these mechanisms require little to no modification of the original source code or configuration.” If we can design our systems to reduce the ceremony required for inspectability, we are effectively enhancing it.
When debugging, the most basic and commonly used techniques are prn
and inline def. However, some code structures make inspection impossible without modifications or special tools. For example, in the following code, we cannot apply prn
directly to inspect the results of (reduce + xs)
and (count xs)
without modifying the function.
(defn mean [xs]
(/ (double (reduce + xs)) (count xs)))
Among the many debugging tools available, I find that hashp requires minimal code changes while being highly intuitive. In the code below, #p
prints the evaluated value of the form after it.
(defn mean [xs]
(/ (double #p (reduce + xs)) #p (count xs)))
Inspectability at Different Levels
The first step most developers take toward improving inspectability is enhancing system logs. This is an important step, as logs contribute to developer-level inspectability. However, two other levels of inspectability are also worth considering: library developer-level and user-level.
At the library developer level, Clojure provides metadata, which can be used for:
- Tagging & Tracing
- Function Documentation & Metadata
- Aspect-Oriented Programming
Consider this example from EmbedKit, where metadata is used for tagging. The request’s metadata is preserved in the response using vary-meta
, allowing developers to retain important context.
(defn- do-request [req]
(vary-meta (http/request req)
assoc
::request req))
Similarly, user-level inspectability is critical. End users rely on documentation rather than source code to understand a system. When a system is inspectable, users can observe internal states to verify their actions.
Common user-level inspectability mechanisms include:
/health
API/metrics
API
Inspectability Level | Purpose | Target Users | Typical Mechanisms |
---|---|---|---|
Developer-Level | Debugging, error tracing, performance monitoring | Application developers | Log (clojure.tools.logging ) |
User-Level | System monitoring, quick issue diagnosis | End users, operations teams | RESTful API (/health , /metrics ) |
Library Developer-Level | Behavior control, localized observability, additional semantics | Library/framework developers | Metadata (meta , with-meta ) |
Events and Behaviors
In Clojure, we use live development—most people call it interactive development, but I prefer “live” because a running program is alive—which inherently provides an inspectability experience comparable to working in a debugger in other languages. Variables in a running system are always accessible, meaning that state is generally inspectable.
However, events and behaviors are not always so easily observable, which is why they deserve special consideration.
For example, in re-frame, I only gained event inspectability after installing re-frame-10x.
What about behavior inspectability? A good example is the bin/dev script used in many Gaiwan projects. Its primary function is deployment, but you can also think of it as an admin process.
The standard design of bin/dev
always includes a dry-run (--dry-run
) option. Any subcommand, when modified with the dry-run option, will not actually execute but will instead display the intended actions on the screen. This makes the upcoming “deployment behavior” inspectable. When we are unsure about the effects of a subcommand, we can quickly run a dry-run to understand what will happen before executing it for real.
Improving Exception Messages
Compared to my experience developing in C++ twenty years ago, having exception messages in Clojure/Java is a huge improvement. At the very least, when an exception occurs, I can start tracing the root cause from the stack trace.
Recently, I encountered an exception message with a full stack trace that was difficult for beginners to read. The stack trace had a total of 126 lines, but most of them were just noise. Only five lines were useful in quickly identifying the root cause. These five lines consisted of:
- The very first line:
Incorrect string value: ...
, which hints that the error is related to a string. - Four other lines containing
APPLICATION_NAMESPACE
, which usually indicate where the issue can be found.
Ultimately, I fixed this exception in the code located at line 45 of src/clj/APPLICATION_NAMESPACE/services/files.clj
. The fix was simply changing path
to (str path)
.
In more complex cases, the number of lines containing APPLICATION_NAMESPACE
may be far greater than four. In such situations, inspectability becomes a challenge since, for a developer, any occurrence of APPLICATION_NAMESPACE
could be the source of the error. If this happens, one approach is to introduce Clojure spec checks between different layers of APPLICATION_NAMESPACE
. This allows exceptions to be caught earlier by spec validation. Eric Normand has analyzed this issue.
[clojure] java.sql.BatchUpdateException: Incorrect string value: '\xAC\xED\x00\x05sr...' for column 'path' at row 1
[clojure] at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
[clojure] at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
[clojure] at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
[clojure] at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
[clojure] at com.mysql.jdbc.Util.getInstance(Util.java:387)
...
[clojure] at APPLICATION_NAMESPACE.db.access.files$update_file_BANG_.invokeStatic(files.clj:15)
[clojure] at APPLICATION_NAMESPACE.db.access.files$update_file_BANG_.doInvoke(files.clj:15)
[clojure] at clojure.lang.RestFn.invoke(RestFn.java:428)
[clojure] at APPLICATION_NAMESPACE.services.files$upload_file_handler.invokeStatic(files.clj:44)
[clojure] at APPLICATION_NAMESPACE.services.files$upload_file_handler.invoke(files.clj:19)
Conclusion
We have explored the importance of inspectability in maintaining and developing software systems. Starting from its definition, we emphasized or extended the definition in various ways—expanding from the developer level to the user level—and proposed several approaches to enhance inspectability during software development.
Finally, don’t forget: if you want to know whether your system has good inspectability, ask the person who takes over its maintenance. I believe they can provide you with highly valuable feedback.