An engine that imports user code at runtime

This engine takes a Python file a user wrote and turns it into a live API without restarting the server.

So there was no choosing where state would live. importlib can't read a database column, so for code to become a module there has to be a real file on disk.

The documentation records it in one line.

Core logic doesn't use a relational database; persistence is delegated to the filesystem — to allow for dynamic module loading.

Most of the bugs filed over the following five months came out of that decision.

From file to API

A file passes through four places on its way to becoming an API.

In an ordinary web server none of these change at runtime, but here all four do.

Works while running, breaks on restart

A user put two files in the same directory and imported one from the other.

# routes/{agent}/sandboxes/bbb.py
from aaa import example_var

It worked while the server was up. After a restart:

ModuleNotFoundError: No module named 'aaa'

The cause was sys.path. Only the watched parent directory was added to the search path — not the directory the module actually sits in. While running, an earlier registration step happened to leave the path correct as a side effect. Restarting removed the coincidence.

The fix was to add the module's own directory to the search path as well. Since the server is always up when you check, this bug doesn't show at checking time. It only shows once the pod restarts.

Registration order splits a singleton in two

The next one was a singleton. Router modules A and B both imported service module C, and C was a singleton.

The registration code re-read modules unconditionally.

if normalized_module_name in sys.modules:
    del sys.modules[normalized_module_name]   # if present, drop it and rebuild

The intent was to make file edits take effect. But deleting a name from sys.modules means the next import builds a new object, while anything already holding the old one keeps holding it.

Registered in the order A → B → C, nothing would have happened. So I flipped the rule: an already-loaded module is reused, and a file is re-read only when it has actually changed.

A container loses its own modules

This one came from outside Python.

ModuleNotFoundError: No module named 'utils.commons.debugger'

This package is a shared utility that ships in the image, so locally it was obviously there, and on Kubernetes it vanished.

The shared volume was mounted over the same path in the image, and a mount hides whatever was at that path. A file plainly present in the image stopped existing the moment the volume attached.

The fix was on the deployment side. An init container copies the shared package from the image onto the volume, and the editing service mounts it read-only. The application code wasn't touched.

All three produced nearly the same error, ModuleNotFoundError, while the causes were the search path, the module cache and a volume mount.

Three patches, then a change of direction

Duplicate registration was held together with local fixes for a while. Simplify the duplicate check; let duplicates through if a router is present; block the regression that caused with a same-day hotfix. All within a few days.

Four months later the approach changed. The single loop that read a file, executed it and pulled out a router was split in two.

Phase 1. Compile every file, create an empty module object, register it in sys.modules. Don't execute.

Phase 2. Execute the registered modules and extract routers.

By the time phase 2 starts every module name is already in sys.modules, so import finds the name whatever the processing order. And since execution only begins after everything has compiled, a syntax error in the third file no longer leaves the first two half-deployed.

What phase 2 actually guarantees

I checked whether the ordering problem was really gone with a small reproduction of the same structure: compile, register as empty modules, then run them with only the execution order changing.

So what two-phase loading solved is name resolution, not execution order. A top-level from X import name still depends on order, while import X with attribute access deferred into a function doesn't. Which one you get depends on how the user writes their code, and that isn't something we can enforce.

Was importing in-process the right call?

sys.modules is process-global mutable state, and we're using it as a deployment state store. Delete a key and someone else's reference goes stale; keep it and edits don't take effect. Between the two we settled on deleting only when the file has actually changed, and if that call is wrong we'll see one of the two symptoms again.

Looking at a single ModuleNotFoundError, nothing in the log tells you whether it came from the search path, the module cache, a volume mount or registration order. A distinct diagnostic per layer would probably be enough to tell them apart.

Looking back, I think the bigger decision was importing user code inside our own process, more than doing without a database. Running it in a separate process or container would have removed every bug in this post, at the cost of call latency and resource isolation. Whether that trade was right would take another look to say, but for now I think this approach stays within what we can handle.

End