r/MachineLearning 11d ago

Project repo2nb 0.2.0, convert a GitHub repo into a Kaggle/Colab notebook (dependency resolution, reverse mode, incremental sync) [P]

repo2nb is an open-source CLI that converts a GitHub repo into a runnable Kaggle or Colab notebook: walks the file tree, resolves dependencies, and generates cells, instead of you doing that by hand for a repo you didn't write (a paper's code, a tutorial, someone else's experiment).

0.2.0 highlights:

  • Dependency resolution tries poetry export, then uv export, then requirements.txt, then falls back to an AST import scan if none of those exist. Output is always a plain %pip install cell regardless of which path it took, so poetry/uv are only ever needed locally at generation time, not on Kaggle/Colab.
  • Reverse mode (repo2nb reverse <notebook>) reconstructs the original repo from a generated notebook, using the per-cell path/hash metadata every generated cell now carries. Validates against directory traversal and won't write into a non-empty directory without --force.
  • Incremental sync (repo2nb sync <repo>) does one-directional (repo to notebook) updates: added files get new cells, edited files update in place, deleted files get removed. --dry-run previews the diff.
  • Added a Colab target with its own auth cell (google.colab.userdata.get) rather than reusing the Kaggle secrets flow.

Install: pip install repo2nb

Repo: https://github.com/David-Magdy/repo2nb

Curious whether the dependency-resolution fallback order (poetry > uv > requirements.txt > import scan) matches what people actually run into, or if there's a common setup it'd get wrong.

Any feedback or opinions are much welcomed!

2 Upvotes

2 comments sorted by

1

u/Altruistic-Length221 11d ago

The reverse mode is actually the part that caught my attention, most tools like this just go one way and call it done. Being able to reconstruct the repo from a notebook makes this way more useful for sharing work without worrying about someone messing up the file structure manually.

The dependency fallback order seems fine for most cases I've seen. Poetry and uv are getting more common but requirements.txt still dominates in research repos, so having that before the import scan makes sense. The AST scan as a last resort is clever but I'd be curious how often it produces broken installs when people use weird imports or conditional dependencies.

One thing I wonder about is how it handles notebooks that have their own pip install cells mixed in with the generated ones. Does sync mode get confused by manual edits to the install cell or does it just leave it alone?

1

u/PolarIceBear_ 11d ago

First of all, thanks for the feedback!

On the AST fallback: yeah, it can mix up your project's own internal imports with real pip packages. Properly telling "this is something I need to install" from "this is just a module inside the repo" would take a lot more logic than a simple scan. In practice it's usually easy to catch though. Just run Run All and pip will fail because that "package" doesn't exist on PyPI.

Sync with manual edits needs a bit more care. Sync only tracks cells that are linked to an actual file in the repo, using the path and hash metadata. So if you add a custom %pip install cell, or something like !cd train/ followed by !python train.py, those aren't coming from any file that got added, changed, or deleted. They're just standalone execution cells. Sync has nothing to match them against, so they won't stick around after a resync or a fresh re-upload.

For now the safe approach is to keep those manual cells outside of what sync manages. It's a real gap though, and something like an explicit "preserve this cell" marker would help.