Coding to comparing filenames and foldernames on android

[u/Technical-Ad-565]

UPDATE: I have temporary solved my immediate problem with using the find and diff commands from Termux.
This got me a usable list of differences. Although I had to move folders around a bit.

In the long run I will use Python for this and further functions.

END UPDATE.

I need to make a program för comparing file names and subfolder names between 2 folders containing something like 100000 files and subfolders. And it's around 650GB of data on the phone.

(Background: I have programmed professionally since 1979, but not on android or pc. Are totally new to Python. I use the phone almost always, very rarely the pc, even for my own business.)

I have searched for and tried extensively with apps for android but there isn't any useful. Apparently mainly due to permission problems.

The main question is: is it at all possible due to the permission problem?

(The solution by transferring the files to the pc have the problem that it takes very long time. And I really want a way to do it relatively often.)

The second question is: are the any existing code (snippets) for this or have you some advices in general?

Thanks on advance for any suggestions.

//Thomas

[u/11krish11]

Yes, it is definitely possible directly on Android using Termux (a Linux terminal environment app available on F-Droid or GitHub).

• Grant Storage Permissions: After installing Termux, run termux-setup-storage and allow "All files access" in Android Settings so scripts can scan your internal storage/SD card without permission blocks.
• Efficient Python Approach: For 100,000+ files, avoid reading file contents. Collect relative paths into Python set structures using os.scandir() or os.walk(), then use set operations:
• import os
• def get_rel_paths(root):
• paths = set()
• for dirpath, dirnames, filenames in os.scandir(root):
• # build relative path set for quick difference lookup
• ...

Fast Comparison: Computing folder_a - folder_b using sets is an $O(n)$ in-memory operation that completes in seconds once the path tree is indexed.

[u/Technical-Ad-565 → u/11krish11]

Thanks. I will (almost) never read file contents. The intended functionality is solely focused on file names, in wich folders they are and the same for folders themselves. Add to that "tagging" all those in lists/crosslists in files. The tagging could be anything descriptive. (Here we are talking about tags that cannot solely be a subordinate to another tag; some simple examples: time, places, people and things.)

[u/11krish11 → u/Technical-Ad-565]

Since you're dealing with multi-attribute tagging across 100,000+ files, using an embedded database like SQLite (built right into Python) will be far more manageable and faster than raw text lists.

• Database schema: Store file paths in one table and tags in another, linked by a standard many-to-many relationship (or use SQLite JSON / comma-separated tags if keeping it simple).
• Multi-criteria filtering: With SQLite, querying files by complex tag combinations (e.g., WHERE tag IN ('place_A', 'person_B')) is nearly instantaneous even on a phone.
• Persistent & portable: The entire index and tag metadata live in a single .db file that you can easily back up or inspect without scanning the filesystem every time.

If you prefer a pure Python data structure to start, a dict mapping relative paths to sets of tags ({ "subfolder/file.ext": {"place", "2024", "person"} }) dumped to a JSON file will also work cleanly.

[u/Technical-Ad-565 → u/11krish11]

Thanks!

[u/11krish11 → u/Technical-Ad-565]

You're Welcome!