Coding to comparing filenames and foldernames on android

[u/Technical-Ad-565]

UPDATE: I have temporary solved my immediate problem with using the find and diff commands from Termux.
This got me a usable list of differences. Although I had to move folders around a bit.

In the long run I will use Python for this and further functions.

END UPDATE.

I need to make a program för comparing file names and subfolder names between 2 folders containing something like 100000 files and subfolders. And it's around 650GB of data on the phone.

(Background: I have programmed professionally since 1979, but not on android or pc. Are totally new to Python. I use the phone almost always, very rarely the pc, even for my own business.)

I have searched for and tried extensively with apps for android but there isn't any useful. Apparently mainly due to permission problems.

The main question is: is it at all possible due to the permission problem?

(The solution by transferring the files to the pc have the problem that it takes very long time. And I really want a way to do it relatively often.)

The second question is: are the any existing code (snippets) for this or have you some advices in general?

Thanks on advance for any suggestions.

//Thomas

[u/Bright-Historian-216]

android is based on the linux kernel. i'm not familiar with android specifically, but you may be able to install python like you would on linux and use it in the same way. though, this may or may not require you to root your phone.

[u/k0rvbert]

Try asking some android devs, there isn't much overlap with python for that platform

[u/Technical-Ad-565 → u/k0rvbert]

Suggestion for a subreddit for that? (I'm assuming there are several possible.)

[u/Negative_Pay_2940]

what phone have 650gb storage

[u/Technical-Ad-565 → u/Negative_Pay_2940]

I have Samsung S25 Ultra with "1TB" storage. In reality though it's about 0.91TB.

[u/FoeHammer99099]

If you're able to get a shell on the phone you should be able to run diff -qr dir1/ dir2/

[u/Technical-Ad-565 → u/FoeHammer99099]

I tried (both with and whitout double quotes) and got this:

~ $ diff -qr "-Thomas/-B/hist/hist0/- -/-Unsorted/" "-Thomas/-B/hist div all/" diff: invalid option -- 'o' diff: Try 'diff --help' for more information.

I don't understand the reference to 'o' (I'm unfamiliar to the thr general command syntax.)

[u/brasticstack → u/Technical-Ad-565]

Try a double dash between -qr and the paths. I think it's trying to interpret the path names as options.

diff -qr -- '/path1' ..etc

EDIT: I also had no idea you could diff files in a path without ls or find.

[u/Technical-Ad-565 → u/brasticstack]

The double dash helped. But it seems it have problem handling hyphens. Will try with some changes in folder names.

[u/brasticstack → u/Technical-Ad-565]

Here's the shell-based method I was thinking of:

``` ls -1 /dir1 | sort > dir1.txt ls -1 /dir2 | sort > dir2.txt cat dir1.txt dir2.txt | uniq -c | egrep -v '<sup>\s+1'</sup>

should show duplicates between dirs.

```

You can also diff the txt files or whatever other method you want to compare.

I'm not at a computer, so that egrep command might need some tweaking to work right.

EDIT: sort -c not count EDIT2: uniq -c so much for being helpful away from my terminal

[u/FoeHammer99099 → u/Technical-Ad-565]

That's weird. Maybe try with single quotes? It's probably the dashes in the paths being misinterpreted as cli options.

[u/Technical-Ad-565 → u/FoeHammer99099]

When I changed all hyphens and blanks in the names to "x" the error message disappeared. (But it of course couldn't find the directories.)

[u/brasticstack]

Your phone has Python installed! You could get the Termux app and run it on the terminal.

Use pathlib to locate and compare filenames, then IMO the fastest method would be moving the files you want to transfer to their own directory, making a tarball of that dir if space permits and copying that. Otherwise it's very slow doing files individually through ADB (single threaded) or MTP (dog slow for some reason.)

[u/Glittering_Box5197]

Yes, it is possible in Python, but on Android the permission/access method is likely to be the harder part, especially for 650 GB of data.

If the two folders are accessible to Python, you don't need to read the contents of all 650 GB just to compare filenames and folder names. You can compare the directory trees by walking the two folders and building relative paths.

For example, the basic approach is:

from pathlib import Path

def get_paths(folder):

return {

p.relative_to(folder)

for p in Path(folder).rglob("*")

}

folder1 = Path("/path/to/folder1")

folder2 = Path("/path/to/folder2")

paths1 = get_paths(folder1)

paths2 = get_paths(folder2)

only_in_1 = paths1 - paths2

only_in_2 = paths2 - paths1

print("Only in folder 1:", len(only_in_1))

print("Only in folder 2:", len(only_in_2))

This compares the relative filenames and subfolder structure, rather than comparing the actual file contents.

With around 100,000 entries, I'd also avoid loading unnecessary file metadata or file contents. If you eventually want to compare whether files themselves are identical, that's a different problem because you'd need to consider size, modification time, hashes, etc.

On Android, I'd first solve the access problem with a small test program that can successfully list the contents of both folders. Once Python can see both directory trees, the comparison itself is fairly straightforward.

Since you're new to Python, I'd start with a program that simply prints the number of files/folders found in each directory before attempting the full comparison.

[u/Technical-Ad-565 → u/Glittering_Box5197]

Thanks for the code example. My first concern will be the permission problem. A test program is of course the first thing I must do. But if someone has tried this and it worked would be helpful to know. Due to lack of time I haven't began with this "project" so I just took on the comparison problem. (My main objective is to be able to handle and register connections between files and folders and "tag" them depending on content and relationship with other files etc.)

[u/LividRequirement4364]

Hey Thomas. To fix the permission problem on Android, try using an app called Termux. Just install it and type the command termux-setup-storage. It will ask you for file access.

For the Python code, look into os.walk() to scan your folders. Because you have so many files, put the file names from each folder into two Python sets. Then you can literally just subtract one set from the other to see the differences instantly. It is super fast. Hope this helps!

[u/Technical-Ad-565 → u/LividRequirement4364]

I actually just installed and used Termux. Used find and diff commands as a temporary stop gap until another/better solution.

As you mentioned my basic concerns is the permission problems. It seems all commercial or easy availible apps is blocked by these (for my needs).

• The question I have do these permission problems also block my Python programs? *

(Thanks for the code suggestions! If I can go the Python way I will create extensive lists in files about files and folders. I have more needs than I mentioned in my post.)

[u/LividRequirement4364 → u/Technical-Ad-565]

Since you already got Termux working, you are in luck. If you run your Python script inside Termux, it automatically shares the exact same permissions that Termux has.

So no, Android will not block your Python code as long as you run it through there. You just need to type pkg install python in Termux to get it set up. After that, your script will be able to scan your folders and write out those long text files without any permission errors.

The only small thing to remember is that newer Android versions completely lock down hidden system folders and app data folders. But for all your regular files and folders, this setup will work perfectly for what you need.

[u/Technical-Ad-565 → u/LividRequirement4364]

Thanks for the very valuable info!
(I have no need to access the app data or system folders and files. I don't like to shot me in the foot. :D )

[u/LividRequirement4364 → u/Technical-Ad-565]

Haha wise choice! Breaking the Android file system is a headache you definitely don't want.

Since you mentioned you will be creating extensive text files for your lists, just one quick Python tip: look up how to use the "with open" statement for writing files. Coming from older languages, you will love it. It automatically closes the file and frees up memory for you even if the script crashes, which is a lifesaver when scanning 650GB of data. Good luck building it man!

[u/11krish11]

Yes, it is definitely possible directly on Android using Termux (a Linux terminal environment app available on F-Droid or GitHub).

• Grant Storage Permissions: After installing Termux, run termux-setup-storage and allow "All files access" in Android Settings so scripts can scan your internal storage/SD card without permission blocks.
• Efficient Python Approach: For 100,000+ files, avoid reading file contents. Collect relative paths into Python set structures using os.scandir() or os.walk(), then use set operations:
• import os
• def get_rel_paths(root):
• paths = set()
• for dirpath, dirnames, filenames in os.scandir(root):
• # build relative path set for quick difference lookup
• ...

Fast Comparison: Computing folder_a - folder_b using sets is an $O(n)$ in-memory operation that completes in seconds once the path tree is indexed.

[u/Technical-Ad-565 → u/11krish11]

Thanks. I will (almost) never read file contents. The intended functionality is solely focused on file names, in wich folders they are and the same for folders themselves. Add to that "tagging" all those in lists/crosslists in files. The tagging could be anything descriptive. (Here we are talking about tags that cannot solely be a subordinate to another tag; some simple examples: time, places, people and things.)

[u/11krish11 → u/Technical-Ad-565]

Since you're dealing with multi-attribute tagging across 100,000+ files, using an embedded database like SQLite (built right into Python) will be far more manageable and faster than raw text lists.

• Database schema: Store file paths in one table and tags in another, linked by a standard many-to-many relationship (or use SQLite JSON / comma-separated tags if keeping it simple).
• Multi-criteria filtering: With SQLite, querying files by complex tag combinations (e.g., WHERE tag IN ('place_A', 'person_B')) is nearly instantaneous even on a phone.
• Persistent & portable: The entire index and tag metadata live in a single .db file that you can easily back up or inspect without scanning the filesystem every time.

If you prefer a pure Python data structure to start, a dict mapping relative paths to sets of tags ({ "subfolder/file.ext": {"place", "2024", "person"} }) dumped to a JSON file will also work cleanly.

[u/Technical-Ad-565 → u/11krish11]

Thanks!

[u/snugar_i]

Why Python? It's not really that common on Android

[u/Technical-Ad-565 → u/snugar_i]

Just because I like it. Based on the overall functionality and that it don't overcomplicate things. Which I like as a programmer. (My code will never be used by other people.)

[u/Turtlestacker]

Im guessing you havent tried codex / Claude?

[u/Technical-Ad-565 → u/Turtlestacker]

I'm sorry but I don't understand. Are you e g suggesting I use AI? (Haven't used it.)

[u/Turtlestacker → u/Technical-Ad-565]

lol
Redditors hate Ai - maybe try it