Over the past few months, we have been working on an SDK-based implementation of the Proton Drive macOS app. This work started shipping in version 2.11.0, and since then we have been continuously improving the performance of file operations so our customers get both strong data protection and an app that does not burn through CPU, memory, or battery unnecessarily.
The improvements came out of an investigation loop we built up over the project: make workloads reproducible, measure the right processes, turn traces into narrow hypotheses, validate those hypotheses with focused tests, and split the fixes by risk. No single large rewrite was involved.
The loop changed both the code and the measurements. In representative traces, one repeated parent-chain lookup path dropped from about 12% of samples to about 2%. A noisy telemetry-write path dropped from roughly 1.4% to 0.5%. In one small-file upload workload, safe tuning cut overall CPU by about 5%; in another, the database and parent-chain improvements raised throughput by about 10%. Those numbers describe specific workloads on specific machines, and each one is the kind of evidence we wanted every optimization to produce.
This post is about that process. For this investigation, "performance" meant more than transfer speed. We cared about:
- Throughput: how many files or bytes get transferred per minute.
- Responsiveness: how quickly Finder and the File Provider extension answer requests.
- CPU usage: especially sustained File Provider CPU during sync.
- Memory growth: especially over long extension lifetimes.
- Battery impact: because CPU and memory pressure translate directly into power use on laptops.
How Proton Drive works on macOS
Proton Drive for macOS is built around Apple's File Provider framework. The visible app is the menu bar application: it handles account state, settings, user-facing sync status, and coordination. The file operations users trigger in Finder are handled by a File Provider extension: creating folders, uploading files, downloading files, moving items, deleting items, and enumerating directories.
The architecture is powerful, but it changes how performance work has to be done.
The extension is a separate, system-managed process. macOS can launch it, suspend it, terminate it, or ask it to service a burst of file-system requests. A performance issue can therefore hide in a place that is not obvious from the main app. If Finder is slow to show a folder, if a batch of small files uploads slowly, or if the process grows in memory over time, the interesting work is often happening inside the File Provider extension.
There is another constraint that matters: Proton Drive is end-to-end encrypted. Metadata and file contents have to be encrypted and decrypted on the client. That means the hot path for a file operation can include database lookups, metadata decryption, key access, progress reporting, logging, File Provider item construction, and network calls. Our aim is to do all of that work as efficiently as possible.
Towards reproducible workloads
The first challenge was that customer workloads are not uniform. Uploading a folder with ten large videos stresses a very different part of the system than uploading a folder with thousands of tiny documents. Small-file workloads are particularly demanding because the per-file overhead is large compared with the file contents themselves. Every file can require metadata work, encryption work, database updates, progress updates, and File Provider notifications.
We needed repeatable workloads before we could trust any performance conclusion.
For that we used our client load-testing harness, a Python-based test runner that can drive the macOS app through realistic file operations. A test scenario is a sequence of steps: start the app, sign in, create local test data, upload a folder, wait for sync completion, mark files online-only, download a folder, pause or resume syncing, move files, delete files, collect logs, and so on.
The harness can generate file sets with known shapes. It supports flat folders, nested folder structures, fixed file sizes, random extensions, reproducible seeds, and very large stress scenarios. One scenario, for example, models a deep folder tree with many small files spread across multiple levels. That kind of workload is useful because it amplifies per-file overhead and makes repeated work visible.
Each run produces a timestamped test run directory. The runner collects application logs, File Provider logs, crash reports, database sizes, and resource metrics. It can also export local Prometheus-style metric logs and turn them into comparison reports. The important metrics include file progress (current/total files, transferred bytes) and resource usage: CPU and memory for the main app, the File Provider extension and the system.
This turned performance work into a controlled experiment. We could run the same scenario against version 2.11.0, a later release, and an experimental branch, then compare the shape of the run instead of relying on whether the app "felt faster."
Isolating a key variable: the machine itself
Reproducible workloads are necessary, but they are not sufficient. The execution environment also has to be representative.
Our load tests originally ran in macOS virtual machines. That made sense for automation: VMs are easier to reset, easier to run in CI, and easier to keep isolated from a developer's local machine. But while investigating performance on Apple Silicon, we found that VM results could have materially different performance profiles from native runs on the same hardware.
The reason is Apple Silicon's asymmetric CPU design. Modern Apple chips have performance cores and efficiency cores, and macOS uses a thread's Quality of Service (QoS) to decide where that work should run. As Howard Oakley explains in a blog post, low-QoS background work normally runs on efficiency cores, while higher-QoS work can use performance cores when they are available.
Virtualization changes that picture. Oakley notes that macOS virtual machines on Apple Silicon are assigned high QoS and run preferentially on performance cores; work that would normally be confined to efficiency cores on the host can therefore run through performance cores inside a VM. His earlier article on virtualization and core use gives a concrete example where a workload constrained on the host runs much faster in a VM because of this difference.
This mattered because sync software deliberately contains background and utility-priority work. File Provider operations, database maintenance, logging, metadata work, and progress reporting do not all have the same urgency. A VM can therefore make some parts of the system look faster, noisier, or differently balanced than they are for customers running the app normally.
So we split the role of VMs from the role of profiling machines. VMs remained useful for functional load testing and reproducible automation. But when the question was "where is CPU time going?" or "is this change representative of a customer's Mac?", we moved the critical measurements to native Apple Silicon hardware and treated VM measurements as a separate signal.
Before optimizing a hot path, confirm it reflects hardware customers actually run. A perfectly reproducible test can still mislead if it runs under a scheduler and core-allocation model customers will never use.
From symptom to cause
The load tests told us when a run was expensive. They did not tell us why.
A metrics chart might show that the File Provider extension used too much CPU during a small-file upload. It might show memory climbing during a long run. It might show file throughput flattening. Those are useful signals, but they are still symptoms.
The next step was to profile the process that was actually doing the work.
Profiling a File Provider extension is awkward enough that it is easy to get inconsistent results. The extension may not be running yet. It may be idle. The main app may be active while the extension is not. A trace might capture the wrong process or miss the interesting window entirely.
To make this repeatable, we built a small wrapper around Apple's Instruments toolkit. It finds or waits for the ProtonDriveFileProviderMac process, can wake it by opening the Proton Drive folder, records with Xcode Instruments' xctrace Time Profiler, exports the samples, collapses them with inferno, demangles Swift symbols, and renders an SVG flamegraph.
The workflow became:
- Generate a known file set.
- Start a known upload, download, or enumeration scenario.
- Attach to the File Provider extension.
- Capture CPU samples for a bounded period.
- Compare flamegraphs across versions or branches.
On its own, the flamegraph only showed us where to look next.
One hypothesis from trace to fix
One useful trace pointed at cryptographic setup for file encryption.
This is a delicate kind of performance finding. Because Proton Drive is end-to-end encrypted, cryptographic work is a core part of the product. Seeing crypto-related functions in a flamegraph doesn't usually mean we can make the crypto cheaper or skip the work. The first question has to be more precise: are we looking at unavoidable per-file encryption work, or are we repeatedly preparing the same key material inside a short-lived operation?
In this case, the trace suggested the second problem. During encryption of folders with many files in it, the app repeatedly needed the same unlocked private key. Keys are stored encrypted and unlocking them requires passphrase-protected key derivation. That derivation is intentionally expensive because it protects key material against brute-force attacks. Paying that cost once when the key is needed is expected. Paying it over and over for the same key during a burst of file operations is a different problem.
The hypothesis became:
- The app was repeating key-unlock setup for the same address key inside a short time window.
- A small in-memory cache could remove that repeated setup while preserving the security boundaries around key lifetime and invalidation.
The second point carried the risk. A cache around unlocked key material behaves differently from a normal performance cache: it changes how long sensitive data stays available in memory. So the fix came down to rules: where the cache lives, how large it can get, when it expires, and which account-state changes have to clear it.
The chosen fix kept the cache inside the session-vault layer, where the app already owns account keys and passphrases. The cache was bounded, short-lived, and in-memory only. It also coalesced concurrent requests for the same key, so a burst of callers would wait for one derivation instead of starting many duplicate derivations.
Validation focused on failure modes as much as speed. Tests covered cache expiry, sign-out, passphrase changes, user-key changes, address-key changes, cache scoping between vault instances, and concurrent callers requesting the same key at the same time. Those tests mattered because a faster trace would not be enough if the cache survived the wrong state transition and corrupted user data.
After the change, repeated key derivation almost disappeared from the trace: the visible stack went from roughly 5% of samples to effectively zero in the measured run. Performance work around encryption has to separate essential cryptographic cost from avoidable repeated setup, and validation has to match the risk the optimization introduces.
Investigating memory growth
CPU flamegraphs are good at showing where time is spent. They are less useful for explaining why a process grows over a long run.
For memory investigations, we used Instruments allocation traces and a DTrace script that tracks malloc/free activity for a process. It prints a heartbeat of outstanding bytes during a run and summarizes allocation sites by bytes and count when tracing stops. Since DTrace stack output is not always symbolicated, we used a companion script to resolve stack addresses with atos.
This let us ask different questions:
- Are outstanding bytes growing steadily during a long scenario?
- Which allocation sites dominate retained memory?
- Does the growth correlate with database contexts, File Provider item construction, logging, or metadata handling?
This pointed to another class of fix: reducing memory accumulation in long-lived Core Data contexts. The key observation was that reused contexts retained managed objects across many operations. The eventual change moved the File Provider extension toward resettable context pools, so contexts could be reused without accumulating state for the lifetime of the process.
When measurement adds to the workload
One of the more useful findings was that our own measurement pipeline could add work to the system.
During sustained progress reporting, performance measurements were being written too eagerly to Core Data. That meant the app was doing database work to sync files and additional database work to record that syncing was happening. In a small-file workload, that per-event cost compounds quickly.
The investigation question was: how much work are we doing to observe the work?
The fix was to buffer performance-measurement writes in memory and flush them in batches, while keeping read paths consistent when data had to be reported. Observability has to be cheap enough to leave on; otherwise it changes the workload it is trying to describe.
Separating safe changes from risky ones
Performance work creates a temptation to bundle many improvements together. That makes results harder to understand and reviews harder to reason about.
We took the opposite approach. Changes were split by risk.
Some fixes were local and low risk: replace a regular expression in a hot path, increase a SQLite cache size, avoid unnecessary response-header processing, batch telemetry writes, or add targeted database indexes with benchmarks.
Other fixes had correctness or security tradeoffs: cache parent chains, cache unlocked keys, change Core Data context lifetime, or reuse decrypted metadata. Those changes needed specific guardrails. A cache needs invalidation tests. A key cache needs strict lifetime and clearing rules. A context-lifetime change needs tests around object usage and operation boundaries.
Several ideas stayed experimental until they had enough evidence and review, and some were discarded as too risky. That was deliberate: a performance investigation should preserve promising hypotheses without forcing all of them into a release.
What changed
The investigation led to improvements across several layers, and each one had to carry its own evidence:
- Database access became more predictable through targeted indexes and batched lookup work. The focused benchmarks showed which point lookups stopped scaling badly with database size, and which broad result-set queries were already better left to SQLite scans.
- Repeated tree traversal was reduced by caching parent-chain information with explicit invalidation. In representative traces, that path dropped from about 12% of samples to about 2%.
- Repeated cryptographic derivation was reduced through bounded key caching. The gain was about 5% in the trace; review centered on lifetime and clearing rules because this touches sensitive material.
- Performance telemetry stopped competing with the workload it measured. The measurement-write path dropped from roughly 1.4% of samples to about 0.5%.
- Long-running File Provider memory behavior improved through resettable Core Data context pools, which treat retained managed objects as a lifetime concern at the context level.
- Small hot-path overheads were removed where profiling showed they mattered. In one representative small-file upload workload, later safe tuning reduced overall CPU by about 5%.
The exact numbers vary by machine, account state, network, and workload shape, but the direction was consistent: once repeated work was visible, we could remove it methodically.
Beyond any single fix, the workflow itself is the durable result. We now have a clearer path from "this feels slow" to "this stack repeats under this workload, this benchmark isolates it, and this change removes it without changing behavior."
What comes next
The Netflix TechBlog has written about catching performance regressions before they ship by running focused performance tests continuously and comparing each result with nearby historical data. We are working towards applying the same broad principle to Proton Drive: performance work should not depend on one-off debugging sessions or intuition.
The next step is to keep turning these investigations into automated guardrails. The load tester already gives us reproducible scenarios and comparable metrics. The profiling tools give us a way to explain regressions when they appear. The long-term goal is to make this loop tighter: detect suspicious changes earlier, explain them faster, and keep regressions from reaching customers.
Performance work gets far more tractable when every optimization traces back to a specific workload, a profile, a hypothesis, and a validation step.
If this kind of work interests you, come join us!
