Skip to content

store accounts async#194

Merged
palmerlao merged 5 commits intodevfrom
palmer/store-accounts-async
Jan 24, 2026
Merged

store accounts async#194
palmerlao merged 5 commits intodevfrom
palmer/store-accounts-async

Conversation

@palmerlao
Copy link
Collaborator

@palmerlao palmerlao commented Jan 23, 2026

This undoes the account prefetcher, which really implemented prefetch and async storage at the same time in the account loader, in favor of implementing just async storage in accounts DB.

image

testing

I crossed two epoch boundaries 3 times each with no incident:

./mithril verify-range \
...
--snapshot /mnt/nvme/snapshots-backup/snapshot-392161343-CV9cWMjcEk6gkh6ZRr5iFRFeLBmreauHJrCywJuVYL2F.tar.zst \
--incremental-snapshot /mnt/nvme/snapshots-backup/incremental-snapshot-392161343-392253715-HZPo3grHCoskYDWmdHX6XXLxBxX66hSEozLwHGNeZggU.tar.zst \
--num-slots 3000 \
...
./mithril verify-range \
...
--snapshot /mnt/nvme/snapshots-backup/snapshot-392663585-8FNmbEu9yZDTjNZbDKY7xxjA584Nch8DMeg53wkzLHu3.tar.zst \
--incremental-snapshot /mnt/nvme/snapshots-backup/incremental-snapshot-392663585-392686279-G2YJnh1A4rES4McB9ceQTFy8cEY1oJL24AXTTnjEj6Ep.tar.zst \
...
--num-slots 2250 \
...

I also ctrl C'd/resumed replay a few times.

performance

I ran this on f4d5dec and c48e6d3 (dev before prefetcher):

./mithril verify-range \
--load-from-snapshot \
--snapshot /mnt/mithril-ledger/snapshots/snapshot-395273312-BMG9gUFWoQVGSh53MQyXC36hHbTHEJzpBhnz2GZPUYb2.tar.zst \
--incremental-snapshot /mnt/mithril-ledger/snapshots/incremental-snapshot-395273312-395368707-9vNrVHXxLqdScJnjEomEjVdWmzAArF6FmaAMkhhH4ytV.tar.zst \
--accounts-path /mnt/mithril-accounts \
--num-slots 1000 \
--rpc=REDACTED \
--config config.toml

The slot replay duration improvement is more modest, only ~10ms. Probably due to LoadBlockAccounts still being on critical path and added coordination. However, this design decouples prefetch and async storage. It also could evolve to keeping modified accounts in memory until a slot is committed. For example, the list could become a tree of uncommitted slot modifications, and we don't have to invoke the store worker on those modifications until we know some subtree is confirmed.

…ntloader"

This reverts commit a0eaffb, reversing
changes made to c48e6d3.
@palmerlao palmerlao force-pushed the palmer/store-accounts-async branch 4 times, most recently from 53492c9 to 0ddec40 Compare January 23, 2026 17:35
@palmerlao palmerlao force-pushed the palmer/store-accounts-async branch from 0ddec40 to f4d5dec Compare January 23, 2026 18:55
@palmerlao palmerlao marked this pull request as ready for review January 24, 2026 02:00
@palmerlao palmerlao merged commit 07e908c into dev Jan 24, 2026
1 check passed
@palmerlao palmerlao deleted the palmer/store-accounts-async branch January 24, 2026 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants