-
Notifications
You must be signed in to change notification settings - Fork 2.2k
payments/db: add Payments benchmarks#10569
Closed
ziggie1984 wants to merge 95 commits intolightningnetwork:elle-payment-sql-series-newfrom
Closed
payments/db: add Payments benchmarks#10569ziggie1984 wants to merge 95 commits intolightningnetwork:elle-payment-sql-series-newfrom
ziggie1984 wants to merge 95 commits intolightningnetwork:elle-payment-sql-series-newfrom
Conversation
Collaborator
Summary
- Add
BenchmarkQueryPaymentsmeasuring theQueryPayments(ListPayments RPC path) across common query patterns against pre-populated 50k-payment databases - Sub-benchmarks:
default_page(100),large_page(1000),reversed,include_incomplete,omit_hops, andfetch_all(all 50k) - Runs against both bbolt and SQLite backends using
b.RunParallel
Also includes supporting benchmarks already in the file:
BenchmarkPayments: full payment lifecycle (Init - Register - Fail - Fetch - Register - Settle - DeleteFailed) against pre-populated testdataBenchmarkConcurrentPaymentFlow: same lifecycle with runtime preloadingTestPopulateDB: generates testdata databases (skipped by default)
Testdata files are not committed; generate locally with:
go test -run TestPopulateDB -v -count=1 -tags test_db_sqlite ./payments/db/...
Then run:
go test -bench BenchmarkQueryPayments -benchtime=100x -count=1 -tags test_db_sqlite ./payments/db/...
go test -bench 'BenchmarkPayments$' -benchtime=100x -count=1 -tags test_db_sqlite ./payments/db/...
Benchmark results (M4 Pro, 50k pre-populated payments, -benchtime=100x)
BenchmarkQueryPayments
goos: darwin
goarch: arm64
cpu: Apple M4 Pro
BenchmarkQueryPayments/bbolt/default_page-14 100 695381 ns/op
BenchmarkQueryPayments/bbolt/large_page-14 100 6663985 ns/op
BenchmarkQueryPayments/bbolt/reversed-14 100 698732 ns/op
BenchmarkQueryPayments/bbolt/include_incomplete-14 100 556392 ns/op
BenchmarkQueryPayments/bbolt/omit_hops-14 100 572189 ns/op
BenchmarkQueryPayments/bbolt/fetch_all-14 100 368034726 ns/op
BenchmarkQueryPayments/sqlite/default_page-14 100 9022407 ns/op
BenchmarkQueryPayments/sqlite/large_page-14 100 55741717 ns/op
BenchmarkQueryPayments/sqlite/reversed-14 100 8892144 ns/op
BenchmarkQueryPayments/sqlite/include_incomplete-14 100 8742358 ns/op
BenchmarkQueryPayments/sqlite/omit_hops-14 100 3629154 ns/op
BenchmarkQueryPayments/sqlite/fetch_all-14 100 2070545420 ns/op
BenchmarkPayments (full lifecycle)
goos: darwin
goarch: arm64
cpu: Apple M4 Pro
BenchmarkPayments/bbolt-14 100 14297547 ns/op
BenchmarkPayments/sqlite-14 100 45085083 ns/op
Key observations
| Sub-bench | BBolt (ms) | SQLite (ms) | Ratio |
|---|---|---|---|
| default_page (100) | 0.70 | 9.02 | 12.9x |
| large_page (1000) | 6.66 | 55.74 | 8.4x |
| reversed (100) | 0.70 | 8.89 | 12.7x |
| include_incomplete (100) | 0.56 | 8.74 | 15.6x |
| omit_hops (100) | 0.57 | 3.63 | 6.4x |
| fetch_all (50k) | 368 | 2071 | 5.6x |
| lifecycle | 14.3 | 45.1 | 3.2x |
omit_hopsgives a ~2.4x speedup on SQLite (hop JOIN/deserialization is the dominant cost)- The bbolt/SQLite gap narrows from ~13x to ~5.6x as page size grows (per-query overhead amortization + page cache effects at this dataset size)
- For the write-heavy lifecycle benchmark, SQLite is only 3.2x slower than bbolt
Test plan
-
go build -tags test_db_sqlite ./payments/db/...compiles
We prepare the code for the sql payment backend. However no
payment db interface method for the sql backend is implemented
yet. This will be done in the following commits. They currently
use the embedded KVStore to satify the build environment.
payment db interface method for the sql backend is implemented
yet. This will be done in the following commits. They currently
use the embedded KVStore to satify the build environment.
This does not include duplicate payments yet. They will be added
when the migration code is introduced for payments.
when the migration code is introduced for payments.
We allow the migration of the payment schema to be applied when
the test_native_sql is active to make sure the tables are properly
contructed.
the test_native_sql is active to make sure the tables are properly
contructed.
Previously a one(intent)-to-many(payment) relationship it is now
changed to a one-to-one relationship because a payment request
only can have 1 payment related to it. Looking into the future
with BOLT12 offers, the fetched invoice from the offer could be
stored here as well and the relationship would still hold.
changed to a one-to-one relationship because a payment request
only can have 1 payment related to it. Looking into the future
with BOLT12 offers, the fetched invoice from the offer could be
stored here as well and the relationship would still hold.
In this commit we add all queries which we will need to insert
payment related data into the db.
payment related data into the db.
We remove the SQLStore from most of the helper functions. This
also makes sure we do not accidentally create a new db tx but use
the provided db SQLQueries parameter.
also makes sure we do not accidentally create a new db tx but use
the provided db SQLQueries parameter.
We rename this variable to paymentsDetailsData because we will
also need to batch load the core payment and intent data in
future commits and this renaming should make it clear that this
does match payment related data but not the core data which is
in the payment and in the intent table.
also need to batch load the core payment and intent data in
future commits and this renaming should make it clear that this
does match payment related data but not the core data which is
in the payment and in the intent table.
We wrap the fetchPayment db call and catch the case where no errors
are found in the db, where we now return the ErrPaymentNotInitiated
error.
are found in the db, where we now return the ErrPaymentNotInitiated
error.
Drop two explicit indexes in the payments migration that duplicate indexes
already created by UNIQUE constraints:
- payment_htlc_attempts(attempt_index) duplicates UNIQUE(attempt_index)
- payment_route_hops(htlc_attempt_index) duplicates
UNIQUE(htlc_attempt_index, hop_index)
This reduces write/index maintenance overhead without changing query
capabilities, since the UNIQUE-backed autoindexes already satisfy these
lookups via exact and leftmost-prefix matching.
Also remove the corresponding DROP INDEX statements in the down migration.
We don't have to write a migration because the the main change is still
not merged into master to we can just change it.
already created by UNIQUE constraints:
- payment_htlc_attempts(attempt_index) duplicates UNIQUE(attempt_index)
- payment_route_hops(htlc_attempt_index) duplicates
UNIQUE(htlc_attempt_index, hop_index)
This reduces write/index maintenance overhead without changing query
capabilities, since the UNIQUE-backed autoindexes already satisfy these
lookups via exact and leftmost-prefix matching.
Also remove the corresponding DROP INDEX statements in the down migration.
We don't have to write a migration because the the main change is still
not merged into master to we can just change it.
Add two composite indexes to the payments schema migration to better match
high-frequency query patterns observed in the payment lifecycle:
- idx_htlc_payment_id_attempt_time on payment_htlc_attempts(payment_id, attempt_time)
Matches batched attempt reads that filter by payment_id and order by
payment_id, attempt_time (FetchHtlcAttemptsForPayments).
- idx_htlc_resolutions_type_attempt_index on
payment_htlc_attempt_resolutions(resolution_type, attempt_index)
Matches failed-attempt cleanup paths that filter by resolution_type and
then join/delete by attempt_index (DeleteFailedAttempts).
Also update the down migration to drop these new indexes.
high-frequency query patterns observed in the payment lifecycle:
- idx_htlc_payment_id_attempt_time on payment_htlc_attempts(payment_id, attempt_time)
Matches batched attempt reads that filter by payment_id and order by
payment_id, attempt_time (FetchHtlcAttemptsForPayments).
- idx_htlc_resolutions_type_attempt_index on
payment_htlc_attempt_resolutions(resolution_type, attempt_index)
Matches failed-attempt cleanup paths that filter by resolution_type and
then join/delete by attempt_index (DeleteFailedAttempts).
Also update the down migration to drop these new indexes.
Add a new omit_hops field to ListPaymentsRequest that allows clients
to skip loading hop-level route data for HTLC attempts, reducing both
query cost and response size. When set, the route is returned with
only route-level fields (TotalTimeLock, TotalAmount, SourcePubKey)
and no individual hop data or hop-level custom records.
to skip loading hop-level route data for HTLC attempts, reducing both
query cost and response size. When set, the route is returned with
only route-level fields (TotalTimeLock, TotalAmount, SourcePubKey)
and no individual hop data or hop-level custom records.
The previous query used an IN subquery that scanned all failed
resolutions across all payments (O(N) where N = total failed attempts
globally). Replace with a correlated EXISTS subquery that only checks
resolutions for the specific payment's attempts, making it O(k) where
k = attempts for this payment (typically 1-5).
resolutions across all payments (O(N) where N = total failed attempts
globally). Replace with a correlated EXISTS subquery that only checks
resolutions for the specific payment's attempts, making it O(k) where
k = attempts for this payment (typically 1-5).
Add benchmark_test.go with benchmarks for the payments database layer,
designed to run against pre-populated 50k-payment databases:
- BenchmarkQueryPayments: measures QueryPayments (the ListPayments RPC
path) across common query patterns: default page, large page, reversed,
include_incomplete, omit_hops, and fetch_all. Runs against both bbolt
and SQLite backends using b.RunParallel.
- BenchmarkPayments: measures the full payment lifecycle
(Init - Register - Fail - Fetch - Register - Settle - DeleteFailed)
against the same pre-populated databases.
- BenchmarkConcurrentPaymentFlow: same lifecycle but preloads payments
at runtime instead of using testdata, useful for fresh-DB profiling.
- TestPopulateDB: generates the pre-populated testdata databases
(skipped by default, run manually).
Testdata files are not committed; generate them locally with:
go test -run TestPopulateDB -v -count=1 -tags test_db_sqlite ./payments/db/...
Then run benchmarks with:
go test -bench BenchmarkQueryPayments -benchtime=100x -count=1 -tags test_db_sqlite ./payments/db/...
designed to run against pre-populated 50k-payment databases:
- BenchmarkQueryPayments: measures QueryPayments (the ListPayments RPC
path) across common query patterns: default page, large page, reversed,
include_incomplete, omit_hops, and fetch_all. Runs against both bbolt
and SQLite backends using b.RunParallel.
- BenchmarkPayments: measures the full payment lifecycle
(Init - Register - Fail - Fetch - Register - Settle - DeleteFailed)
against the same pre-populated databases.
- BenchmarkConcurrentPaymentFlow: same lifecycle but preloads payments
at runtime instead of using testdata, useful for fresh-DB profiling.
- TestPopulateDB: generates the pre-populated testdata databases
(skipped by default, run manually).
Testdata files are not committed; generate them locally with:
go test -run TestPopulateDB -v -count=1 -tags test_db_sqlite ./payments/db/...
Then run benchmarks with:
go test -bench BenchmarkQueryPayments -benchtime=100x -count=1 -tags test_db_sqlite ./payments/db/...
ziggie1984
changed the title
payments/db: add QueryPayments benchmarks
payments/db: add Payments benchmarks
Feb 12, 2026
Summary of ChangesHello @ziggie1984, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces performance benchmarks for payments database operations, enhances the Highlights
New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with and on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces benchmarks for payment database operations and adds a new omit_hops flag to the ListPayments RPC to improve query performance. The changes are well-implemented across the CLI, RPC, and database layers. The new benchmarks are comprehensive and cover various scenarios, including the full payment lifecycle and different query patterns.
Additionally, this PR includes a significant amount of code for migrating payments from the KV store to a native SQL backend. This migration logic appears to be well-structured and follows existing patterns in the codebase. The database schema and query optimizations are sound and should improve performance.
Overall, the changes are of high quality, and I have no specific comments for improvement.
ziggie1984
force-pushed
the
elle-payment-sql-series-new
branch
from
February 15, 2026 08:59
2104014 to
2538726
Compare
ziggie1984
force-pushed
the
elle-payment-sql-series-new
branch
6 times, most recently
from
February 25, 2026 17:36
78ba5e4 to
e9a8826
Compare
Collaborator
|
@ziggie1984, remember to re-request review from reviewers when ready |
Collaborator
Author
|
closing for now, we did the benchmark tests within the smoker environment. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.