Enable lnd pod logging #691

macgyver13 · 2025-03-09T15:30:10Z

This update provides a new strategy for adding containers to a lnd pod via extraContainers. A reference implementation is integrated into test/data/ln and test/data/logging.

Add lnd-exporter metric capture to test/data/ln (activated by metricsExport: true)
Extend lnd pod template to support extraContainers
Removes broken reference to circuitBreaker

Extend lnd pod template with extraContainers

willcl-ark · 2025-03-09T20:30:15Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

macgyver13 · 2025-03-09T20:43:00Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#688 (Circuit breaker plugin by Camillarhi)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

#691 should proceed before #688 IMO

pinheadmz

This is looking good so far. Try to make as few changes as possible to test/data/ln since there are tests that expect that network to have specific properties. Ideally you'd just be able to add exporters to the existing LN nodes in that network and then add a test (either in ln_test.py or 'ln_basic_test.py) similar to the fork observer check in services_test.py

I'll try to allow-list you so CI runs automatically when you push. Be sure to run the tests locally as well!

docs/logging_monitoring.md

- Reduced complexity of ln network.yaml config to align with ln_basic_test

pinheadmz

Nice!!

pinheadmz · 2025-03-12T14:45:04Z

test/data/logging/network.yaml

+          imagePullPolicy: IfNotPresent
+          env:
+            - name: ADMIN_MACAROON_HEX
+              value: 0201036c6e6402f801030a1062beabbf2a614b112128afa0c0b4fdd61201301a160a0761646472657373120472656164120577726974651a130a04696e666f120472656164120577726974651a170a08696e766f69636573120472656164120577726974651a210a086d616361726f6f6e120867656e6572617465120472656164120577726974651a160a076d657373616765120472656164120577726974651a170a086f6666636861696e120472656164120577726974651a160a076f6e636861696e120472656164120577726974651a140a057065657273120472656164120577726974651a180a067369676e6572120867656e657261746512047265616400000620b17be53e367290871681055d0de15587f6d1cd47d1248fe2662ae27f62cfbdc6


for a follow up PR maybe we can figure out how to make this value (and the tls.cert) a global value for all helm charts in the system

lnd-exporter assumes this value by default so it can technically be removed everywhere in warnet where lnd-exporter is referenced - just added for completeness.

Oh hm. Probably not the best idea to maintain a constant value in two separate repos, in case it ever changes in warnet ...?

Agree that is messy so lets go with your global helm chart approach, I know that is already implemented for chain (regtest)

I don't really understand what the global helm approach would be but if it can be deleted then let's do that?

test/data/ln/node-defaults.yaml

pinheadmz

ACK aa4d3b2

Show Signature

pinheadmz's public key is on openpgp.org

m3dwards · 2025-03-12T15:20:52Z

utACK aa4d3b2

-shared volume for containers in same pod -lookup configmap based on label for separate pods

…puter. pipeline artifacts did not provide any useful information

test/data/ln/node-defaults.yaml

pinheadmz · 2025-03-17T15:32:46Z

ACK this is great thanks!

I added #696 locally and will probably merge them together. Then I viewed the lnd dashboard after making a few changes to the test/data/ln network so there'd be something to look at:

diff --git a/resources/plugins/simln/charts/simln/templates/pod.yaml b/resources/plugins/simln/charts/simln/templates/pod.yaml
index 69790c9e..bda0344f 100644
--- a/resources/plugins/simln/charts/simln/templates/pod.yaml
+++ b/resources/plugins/simln/charts/simln/templates/pod.yaml
@@ -36,7 +36,7 @@ spec:
       args:
         - >
           cd /working;
-          sim-cli
+          sim-cli --capacity-multiplier=30
       volumeMounts:
         - name: {{ .Values.workingVolume.name }}
           mountPath: {{ .Values.workingVolume.mountPath }}
diff --git a/test/data/ln/network.yaml b/test/data/ln/network.yaml
index 5d30686d..bbf07636 100644
--- a/test/data/ln/network.yaml
+++ b/test/data/ln/network.yaml
@@ -19,8 +19,8 @@ nodes:
             block: 300
             index: 1
           target: tank-0004-ln
-          capacity: 100000
-          push_amt: 50000
+          capacity: 10000000
+          push_amt: 500000
   - name: tank-0004
     addnode:
       - tank-0000
@@ -30,8 +30,16 @@ nodes:
             block: 300
             index: 2
           target: tank-0005-ln
-          capacity: 50000
-          push_amt: 25000
+          capacity: 5000000
+          push_amt: 250000
   - name: tank-0005
     addnode:
-      - tank-0000
\ No newline at end of file
+      - tank-0000
+
+plugins:
+  postDeploy:
+    simln:
+      entrypoint: "../../../resources/plugins/simln"
+
+caddy:
+  enabled: true
\ No newline at end of file
diff --git a/test/data/ln/node-defaults.yaml b/test/data/ln/node-defaults.yaml
index 62e05199..d82ef96f 100644
--- a/test/data/ln/node-defaults.yaml
+++ b/test/data/ln/node-defaults.yaml
@@ -5,8 +5,8 @@ image:
   repository: bitcoindevproject/bitcoin
   pullPolicy: IfNotPresent
   tag: "27.0"
-collectLogs: false
-metricsExport: false
+collectLogs: true
+metricsExport: true
 
 #LN configs
 ln:
@@ -16,7 +16,7 @@ lnd:
     color=#000000
   config: |
     bitcoin.timelockdelta=33
-  metricsExport: false
+  metricsExport: true
   prometheusMetricsPort: 9332
   extraContainers:
     - name: lnd-exporter

macgyver13 added 3 commits March 9, 2025 11:21

Add lnd exporter test data

46900fe

Extend lnd pod template with extraContainers

set "lnd" as default k8s container to support kubectl exec commands

19076ce

correctly define default annotation

0ca00d0

willcl-ark mentioned this pull request Mar 9, 2025

Circuit breaker plugin #688

Draft

pinheadmz reviewed Mar 10, 2025

View reviewed changes

docs/logging_monitoring.md Outdated Show resolved Hide resolved

macgyver13 added 2 commits March 10, 2025 20:39

- Add grafana metric validation test to logging_test

019f2bc

- Reduced complexity of ln network.yaml config to align with ln_basic_test

build lnd-exporter docker on linux - version 0.1.2

aa4d3b2

macgyver13 requested a review from pinheadmz March 12, 2025 13:08

pinheadmz reviewed Mar 12, 2025

View reviewed changes

test/data/ln/node-defaults.yaml Show resolved Hide resolved

pinheadmz approved these changes Mar 12, 2025

View reviewed changes

macgyver13 added 2 commits March 13, 2025 09:13

update to support single macaroon reference

f62fff1

-shared volume for containers in same pod -lookup configmap based on label for separate pods

sanity check re-run pipeline - unable to reproduce error on local com…

33b4e16

…puter. pipeline artifacts did not provide any useful information

macgyver13 mentioned this pull request Mar 14, 2025

Add sample lnd dashboard to grafana #696

Merged

pinheadmz reviewed Mar 17, 2025

View reviewed changes

test/data/ln/node-defaults.yaml Show resolved Hide resolved

pinheadmz merged commit 8a2e34c into bitcoin-dev-project:main Mar 17, 2025
15 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable lnd pod logging #691

Enable lnd pod logging #691

macgyver13 commented Mar 9, 2025 •

edited

Loading

willcl-ark commented Mar 9, 2025 •

edited by bdp-DrahtBot

Loading

macgyver13 commented Mar 9, 2025 •

edited

Loading

Conflicts

pinheadmz left a comment

pinheadmz left a comment

pinheadmz Mar 12, 2025

macgyver13 Mar 12, 2025

pinheadmz Mar 12, 2025

macgyver13 Mar 12, 2025

m3dwards Mar 12, 2025

pinheadmz left a comment

m3dwards commented Mar 12, 2025

pinheadmz commented Mar 17, 2025

Enable lnd pod logging #691

Enable lnd pod logging #691

Conversation

macgyver13 commented Mar 9, 2025 • edited Loading

willcl-ark commented Mar 9, 2025 • edited by bdp-DrahtBot Loading

Conflicts

macgyver13 commented Mar 9, 2025 • edited Loading

Conflicts

pinheadmz left a comment

Choose a reason for hiding this comment

pinheadmz left a comment

Choose a reason for hiding this comment

pinheadmz Mar 12, 2025

Choose a reason for hiding this comment

macgyver13 Mar 12, 2025

Choose a reason for hiding this comment

pinheadmz Mar 12, 2025

Choose a reason for hiding this comment

macgyver13 Mar 12, 2025

Choose a reason for hiding this comment

m3dwards Mar 12, 2025

Choose a reason for hiding this comment

pinheadmz left a comment

Choose a reason for hiding this comment

m3dwards commented Mar 12, 2025

pinheadmz commented Mar 17, 2025

macgyver13 commented Mar 9, 2025 •

edited

Loading

willcl-ark commented Mar 9, 2025 •

edited by bdp-DrahtBot

Loading

macgyver13 commented Mar 9, 2025 •

edited

Loading