codius/codius-chat

Codius v1.2.5 is out! View https://medium.com/codius/codiusd-1-2-0-upgrading-and-websocket-support-42f712a0cee9 for details. Having trouble? Try the post at https://ilp-ix.link/1-2-1-selftest-fix/ from @WilcoC_gitlab

Brandon Wilson

@wilsonianb

I want to try out both kata + firecracker and kata microvm as alternatives which I expect would improve service cold boot times
but for now kata-qemu with its default is the simplest option for the installer

I'm also hoping to have an easy means of installing codius on a GKE cluster with gvisor (instead of kata) as the runtime
https://cloud.google.com/kubernetes-engine/docs/how-to/sandbox-pods

CoraxR

@CoraxR

@Sidewinder3514 do you have ilp pointers from gatehub ? have you entered as "https://ilp.gatehub.wallet ?When I open your hosts with coil sub nothing happens. I mean I see only "Awaiting Web Monetization...

Sidewinder3514

@Sidewinder3514

@CoraxR in the setup in centos7 I have put the pointer in the https format. The payment pointer currently needs to be entered as a URL (https://wallet/user as opposed to $wallet/user). I not done that in the coil website itself. So $ format. Is that what you are trying to say?

Sidewinder3514

@Sidewinder3514

https://sidewinder.internos.be

I you go there it's annother message I think?

CoraxR

@CoraxR

I meant I have coil subscription 5$/month. So If i visit your host - I should see token/bearer , some information about how to upload service on your host. Thats what I see on Brandon's hosts test.suidoc.net , proxy.suidoc.net, preview.suidoc.net. And also you should receive small amounts of money( perhaps xrp or else, depends on your paymentpointer) per request your host . But when I visit your host url https://sidewinder.internos.be I see pending message"Awaiting Web Monetization..." and nothing else. Also could you check your balance ? Did you receive some money?

Sidewinder3514

@Sidewinder3514

@CoraxR Sommething is coming in I think

Gatehub is showing:

@CoraxR But indeed sidewinder.internos.be is showing Awaiting Web Monetization...

and sidewinder2.internos.be nothing

Strange things

@wilsonianb Any clues?

Sidewinder3514

@Sidewinder3514

for the sidewinde2.internos.be I did "systemctl restart k3s.service"

And is now also showing "Awaiting Web Monetization..."

Sidewinder3514

@Sidewinder3514

@CoraxR Sommething is coming in I think

On my Interledger Payment Pointer in my Gatehub wallet

Brandon Wilson

@wilsonianb

Looking at my Coil extension logs, the SPSP request to https://sidewinder.internos.be/.well-known/pay is returning 503
Can you confirm that the receipt verifier pod is running and that the payment_pointer_url is a url (https://ilp.gatehub.net/XXXXXX not $ilp.gatehub.net/XXXXXX) in the the configmap

kubectl describe configmap -n codius-system codius-config

you could also check the receipt-verifier logs

kubectl logs  -n codius-system   receipt-verifier-XXXXX receipt-verifier

Sidewinder3514

@Sidewinder3514

Ok checking the logs

This one is for sidewinder.internos.be

This one is for sidewinder2.internos.be

Looks normal I think?

Brandon Wilson

@wilsonianb

Thanks. The gatehub payment pointer url looks fine. I can query it directly.
Can you check the traefik ingress logs for anything related to the receipt-verifier

kubectl logs -n kube-system     traefik-XXXXX

Sidewinder3514

@Sidewinder3514

@wilsonianb I mailed you the full result file

But this a brief synopsis :-)

[root@sidewinder ~]# kubectl logs -n kube-system traefik-6c96bd9f66-fxs6m
{"level":"info","msg":"Using TOML configuration file /config/traefik.toml","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"No tls.defaultCertificate given for https: using the first item in tls.certificates as a fallback.","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Traefik version v1.7.19 built on 2019-10-28_02:07:32PM","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"\nStats collection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMore details on: https://docs.traefik.io/v1.7/basics/#collected-data\n","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Preparing server https \u0026{Address::443 TLS:0xc000492630 Redirect:\u003cnil\u003e Auth:\u003cnil\u003e WhitelistSourceRange:[] WhiteList:\u003cnil\u003e Compress:true ProxyProtocol:\u003cnil\u003e ForwardedHeaders:0xc00089f700} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Preparing server prometheus \u0026{Address::9100 TLS:\u003cnil\u003e Redirect:\u003cnil\u003e Auth:\u003cnil\u003e WhitelistSourceRange:[] WhiteList:\u003cnil\u003e Compress:false ProxyProtocol:\u003cnil\u003e ForwardedHeaders:0xc00089f740} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Preparing server http \u0026{Address::80 TLS:\u003cnil\u003e Redirect:\u003cnil\u003e Auth:\u003cnil\u003e WhitelistSourceRange:[] WhiteList:\u003cnil\u003e Compress:true ProxyProtocol:\u003cnil\u003e ForwardedHeaders:0xc00089f6e0} with readTimeout=0s writeTimeout=0s idleTimeout=3m0s","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Starting provider configuration.ProviderAggregator {}","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Starting server on :443","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Starting server on :9100","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Starting server on :80","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Starting provider *kubernetes.Provider {\"Watch\":true,\"Filename\":\"\",\"Constraints\":[],\"Trace\":false,\"TemplateVersion\":0,\"DebugLogGeneratedTemplate\":false,\"Endpoint\":\"\",\"Token\":\"\",\"CertAuthFilePath\":\"\",\"DisablePassHostHeaders\":false,\"EnablePassTLSCert\":false,\"Namespaces\":null,\"LabelSelector\":\"\",\"IngressClass\":\"\",\"IngressEndpoint\":{\"IP\":\"\",\"Hostname\":\"\",\"PublishedService\":\"kube-system/traefik\"},\"ThrottleDuration\":0}","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"ingress label selector is: \"\"","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Creating in-cluster Provider client","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Server configuration reloaded on :9100","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Server configuration reloaded on :80","time":"2020-07-25T13:03:49Z"}
{"level":"info","msg":"Server configuration reloaded on :443","time":"2020-07-25T13:03:49Z"}
E0725 13:04:33.683804 1 reflector.go:322] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to watch *v1beta1.Ingress: Get https://10.43.0.1:443/apis/extensions/v1beta1/ingresses?resourceVersion=3&timeoutSeconds=563&watch=true: dial tcp 10.43.0.1:443: connect: connection refused
E0725 13:04:33.683931 1 reflector.go:322] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to watch *v1.Secret: Get https://10.43.0.1:443/api/v1/secrets?resourceVersion=708&timeoutSeconds=588&watch=true: dial tcp 10.43.0.1:443: connect: connection refused
E0725 13:04:33.684237 1 reflector.go:322] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to watch *v1.Endpoints: Get https://10.43.0.1:443/api/v1/endpoints?resourceVersion=748&timeoutSeconds=560&watch=true: dial tcp 10.43.0.1:443: connect: connection refused
E0725 13:04:33.684290 1 reflector.go:322] github.com/containous/traefik/vendor/k8s.io/client-go/informers/factory.go:86: Failed to watch *v1.Servi

This is for sidewinder.internos.be

Brandon Wilson

@wilsonianb

Can you try increasing the ingress logging by adding logLevel: "debug" to /var/lib/rancher/k3s/server/manifests/traefik-mod.yaml?
Modifying the file should cause the traefik pod to reload. After which can you visit the sidewinder.internos.be again and check the logs?

Sidewinder3514

@Sidewinder3514

@wilsonianb could give me more explanation?

I added # static configuration
log:
level: DEBUG
filePath: /var/lib/rancher/k3s/server/manifests/traefik-mod.yaml
format: json

directly into traefik-mod.yaml file

Brandon Wilson

@wilsonianb

I expect /var/lib/rancher/k3s/server/manifests/traefik-mod.yaml would like this with logLevel: "debug" added

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: traefik
  namespace: kube-system
spec:
  chart: https://%{KUBERNETES_API}%/static/charts/traefik-1.81.0.tgz
  valuesContent: |-
    rbac:
      enabled: true
    ssl:
      insecureSkipVerify: true
      enabled: true
    metrics:
      prometheus:
        enabled: true
    kubernetes:
      ingressEndpoint:
        useDefaultPublishedService: true
    image: "rancher/library-traefik"
    tolerations:
      - key: "CriticalAddonsOnly"
        operator: "Exists"
      - key: "node-role.kubernetes.io/master"
        operator: "Exists"
        effect: "NoSchedule"
    logLevel: "debug"

here's the configuration doc for reference:
https://github.com/helm/charts/tree/master/stable/traefik#configuration

Sidewinder3514

@Sidewinder3514

and when saving the file with the "debug" parameter is auto active?

or need reboot system?

Brandon Wilson

@wilsonianb

traefik will auto re-deploy. no need to reboot
https://rancher.com/docs/k3s/latest/en/helm/#deploying-manifests-and-helm-charts

Sidewinder3514

@Sidewinder3514

same message: Pay with Web Monetization to access this Codius host.

Brandon Wilson

@wilsonianb

I just re-tried both of your hosts
I expect there to be traefik logs for /.well-known/pay and/or the receipt verifier

feel free to dump the logs to a file and send them my way

Sidewinder3514

@Sidewinder3514

Ok thank for info, need to sleep now it's rather late here in Belgium and had a long working day

will look at it tomorrow

Brandon Wilson

@wilsonianb

:+1:

Sidewinder3514

@Sidewinder3514

could this be a problem?

Aug 01 08:26:12 sidewinder.internos.be k3s[6429]: E0801 08:26:12.224224 6429 dns.go:135] Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is: 195.135.195.135 195.8.195.8 2a01:7c8:7000:195:0:135:195:135

Brandon Wilson

@wilsonianb

hmm maybe
I don't have that in my k3s logs
https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues

Brandon Wilson

@wilsonianb

sounds like you could take a look at /etc/netplan/SOME_FILE.yaml / /etc/resolv.conf and see what was omitted
https://bugzilla.redhat.com/show_bug.cgi?id=1791008#c15

Where communities thrive

People

Repo info

Activity