[Jaeger & Jaeger Operator] Cannot skip SSL verify for existing Elasticsearch

[doge95]

Describe the bug
Cannot skip SSL verify for existing Elasticsearch when setting es.tls.skip-host-verify: true
Got the error message:

{"level":"info","ts":1648557710.6038692,"caller":"flags/admin.go:96","msg":"Admin server started","http.host-port":"[::]:14269","health-status":"unavailable"}
{"level":"fatal","ts":1648557715.7408428,"caller":"./main.go:80","msg":"Failed to init storage factory","error":"failed to create primary Elasticsearch client: health check timeout: Head \"https://admin-cluster-client.default.svc.cluster.local:9200\": x509: certificate is valid for node-0.example.com, localhost, not admin-cluster-client.default.svc.cluster.local: no Elasticsearch node available","stacktrace":"main.main.func1\n\t./main.go:80\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.3.0/command.go:856\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.3.0/command.go:974\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.3.0/command.go:902\nmain.main\n\t./main.go:147\nruntime.main\n\truntime/proc.go:255"}

To Reproduce
My Jaeger values.yaml

provisionDataStore:
  cassandra: false
storage:
  type: elasticsearch
  elasticsearch:
    scheme: https
    host: admin-cluster-client.default.svc.cluster.local
    port: 9200
    user: elastic
    password: <password>
agent:
  enabled: false
query: 
  cmdlineParams:
    es.tls.skip-host-verify: true
collector:
  cmdlineParams:
    es.tls.skip-host-verify: true

I also tried jaeger-operator but got the same error.

apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: simple-prod
spec:
  strategy: production
  storage:
    type: elasticsearch
    options:
      es:
        server-urls: https://admin-cluster-client.default.svc.cluster.local:9200
        username: elastic
        password: <password>
        tls:
          skip-host-verify: true

Expected behavior
Skip verifying the SSL

Version (please complete the following information):

• OS: Linux
• Jaeger version: 1.30.0
• Deployment: Kubernetes

[mk-raven]

I have the same problem, which makes it impossible to use jaeger helm chart in our clusters.
Please fix it! This problem live about 6 month. Why You do nothing for fix this?

[mehta-ankit]

@doge95 just curious why host value you use is this: host: admin-cluster-client.default.svc.cluster.local ?
I just use the svc name as host value and environment variable is created in helpers.tpl using scheme, host and port.

See here: https://github.com/jaegertracing/helm-charts/blob/main/charts/jaeger/templates/_helpers.tpl#L352 and https://github.com/jaegertracing/helm-charts/blob/main/charts/jaeger/templates/_helpers.tpl#L278

[mehta-ankit]

Also i see some issue's that were opened in the past on jaeger repo.
jaegertracing/jaeger#1582
jaegertracing/jaeger#1590
Can you make sure all proper flags are being passed : https://www.jaegertracing.io/docs/1.40/cli/#jaeger-query-elasticsearch
Either ways I personally dnt think its a helm chart issue, it all depends on what JAeger expects as a CLI flag and as client cert's.

[mk-raven]

@mehta-ankit I use it with this vars and have errors, but if know correctly
cli flags --es.tls.enabled default - false

collector vars:

collector:
  podSecurityContext: 
    runAsNonRoot: true
    runAsUser: 1000
    runAsGroup: 1000
  securityContext: 
    readOnlyRootFilesystem: true
    runAsNonRoot: true
    runAsUser: 1000
    runAsGroup: 1000
    allowPrivilegeEscalation: false
  enabled: true
  replicaCount: 3
  autoscaling:
    enabled: true
    minReplicas: 3
    maxReplicas: 6
    targetCPUUtilizationPercentage: 80
    targetMemoryUtilizationPercentage: 80
  resources: 
    limits:
      cpu: 1
      memory: 1Gi
    requests:
      cpu: 500m
      memory: 512Mi

jaeger-collector
{"level":"fatal","ts":1672157003.5803607,"caller":"./main.go:82","msg":"Failed to init storage factory","error":"failed to create primary Elasticsearch client: health check timeout: Head \"https://log.elk.lan:9260\": x509: certificate is not valid for any names, but wanted to match log.elk.lan: no Elasticsearch node available","stacktrace":"main.main.func1\n\t./main.go:82\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.6.1/command.go:916\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.6.1/command.go:1044\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.6.1/command.go:968\nmain.main\n\t./main.go:155\nruntime.main\n\truntime/proc.go:250"}

[mk-raven]

Also for jaeger Helm Chart the last appVersion: 1.39.0

[mlkmhd]

That's a problem I'm also facing

[babuaws100]

Give it as command line args

 containers:
      - name: jaeger-collector
        image: jaegertracing/jaeger-collector:1.51.0
        imagePullPolicy: IfNotPresent
        args:
          - --es.tls.enabled=true
          - --es.tls.skip-host-verify=true

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. To keep it open either add a comment or the label do-not-expire.

[github-actions]

This issue has been automatically closed due to inactivity.