NFS VolumeStore FQDN not working

[alatdev]

VIC version:

1.4.0

Deployment details:

What was the vic-machine create command used to deploy the VCH?
vic-machine-windows create --name lvte11 --container-name-convention lvte11_{name} --compute-resource vmcl01 --image-store vmfs12_vmcl01 --base-image-size 8GB --volume-store vmfs12_vmcl01:vsphere --volume-store vmfs12_vmcl01:default --bridge-network lvte11-bridge --bridge-network-range 172.16.0.0/12 --public-network 10.10.x.x --public-network-ip 10.10.1.236/16 --public-network-gateway 10.10.0.10 --dns-server 10.1.44.40 --container-network 10.10.x.x:10.10.x.x --container-network-ip-range 10.10.x.x:10.10.2.230-10.10.2.235 --container-network-gateway 10.10.x.x:10.10.0.10/16 --container-network-dns 10.10.x.x:10.1.44.40 --container-network-firewall 10.10.x.x:published --tls-cname lvte11 --certificate-key-size 2048 --tls-ca Land_Salzburg_Trust_Sub-04_CA.crt --user xxx --thumbprint xxx --target lvvm02.land-sbg.gv.at --registry-ca registry_ca.crt --ops-user xxx --affinity-vm-group --volume-store nfs://lvte24.land-sbg.gv.at/lvte11:nfs

Steps to reproduce:
docker volume create --opt VolumeStore=nfs --name test_ubuntu
docker run -it -v test_ubuntu:/mnt/test --net=10.10.x.x lvte05.land-sbg.gv.at/default-project/alpine:latest

Actual behavior:
When using FQDN to the NFS-Share, the container won't start. I cannot set a DNS-Search-List.
docker: Error response from daemon: Server error from portlayer: unable to wait for process launch status: container VM has unexpectedly powered off.

If I use the IP address, it works.

Expected behavior:
Container can reach NFS Share by FQDN.

Logs:

container-logs_fqdn_notworking.zip

bug2104495

[hickeng]

The error showing in the tether.debug file for the containerVM is:

Jun 12 2018 14:35:17.917Z ERROR mounting nfs://lvte24.land-sbg.gv.at/lvte11/volumes/test_ubuntu on /mnt/test failed: invalid argument

I have not confirmed, but believe that the mount syscall should be passed an IP address and not an unresolved name.

Approaches:

1. resolve the name in containerVM at boot time
2. resolve in the endpointVM when the container is created
3. resolve in endpointVM, add to container as hint, and use dns files resolution order in container.

If chosing (1) or (2) please open a follow up issue for (3).

(1) adds a dependency on a correctly configured and responsive DNS server that knows of the FQND on all container-networks.

(3) can be achieved by a change to config OR fixing #4716 and being explicit about our resolver priorities (dns file instead of the default file dns) for this one lookup. @gigawhitlocks has knowledge of the built in Go resolver logic.

Starting point hint for (1):

vic/lib/tether/ops_linux.go

Line 951 in 346982a

rawSource.WriteString(source.Hostname())