Force endpoint delete

[mavenugo]

There are cases in multi-host networking where an ungraceful host shutdown (and the docker daemon in that host never comes back up again) could result in endpoint resources (such as ip-address) being locked up. This PR provides an ability for higher level orchestration systems to force an endpoint cleanup from the datastore and hence the resources can be reused.

Signed-off-by: Madhu Venugopal madhu@docker.com

[mavenugo]

@mrjana @vieux as discussed, {with this PR + appropriate changes in docker/docker} swarm or other higher level orchestration systems will be able to reclaim the endpoint resources.
This along with Preferred IP will satisfy the container rescheduling needs as described in docker-archive/classicswarm#1578.

[vieux]

LGTM

[mavenugo]

@mrjana PTAL. Took care of all the changes as we discussed. (including the IT)

[mrjana]

Can you please make this error message more clearer for the user? Some thing like the remote host %s where the connection is active is alive? Also please remove the : and add it next to where you say host

[mavenugo]

@vieux based on the design-review comments from @mrjana we decided to add this check to prevent the -f from cleaning up the endpoint when the remote host holding the endpoint is still alive. At the engine level, the node liveness is a factor of discovery timer (by default 20 * 3 sec). This means, swarm cannot push a force delete till the engine catches up with the node failure.
We need this check in order for protecting the engine from invalid forced delete which can lead to inconsistent states and the user can get confused.

[vieux]

I disagree with this change, force is force.

In engine you can docker rmi -f <image> even if the engine has containers using this image.
I agree it can lead to inconsistent states, but the user did force it.

[mrjana]

LGTM