handshakeComplete() resends the initial CONNECTOR_INFO request while the group is shutting down #125

Noticed while fixing #124; independent of it (the same happens when the initial request had already been sent before the shutdown).

What happens

The failure branch of WebSocketConnectionGroup.handshakeComplete() resubmits the initial CONNECTOR_INFO request on any failure of the first one, cancellation included:

request.getPromise().then(success, new Function<RuntimeException, Boolean, RuntimeException>() {
    public Boolean apply(RuntimeException e) {
        logger.ok("Resending initial 'CONNECTOR_INFO' request", e);
        ControlMessageRequest retry = trySubmitRequest(connectorInfoRequestFactory());   // WebSocketConnectionGroup.java:156
        ...

WebSocketConnectionGroup.shutdown() cancels every registered remote request with getPromise().cancel(true). PromiseImpl runs the promise listeners on the cancelling thread, and then(onResult, onException) calls onException for the CANCELLED state, so the retry's trySubmitRequest runs synchronously inside shutdown()'s loop over remoteRequests - before principalIsShuttingDown() gets to remove the sockets from webSockets. The retry is therefore sent over the still-open connection and registered in remoteRequests of a group that is being torn down.

Consequence

Not a crash, but:

• ConcurrentSkipListMap iteration is weakly consistent; request ids grow monotonically, so the loop usually also visits the new entry and cancels it with cancel(true) - the closing connection carries one extra CONNECTOR_INFO request and a CancelOpRequest for it, and the server side executes and then cancels an operation for a client that is gone.
• If the iterator does not see the new entry, the retry stays registered and pending in remoteRequests of the dead group; nothing cancels it afterwards (the sockets' close listeners only remove the socket).

Suggested fix

Do not retry when the first request was cancelled (promise.isCancelled() / the exception is the cancellation exception), or when the group is no longer accepting requests - shutdown() could record that principals is empty and trySubmitRequest / handshakeComplete() check it. The retry is meant for a transport failure of the first request, not for the group's own shutdown.