Skip to content

RuntimeError: Invalid error code #727

Closed

Description

@nkormakov

Expected Behavior

No error, connection should be closed.

Actual Behavior

RuntimeError: Invalid error code

Snippet to Reproduce the Problem

zk = KazooClient(hosts = f'{ip}', verify_certs = False, timeout = 10)
zk.start()
zk.get_children(target_path)

It is happening when zookeeper running with a few extra jvm parameters

-Djava.security.auth.login.config=/etc/kafka/zookeeper_jaas.conf
-Dzookeeper.allowSaslFailedClients=false
-Dzookeeper.sessionRequireClientSASLAuth=true

Default library doesn't handle well error code -124 which declares that client should authorise himself. Our application is ok with closing connection altogether and not messing with it.
I've fixed it with monkey patching:

def new_read_response(self, header, buffer, offset):
    client = self.client
    request, async_object, xid = client._pending.popleft()
    if header.zxid and header.zxid > 0:
        client.last_zxid = header.zxid
    if header.xid != xid:
        exc = RuntimeError('xids do not match, expected %r '
                           'received %r', xid, header.xid)
        async_object.set_exception(exc)
        raise exc

    # Determine if it's an exists request and a no node error
    exists_error = (header.err == NoNodeError.code and
                    request.type == Exists.type)

    # Set the exception if it's not an exists error
    if header.err and not exists_error:
        # -124 error handler
        if header.err == -124:
            async_object.set(False)
            async_object.set_exception(NoAuthError())
            return CLOSE_RESPONSE
        callback_exception = EXCEPTIONS[header.err]()
        self.logger.debug(
            'Received error(xid=%s) %r', xid, callback_exception)
        if async_object:
            async_object.set_exception(callback_exception)
    elif request and async_object:
        if exists_error:
            # It's a NoNodeError, which is fine for an exists
            # request
            async_object.set(None)
        else:
            try:
                response = request.deserialize(buffer, offset)
            except Exception as exc:
                self.logger.exception(
                    "Exception raised during deserialization "
                    "of request: %s", request)
                async_object.set_exception(exc)
                return
            self.logger.debug(
                'Received response(xid=%s): %r', xid, response)

            # We special case a Transaction as we have to unchroot things
            if request.type == Transaction.type:
                response = Transaction.unchroot(client, response)

            async_object.set(response)

        # Determine if watchers should be registered
        watcher = getattr(request, 'watcher', None)
        if not client._stopped.is_set() and watcher:
            if isinstance(request, (GetChildren, GetChildren2)):
                client._child_watchers[request.path].add(watcher)
            else:
                client._data_watchers[request.path].add(watcher)

    if isinstance(request, Close):
        self.logger.log(BLATHER, 'Read close response')
        return CLOSE_RESPONSE

kazoo.protocol.connection.ConnectionHandler._read_response = new_read_response

Logs with logging in DEBUG mode

[2023-08-08 13:59:25,212][INFO]: connection.py:639:_connect - Connecting to 192.xxx.xxx.xxx(192.xxx.xxx.xxx):2181, use_ssl: False
[2023-08-08 13:59:25,223][DEBUG]: connection.py:312:_submit - Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None)
[2023-08-08 13:59:25,236][INFO]: client.py:532:_session_callback - Zookeeper connection established, state: CONNECTED
[2023-08-08 13:59:25,236][DEBUG]: connection.py:312:_submit - Sending request(xid=1): GetChildren(path='/', watcher=None)
[2023-08-08 13:59:25,248][ERROR]: connection.py:631:_connect_attempt - Unhandled exception in connection loop
Traceback (most recent call last):
  File "/opt/scanners/venv/lib/python3.10/site-packages/kazoo/protocol/connection.py", line 602, in _connect_attempt
    response = self._read_socket(read_timeout)
  File "/opt/scanners/venv/lib/python3.10/site-packages/kazoo/protocol/connection.py", line 453, in _read_socket
    return self._read_response(header, buffer, offset)
  File "/opt/scanners/m_zookeeper.py", line 41, in new_read_response
    callback_exception = EXCEPTIONS[header.err]()
  File "/opt/scanners/venv/lib/python3.10/site-packages/kazoo/exceptions.py", line 54, in _invalid_error_code
    raise RuntimeError('Invalid error code')
RuntimeError: Invalid error code
[2023-08-08 13:59:25,248][INFO]: client.py:537:_session_callback - Zookeeper session closed, state: CLOSED
Exception in thread Thread-3 (zk_loop):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/scanners/venv/lib/python3.10/site-packages/kazoo/protocol/connection.py", line 512, in zk_loop
    if retry(self._connect_loop, retry) is STOP_CONNECTING:
  File "/opt/scanners/venv/lib/python3.10/site-packages/kazoo/retry.py", line 126, in __call__
    return func(*args, **kwargs)
  File "/opt/scanners/venv/lib/python3.10/site-packages/kazoo/protocol/connection.py", line 552, in _connect_loop
    status = self._connect_attempt(host, hostip, port, retry)
  File "/opt/scanners/venv/lib/python3.10/site-packages/kazoo/protocol/connection.py", line 602, in _connect_attempt
    response = self._read_socket(read_timeout)
  File "/opt/scanners/venv/lib/python3.10/site-packages/kazoo/protocol/connection.py", line 453, in _read_socket
    return self._read_response(header, buffer, offset)
  File "/opt/scanners/m_zookeeper.py", line 41, in new_read_response
    callback_exception = EXCEPTIONS[header.err]()
  File "/opt/scanners/venv/lib/python3.10/site-packages/kazoo/exceptions.py", line 54, in _invalid_error_code
    raise RuntimeError('Invalid error code')
RuntimeError: Invalid error code

Specifications

  • Kazoo version: 2.9.0
  • Result of pip list command: kazoo only
  • Zookeeper version: Confluent Zookeper latest
  • Zookeeper configuration: put here any useful ZK configuration (authentication, encryption, number of ZK members, number of (concurrent?) clients, Java version, krb5 version, etc.)
  • Python version: 3.9
  • OS: ubuntu 22

Activity

  1. StephenSorriaux commented on Aug 9, 2023

    @StephenSorriaux
    Member

    Hello,

    Thank you for the issue. Indeed the ZSESSIONCLOSEDREQUIRESASLAUTH error (-124) is not handled properly (reminds me of a PR where @ztzg said some errors were not handled at all in the lib).

    Some documentation for reference: https://github.com/apache/zookeeper/blob/master/zookeeper-docs/src/main/resources/markdown/zookeeperAdmin.md?plain=1#L1615

  2. nkormakov commented on Aug 9, 2023

    @nkormakov
    Author

    Hi Stephen!

    What is the proper way of handling it will be? I'm aware that my workaround is not up to standards yet, but it does not produces some secondary errors or halts

  3. ceache commented on Aug 9, 2023

    @ceache
    Contributor
  4. nkormakov commented on Aug 9, 2023

    @nkormakov
    Author

    Ok, I'll look into that and make a PR for you to judge.

  5. jeffwidman commented on Feb 6, 2024

    @jeffwidman
    Member

    Fixed by #743

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions