dependencies issue when installing ansible

[mdraevich]

I have been trying solve this issue for several days, but there is end up no result.

• I downloaded stackstorm for docker from github repository and did the following:

docker-compose up -d
docker-compose exec st2client bash

sudo apt-get update
sudo apt-get install libkrb5-dev gcc python3-dev python-dev

st2 login st2admin -p mypass
st2 pack install ansible

• During installation I got the following error messages:

root@bc772a9689a3:/opt/stackstorm# st2 pack install ansible 

For the "ansible" pack, the following content will be registered:

actions   |  8
rules     |  0
sensors   |  0
aliases   |  0
triggers  |  0

Installation may take a while for packs with many items.

	[ succeeded ] init_task
	[ succeeded ] download_pack
	[ succeeded ] make_a_prerun
	[ succeeded ] get_pack_dependencies
	[ succeeded ] check_dependency_and_conflict_list
	[  failed   ] install_pack_requirements

id: 5fb83ef40c5d67e8030fdeb0
action.ref: packs.install
parameters: 
  packs:
  - ansible
  python3: false
status: failed
start_timestamp: Fri, 20 Nov 2020 22:11:00 UTC
end_timestamp: Fri, 20 Nov 2020 22:15:40 UTC
result: 
  errors:
  - message: Execution failed. See result for details.
    result:
      exit_code: 1
      result: None
      stderr: "st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Setting up virtualenv for pack "ansible" (/opt/stackstorm/packs/ansible)
st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Removing virtualenv in "/opt/stackstorm/virtualenvs/ansible"
st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Creating virtualenv for pack "ansible" in "/opt/stackstorm/virtualenvs/ansible"
st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Creating virtualenv in "/opt/stackstorm/virtualenvs/ansible" using Python binary "/opt/stackstorm/st2/bin/python"
st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Running command "/opt/stackstorm/st2/bin/virtualenv -p /opt/stackstorm/st2/bin/python --always-copy --no-download /opt/stackstorm/virtualenvs/ansible" to create virtualenv.
st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Installing base requirements
st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Installing requirement six>=1.9.0,<2.0 with command /opt/stackstorm/virtualenvs/ansible/bin/pip install six>=1.9.0,<2.0.
st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Installing pack specific requirements from "/opt/stackstorm/packs/ansible/requirements.txt"
st2.actions.python.SetupVirtualEnvironmentAction: DEBUG    Installing requirements from file /opt/stackstorm/packs/ansible/requirements.txt with command /opt/stackstorm/virtualenvs/ansible/bin/pip install -U -r /opt/stackstorm/packs/ansible/requirements.txt.
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/python_runner/python_action_wrapper.py", line 334, in <module>
    obj.run()
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/python_runner/python_action_wrapper.py", line 193, in run
    output = action.run(**self._parameters)
  File "/opt/stackstorm/packs/packs/actions/pack_mgmt/setup_virtualenv.py", line 89, in run
    no_download=no_download)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/virtualenvs.py", line 123, in setup_pack_virtualenv
    logger=logger)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/virtualenvs.py", line 273, in install_requirements
    (requirements_file_path, stdout, stderr))
Exception: Failed to install requirements from "/opt/stackstorm/packs/ansible/requirements.txt": Collecting ansible>=1.9 (from -r /opt/stackstorm/packs/ansible/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/9c/f4/c156b10d7ae90ba6b99b1b126f7d30628adc1e733a6fbd63569852948f21/ansible-2.10.3.tar.gz (28.0MB)
Collecting pywinrm[credssp,kerberos]>=0.2.2 (from -r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/fc/88/be0ea1af44c3bcc54e4c41e4056986743551693c77dfe50b48a3f4ba1bf7/pywinrm-0.4.1.tar.gz
Collecting netaddr>=0.7.19 (from -r /opt/stackstorm/packs/ansible/requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/ff/cd/9cdfea8fc45c56680b798db6a55fa60a22e2d3d3ccf54fc729d083b50ce4/netaddr-0.8.0-py2.py3-none-any.whl (1.9MB)
Collecting ansible-base<2.11,>=2.10.3 (from ansible>=1.9->-r /opt/stackstorm/packs/ansible/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/2e/d1/92422f8f53ae2d4e75ebdc2be2186a9ca2796b5d9679f20ed0239e86d8cf/ansible-base-2.10.3.tar.gz (5.8MB)
Collecting xmltodict (from pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl
Collecting requests>=2.9.1 (from pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/39/fc/f91eac5a39a65f75a7adb58eac7fa78871ea9872283fb9c44e6545998134/requests-2.25.0-py2.py3-none-any.whl (61kB)
Collecting requests_ntlm>=0.3.0 (from pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
Requirement already satisfied, skipping upgrade: six in ./virtualenvs/ansible/lib/python3.6/site-packages (from pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2)) (1.15.0)
Collecting requests-credssp>=1.0.0 (from pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/5c/fa/8c4742147b7071f1309e250a9f82949dd199c3b636b0dc565a47bce836eb/requests_credssp-1.2.0-py2.py3-none-any.whl
Collecting pykerberos<2.0.0,>=1.2.1 (from pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/9a/b8/1ec56b6fa8a2e2a81420bd3d90e70b59fc83f6b857fb2c2c37accddc8be3/pykerberos-1.2.1.tar.gz
Collecting importlib-resources; python_version < "3.7" (from netaddr>=0.7.19->-r /opt/stackstorm/packs/ansible/requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/c5/1f/ec86d2a5c48ac6490d4471b297885603cf0e8da89d5ffbf0bce6e57f4d64/importlib_resources-3.3.0-py2.py3-none-any.whl
Collecting jinja2 (from ansible-base<2.11,>=2.10.3->ansible>=1.9->-r /opt/stackstorm/packs/ansible/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/30/9e/f663a2aa66a09d838042ae1a2c5659828bb9b41ea3a6efa20a20fd92b121/Jinja2-2.11.2-py2.py3-none-any.whl (125kB)
Collecting PyYAML (from ansible-base<2.11,>=2.10.3->ansible>=1.9->-r /opt/stackstorm/packs/ansible/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/64/c2/b80047c7ac2478f9501676c988a5411ed5572f35d1beff9cae07d321512c/PyYAML-5.3.1.tar.gz (269kB)
Collecting cryptography (from ansible-base<2.11,>=2.10.3->ansible>=1.9->-r /opt/stackstorm/packs/ansible/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/4c/a2/6565c5271a79e3c96d7a079053b4d8408a740d4bf365f0f5f244a807bd09/cryptography-3.2.1-cp35-abi3-manylinux2010_x86_64.whl (2.6MB)
Collecting packaging (from ansible-base<2.11,>=2.10.3->ansible>=1.9->-r /opt/stackstorm/packs/ansible/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/46/19/c5ab91b1b05cfe63cccd5cfc971db9214c6dd6ced54e33c30d5af1d2bc43/packaging-20.4-py2.py3-none-any.whl
Collecting chardet<4,>=3.0.2 (from requests>=2.9.1->pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
Collecting urllib3<1.27,>=1.21.1 (from requests>=2.9.1->pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/f5/71/45d36a8df68f3ebb098d6861b2c017f3d094538c0fb98fa61d4dc43e69b9/urllib3-1.26.2-py2.py3-none-any.whl (136kB)
Collecting certifi>=2017.4.17 (from requests>=2.9.1->pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/c1/6f/3d85f0850962279a7e4c622695d7b3171e95ac65308a57d3b29738b27149/certifi-2020.11.8-py2.py3-none-any.whl (155kB)
Collecting idna<3,>=2.5 (from requests>=2.9.1->pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/a2/38/928ddce2273eaa564f6f50de919327bf3a00f091b5baba8dfa9460f3a8a8/idna-2.10-py2.py3-none-any.whl (58kB)
Collecting ntlm-auth>=1.0.2 (from requests_ntlm>=0.3.0->pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/ff/84/97c550164b54942b0e908c31ef09d9469f3ba4cd7332a671e2125732f63b/ntlm_auth-1.5.0-py2.py3-none-any.whl
Collecting pyasn1>=0.3.1 (from requests-credssp>=1.0.0->pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/62/1e/a94a8d635fa3ce4cfc7f506003548d0a2447ae76fd5ca53932970fe3053f/pyasn1-0.4.8-py2.py3-none-any.whl (77kB)
Collecting pyOpenSSL>=16.0.0 (from requests-credssp>=1.0.0->pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/9e/de/f8342b68fa9e981d348039954657bdf681b2ab93de27443be51865ffa310/pyOpenSSL-19.1.0-py2.py3-none-any.whl (53kB)
Collecting pyspnego (from requests-credssp>=1.0.0->pywinrm[credssp,kerberos]>=0.2.2->-r /opt/stackstorm/packs/ansible/requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/0a/76/18d75e793e52cb24880cbdaad78bd410e4902b79f487f5f8a695cac5e223/pyspnego-0.1.3-py2.py3-none-any.whl (107kB)
Collecting zipp>=0.4; python_version < "3.8" (from importlib-resources; python_version < "3.7"->netaddr>=0.7.19->-r /opt/stackstorm/packs/ansible/requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/41/ad/6a4f1a124b325618a7fb758b885b68ff7b058eec47d9220a12ab38d90b1f/zipp-3.4.0-py3-none-any.whl
Collecting MarkupSafe>=0.23 (from jinja2->ansible-base<2.11,>=2.10.3->ansible>=1.9->-r /opt/stackstorm/packs/ansible/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting cffi!=1.11.3,>=1.8 (from cryptography->ansible-base<2.11,>=2.10.3->ansible>=1.9->-r /opt/stackstorm/packs/ansible/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/50/ca/bbca0fd95b24a1d4f0d2e016f09f35ae68d4fe72bf34cc538d0a0d2d3e10/cffi-1.14.3-cp36-cp36m-manylinux1_x86_64.whl (400kB)
Collecting pyparsing>=2.0.2 (from packaging->ansible-base<2.11,>=2.10.3->ansible>=1.9->-r /opt/stackstorm/packs/ansible/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/8a/bb/488841f56197b13700afd5658fc279a2025a39e22449b7cf29864669b15d/pyparsing-2.4.7-py2.py3-none-any.whl (67kB)
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography->ansible-base<2.11,>=2.10.3->ansible>=1.9->-r /opt/stackstorm/packs/ansible/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/ae/e7/d9c3a176ca4b02024debf82342dab36efadfc5776f9c8db077e8f6e71821/pycparser-2.20-py2.py3-none-any.whl (112kB)
Building wheels for collected packages: ansible, pywinrm, ansible-base, pykerberos, PyYAML
  Building wheel for ansible (setup.py): started
  Building wheel for ansible (setup.py): finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/41/79/b5/1f6f77b8a80eab71e9edd46169ab72940d15dba1e346a64134
  Building wheel for pywinrm (setup.py): started
  Building wheel for pywinrm (setup.py): finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/8f/db/f3/3028984b926c1c0165d7be4ead1092d219bee6ef71e9219578
  Building wheel for ansible-base (setup.py): started
  Building wheel for ansible-base (setup.py): finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/46/c4/89/c154e9a421044e3ac0712b644709c44e8cd5767b6162825a09
  Building wheel for pykerberos (setup.py): started
  Building wheel for pykerberos (setup.py): finished with status 'error'
  Running setup.py clean for pykerberos
  Building wheel for PyYAML (setup.py): started
  Building wheel for PyYAML (setup.py): finished with status 'done'
  Stored in directory: /root/.cache/pip/wheels/a7/c1/ea/cf5bd31012e735dc1dfea3131a2d5eae7978b251083d6247bd
Successfully built ansible pywinrm ansible-base PyYAML
Failed to build pykerberos
Installing collected packages: MarkupSafe, jinja2, PyYAML, pycparser, cffi, cryptography, pyparsing, packaging, ansible-base, ansible, xmltodict, chardet, urllib3, certifi, idna, requests, ntlm-auth, requests-ntlm, pyasn1, pyOpenSSL, pyspnego, requests-credssp, pykerberos, pywinrm, zipp, importlib-resources, netaddr
  Running setup.py install for pykerberos: started
    Running setup.py install for pykerberos: finished with status 'error'
 (stderr:   ERROR: Complete output from command /opt/stackstorm/virtualenvs/ansible/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-ekbnisby/pykerberos/setup.py'"'"';f=getattr(\okenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\
'"'"', '"'"'\
'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-1lt86pfy --python-tag cp36:
  ERROR: running bdist_wheel
  running build
  running build_ext
  building 'kerberos' extension
  creating build
  creating build/temp.linux-x86_64-3.6
  creating build/temp.linux-x86_64-3.6/src
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -I/opt/stackstorm/virtualenvs/ansible/include/python3.6m -c src/kerberos.c -o build/temp.linux-x86_64-3.6/src/kerberos.o
  unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for pykerberos
    ERROR: Complete output from command /opt/stackstorm/virtualenvs/ansible/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-ekbnisby/pykerberos/setup.py'"'"';f=getattr(tokenize\ '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\
'"'"', '"'"'\
'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-96df44pv/install-record.txt --single-version-externally-managed --compile --install-headers /opt/stackstorm/virtualenvs/ansible/include/site/python3.6/pykerberos:
    ERROR: running install
    running build
    running build_ext
    building 'kerberos' extension
    creating build
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/src
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -I/opt/stackstorm/virtualenvs/ansible/include/python3.6m -c src/kerberos.c -o build/temp.linux-x86_64-3.6/src/kerberos.o
    unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command "/opt/stackstorm/virtualenvs/ansible/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-ekbnisby/pykerberos/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(\_file__);code=f.read().replace('"'"'\
'"'"', '"'"'\
'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-96df44pv/install-record.txt --single-version-externally-managed --compile --install-headers /opt/stackstorm/virtualenvs/ansible/include/site/python3.6/pykerberos" failed with error code 1 in /tmp/pip-install-ekbnisby/pykerberos/
WARNING: You are using pip version 19.1.1, however version 20.2.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
)
"
      stdout: ''
    task_id: install_pack_requirements
    type: error
  output:
    conflict_list: []
    message: ''
    packs_list:
    - ansible
    warning_list: null
+--------------------------+------------------------+------------------------------------+-----------------------------+-------------------------------+
| id                       | status                 | task                               | action                      | start_timestamp               |
+--------------------------+------------------------+------------------------------------+-----------------------------+-------------------------------+
| 5fb83ef54d9ec79272e88120 | succeeded (0s elapsed) | init_task                          | core.noop                   | Fri, 20 Nov 2020 22:11:01 UTC |
| 5fb83ef64d9ec79272e88130 | succeeded (3s elapsed) | download_pack                      | packs.download              | Fri, 20 Nov 2020 22:11:02 UTC |
| 5fb83ef94d9ec79272e88140 | succeeded (0s elapsed) | make_a_prerun                      | packs.virtualenv_prerun     | Fri, 20 Nov 2020 22:11:05 UTC |
| 5fb83efa4d9ec79272e88150 | succeeded (1s elapsed) | get_pack_dependencies              | packs.get_pack_dependencies | Fri, 20 Nov 2020 22:11:06 UTC |
| 5fb83efb4d9ec79272e88160 | succeeded (0s elapsed) | check_dependency_and_conflict_list | core.noop                   | Fri, 20 Nov 2020 22:11:07 UTC |
| 5fb83efc4d9ec79272e88170 | failed (272s elapsed)  | install_pack_requirements          | packs.setup_virtualenv      | Fri, 20 Nov 2020 22:11:08 UTC |
+--------------------------+------------------------+------------------------------------+-----------------------------+-------------------------------+

What dependencies are missed? I haven't found anymore...

Waiting for your answer, thanks in advance!

[arm4b]

Looks like https://github.com/stackstorm/st2-docker repository is more relevant to this topic as you're running stackstorm in Docker and not deploying/configuring it with Ansible.

The issue is that you're trying to install os-level dependencies for the Ansible pack in the st2client container, while there are other 13 containers/microservices of stackstorm running that might need that dependency. The actual execution happening in the st2actionrunner container, - you'll need that OS dependency in there. It could be not that obvious and I'd also recommend to use a VM like Vagrant (https://docs.stackstorm.com/install/vagrant.html) if you're just getting started with StackStorm as Docker might bring more blockers during a quick start.

Hope that helps.
I'd also suggest to ask questions on Slack and Forum https://stackstorm.com/#community as Github Issues are for bugs and feature requests. By just asking in Slack community might give you correct pointers immediately instead of you trying to figure it out for days!

[mdraevich]

Oh, i'm so sorry, I missed your reply here. Thanks a lot!

[guo3561101]

You should do it in this st2actionrunners container install sudo apt-get install libkrb5-dev gcc python3-dev python-dev