CELERY_RESULT_BACKEND = 'django-cache' is not working when CELERY_CACHE_BACKEND='default' points to a Database

[ShivKJ]

Hi,

I am trying to use Database for CELERY_RESULT_BACKEND and following this doc.

I set the following in settings.py

CELERY_RESULT_BACKEND = 'django-cache'
CELERY_CACHE_BACKEND = 'default'

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": 'redis://localhost/0',
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
            'MAX_ENTRIES': 1000
        },
    }
}

results are getting saved in redis correctly.

However, when I use Database in caching,

CELERY_RESULT_BACKEND = 'django-cache'
CELERY_CACHE_BACKEND = 'default'

CACHES = {
    'default': {
        'BACKEND': 'django.core.cache.backends.db.DatabaseCache',
        'LOCATION': 'cacher',
    }
}

then I get the following error (after doing django migration),

[ ERROR/MainProcess] Pool callback raised exception: TypeError('ord() expected string of length 1, but int found')
Traceback (most recent call last):
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/billiard/pool.py", line 1796, in safe_apply_callback
    fun(*args, **kwargs)
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/celery/worker/request.py", line 571, in on_failure
    self.task.backend.mark_as_failure(
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/celery/backends/base.py", line 171, in mark_as_failure
    self.store_result(task_id, exc, state,
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/celery/backends/base.py", line 482, in store_result
    self._store_result(task_id, result, state, traceback,
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/celery/backends/base.py", line 903, in _store_result
    current_meta = self._get_task_meta_for(task_id)
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/celery/backends/base.py", line 925, in _get_task_meta_for
    meta = self.get(self.get_key_for_task(task_id))
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/django_celery_results/backends/cache.py", line 18, in get
    return self.cache_backend.get(key)
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/django/core/cache/backends/db.py", line 51, in get
    return self.get_many([key], version).get(key, default)
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/django/core/cache/backends/db.py", line 59, in get_many
    self.validate_key(key)
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/django/core/cache/backends/base.py", line 249, in validate_key
    for warning in memcache_key_warnings(key):
  File "/home/abc/.virtualenvs/exp/lib/python3.9/site-packages/django/core/cache/backends/base.py", line 287, in memcache_key_warnings
    if ord(char) < 33 or ord(char) == 127:
TypeError: ord() expected string of length 1, but int found

Is this the expected behavior with Database as backend cache?

---

Please note that if I use,

CELERY_RESULT_BACKEND = 'django-db'

Then it would work, as django_celery_results creates tables for it and results will be stored correctly. But this method is different from that of using django-cache

[badeendjuh]

I'm running into the same error. From my experimenting I have found so far that:

• The problem occurs only when using CELERY_RESULT_BACKEND='django-cache' and the django cache driver django.core.cache.backends.db.DatabaseCache.
• Naming the cache entry anything other then 'default' does not change the error.
• Using a different cache driver like django.core.cache.backends.filebased.FileBasedCache works fine
• The stack trace is about retrieving the results but the problem seems to be that no results are stored to begin with. The cache table always remains empty.
• Setting and getting cache items using the Django Cache API directly works fine.

[badeendjuh]

I've narrowed down the problem to django_celery_results/backends/cache.py. The CacheBackend class feeds a key item to get/set that is of type byte. The Django Cache API does not except this, it wants a str type key.

The following code django_celery_results/backends/cache.py fixes it for me.
This is similar to a fix done for CoucheDB: celery/celery#6781

"""Celery cache backend using the Django Cache Framework."""
from __future__ import absolute_import, unicode_literals

from django.core.cache import cache as default_cache, caches
from kombu.utils.encoding import bytes_to_str
from celery.backends.base import KeyValueStoreBackend


class CacheBackend(KeyValueStoreBackend):
   """Backend using the Django cache framework to store task metadata."""

   def __init__(self, *args, **kwargs):
       super(CacheBackend, self).__init__(*args, **kwargs)

       # Must make sure backend doesn't convert exceptions to dict.
       self.serializer = 'pickle'

   def get(self, key):
       key = bytes_to_str(key)
       return self.cache_backend.get(key)

   def set(self, key, value):
       key = bytes_to_str(key)
       self.cache_backend.set(key, value, self.expires)

   def delete(self, key):
       key = bytes_to_str(key)
       self.cache_backend.delete(key)

   def encode(self, data):
       return data

   def decode(self, data):
       return data

   @property
   def cache_backend(self):
       backend = self.app.conf.cache_backend
       return caches[backend] if backend else default_cache

I will see if I can put this is a pull request, but no promises because I'm not familiar with the github pull mechanism