Decryption corrupted all files — serious file loss

[mmaedler]

Steps to reproduce

1. Enable server-side encryption via occ encryption encrypt:all
2. Disable server-side encryption via occ encryption decrypt:all
3. All files being opened trigger "Bad Signature error in log" and seem corrupted

Expected behaviour

Files should be decrypted and accessible

Actual behaviour

Files are corrupted and cannot be opened anymore. Due to that I have lost important files.

Server configuration

Operating system: Ubuntu 16.04 server

Web server: nginx

Database: mysql

PHP version: 7.0

Nextcloud version: 12.0.4

Updated from an older Nextcloud/ownCloud or fresh install: Updated

Where did you install Nextcloud from:

Signing status:

Signing status

No errors have been found.

List of activated apps:

App list

Enabled:
  - activity: 2.5.2
  - admin_audit: 1.2.0
  - admin_notifications: 1.0.1
  - announcementcenter: 3.1.1
  - bruteforcesettings: 1.0.3
  - calendar: 1.5.7
  - comments: 1.2.0
  - contacts: 2.0.1
  - dav: 1.3.0
  - encryption: 1.6.0
  - external: 2.0.3
  - federatedfilesharing: 1.2.0
  - federation: 1.2.0
  - files: 1.7.2
  - files_accesscontrol: 1.2.5
  - files_automatedtagging: 1.2.2
  - files_external: 1.3.0
  - files_pdfviewer: 1.1.1
  - files_sharing: 1.4.0
  - files_texteditor: 2.4.1
  - files_trashbin: 1.2.0
  - files_versions: 1.5.0
  - files_videoplayer: 1.1.0
  - firstrunwizard: 2.1
  - gallery: 17.0.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.0.0
  - nextcloud_announcements: 1.1
  - notifications: 2.0.0
  - oauth2: 1.0.5
  - password_policy: 1.2.2
  - provisioning_api: 1.2.0
  - quota_warning: 1.1.1
  - serverinfo: 1.2.0
  - sharebymail: 1.2.0
  - socialsharing_email: 1.0.3
  - survey_client: 1.0.0
  - systemtags: 1.2.0
  - theming: 1.3.0
  - twofactor_backupcodes: 1.1.1
  - updatenotification: 1.2.0
  - workflowengine: 1.2.0

Nextcloud configuration:

Config report

{
    "system": {
        "instanceid": "occ76c8edd49",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "\/var\/www\/nextcloud\/data",
        "dbtype": "mysql",
        "version": "12.0.4.3",
        "dbname": "owncloud",
        "dbhost": "127.0.0.1",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "loglevel": 3,
        "logtimezone": "Europe\/Berlin",
        "maintenance": false,
        "theme": "",
        "appstoreenabled": true,
        "appstoreurl": "https:\/\/apps.nextcloud.com\/api\/v0",
        "trusted_domains": [
            "oc.betaserv.net"
        ],
        "mail_smtpmode": "php",
        "mail_smtpsecure": "ssl",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "forcessl": true,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "\/run\/redis\/redis.sock",
            "port": 0,
            "dbindex": 0,
            "timeout": 1.5
        },
        "appstore.experimental.enabled": true,
        "trashbin_retention_obligation": "auto",
        "updater.release.channel": "stable",
        "mail_from_address": "Nextcloud",
        "mail_domain": "",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpauth": 1,
        "mail_smtphost": "",
        "mail_smtpport": "587",
        "preview-libreoffice-path": "\/lib\/libreoffice\/program\/soffice",
        "singleuser": true,
        "updatechecker": true,
        "updater.server.url": "https:\/\/updates.nextcloud.com\/updater_server\/",
        "token_auth_enforced": true,
        "overwrite.cli.url": "https:\/\/oc.betaserv.net"
    }
}

**Are you using encryption:** yes and no

[AlexCloudDev]

hi, same here :(

we ran our nc instance with enabled server-side encryption since owncloud 7 times and disabled it 3 days ago via
"occ encryption decrypt:all"

Currently we are on nc13, php7.
User data encryption is still enabled.

After that we get "OCP\Encryption\Exceptions\GenericEncryptionException: Bad Signature" errors at every file we try to open. Files are still there but inaccessible.
The "occ encryption decrypt:all" command finished without errors.

pls help
thx

log:

OCP\Encryption\Exceptions\GenericEncryptionException: Bad Signature
/var/www/html/apps/encryption/lib/Crypto/Crypt.php - line 465: OCA\Encryption\Crypto\Crypt->checkSignature('IbvrdqBYkdwoRjr...', '+dBH\x0E\xCA\xBD\xD4U\xC2\xAD>\x0E\xA7z...', 'c5578ba7708b5b6...')
/var/www/html/apps/encryption/lib/Crypto/Encryption.php - line 380: OCA\Encryption\Crypto\Crypt->symmetricDecryptFileContent('IbvrdqBYkdwoRjr...', '+dBH\x0E\xCA\xBD\xD4U\xC2\xAD>\x0E\xA7z...', 'AES-256-CTR', 0, 0)
/var/www/html/lib/private/Files/Stream/Encryption.php - line 464: OCA\Encryption\Crypto\Encryption->decrypt(*** sensitive parameters replaced )
/var/www/html/lib/private/Files/Stream/Encryption.php - line 295: OC\Files\Stream\Encryption->readCache()
[internal function] OC\Files\Stream\Encryption->stream_read(8192)
/var/www/html/3rdparty/icewind/streams/src/Wrapper.php - line 83: fread(Resource id #42, 8192)
/var/www/html/3rdparty/icewind/streams/src/CallbackWrapper.php - line 91: Icewind\Streams\Wrapper->stream_read(8192)
[internal function] Icewind\Streams\CallbackWrapper->stream_read(8192)
/var/www/html/lib/private/Files/View.php - line 425: fread(Resource id #45, 8192)
/var/www/html/lib/private/legacy/files.php - line 310: OC\Files\View->readfile('//ts3.txt')
/var/www/html/lib/private/legacy/files.php - line 122: OC_Files getSingleFile(Object(OC\Files\View), '/', 'ts3.txt', Array)
/var/www/html/apps/files/ajax/download.php - line 64: OC_Files get('/', 'ts3.txt', Array)
/var/www/html/lib/private/Route/Route.php - line 155: require_once('/var/www/html/a...')
[internal function] OC\Route\Route->OC\Route{closure}( sensitive parameters replaced ***)
/var/www/html/lib/private/Route/Router.php - line 297: call_user_func(Object(Closure), Array)
/var/www/html/lib/base.php - line 998: OC\Route\Router->match('/apps/files/aja...')
/var/www/html/index.php - line 37: OC handleRequest()
{main}

[mmaedler]

Hi —
is there any update regarding this topic? I really need my files back...
Thanks!

[tessus]

Has anyone found a solution? Is this being investigated?

Now and then I truly believe that priorities are totally screwed up with this project. Sometimes a proper icon placement seems more important than making a basic feature work.

[blizzz]

@schiessle

[FlorianFranzen]

I am facing the same issue on nextcloud 13.0.1.

The problem seems to arises when encryption:decrypt-all is run in maintenance mode, during which the encryption module is actually disabled.

As a result, you will not be asked for the recovery password before the decryption and none of the files will be decrypted, but marked as such in the database, resulting in aforementioned error.

To reproduce the issue I just tested this with a fresh docker-based installation:

1. Enable encryption.
2. Upload a new file
3. Go to maintenance mode
4. Run decrypt-all

Result: File no longer accessible with error above (after leaving maintenance mode).

To get access to the file again after the failed decryption it was enough to set the files encrypted column in the filecache table back to 1 (i.e. update oc_filecache set encrypted = 1 where fileid = <FILE_ID> if you have the file id).

@schiessle Is there a quick way to detect these files and to fix their DB entries without having to restore backups?

[mmaedler]

Interesting find @FlorianFranzen ! Is it possible to simple change the encrypted-column back to 1 for all files (since I guess none of them is accessible at the moment) in one go?

Thanks!

[FlorianFranzen]

Disclaimer: I am not an nextcloud developer and you should have backups of all your data and database before you start messing with nextcloud's internal structure.

@mmaedler Yes, you could. The problem is just, that not all entries in the file cache are files (but dirs, etc.), whose encrypted flag should probably not be set.

I am working at a fix at the moment that avoids working on the database directly. I failed to get any help from any actual developers, but given the projects bad reputation that is not surprising at all.

It seems that even if the file is not marked encrypted in file cache, the encryption stream wrapper is called somehow (probably based on file content) but then fails because the file cache marks the file as not encrypted.

[AlexCloudDev]

Is there a way to decrypt the files via "occ encryption decrypt:all" without entering the maintenance mode to avoid the problem?

Thx for your help guys :)

[FlorianFranzen]

@AlexCloudDev Yes, some quick test seems to suggest that it is save. decrypt-all actually enables maintenance mode during the decryption, so there is no need for it anyway.

[mmaedler]

I tried what you suggested earlier and when running occ decrypt-all I get

Server side encryption not enabled. Nothing to do.

I guess this is because I ran the decrypt-all command before (and then ended up with this mess of files). Shall I enable encryption again?

Thanks,

Moritz

[FlorianFranzen]

@mmaedler: Just enable encryption for a brief moment, decrypt-all will disable it again anyway. Enabling encryption does only mean, that all new files added will be encrypted.

[FlorianFranzen]

@schiessle: This is quite a serious bug, that can be easily replicate in a few steps in a fresh install. Care to take a look or to at least comment on this issue?

[FlorianFranzen]

@MorrisJobke @rullzer You seem to be some sort of nextcloud maintainer. I already tried to talk to people on IRC but nobody seems to care. Would anybody care to comment on this issue?