Random file corruption (possibly related to shell prompt integration)

[ulope]

I've integrated TW into my shell prompt. The commands that are run for each prompt are the following:

• task _get rc.context
• task +READY count
• task +READY +DUE count
• task +READY due.before:today count

Since about two weeks ago or so I get random TW data file corruptions every few days.
For example:

Unrecognized Taskwarrior file format or blank line in data. in /Users/ulo/.task/pending.data at line 1

The only significant change has been that I upgraded to macOS 10.14.

My wild and totally unsubstantiated guess is that those commands are run concurrently and somehow don't properly lock files before modifying them.

[ulope]

I just realized that this is very likely a duplicate of #1735

[pawamoy]

Got this as well.

$ task 111 start
Starting task 111 'Copy kanban task into taskwarrior'.
Started 1 task.
$ task add Wash costumes and put them back in a box
Unrecognized taskwarrior file format. in /home/pawamoy/.task/pending.data at line 11

Now all my files are f***** up 🙁

I am myself using task in conky, so I wonder if concurrent accesses to the files could be the source of the problem. My conky scripts do not write tasks, only read them (print reports).

This is very annoying. Thankfully I had just a few tasks, and bugwarrior imported ones (that can be reimported easily). But now I'm not sure I want to continue using taskwarrior if I get random file corruptions every once and a while...

[pawamoy]

Linking possibly related issues: #1785, #1420.

Workaround from comments: version-control your task data, with a git commit on the on-exit hook. If files get corrupted, reverse to previous commit to fix them. Too late for me though 🙁

[pawamoy]

I eventually used another workaround with flock.

In conky:

${execpi 60 flock /home/me/.task task limit:10 list}

In bash:

alias task='flock ~/.task task'

[pbeckingham]

@pawamoy Good workaround.

Simultaneous writes cause problems. Current locking mechanism isn't adequate, but we're not sure why.

[tbabej]

There is some ongoing work to improve robustness in GothenburgBitFactory/libshared#64

[djmitche]

This is not an issue with 3.0, as it uses SQLite, which is robust to concurrent access.