Enable use of Parallel TS of c++17

[breznak]

After #55
https://www.phoronix.com/scan.php?page=news_item&px=Intel-Parallel-STL-libstdc-libc

Paralel TS, as defined in c++17, should soon be available in GCC, Clang (supported in MSVC already 👍 )
I'm very interested to test its performance.

• it allows code from to be run in parallel, vectorized imlicitely.

---

EDIT:

• requires C++17 Switch to c++17 #55 and transform from <algorithms>
• blocked by need of more widespread support for C++17 TS Parallel
  • currently only MSVC; stdlibc++ (gcc), and libc++ (clang) in progress
    https://www.phoronix.com/scan.php?page=news_item&px=Intel-Parallel-STL-libstdc-libc

PROs:

• easy to implement
• usable in all code (allies to small sniplet, loop)
• seamless

CONs:

• gains may not be that big
• blocked by lacking compiler/libstdc++ support, ETA 2019 earliest?

---

I will experiment with this, some progress outline, @breznak
TODO:

• obtain gcc-9 for ubuntu -- from PPA
  • codebase compiles with g++-9
• decided for c++17 standard Parallel TS, currently requires g++-9 & TBB
  • install TBB >= 2018.6 from package pinning from Eoan
  • if rolled for generic use, build as a dependency from cmake: https://github.com/intel/tbb
• demo code on sorting
• SP optimizations
• header that sorts support if possible, same as boost-vs-cpp17
  • design some control over used thread counts based on num CPU cores
• test parallel execution on CI

[ctrl-z-9000-times]

I looked into this and it look pretty neat! It's always nice to see more ways to use the extra CPU cores that come with most computers now a days.

Some things to consider:

• Parallel code is difficult to get right. Although this approach does look really simple...
• The bottlenecks in this program might be memory accesses, in which case this won't help very much. Need profiling data before doing this.

Naively, I think we could get the most bang/buck by parallelizing the methods:

• Connections::computeActivity and NOT using mutex locks. This method is fault tolerant and does not need to be exact.
• Connections::adaptSegment which will need mutex lock on conenctions object only when a synapse changes its connected/potential state, so that it can modify the presynaptic-maps. This method would not be parallel, but rather would be thread safe and assume that the caller is parallel. Alternatively if this method were parallel it might be safe to eschew the mutex locks, but I think this alternative would have less performance gain.

[breznak]

Parallel code is difficult to get right. Although this approach does look really simple...

right, that should be the advantage of this approach, c++ would parallelize select (for-loops) for you implicitly.

I see 3 different levels of parallelisation that could be implemented:

• low-level: , for-loops, implicit, by c++17 Parallel
  • this reduces (if??) execution time of each,smallest code-unit
• hand-coded: efficient, makes code much more difficult
• high-level: NetworkAPI Regions, classes SP,TM,Encoder,Classifier in a pipeline could run in parallel
  • reduces pipeline time to the min(time of each unit)

Connections::computeActivity and NOT using mutex locks. This method is fault tolerant and does not need to be exact.

these are valuable hints, thank you!

Another thing I'd be interested is making ALL the code run in parallel (each column, ideally), this would be interesting from biological POV as the algos would be async, from implementation/future POV that 64+ threaded CPUs are becoming more and more common,

• in implementation, this would be exploring atomic keyword and more atomic data structures, FB has a great library for this: https://github.com/facebook/folly/tree/master/folly
  AtomicBitSet is something I have in mind for SDR internal data structure

[breznak]

FYI, c++17 TS:Parallel in gcc-9 is now reaching availability, I've installed for Ubuntu from a PPA, our codebase compiles with g++9 #523
When I get some time I'd like to experiment with this approach more.