run YSB benchmark with 32 threads

[chenzongxiong]

Hi,
When I try to run YSB with 32 threads, there is no throughput on console.

./run.sh uk.ac.imperial.lsds.saber.experiments.benchmarks.yahoo.YahooBenchmarkApp 32

From the log, I think it's due to the buffer not released. However, I didn't find out the root cause.

One more thing I need to mention is that, this APP run well with 1 to 8 threads.

[grtheod]

Hi, Unfortunately, I am away in an internship and I don't have much time to look at the code right now. The first thing that you have to keep in mind is that SABER is not NUMA-aware (we are working on it in another project). So when you cross a socket the performance will drop dramatically, especially in queries such as YahooBenchmark where you spend most of your time on transferring of data between operators (it's not computational intensive). Still, this doesn't reply to your question. The problem lies in the merge phase. In the current version, there is a race condition when accessing the atomics in the ResultCollector/ResultHandler and you are pipelining operators (as happens here). The proper solution would be to create individual merge tasks (this is not how it works now) and add them in the GlobalTaskQueue, to ensure that they are going to be executed. This is something I am working on and I haven't finished yet. I am sorry I didn't provide a solution for now, but I can keep you updated with the progress of our new project if you are interested. Cheers, George Στις Πέμ, 12 Σεπ 2019 στις 12:41 μ.μ., ο/η zxchen <notifications@github.com> έγραψε:

…

Hi, When I try to run YSB with 32 threads, there is no throughput on console. ./run.sh uk.ac.imperial.lsds.saber.experiments.benchmarks.yahoo.YahooBenchmarkApp 32 [image: image] <https://user-images.githubusercontent.com/8354165/64777109-caa30e00-d559-11e9-83cd-908eb805aff9.png> [image: image] <https://user-images.githubusercontent.com/8354165/64777389-59178f80-d55a-11e9-9f5b-7517e1b4191c.png> From the log, I think it's due to the buffer not released. However, I didn't find out the root cause. One more thing I need to mention is that, this APP run well with 1 to 8 threads. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3?email_source=notifications&email_token=ADTBN2HJGKMVC2WJHKEQDPDQJIMEXA5CNFSM4IWCFTT2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HK6NCXA>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADTBN2ECNTVKYM5TE3EOPCTQJIMEXANCNFSM4IWCFTTQ> .