Apache giraph job must be assigned the number of workers(which is basically count of mappers). But with how much workers should we assign for best runtime of our job?
Well few tips I got from my friend semih, from standard university who also developed GPS, an alternative to apache giraph project.
Tips:
Each node should be configured to have mapper count same as number of processor cores. So if you have 2 core in the processor of your hadoop node than set mapred.map.tasks=2.
Each job must be assigned workers in the multiple of hadoop nodes. Let say you have 8 node cluster than you must assign "-w" value as 7 or 15 or 23 etc.. Note: worker count start from zero(0).
Again thanks to semih, phD scholar from Stanford University for these tips.
No comments:
Post a Comment