Well, perlbrew accept the "-j" option.
Ex:
perlbrew install -j 5 perl-5.12.3
It will pass to "make" the "-j" option. Use your cpu number + 1 to have an optimised compilation. Done
View the discussion thread.