[Bioperl-l] out of memory issue
David.Messina at sbc.su.se
Wed Apr 28 11:23:56 EDT 2010
The problem is quite clear, that you're running out of memory, so you need to find out why.
A lot depends on how big your IDs are and the associated values you're storing. And Perl also has some overhead of its own that might taking a lot more space than expected.
Try installing the Devel::Size module and using its total_size function.
Look at how many bytes a hash of, say 10,000 or 100,000 of your records instead of all 38 million.
use Devel::Size qw(total_size);
print "Empty: ", total_size (\%hash), " bytes\n";
# fill %hash here
print "Full: ", total_size (\%hash), " bytes\n";
More information about the Bioperl-l