[Bioperl-l] perl memory problem
jason.stajich at duke.edu
Wed Oct 26 13:29:42 EDT 2005
On Oct 26, 2005, at 11:55 AM, Zaigang Liu wrote:
> Hello, everyone,
> I am new to this list and I am looking for help for one problem I met.
> I am trying to run a perl program doing calculation on large data
> set. It
> uses an array to store the input data and the array size will reach 8G
> after all the data is loaded by my estimation. The process always
> stops and
> gives a "out of memory" error when the memory usage reached about
> 2G. I
> used a solaris machine with 40G memory so it is not really run out of
> memory when the program stopped.
there is probably a limit on how much memory perl can address in the
version you have compiled.
Have you considered using a data structure which does not require the
entire array to be in memory at once. the DB_File module is very
useful for this as you can tie your array to a flatfile Berkeley DB.
You can also use something like SQLite or a full fledged relational
database instead if you are crunching such a large number of items.
> Does anyone know where could this program come from and how to fix it.
> Thanks in advance,
> This communication is for use by the intended recipient and contains
> information that may be Privileged, confidential or copyrighted under
> applicable law. If you are not the intended recipient, you are hereby
> formally notified that any use, copying or distribution of this e-
> in whole or in part, is strictly prohibited. Please notify the
> sender by
> return e-mail and delete this e-mail from your system. Unless
> and conspicuously designated as "E-Contract Intended", this e-mail
> not constitute a contract offer, a contract amendment, or an
> of a contract offer. This e-mail does not constitute a consent to the
> use of sender's contact information for direct marketing purposes
> or for
> transfers of data to third parties.
> Francais Deutsch Italiano Espanol Portugues Japanese Chinese
> Bioperl-l mailing list
> Bioperl-l at portal.open-bio.org
More information about the Bioperl-l