[Bioperl-l] dealing with large files
cjfields at uiuc.edu
Thu Dec 20 15:27:45 EST 2007
On Dec 20, 2007, at 11:06 AM, Sendu Bala wrote:
> Chris Fields wrote:
>> The only way I can think of to fix this would be (as Jason also
>> suggested) lightweight objects, or something like the lazy sequence
>> object ala the SwissKnife suite (which only bring what you want
>> into memory).
>> Related to that, I have been testing something like that, which
>> uses iterators to pass in chunks of data from a stream to handlers
>> to build a sequence object. Wouldn't be too hard to reconfigure
>> that to return file positions as well. Maybe for the 1.7 release...
> Bio::PullParserI is your friend.
I'm looking into that, yes. I'm thinking of something like a generic
lazy sequence class with an embedded Handler/PullParser object which
processes stuff on the fly.
Oh, when I have a bit more time...
More information about the Bioperl-l