Download

To download, run "git clone http://www.sbg.bio.ic.ac.uk/~sm2708/software/git/perl-filequeue.git/".

Name

FileQueue.pm -- Simple file queue for multiple processes.

Synopsis

    my $queue = IPC::FileQueue->new(filename => "queue");
    my $queue = IPC::FileQueue->new(filename => "queue", line_size => 10);

Description

An IPC::FileQueue object represents a file on disk that contains one queue item per line. The file is locked when being read from or written to, and so it is safe for multiple processes to use one queue simultaneously.

This module is intended as a simple queue for use with in-house compute clusters, where it is desirable to have many processes processing separate items. If the length of time required to process each item is not known, then simply splitting the items into predfined chunks for each process may be inefficient. This module allows processes to grab items as needed.

This queue is fairly fast, as it make use of the append filemode and the "truncate" in perlfunc function, meaning that the file does not need to read into memory or even iterated through. Tests indicate that this module is significantly faster than both Tie::File and naive methods.

Attributes

IPC::FileQueue accepts two parameters to its constructor:

filename (Mandatory): Name of the file in which items will be stored.
line_size (Default = 10): Approximate number of characters in a line. The queue will work even if this is wildly inaccurate, but it may be slower than necessary.

Methods