File::Map(3) User Contributed Perl Documentation File::Map(3)NAMEFile::Map - Memory mapping made simple and safe.
VERSION
Version 0.31
SYNOPSIS
use File::Map 'map_file';
map_file my $map, $filename;
if ($map ne "foobar") {
$map =~ s/bar/quz/g;
substr $map, 1024, 11, "Hello world";
}
DESCRIPTIONFile::Map maps files or anonymous memory into perl variables.
Advantages of memory mapping
· Unlike normal perl variables, mapped memory is shared between
threads or forked processes.
· It is an efficient way to slurp an entire file. Unlike for example
File::Slurp, this module returns almost immediately, loading the
pages lazily on access. This means you only 'pay' for the parts of
the file you actually use.
· Perl usually doesn't return memory to the system while running,
mapped memory can be returned.
Advantages of this module over other similar modules
· Safety and Speed
This module is safe yet fast. Alternatives are either fast but can
cause segfaults or loose the mapping when not used correctly, or
are safe but rather slow. File::Map is as fast as a normal string
yet safe.
· Simplicity
It offers a simple interface targeted at common usage patterns
· Files are mapped into a variable that can be read just like any
other variable, and it can be written to using standard Perl
techniques such as regexps and "substr".
· Files can be mapped using a set of simple functions. There is
no need to know weird constants or the order of 6 arguments.
· It will automatically unmap the file when the scalar gets
destroyed. This works correctly even in multi-threaded
programs.
· Portability
File::Map supports Unix, Windows and VMS.
· Thread synchronization
It has built-in support for thread synchronization.
FUNCTIONS
Mapping
The following functions for mapping a variable are available for
exportation.
· map_handle $lvalue, $filehandle, $mode = '<', $offset = 0, $length
= -s(*handle) - $offset
Use a filehandle to map into an lvalue. $filehandle should be a
scalar filehandle. $mode uses the same format as "open" does (it
currently accepts "<", "+<", ">" and "+>"). $offset and $length are
byte positions in the file, and default to mapping the whole file.
· map_file $lvalue, $filename, $mode = '<', $offset = 0, $length =
-s($filename) - $offset
Open a file and map it into an lvalue. Other than $filename, all
arguments work as in map_handle.
· map_anonymous $lvalue, $length
Map an anonymous piece of memory.
· sys_map $lvalue, $length, $protection, $flags, $filehandle, $offset
= 0
Low level map operation. It accepts the same constants as mmap does
(except its first argument obviously). If you don't know how mmap
works you probably shouldn't be using this.
· unmap $lvalue
Unmap a variable. Note that normally this is not necessary as
variables are unmapped automatically at destruction, but it is
included for completeness.
· remap $lvalue, $new_size
Try to remap $lvalue to a new size. It may fail if there is not
sufficient space to expand a mapping at its current location. This
call is linux specific and not supported on other systems.
Auxiliary
· sync $lvalue, $synchronous = 1
Flush changes made to the memory map back to disk. Mappings are
always flushed when unmapped, so this is usually not necessary. If
$synchronous is true and your operating system supports it, the
flushing will be done synchronously.
· pin $lvalue
Disable paging for this map, thus locking it in physical memory.
Depending on your operating system there may be limits on pinning.
· unpin $lvalue
Unlock the map from physical memory.
· advise $lvalue, $advice
Advise a certain memory usage pattern. This is not implemented on
all operating systems, and may be a no-op. The following values for
$advice are always accepted:.
· normal
Specifies that the application has no advice to give on its
behavior with respect to the mapped variable. It is the default
characteristic if no advice is given.
· random
Specifies that the application expects to access the mapped
variable in a random order.
· sequential
Specifies that the application expects to access the mapped
variable sequentially from start to end.
· willneed
Specifies that the application expects to access the mapped
variable in the near future.
· dontneed
Specifies that the application expects that it will not access
the mapped variable in the near future.
On some systems there may be more values available, but this can
not be relied on. Unknown values for $advice will cause a warning
but are further ignored.
· protect $lvalue, $mode
Change the memory protection of the mapping. $mode takes the same
format as, but also accepts sys_map style constants.
Locking
These locking functions provide locking for threads for the mapped
region. The mapped region has an internal lock and condition variable.
The condition variable functions("wait_until", "notify", "broadcast")
can only be used inside a locked block. If your perl has been compiled
without thread support the condition functions will not be available.
· lock_map $lvalue
Lock $lvalue until the end of the scope. If your perl does not
support threads, this will be a no-op.
· wait_until { block } $lvalue
Wait for block to become true. After every failed attempt, wait for
a signal. It returns the value returned by the block.
· notify $lvalue
This will signal to one listener that the map is available.
· broadcast $lvalue
This will signal to all listeners that the map is available.
CONSTANTS
PROT_NONE, PROT_READ, PROT_WRITE, PROT_EXEC, MAP_ANONYMOUS, MAP_SHARED,
MAP_PRIVATE, MAP_ANON, MAP_FILE
These constants are used for sys_map. If you think you need them
your mmap manpage will explain them, but in most cases you can skip
sys_map altogether.
EXPORTS
All previously mentioned functions are available for exportation, but
none are exported by default. Some functions may not be available on
your OS or your version of perl as specified above. A number of tags
are defined to make importation easier.
· :map
map_handle, map_file, map_anonymous, sys_map, unmap
· :extra
remap, sync, pin, unpin, advise, protect
· :lock
lock_map, wait_until, notify, broadcast
· :constants
PROT_NONE, PROT_READ, PROT_WRITE, PROT_EXEC, MAP_ANONYMOUS,
MAP_SHARED, MAP_PRIVATE, MAP_ANON, MAP_FILE
· :all
All functions defined in this module.
DIAGNOSTICS
In this overview %f is the name of the function that produced the
error, and %e is some error from your OS.
Exceptions
· Could not %f: this variable is not memory mapped
An attempt was made to "sync", "remap", "unmap", "pin", "unpin",
"advise" or "lock_map" an unmapped variable.
· Could not %f: %e
Your OS didn't allow File::Map to do what you asked it to do for
the reason specified in %e.
· Trying to %f on an unlocked map
You tried to "wait_until", "notify" or "broadcast" on an unlocked
variable.
· Zero length not allowed for anonymous map
A zero length anonymous map is not possible (or in any way useful).
· Can't remap a shared mapping
An attempts was made to remap a mapping that is shared among
different threads, this is not possible.
· Window ($start, $end) is outside the file
The offset and/or length you specified were invalid for this file.
Warnings
· Writing directly to a memory mapped file is not recommended
Due to the way perl works internally, it's not possible to write a
mapping implementation that allows direct assignment yet performs
well. As a compromise, File::Map is capable of fixing up the mess
if you do it nonetheless, but it will warn you that you're doing
something you shouldn't. This warning is only given when "use
warnings 'substr'" is in effect.
· Truncating new value to size of the memory map
This warning is additional to the previous one, warning you that
you're losing data. This warning is only given when "use warnings
'substr'" is in effect.
· Shouldn't mmap non-binary filehandle: layer '%s' is not binary
You tried to to map a filehandle that has some encoding layer.
Encoding layers are not supported by File::Map. This warning is
only given when "use warnings 'layer'" is in effect.
· Unknown advice '%s'
You gave advise an advice it didn't know. This is probably either a
typo or a portability issue. This warning is only given when "use
warnings 'portable'" is in effect.
· Syncing a readonly map makes no sense
"sync" flushes changes to the map to the filesystem. This obviously
is of little use when you can't change the map. This warning is
only given when "use warnings 'io'" is in effect.
· Can't overwrite an empty map
Overwriting an empty map is rather nonsensical, hence a warning is
given when this is tried. This warning is only given when "use
warnings 'substr'" is in effect.
DEPENDENCIES
This module depends on perl 5.8 and Const::Fast.
PITFALLS
On perl versions before 5.11.5 many string functions including "substr"
are limited to 32bit logic
<http://rt.perl.org/rt3//Public/Bug/Display.html?id=72784>, even on
64bit architectures. Effectively this means you can't use them on
strings bigger than 2GB. If you are working with such large files, I
strongly recommend upgrading to 5.12.
This module assumes the file is binary data and doesn't do any encoding
or newline transformation for you. Most importantly this means that:
· On Windows, you have to remember to make your filehandles binary
before passing them to "map_handle" or "sys_map". It may warn on a
filehandle doing "crlf" transformation as it would return a different
value than reading it normally would. This can be remedied by using
open modes or binmode, see PerlIO for more information.
· You can make it a unicode string in-place by using utf8::decode if
it's valid utf-8, but writing to it requires you to really know what
you're doing. Currently "utf8" filehandles are not dealt with
gracefully, though in a future version that may change.
You probably don't want to use ">" as a mode. This does not give you
reading permissions on many architectures, resulting in segmentation
faults when trying to read a variable (confusingly, it will work on
some others like x86).
BUGS AND LIMITATIONS
There is an off-by-one bug in Perl's regexp engine, as explained here
<http://rt.perl.org/rt3//Public/Bug/Display.html?id=73542>. If the
length of the file is an exact multiple of the page size, some regexps
can trigger a segmentation fault. This can not be fixed in this module
though.
As any piece of software, bugs are likely to exist here. Bug reports
are welcome.
Please report any bugs or feature requests to "bug-file-map at
rt.cpan.org", or through the web interface at
http://rt.cpan.org/NoAuth/ReportBug.html?Queue=File-Map
<http://rt.cpan.org/NoAuth/ReportBug.html?Queue=File-Map>. I will be
notified, and then you'll automatically be notified of progress on your
bug as I make changes.
SEE ALSO
· Sys::Mmap, the original Perl mmap module
· mmap(2), your mmap man page
· Win32::MMF
· CreateFileMapping at MSDN:
http://msdn.microsoft.com/en-us/library/aa366537(VS.85).aspx
<http://msdn.microsoft.com/en-us/library/aa366537(VS.85).aspx>
AUTHOR
Leon Timmermans, "<leont at cpan.org>"
SUPPORT
You can find documentation for this module with the perldoc command.
perldoc File::Map
You can also look for information at:
· RT: CPAN's request tracker
http://rt.cpan.org/NoAuth/Bugs.html?Dist=File-Map
<http://rt.cpan.org/NoAuth/Bugs.html?Dist=File-Map>
· AnnoCPAN: Annotated CPAN documentation
http://annocpan.org/dist/File-Map <http://annocpan.org/dist/File-
Map>
· CPAN Ratings
http://cpanratings.perl.org/d/File-Map
<http://cpanratings.perl.org/d/File-Map>
· Search CPAN
http://search.cpan.org/dist/File-Map
<http://search.cpan.org/dist/File-Map>
COPYRIGHT AND LICENSE
Copyright 2008, 2009, 2010 Leon Timmermans, all rights reserved.
This program is free software; you can redistribute it and/or modify it
under the same terms as perl itself.
perl v5.14.1 2011-07-19 File::Map(3)
