MogileFS::Client(3) User Contributed Perl Documentation MogileFS::Client(3)NAMEMogileFS::Client - Client library for the MogileFS distributed file
system.
SYNOPSIS
use MogileFS::Client;
# create client object w/ server-configured namespace
# and IPs of trackers
$mogc = MogileFS::Client->new(domain => "foo.com::my_namespace",
hosts => ['10.0.0.2:7001', '10.0.0.3:7001']);
# create a file
# mogile is a flat namespace. no paths.
$key = "image_of_userid:$userid";
# must be configured on server
$class = "user_images";
$fh = $mogc->new_file($key, $class);
print $fh $data;
unless ($fh->close) {
die "Error writing file: " . $mogc->errcode . ": " . $mogc->errstr;
}
# Find the URLs that the file was replicated to.
# May change over time.
@urls = $mogc->get_paths($key);
# no longer want it?
$mogc->delete($key);
DESCRIPTION
This module is a client library for the MogileFS distributed file
system. The class method 'new' creates a client object against a
particular mogilefs tracker and domain. This object may then be used to
store and retrieve content easily from MogileFS.
METHODS
new
$client = MogileFS::Client->new( %OPTIONS );
Creates a new MogileFS::Client object.
Returns MogileFS::Client object on success, or dies on failure.
OPTIONS:
hosts
Arrayref of 'host:port' strings to connect to as backend trackers
in this client.
domain
String representing the mogile domain which this MogileFS client is
associated with. (All create/delete/fetch operations will be
performed against this mogile domain). See the mogadm shell command
and its 'domain' category of operations for information on
manipulating the list of possible domains on a MogileFS system.
reload
$mogc->reload( %OPTIONS )
Re-init the object, like you'd just reconstructed it with 'new', but
change it in-place instead. Useful if you have a system which reloads
a config file, and you want to update a singleton $mogc handle's config
value.
last_tracker
Returns a scalar of form "ip:port", representing the last mogilefsd
'tracker' server which was talked to.
errstr
Returns string representation of the last error that occurred. It
includes the error code (same as method 'errcode') and a space before
the optional English error message.
This isn't necessarily guaranteed to reset after a successful
operation. Only call it after another operation returns an error.
errcode
Returns an error code. Not a number, but a string identifier (e.g.
"no_domain") which is stable for use in error handling logic.
This isn't necessarily guaranteed to reset after a successful
operation. Only call it after another operation returns an error.
readonly
$is_readonly = $mogc->readonly
$mogc->readonly(1)
Getter/setter to mark this client object as read-only. Purely a local
operation/restriction, doesn't do a network operation to the mogilefsd
server.
new_file
$mogc->new_file($key)
$mogc->new_file($key, $class)
$mogc->new_file($key, $class, $content_length)
$mogc->new_file($key, $class, $content_length , $opts_hashref)
Start creating a new filehandle with the given key, and option given
class and options.
Returns a filehandle you should then print to, and later close to
complete the operation. NOTE: check the return value from close! If
your close didn't succeed, the file didn't get saved!
$opts_hashref can contain keys:
fid Explicitly specify the fid number to use, rather than it being
automatically allocated.
create_open_args
Hashref of extra key/value pairs to send to mogilefsd in
create_open phase.
create_close_args
Hashref of extra key/value pairs to send to mogilefsd in
create_close phase.
largefile
Use MogileFS::ClientHTTPFile which will not load the entire file
into memory like the default MogileFS::NewHTTPFile but requires
that the storage node HTTP servers support the Content-Range header
in PUT requests and is a little slower.
edit_file
$mogc->edit_file($key, $opts_hashref)
Edit the file with the the given key.
NOTE: edit_file is currently EXPERIMENTAL and not recommended for
production use. MogileFS is primarily designed for storing files for
later retrieval, rather than editing. Use of this function may lead to
poor performance and, until it has been proven mature, should be
considered to also potentially cause data loss.
NOTE: use of this function requires support for the DAV 'MOVE' verb and
partial PUT (i.e. Content-Range in PUT) on the back-end storage servers
(e.g. apache with mod_dav).
Returns a seekable filehandle you can read/write to. Calling this
function may invalidate some or all URLs you currently have for this
key, so you should call ->get_paths again afterwards if you need them.
On close of the filehandle, the new file contents will replace the
previous contents (and again invalidate any existing URLs).
By default, the file contents are preserved on open, but you may
specify the overwrite option to zero the file first. The seek position
is at the beginning of the file, but you may seek to the end to append.
$opts_hashref can contain keys:
overwrite
The edit will overwrite the file, equivalent to opening with '>'.
Default: false.
read_file
$mogc->read_file($key)
Read the file with the the given key.
Returns a seekable filehandle you can read() from. Note that you cannot
read line by line using <$fh> notation.
Takes the same options as get_paths (which is called internally to get
the URIs to read from).
store_file
$mogc->store_file($key, $class, $fh_or_filename[, $opts_hashref])
Wrapper around new_file, print, and close.
Given a key, class, and a filehandle or filename, stores the file
contents in MogileFS. Returns the number of bytes stored on success,
undef on failure.
$opts_hashref can contain keys for new_file, and also the following:
chunk_size
Number of bytes to read and write and write at once out of the
larger file. Defaults to 8192 bytes. Increasing this can increase
performance at the cost of more memory used while uploading the
file. Note that this mostly helps when using largefile => 1
store_content
$mogc->store_content($key, $class, $content[, $opts]);
Wrapper around new_file, print, and close. Given a key, class, and
file contents (scalar or scalarref), stores the file contents in
MogileFS. Returns the number of bytes stored on success, undef on
failure.
get_paths
@paths = $mogc->get_paths($key)
@paths = $mogc->get_paths($key, $no_verify_bool); # old way
@paths = $mogc->get_paths($key, { noverify => $bool }); # new way
Given a key, returns an array of all the locations (HTTP URLs) that the
file has been replicated to.
noverify
If the "no verify" option is set, the mogilefsd tracker doesn't
verify that the first item returned in the list is up/alive.
Skipping that check is faster, so use "noverify" if your
application can do it faster/smarter. For instance, when giving
Perlbal a list of URLs to reproxy to, Perlbal can intelligently
find one that's alive, so use noverify and get out of mod_perl or
whatever as soon as possible.
zone
If the zone option is set to 'alt', the mogilefsd tracker will use
the alternative IP for each host if available, while constructing
the paths.
pathcount
If the pathcount option is set to a positive integer greater than
2, the mogilefsd tracker will attempt to return that many different
paths (if available) to the same file. If not present or out of
range, this value defaults to 2.
get_file_data
$dataref = $mogc->get_file_data($key)
Wrapper around get_paths & LWP, which returns scalarref of file
contents in a scalarref.
Don't use for large data, as it all comes back to you in one string.
delete
$mogc->delete($key);
Delete a key from MogileFS.
rename
$mogc->rename($oldkey, $newkey);
Rename file (key) in MogileFS from oldkey to newkey. Returns true on
success, failure otherwise.
file_debug
my $info_gob = $mogc->file_debug(fid => $fid);
... or ...
my $info_gob = $mogc->file_debug(key => $key);
Thoroughly search for any database notes about a particular fid.
Searchable by raw fidid, or by domain and key. Use sparingly. Command
hits the master database numerous times, and if you're using it in
production something is likely very wrong.
To be used with troubleshooting broken/odd files and errors from
mogilefsd.
file_info
my $fid = $mogc->file_info($key, { devices => 0 });
Used to return metadata about a file. Returns the domain, class,
expected length, devcount, etc. Optionally device ids (not paths) can
be returned as well.
Should be used for informational purposes, and not usually for
dynamically serving files.
list_keys
$keys = $mogc->list_keys($prefix, $after[, $limit]);
($after, $keys) = $mogc->list_keys($prefix, $after[, $limit]);
Used to get a list of keys matching a certain prefix.
$prefix specifies what you want to get a list of. $after is the item
specified as a return value from this function last time you called it.
$limit is optional and defaults to 1000 keys returned.
In list context, returns ($after, $keys). In scalar context, returns
arrayref of keys. The value $after is to be used as $after when you
call this function again.
When there are no more keys in the list, you will get back undef or an
empty list.
foreach_key
$mogc->foreach_key( %OPTIONS, sub { my $key = shift; ... } );
$mogc->foreach_key( prefix => "foo:", sub { my $key = shift; ... } );
Functional interface/wrapper around list_keys.
Given some %OPTIONS (currently only one, "prefix"), calls your callback
for each key matching the provided prefix.
update_class
$mogc->update_class($key, $newclass);
Update the replication class of a pre-existing file, causing the file
to become more or less replicated.
set_pref_ip
$mogc->set_pref_ip({ "10.0.0.2" => "10.2.0.2" });
Weird option for old, weird network architecture. Sets a mapping table
of preferred alternate IPs, if reachable. For instance, if trying to
connect to 10.0.0.2 in the above example, the module would instead try
to connect to 10.2.0.2 quickly first, then then fall back to 10.0.0.2
if 10.2.0.2 wasn't reachable.
PLUGIN METHODS
WRITEME
HOOKS
add_hook
WRITEME
add_backend_hook
WRITEME
SEE ALSO
<http://www.danga.com/mogilefs/>
COPYRIGHT
This module is Copyright 2003-2004 Brad Fitzpatrick, and copyright
2005-2007 Six Apart, Ltd.
All rights reserved.
You may distribute under the terms of either the GNU General Public
License or the Artistic License, as specified in the Perl README file.
WARRANTY
This is free software. IT COMES WITHOUT WARRANTY OF ANY KIND.
AUTHORS
Brad Fitzpatrick <brad@danga.com>
Brad Whitaker <whitaker@danga.com>
Mark Smith <marksmith@danga.com>
perl v5.14.1 2011-01-08 MogileFS::Client(3)
