LONDISTE3(1)LONDISTE3(1)NAMElondiste3 - tool for managing trigger-based replication for PostgreSQL.
SYNOPSISlondiste3 <config.ini> command [subcommand] [options]
DESCRIPTION
Londiste allows you to setup and administer the replication, and is run
as a daemon to handle the replication itself. (Londiste is just a
complex PgQ Consumer).
See EXAMPLES below to start your first trigger-based replication in a
few steps.
The command is one of Node, Replication, Information or Internal
commands listed below.
Londiste introduces the notion of takeover. It is the action when a
local node takes over the work of another node. The other node can be a
root node or a branch node and it can be dead or alive when the action
is run.
Londiste also allows (among many other features): cascading
replication, partial replication and custom handlers for replication.
GENERAL OPTIONS-V, --version
Print version info and exit.
-h, --help
Show this help message and exit.
-q, --quiet
Log only errors and warnings.
-v, --verbose
Log verbosely.
-d, --daemon
Run in daemon mode (go background).
SPECIFIC OPTIONS--ini
Display sample ini file.
--set="param=val[,param=value]"
Override config setting.
DAEMON OPTIONS-r, --reload
Reload config (send SIGHUP).
-s, --stop
Stop program safely (send SIGINT).
-k, --kill
Kill program immediately (send SIGTERM).
REPLICATION EXTRA ARGUMENTS--rewind
Change queue position according to destination.
--reset
Reset queue position on destination side.
NODE INITIALIZATION COMMANDS
Initialization commands will set up "public connect string" for current
node. It is a connect string that other nodes will use to connect to
current node. The local Londiste itself uses db option from config file
to connect to local node, which can have different user rights than
scripts coming over public connect string.
Connect strings can be set in either command line or config file.
Command line overrides config. Setting them up in config might be more
comfortable. See londiste3--ini for details.
create-root <node> [<public_connstr_for_node>]
Initializes a Master node.
The <node> is the name of the node, it should be unique.
create-branch <node> [<public_connstr_for_node>]
[--provider=<connstr_to_provider>]
Initializes a Slave node which can be used as a reference for other
nodes.
The <node> is the name of the node, it should be unique. The
<public_connstr_for_node> argument is the connection string to the
database on the current node and <connstr_to_provider> is the
connection string to the provider database (it can be a root node or a
branch node).
create-leaf <node> [<public_connstr_for_node>]
[--provider=<connstr_to_provider>]
Initializes a slave node which can not be used as a reference for other
nodes.
The <node> is the name of the node, it should be unique. The
<public_connstr_for_node> argument is the connection string to the
database on the current node and <connstr_to_provider> is the
connection string to the provider database (it can be a root node or a
branch node).
--merge=qname
combined queue name
NODE ADMINISTRATION COMMANDS
pause
Pause the consumer: the replication of the events is stopped and can be
resumed later.
resume
When the consumer has been paused, let it replay again.
change-provider <tonode>
Make <tonode> become the new provider for the current node.
TODO: londiste.py need update (param change from --provider)
takeover <fromnode> [--target=<tonode>] [--all] [--dead]
This command allows to achieve failover and switchover for any of your
providers (root or branch nodes).
--target=tonode
Target node of the takeover.
--all
In addition to take over the work from the fromnode, make other
nodes change their provider to the current node.
--dead
Don’t wait to take the new role and flag the fromnode as dead.
--dead=deadnode
Assume node is dead. TODO : why use this one ?
--dead-root
Old node was root.
--dead-branch
Old node was branch
resurrect
In case root was down and taken over with --dead-root, this command
fixes queue contents on that old root to match the rest of cascade.
Events that did not propagate to rest of the cascade before failure
(lost events) are dumped into file in JSON format and deleted from
queue. Then the node is registered into cascade and it’s worker will be
paused.
It requires that there is another active root in cascade and there is
consumer named NODENAME.gravestone registered on same node, it uses
that to get position where rest of the cascade moved on.
It does not touch actual tables, which means there must be external
mechanism to survive unsynced tables. Options:
· Ignore lost events. May need trigger/rule on tables to handle
conflicts.
· Replay the lost events on new root. May need trigger/rule on tables
to handle conflicts.
· Roll back table changes. May need old version of row stored in
events. (Achieved with backup parameter to pgq.logutriga)
drop-node <node>
Remove the node from the Londiste replication.
Londiste triggers on the node are removed but Londiste or PgQ are not
removed.
tag-dead <node>
Tag the node as dead, the command can be run from any node in the
replication.
tag-alive <node>
Tag the node as alive, the command can be run from any node in the
replication.
INFORMATION COMMANDS
status
Show status of the replication viewed by the current node.
The output is a tree of the members of the replication with their lags,
last tick, status and the number of tables in state: ok/half/ignored
(replicated, initial copy not finnished, table not replicated locally).
members
Show members of the replication viewed by the current node.
Output the nodes name, status and node location (connection string to
the node).
show-consumers [--node]
TODO: command is not working
REPLICATION DAEMON COMMAND
worker
Replay events to subscriber: it is needed to make the replication
active as it will start to replay the events.
REPLICATION ADMINISTRATION COMMANDS
add-table <table> [args]
Add the table to the replication.
See ADD ARGUMENTS below for the list of possible arguments.
remove-table <table>
Remove the table from the replication.
add-seq <seq> [args]
Add the sequence to the replication.
See ADD ARGUMENTS below for the list of possible arguments.
remove-seq <seq>
Remove the sequence from the replication.
tables
Show all tables on provider.
seqs
Show all sequences on provider.
missing
List tables subscriber has not yet attached to.
resync <table>
Do full copy of the table, again.
ADD ARGUMENTS--all
Include all possible tables.
--wait-sync
Wait until newly added tables are synced fully.
--dest-table=table
Redirect changes to different table.
--force
Ignore table differences.
--expect-sync
No copy needed.
--skip-truncate
Keep old data.
--create
Create table/sequence if not exist, with minimal schema.
--create-full
Create table/sequence if not exist, with full schema.
--trigger-flags=trigger_flags
Trigger creation flags, see below for details.
--trigger-arg=trigger_arg
Custom trigger arg (can be specified multiply times).
--no-triggers
Dont put triggers on table (makes sense on leaf node).
--handler=handler
Custom handler for table.
--handler-arg=handler_arg
Argument to custom handler.
--copy-node=NODE_NAME
Do initial copy from that node instead from provider. Useful if
provider does not contain table data locally or is simply under
load.
--merge-all
Merge tables from all source queues.
--no-merge
Don’t merge tables from source queues.
--max-parallel-copy=max_parallel_copy
Max number of parallel copy processes.
--skip-non-existing
Skip objects that do not exist.
Trigger creation flags (default: AIUDL):
· I - ON INSERT
· U - ON UPDATE
· D - ON DELETE
· Q - use pgq.sqltriga() as trigger function
· L - use pgq.logutriga() as trigger function
· B - BEFORE
· A - AFTER
· S - SKIP
REPLICATION EXTRA COMMANDS
check
Compare table structure on both sides.
fkeys
Print out fkey drop/create commands.
compare [<table>]
Compare table contents on both sides.
--count-only
Just count rows, do not compare data.
repair [<table>] [--force]
Repair data on subscriber.
--force
Ignore lag.
execute [filepath]
Execute SQL files on each node of the cascaded queue. The SQL file is
executed locally in single transaction and inserted into queue in same
transaction. Thus guaranteeing that is will be replayed in subscriber
databases at correct position.
The filename is stored in londiste.applied_execute table, and checked
before execution. If same filename already exists, the SQL execution is
skipped.
SQL meta-data attributes
SQL file can contain attributes that limit where the SQL is
executed:
--*--
--*-- Local-Table: mytable, othertable,
--*-- thirdtable
--*-- Local-Sequence: thisseq
--*--
The magic comments are searched only in file start, before any
actual SQL statement is seen. Empty lines and lines with regular
SQL comments are ignored.
Supported keys:
Local-Table
Table must be added to local node with add-table.
Local-Sequence
Sequence must be added to local node with add-seq.
Local-Destination
Table must be added to local node and actual destination table
must exists. This is for cases where table is added to some
nodes with handler that does not need actual table to exist.
Need-Table
Physical table must exist in database. It does not matter if it
is replicated or not.
Need-Sequence
Sequence must exist in database.
Need-Function
Database function must exists. The function name is in form
function_name(nargs). If the (nargs) portion is missed then
nargs is taken as 0.
Need-View
A view must exist in database.
Need-Schema
Schema mist exist in database.
Londiste supports table renaming, where table is attached to queue
with one name but events are applied to local table with different
name. To make this work with EXECUTE, the Local-Toble and
Local-Destination support tag replacement, where queue’s table name
that is mentioned in attribute is replaced with actual table name
in local database:
--*-- Local-Table: mytable
ALTER TABLE @mytable@ ...;
show-handlers [handler]
Show info about all or a specific handler.
wait-sync
Wait until all added tables are copied over.
wait-provider
Wait until local node passes latest queue position on provider.
wait-root
Wait until local node passes latest queue position on root.
INTERNAL COMMAND
copy
Copy table logic.
EXIT STATUS
0
Successful program execution.
ENVIRONMENT
PostgreSQL environment variables can be used.
EXAMPLES
Londiste is provided with HowTos to help you make your fisrt steps:
· How to set up simple replication.
· How to set up cascaded replication.
· How to set up table partitioning (handlers).
04/01/2014 LONDISTE3(1)
