silk-plugin(3) SiLK Tool Suite silk-plugin(3)NAMEsilk-plugin - Creating a SiLK run-time plug-in using C
SYNOPSIS
sk_cc=`silk_config --compiler`
sk_cflags=`silk_config --cflags`
$sk_cc $sk_cflags -shared -o FILENAME.so FILENAME.c
rwfilter --plugin=FILENAME.so [--plugin=FILENAME.so ...] ...
rwcut --plugin=FILENAME.so [--plugin=FILENAME.so ...]
--fields=FIELDS ...
rwgroup --plugin=FILENAME.so [--plugin=FILENAME.so ...]
--id-fields=FIELDS ...
rwsort --plugin=FILENAME.so [--plugin=FILENAME.so ...]
--fields=FIELDS ...
rwstats --plugin=FILENAME.so [--plugin=FILENAME.so ...]
--fields=FIELDS --values=VALUES ...
rwuniq --plugin=FILENAME.so [--plugin=FILENAME.so ...]
--fields=FIELDS --values=VALUES ...
DESCRIPTION
Several of the SiLK analysis tools allow the user to augment the tools'
functionality through the use of plug-ins that get loaded at run-time.
These tools are:
rwfilter(1)
Supports adding new switches to determine whether each SiLK Flow
record should be written in the --pass or the --fail output stream.
rwcut(1)
Supports adding new output fields that, when selected using the
--fields switch, appear as a column in the output.
rwsort(1)
Supports adding new key fields that, when selected using the
--fields switch, are used to determine the order in which records
are sorted.
rwgroup(1)
Supports adding new key fields that, when selected using the
--id-fields switch, are used to determine how records are grouped.
rwuniq(1)
Supports adding new key fields that, when selected using the
--fields switch, are used to bin (i.e., group) the records. In
addition, rwuniq supports adding new aggregate value fields that,
when selected using the --values switch, will be computed for each
bin. The key and value fields will appear in the output.
rwstats(1)
Supports adding new key fields that, when selected using the
--fields switch, are used to bin (i.e., group) the records. In
addition, rwstats supports adding new aggregate value fields that,
when selected using the --values switch, will be computed for each
bin and can be used to determine the top-N (or bottom-N) bins. The
key and value fields will appear in the output for bins that meet
the top-N threshold.
rwptoflow(1)
Supports adding functionality to ignore packets in the pcap(3)
input stream or to modify the SiLK Flow records as the records are
generated.
In addition, all of the above tools support adding new command line
switches that can be used to initialize the plug-in itself (for
example, to load an auxiliary file that the plug-in requires).
The plug-ins for all tools except rwptoflow can be written in either C
or using PySiLK (the SiLK Python extension, see pysilk(3)). Although
the execution time for PySiLK plug-ins is slower than for C plug-ins,
we encourage you to use PySiLK for your plug-ins since the time-to-
result can be faster for PySiLK: The faster development time in Python
typically more than compensates for the slower execution time. Once
you find that your PySiLK plug-in is seeing a great deal of use, or
that PySiLK is just too slow for the amount of data you are processing,
then re-write the plug-in using C. Even when you intend to write a
plug-in using C, it can be helpful to prototype your plug-in using
PySiLK.
The remainder of this document explains how to create a plug-in for the
SiLK analysis tools (except rwptoflow) using the C programming
language. For information on creating a plug-in using PySiLK, see
silkpython(3).
A template file for plug-ins is included in the SiLK source tree, in
the silk-VERSION/src/template/c-plugin.c file.
The setup function
When you provide "--plugin=my-plugin.so" on the command line to an
application, the application loads the my-plugin.so file and calls a
setup function in that file to determine the new switches and/or fields
that "my-plugin.so" provides.
This setup function is called with three arguments: the first two
describe the version of the plug-in API, and the third is a pointer
that is currently unused.
skplugin_err_t SKPLUGIN_SETUP_FN(
uint16_t major_version,
uint16_t minor_version,
void *plug_in_data)
{
...
}
There are several tasks this setup function may do: (1) check the API
version, (2) register new command line switches (if any), (3) register
new filters (if any), and (4) register new fields (if any). Let's
describe these in more detail.
(1) Check the API version
The setup function should ensure that the plug-in and the application
agree on the API to use. This provides protection in case the SiLK API
to plug-ins changes in the future. To make this determination, call
the skpinSimpleCheckVersion() function. A typical invocation is shown
here, where the "major_version" and "minor_version" were passed into
the SKPLUGIN_SETUP_FN, and "PLUGIN_API_VERSION_MAJOR" and
"PLUGIN_API_VERSION_MINOR" are macros defined in the template file to
the current version of the API.
#define PLUGIN_API_VERSION_MAJOR 1
#define PLUGIN_API_VERSION_MINOR 0
/* Check the plug-in API version */
rv = skpinSimpleCheckVersion(major_version, minor_version,
PLUGIN_API_VERSION_MAJOR,
PLUGIN_API_VERSION_MINOR,
skAppPrintErr);
if (rv != SKPLUGIN_OK) {
return rv;
}
(2) Register command line switches
If the plug-in wants to define new command line switches, those
switches must be registered in the setup function. A typical use of a
command line switch is to allow the user to configure the plug-in; for
example, the switch may allow the user to specify the location of an
auxiliary input file that the plug-in requires, or to set a parameter
used by the plug-in.
A second use for a command line switch is more subtle. When creating a
plug-in for rwfilter, you may want your plug-in to provide several
similar features, and only enable each feature when the user requests
it via a command line switch. For this case, you want to delay
registering the filter until the command line switch is seen, in which
case the filter registration function should be invoked in the switch's
callback function.
Information on registering a command line switch is available below
("Registering command line switches").
(3) Register filters
You only need to register filters when the plug-in will be used by
rwfilter(1). You may choose to register the filters in the setup
function; if you do, the filter will always be used when the plug-in is
loaded by rwfilter. If you the plug-in provides several filtering
functions that the user may choose from via command line switches, you
should call the filter registration function in the callback function
for the command line switch.
See "Registering filter functions" for details on registering a
function to use with rwfilter.
(4) Register fields
If you want your plug-in to create a new printable field for rwcut(1),
a new sorting field for rwsort(1), a new grouping field for rwgroup(1),
rwstats(1), or rwuniq(1), or a new aggregate value field for rwstats or
rwuniq, you should register those fields in the setup function. (While
you can register the fields in a switch's callback function, there is
usually little reason to do so.)
There are two interfaces to registering a new field:
1. The advanced interface provides complete control over how the field
is defined, and allows (or forces) you to specify exactly how to
map from a SiLK Flow record to a binary representation to a textual
representation. To use the advanced interface you will need to
define several functions and fill in a C structure with pointers to
those functions. This interface is described in the "Advanced
field registration function" section below.
2. The simple interface can be used to define fields that map to an
integer value, an IP address, or text that is index by an integer
value. To use this interface, you need to define only one or two
functions. The simple interface should handle many common cases,
and it is described in "Simple field registration functions".
Registering command line switches
When you register a switch, the two important pieces of information you
must provide are a name for the switch and a callback function. When
the application encounters the command line switch registered by your
plug-in, the application will invoke the callback function with the
parameter that the user provided (if any) to the command line switch.
To register a command line switch, call the skpinRegOption2() function:
skplugin_err_t skpinRegOption2(
const char *option_name,
skplugin_arg_mode_t mode,
const char *option_help_string,
skplugin_help_fn_t option_help_fn,
skplugin_option_fn_t opt_process_fn,
void *opt_callback_data,
int num_fn_mask,
...); /* list of skplugin_fn_mask_t */
The parameters are
"option_name"
Specifies the command line switch to create. Do not include the
leading "--" characters in the name.
"mode"
Determines whether the switch takes an argument. It should be one
of
"NO_ARG"
when the command line option acts as an on/off switch
"OPTIONAL_ARG"
when the command line option has a default value, or
"REQUIRED_ARG"
when the user of the plug-in must provide an argument to the
command line option.
"option_help_string"
This parameter specifies the usage string to print when the user
requests --help from the application. This parameter may be NULL.
Alternatively, you may instruct the application to generate a help
string by invoking a callback function your plug-in provides, as
described next.
"option_help_fn"
This parameter specifies a pointer to a function that the
application will to call to print a help message for the command
line switch when the user requests --help from the application.
This parameter may be NULL; if it is not NULL, the
"option_help_string" value is ignored. The signature of the
function to provide is
void option_help_fn(
FILE *file_handle,
const struct option *option,
void *opt_callback_data);
The "file_handle" argument is where the function should print its
help message. The "opt_callback_data" is the value provided to
skpinRegOption2() when the option was registered. The "struct
option" parameter has two members of interest: "name" contains the
number used to register the option, and "has_arg" contains the
"mode" that was used when the option was specified.
"opt_process_fn"
Specifies the callback function, whose signature is
skplugin_err_t opt_process_fn(
const char *opt_arg,
void *opt_callback_data);
The application will call
"opt_process_fn(opt_arg,opt_callback_data)" when --option_name is
seen as a command line argument. "opt_arg" will be the parameter
the user passed to the switch, or it will be NULL if no parameter
was given.
"opt_callback_data"
Will be passed back unchanged to the plug-in as a parameter in the
opt_process_fn() and option_help_fn() callback functions.
"num_fn_mask"
Specifies the number of "skplugin_fn_mask_t" values specified as
the final argument(s) to skpinRegOption2().
"..."
Specifies a list of "skplugin_fn_mask_t" values. The length of
this list must be specified in the "num_fn_mask" parameter. A
plug-in file (e.g., my-plugin.so) can be loaded into any SiLK tool
that supports plug-ins, but you may want a command line switch to
appear only in certain applications. For example, the flowrate(3)
plug-in can be used in both rwfilter and rwcut. When used by
rwfilter, flowrate provides a --bytes-per-second switch; when used
by rwcut, that switch is not available, and instead the bytes/sec
field becomes available. This list determines in which
applications the switch gets defined, and the list should contain
the "SKPLUGIN_FN_*" or "SKPLUGIN_APP_*" macros defined in
skplugin.h. To make the switch available in all applications,
specify "SKPLUGIN_FN_ANY". When skpinRegOption2() is called in an
the application that does not match a value in this list, the
function returns SKPLUGIN_ERR_DID_NOT_REGISTER, indicating that
this option is not applicable to the application.
Registering filter functions
When you register a filter function, you are specifying a function that
rwfilter will call for every SiLK Flow record that rwfilter reads from
its input files. If the function returns "SKPLUGIN_FILTER_PASS",
rwfilter writes the record into the stream(s) specified by --pass. The
record goes to the --fail streams if the function returns
"SKPLUGIN_FILTER_FAIL".
(The previous paragraph is true only when the plug-in is the only
filtering predicate. When multiple tests are specified on the rwfilter
command line, rwfilter will put the record into the fail destination as
soon as any test fails. If there are multiple tests, your plug-in
function will only see records that have not yet failed a test. If a
plug-in filter function follows your function, it may fail a record
that your filter function passed.)
To register a filter function, call the following function:
skplugin_err_t skpinRegFilter(
skplugin_filter_t **filter_handle,
const skplugin_callbacks_t *regdata,
void *cbdata);
"filter_handle"
When this parameter is not NULL, skpinRegFilter() will set the
location it references to the newly created filter. Currently, no
other function accepts the "skplugin_filter_t" as an argument.
"cbdata"
This parameter will be passed back unchanged to the plug-in as a
parameter in the various callback functions. It may be NULL.
"regdata"
This structure has a member for every possible callback function
the SiLK plug-in API supports. When used by skpinRegFilter(), the
following members are supported.
"filter"
rwfilter invokes this function for each SiLK flow record. If
the function returns SKPLUGIN_FILTER_PASS, the record is
accepted; if it returns SKPLUGIN_FILTER_FAIL, the record is
rejected. The type of the function is a
"skplugin_filter_fn_t", and its signature is:
skplugin_err_t filter(
const rwRec *rec,
void *cbdata,
void **extra);
where "rec" is the SiLK Flow record, "cbdata" is the "cbdata"
specified in skpinRegFilter(), and "extra" will likely be
unused.
"init"
rwfilter invokes this function for all registered filter
predicates. It is called after argument processing and before
reading records. The function's type is
"skplugin_callback_fn_t" and the function pointer may be NULL.
The callback's signature is
skplugin_err_t init(
void *cbdata);
"cleanup"
When this function pointer is non-NULL, rwfilter calls this
function after all records have been processed. This function
has the same type and signature as the "init" function.
The function's return value will be "SKPLUGIN_OK" unless the "filter"
member of the "regdata" structure is NULL.
If your plug-in registers a filter function and the plug-in is used in
an application other that rwfilter, the call to skpinRegFilter() is a
no-op.
Simple field registration functions
Using a plug-in, you can augment the keys available in the --fields
switch on rwcut(1), rwgroup(1), rwsort(1), rwstats(1), and rwuniq(1),
and provide new aggregate value fields for the --values switch on
rwstats and rwuniq.
The standard field registration function, skpinRegField(), is
powerful---for example, you can control exactly how the value you
compute will be printed. However, that power comes with complexity.
Many times, all your plug-in needs to do is to compute a value, and
having to write a function to print a number is work with little
reward. The functions in this section handle the registration of
common field types.
All of these functions require a name for the new field. The name is
used as one of the arguments to the --fields or --values switch, and
the name will also be used as the title when the field is printed (as
in rwcut). Field names are case insensitive, and all field names must
be unique within an application. You will get a run-time error if you
attempt to create a field whose name already exists. (In rwuniq and
rwstats, you may have a --fields key and a --values aggregate value
with the same name.)
The callback functions dealing with integers use "uint64_t" for
convenience, but internally the value will be stored in a smaller
integer field if possible. Specifying the "max" parameter to the
largest value you actually use may allow SiLK to use a smaller integer
field.
The functions in this section return "SKPLUGIN_OK" unless the callback
function is NULL.
Integer key field
The following function is used to register a key field whose value is
an unsigned 64 bit integer.
skplugin_err_t skpinRegIntField(
const char *name,
uint64_t min,
uint64_t max,
skplugin_int_field_fn_t rec_to_int,
size_t width);
"name"
The name of the new key field.
"min"
A number representing the minimum integer value for the field.
"max"
A number representing the maximum integer value for the field. If
"max" is 0, a value of UINT64_MAX is used instead.
"rec_to_int"
A callback function that accepts a SiLK Flow record as its sole
argument, and returns an unsigned integer (in host byte order)
which represents the value of the "name" field for the given
record. The signature is
uint64_t rec_to_int(
const rwRec *rec);
"width"
The column width to use when displaying the field. If "width" is
0, it will be computed to be the number of digits necessary to
display the integer "max".
IPv4 key field
The following function registers a new key field whose value is an IPv4
address.
skplugin_err_t skpinRegIPv4Field(
const char *name,
skplugin_ipv4_field_fn_t rec_to_ipv4,
size_t width);
"name"
The name of the new key field.
"rec_to_ipv4"
A callback function that accepts a SiLK Flow record as its sole
argument, and returns a 32 bit integer (in host byte order) which
represents the IPv4 addresses for the "name" field for the given
record. The signature is
uint32_t rec_to_ipv4(
const rwRec *rec);
"width"
The column width to use when displaying the field. If "width" is
0, it will be set to 15.
IP key field
The following function is used to register a key field whose value is
any IP address (an "skipaddr_t").
skplugin_err_t skpinRegIPAddressField(
const char *name,
skplugin_ip_field_fn_t rec_to_ipaddr,
size_t width);
"name"
The name of the new key field.
"rec_to_ipaddr"
A callback function that accepts a SiLK Flow record and an
"skipaddr_t" as arguments. The function should fill in the IP
address as required for the "name" field. The signature is
void rec_to_ipaddr(
skipaddr_t *dest,
const rwRec *rec);
"width"
The column width to use when displaying the field. If "width" is
0, it will be set to 39 when SiLK has support for IPv6 addresses,
or 15 otherwise.
Text key field (from an integer)
The following function is used to register a key field whose value is
an unsigned 64 bit integer (similar to skpinRegIntField()), but where
the printed representation of the field is determined by a second
callback function. This allows the plug-in to create arbitrary text
for the field.
skplugin_err_t skpinRegTextField(
const char *name,
uint64_t min,
uint64_t max,
skplugin_int_field_fn_t value_fn,
skplugin_text_field_fn_t text_fn,
size_t width);
"name"
The name of the new key field.
"min"
A number representing the minimum integer value for the field.
"max"
A number representing the maximum integer value for the field. If
"max" is 0, a value of UINT64_MAX is used instead.
"value_fn"
A callback function that accepts a SiLK Flow record as its sole
argument, and returns an unsigned integer (in host byte order)
which represents the value of the "name" field for the given
record. The signature is
uint64_t rec_to_int(
const rwRec *rec);
"text_fn"
A callback function that provides the textual representation of the
value returned by "value_fn". The function's signature is
void text_fn(
char *dest,
size_t dest_len,
uint64_t val);
The callback should fill the character array "dest" with the
printable representation of "val". The number of characters in
"dest" is given by "dest_len". Note that "dest_len" may be
different than the parameter "width" passed to skpinRegTextField(),
and "text_fn" must NUL-terminate the string.
"width"
The column width to use when displaying the field.
Text key field (from a list)
The following function is used to register a field whose value is one
of a list of strings. The plug-in provides the list of strings and a
callback that takes a SiLK Flow record and returns an index into the
list of strings.
skplugin_err_t skpinRegStringListField(
const char *name,
const char **list,
size_t entries,
const char *default_value,
skplugin_int_field_fn_t rec_to_index,
size_t width);
"name"
The name of the new key field.
"list"
List is the list of strings. The list should either be NULL
terminated, or "entries" should have a non-zero value.
"entries"
The number of entries in "list". If "entries" is 0, SiLK
determines the number of entries by traversing "list" until it
finds a element whose value is NULL.
"default_value"
The value to use when "rec_to_index" returns an invalid value.
"rec_to_index"
A callback function that accepts a SiLK Flow record as its sole
argument, and returns an unsigned integer (in host byte order)
which represents an index into "list". If the return value is
beyond the end of "list", "default_value" will be used instead.
The signature of this callback function is
uint64_t rec_to_int(
const rwRec *rec);
"width"
The column width to use when displaying the field. If "width" is
0, it is defaulted to the width of the longest string in "list" and
"default_value".
Integer sum aggregate value field
The following function registers an aggregate value field that
maintains a running unsigned integer sum. That is, the values returned
by the callback are summed for every SiLK Flow record that matches a
bin's key. The sum is printed when the bin is printed.
skplugin_err_t skpinRegIntSumAggregator(
const char *name,
uint64_t max,
skplugin_int_field_fn_t rec_to_int,
size_t width);
"name"
The name of the new aggregate value field.
"max"
A number representing the maximum integer value for the field. If
"max" is 0, a value of UINT64_MAX is used instead.
"rec_to_int"
A callback function that accepts a SiLK Flow record as its sole
argument, and returns an unsigned integer (in host byte order)
which represents the value of the "name" value field for the given
record. The signature is
uint64_t rec_to_int(
const rwRec *rec);
"width"
The column width to use when displaying the value. If "width" is
0, it will be computed to be the number of digits necessary to
display the integer "max".
Integer minimum or maximum aggregate value field
The following function registers an aggregate value field that
maintains the minimum integer value seen among all values returned by
the callback function.
skplugin_err_t skpinRegIntMinAggregator(
const char *name,
uint64_t max,
skplugin_int_field_fn_t rec_to_int,
size_t width);
This function is similar, except it maintains the maximum value.
skplugin_err_t skpinRegIntMaxAggregator(
const char *name,
uint64_t max,
skplugin_int_field_fn_t rec_to_int,
size_t width);
"name"
The name of the new aggregate value field.
"max"
A number representing the maximum integer value for the field. If
"max" is 0, a value of UINT64_MAX is used instead.
"rec_to_int"
A callback function that accepts a SiLK Flow record as its sole
argument, and returns an unsigned integer (in host byte order)
which represents the value of the "name" value field for the given
record. The signature is
uint64_t rec_to_int(
const rwRec *rec);
"width"
The column width to use when displaying the value. If "width" is
0, it will be computed to be the number of digits necessary to
display the integer "max".
Unsigned integer aggregate value field
The following function registers an aggregate value field that can be
represented by a 64 bit integer. The plug-in must register two
callback functions. The first takes a SiLK Flow record and returns an
integer value; the second takes two integer values (as returned by the
first callback function) and combines them to form a new aggregate
value.
skplugin_err_t skpinRegIntAggregator(
const char *name,
uint64_t max,
skplugin_int_field_fn_t rec_to_int,
skplugin_agg_fn_t agg,
uint64_t initial,
size_t width);
"name"
The name of the new aggregate value field.
"max"
A number representing the maximum integer value for the field. If
"max" is 0, a value of UINT64_MAX is used instead.
"rec_to_int"
A callback function that accepts a SiLK Flow record as its sole
argument, and returns an unsigned integer (in host byte order)
which represents the value of the "name" value field for the given
record. The signature is
uint64_t rec_to_int(
const rwRec *rec);
"agg"
A callback function that combines (aggregates) two values. For
example, if you wanted to create a new aggregate value that
contained a bit-wise OR of the TCP flags seen on every packet, your
"agg" function would OR the values. The signature is
uint64_t agg(
uint64_t current,
uint64_t operand);
"initial"
Specifies the initial value for the aggregate value. The first
time the "agg" function is called on a bin, "operand" will be the
value returned by "rec_to_int", and "current" will be the value
given in "initial". The value in "initial" must be less than or
equal to the value in "max".
"width"
The column width to use when displaying the value. If "width" is
0, it will be computed to be the number of digits necessary to
display the integer "max".
Advanced field registration function
When the simple field registration functions do not provide what you
need, you can use the skpinRegField() function that gives you complete
control over the field.
skpinRegField() registers a new derived field for record processing.
The plug-in must supply the name of the new field. The name is used as
one of the arguments to the --fields switch (for key fields) or
--values switch (for aggregate value fields). Field names are case
insensitive, and all field names must be unique within an application.
You will get a run-time error if you attempt to create a field whose
name already exists. (In rwuniq and rwstats, you may have a --fields
key and a --values aggregate value with the same name.)
The skpinRegField() function requires you initialize and pass in a
structure. In this structure you will specify the callback functions
that the application will call, as well as additional information
required by some applications. Although the structure is complex, not
all applications use all members.
If the plug-in is loaded by an application that does not support fields
(such as rwfilter), the function is a no-op.
The advanced field registration function is
skplugin_err_t skpinRegField(
skplugin_field_t **return_field,
const char *name,
const char *description,
const skplugin_callbacks_t *regdata,
void *cbdata);
"return_field"
When this value is not NULL, skpinRegField() will set the location
it references to the newly created field.
"name"
This sets the primary name of the field, and by default will be the
title used when printing the field.
"description"
The "description" provides a textual description of the field.
Currently this is unused.
"regdata"
The "regdata" structure provides the application with the callback
functions and additional information it needs to use the plug-in.
The members that must be set vary by application. It is described
in more detail below.
"cbdata"
This parameter will be passed back unchanged to the plug-in as a
parameter in the various callback functions. It may be NULL.
The structure used by the skpinRegField() (and skpinRegFilter())
functions to specify callback functions is shown here:
typedef struct skplugin_callbacks_st {
skplugin_callback_fn_t init;
skplugin_callback_fn_t cleanup;
size_t column_width;
size_t bin_bytes;
skplugin_text_fn_t rec_to_text;
skplugin_bin_fn_t rec_to_bin;
skplugin_bin_fn_t add_rec_to_bin;
skplugin_bin_to_text_fn_t bin_to_text;
skplugin_bin_merge_fn_t bin_merge;
skplugin_bin_cmp_fn_t bin_compare;
skplugin_filter_fn_t filter;
skplugin_transform_fn_t transform;
const uint8_t *initial;
const char **extra;
} skplugin_callbacks_t;
All of the callback functions reference in this structure take "cbdata"
as a parameter, which is the value that was specified in the call to
skpinRegField(). The "extra" parameter to the callback functions is
used in complex plug-ins and can be ignored.
The members of the structure are:
"init"
This specifies a callback function which the application will call
when it has determined this field will be used. (In the case of
skpinRegFilter(), the function is called for all registered
filters.) The application calls the function before processing
data. It may be NULL; the signature of the callback function is
skplugin_err_t init(
void *cbdata);
"cleanup"
When this callback function is not NULL, the application will call
it after all records have been processed. It has the same
signature as the "init" function.
"column_width"
The number of characters (not including trailing NUL) required to
hold a string representation of the longest value of the field.
This value can be 0 if not used (e.g., rwsort does not print
fields), or if it will be set later using skpinSetFieldWidths().
"bin_bytes"
The number of bytes (octets) required to hold a binary
representation of a value of the field. This value can be 0 if not
used (e.g., rwcut does not use binary values), or if it will be set
later using skpinSetFieldWidths().
"rec_to_text"
The rwcut application uses this callback function to fetch the
textual value for the field given a SiLK Flow record. The
signature of this function is
skplugin_err_t rec_to_text(
const rwRec *rec,
char *dest,
size_t width,
void *cbdata,
void **extra);
The callback function should fill the character array "dest" with
the textual value, and the value should be NUL-terminated. "width"
specifies the overall size of "dest", and it may not have the same
value as specified by the "column_width" member. For proper
formatting, the callback function should write no more than
"column_width" characters into "dest". Note that if an application
requires a "rec_to_bin" function and "rec_to_bin" is NULL, the
application will use "rec_to_text" if it is provided. The
application will use "column_width" as the width for binary values
(zeroing out the destination area before it is written to).
"rec_to_bin"
This callback function is used by the application to fetch the
binary value for this field given the SiLK Flow record. The
signature of this function is:
skplugin_err_t rec_to_bin(
const rwRec *rec,
uint8_t *dest,
void *cbdata,
void **extra);
The callback function should write exactly "bin_bytes" of data into
"dest" (where "bin_bytes" was specified in the call to
skpinRegField() or skpinSetFieldWidths()). See also the
"rec_to_text" member.
"add_rec_to_bin"
This callback function is used by rwuniq and rwstats when computing
aggregate value fields. The application expects this function to
get the binary value for this field from the SiLK Flow record and
merge it (e.g., add it) to the current value. That is, the
function should update the value in "current_and_new_value" with
the value that comes from the current "rec". The signature is:
skplugin_err_t add_rec_to_bin(
const rwRec *rec,
uint8_t *current_and_new_value,
void *cbdata,
void **extra);
The callback function should write exactly "bin_bytes" of data into
"current_and_new_value".
"bin_to_text"
This callback function is used to get a textual representation of a
binary value that was set by a prior call to the "rec_to_bin" or
"add_rec_to_bin" functions. The function signature is
skplugin_err_t bin_to_text(
const uint8_t *bin,
char *dest,
size_t width,
void *cbdata);
The binary input value is in "bin", and it is exactly "bin_bytes"
in length. The textual output must be written to "dest". The
overall size of "dest" is given by "width", which may be different
than the "column_width" value that was previously specified. For
proper formatting, the callback function should write no more than
"column_width" characters into "dest".
"bin_merge"
When rwstats and rwuniq are unable to store all values in memory,
the applications write their current state to temporary files on
disk. Once all input data has been processed, the temporary files
are combined to produce the output. When a key appears in multiple
temporary files, the aggregate values must be merged (for example,
the byte count for two keys would be added). This callback
function is used to merge aggregate value fields defined by the
plug-in. The function signature is below. The "src1_and_dest"
parameter will contain a binary aggregate value from one of the
files, and the "src2" parameter a value from the other. These
should be combined and the (binary) result written to
"src1_and_dest". The byte length of both parameters is
"bin_bytes".
skplugin_err_t bin_merge(
uint8_t *src1_and_dest,
const uint8_t *src2,
void *cbdata);
"bin_compare"
This callback function is used by rwstats when determining the top-
N (or bottom-N) bins based on the binary aggregate values. The
function accepts two binary values, "value_a" and "value_b", each
of length "bin_bytes". The function must set "cmp_result" to an
integer less than 0, equal 0, or greater than 0 to indicate whether
"value_a" is less than, equal to, or greater than "value_b",
respectively. If this function is NULL, memcmp() will be used on
the binary values instead.
skplugin_err_t bin_compare(
int *cmp_result,
const uint8_t *value_a,
const uint8_t *value_b,
void *cbdata);
"filter"
This callback function is only required when the plug-in will be
used by rwfilter, as described above. When defining a field,
"filter" is ignored.
"transform"
This callback function is only required when the plug-in will be
used by rwptoflow. This callback allows the plug-in to modify the
SiLK Flow record, "rec", before it is written to the output. The
callback function should modify "rec" in place; the signature is
skplugin_err_t transform(
rwRec *rec,
void *cbdata,
void **extra);
"initial"
When the "initial" member is not NULL, it should point to a value
containing at least "bin_bytes" bytes. These bytes will be used to
initialize the binary aggregate value. As an example use case,
when the plug-in is computing a minimum, it may choose to
initialize the field to contain the maximum value. When "initial"
is NULL, binary aggregate values are initialized using bzero().
"extra"
This member is usually NULL. When not NULL, it points to a NULL-
terminated constant array of strings representing "extra
arguments". These are not often used, and they will not be
discussed in this manual page.
Once a field is registered, you may make changes to it by calling the
additional functions described below. In each of these functions, the
"field" parameter is the handle returned when the field was registered.
By default, the "name" will also be used as the field's title. To
specify a different title, the plug-in may call
skplugin_err_t skpinSetFieldTitle(
skplugin_field_t field,
const char title);
To create an alternate name for the field (that is, a name that can be
used in the --fields or --values switches) call
skplugin_err_t skpinAddFieldAlias(
skplugin_field_t field,
const char alias);
To set or modify the textual and binary widths for a field, use the
following function. This function should called in the field's "init"
callback function.
skplugin_err_t skpinSetFieldWidths(
skplugin_field_t field,
size_t field_width_text,
size_t field_width_bin);
The following table shows when a member of the "skplugin_callbacks_t"
structure is required or optional. (Where the table shows
"column_width" and "bin_bytes" as required, the values can be set in
the structure or via the skpinSetFieldWidths() function.)
rwfilter rwcut rwgroup rwsort rwstats rwuniq rwptoflow
init r f f f f,a f,a r
cleanup r f f f f,a f,a r
column_width . F . . F,A F,A .
bin_bytes . . F F F,A F,A .
rec_to_text . F . . . . .
rec_to_bin . . F F F F .
add_rec_to_bin . . . . A A .
bin_to_text . . . . F,A F,A .
bin_merge . . . . A A .
bin_compare . . . . A . .
initial . . . . a a .
filter R . . . . . .
transform . . . . . . R
extra r f f f f,a f,a r
The legend is
F required for a key field
A required for an aggregate value field
R required for a non-field application (e.g., rwfilter)
f optional for a key field
a optional for an aggregate value field
r optional for a non-field application
. ignored
Miscellaneous functions
The following registers a cleanup function for the plug-in. This
function will be called by the application after any field- or filter-
specific cleanup functions are called. Specifically, this is the last
callback that the application will invoke on a plug-in.
skplugin_err_t skpinRegCleanup(
skplugin_cleanup_fn_t cleanup);
The signature of the "cleanup" function is:
void cleanup(void);
The plug-in author should invoke the following function to tell
rwfilter that this plug-in is not thread safe. Calling this function
causes rwfilter not use multiple threads; as such, this function should
only be called when the plug-in has registered an active filter
function.
void skpinSetThreadNonSafe(void);
Compiling the plug-in
Once you have finished writing the C code for the plug-in, save it in a
file. The following uses the name my-plugin.c for the name of this
file.
In the following, the leading dollar sign ("$") followed by a space
represents the shell prompt. The text after the dollar sign represents
the command line. Lines have been wrapped for improved readability,
and the back slash ("\") is used to indicate a wrapped line.
When compiling a plug-in, you should use the same compiler and
compiler-options as when SiLK was compiled. The silk_config(1) utility
can be used to obtain that information. To store the compiler used to
compile SiLK into the variable "sk_cc", specify the following at a
shell prompt (note that those are backquotes, and this assumes a
Bourne-compatible shell):
$ sk_cc=`silk_config --compiler`
To get the compiler flags used to compile SiLK:
$ sk_cflags=`silk_config --cflags`
Using those two variables, you can now compile the plug-in. The
following will work on Linux and Mac OS X:
$ $sk_cc $sk_cflags -shared -o my-plugin.so my-plugin.c
For Mac OS X:
$ $sk_cc $sk_cflags -bundle -flat_namespace -undefined suppress \
-o my-plugin.so my-plugin.c
If there are compilation errors, fix them and compile again.
Notes: The preceding assumed you were building the plug-in after having
installed SiLK. The paths given by silk_config do not work if SiLK has
not been installed. To compile the plug-in, you must have access to
the SiLK header files. (If you are using an RPM installation of SiLK,
ensure that the "silk-devel" RPM is installed.)
Once you have created the my-plugin.so file, you can load it into an
application by using the --plugin switch on the application as shown in
the "SYNOPSIS". When loading a plug-in from the current directly, it
is best to prefix the filename with "./":
$ rwcut --plugin=./my-plugin.so ...
If there are problems loading the plug-in into the application, you can
trace the actions the application is doing by setting the
SILK_PLUGIN_DEBUG environment variable:
$ SILK_PLUGIN_DEBUG=1 rwcut --plugin=./my-plugin.so ...
EXAMPLES
rwfilter
Suppose you want to find traffic destined to a particular host,
10.0.0.23, that is either ICMP or coming from 1434/udp. If you attempt
to use:
$ rwfilter --daddr=10.0.0.23 --proto=1,17 --sport=1434 \
--pass=outfile.rw flowrec.rw
the --sport option will not match any of the ICMP traffic, and your
result will not contain ICMP records. To avoid having to use two
invocations of rwfilter, you can create the following plug-in to do the
entire check in a single pass:
#include <silk/silk.h>
#include <silk/rwrec.h>
#include <silk/skipaddr.h>
#include <silk/skplugin.h>
#include <silk/utils.h>
/* These variables specify the version of the SiLK plug-in API. */
#define PLUGIN_API_VERSION_MAJOR 1
#define PLUGIN_API_VERSION_MINOR 0
/* ip to search for */
static skipaddr_t ipaddr;
/*
* status = filter(rwrec, reg_data, extra);
*
* The function should examine the SiLK flow record and return
* SKPLUGIN_FILTER_PASS to write the rwRec to the
* pass-destination(s) or SKPLUGIN_FILTER_FAIL to write it to the
* fail-destination(s).
*/
static skplugin_err_t filter(
const rwRec *rwrec,
void *reg_data,
void **extra)
{
skipaddr_t dip;
rwRecMemGetDIP(rwrec, &dip);
if (0 == skipaddrCompare(&dip, &ipaddr)
&& (rwRecGetProto(rwrec) == 1
|| (rwRecGetProto(rwrec) == 17
&& rwRecGetSPort(rwrec) == 1434)))
{
return SKPLUGIN_FILTER_PASS;
}
return SKPLUGIN_FILTER_FAIL;
}
/* The set-up function that the application will call. */
skplugin_err_t SKPLUGIN_SETUP_FN(
uint16_t major_version,
uint16_t minor_version,
void *plug_in_data)
{
uint32_t ipv4;
skplugin_err_t rv;
skplugin_callbacks_t regdata;
/* Check the plug-in API version */
rv = skpinSimpleCheckVersion(major_version, minor_version,
PLUGIN_API_VERSION_MAJOR,
PLUGIN_API_VERSION_MINOR,
skAppPrintErr);
if (rv != SKPLUGIN_OK) {
return rv;
}
/* set global ipaddr */
ipv4 = ((10 << 24) | 23);
skipaddrSetV4(&ipaddr, &ipv4);
/* register the filter */
memset(®data, 0, sizeof(regdata));
regdata.filter = filter;
return skpinRegFilter(NULL, ®data, NULL);
}
Once this file is created and compiled, you can use it from rwfilter as
shown here:
$ rwfilter --plugin=./my-plugin.so --pass=outfile.rw flowrec.rw
Additional examples
For additional examples, see the source files in
silk-VERSION/src/plugins.
ENVIRONMENT
SILK_PATH
This environment variable gives the root of the install tree. When
searching for plug-ins, a SiLK application may use this environment
variable. See the "FILES" section for details.
SILK_PLUGIN_DEBUG
When set to 1, the SiLK applications print status messages to the
standard error as they attempt to find and open each plug-in. In
addition, when an attempt to register a field fails, the
application prints a message specifying the additional function(s)
that must be defined to register the field in the application. Be
aware that the output can be rather verbose.
FILES
${SILK_PATH}/lib64/silk/
${SILK_PATH}/lib64/
${SILK_PATH}/lib/silk/
${SILK_PATH}/lib/
/usr/local/lib64/silk/
/usr/local/lib64/
/usr/local/lib/silk/
/usr/local/lib/
Directories that a SiLK application checks when attempting to load
a plug-in.
SEE ALSOrwfilter(1), rwcut(1), rwgroup(1), rwsort(1), rwstats(1), rwuniq(1),
silk_config(1), rwptoflow(1), pysilk(3), silkpython(3), flowrate(3),
silk(7), pcap(3)SiLK 3.11.0.1 2016-02-19 silk-plugin(3)
