YAFDPI(1) Yet Another Flowmeter YAFDPI(1)NAME
yaf deep packet inspection
DESCRIPTION
yaf can examine packet payloads, capture useful information for a
specific protocol, and export it in a protocol-specific template within
yaf's SubTemplateMultiList if yaf is built with plugin support enabled
(using the --enable-plugins option to ./configure). It may be
necessary to set the LTDL_LIBRARY_PATH environment variable if the
plugins were installed in a nonstandard location.
The DPI plugin requires payload capture to be enabled with the
--max-payload option. A minimum payload capture length of 384 octets
is recommended for best results. --applabel is also required, as the
application label determines how the inspection will execute.
DPI in yaf is directly related to application labeling as it will only
perform DPI if a match was found during the application labeling phase,
and it will only execute an inspection specific to the protocol denoted
by the application label.
In order to enable DPI in yaf the following should be added to the
command line:
"--plugin-name=/usr/local/lib/yaf/dpacketplugin.la"
You can also add the option switch to specify which protocols to
perform DPI:
"--plugin-opts="53 80 21""
The above will perform DPI for DNS, HTTP, and FTP.
DPI operates differently depending on whether the protocol is plugin-
based or regex-based in the yafApplabelRules.conf file. If the
protocol uses a regex rule for application labeling, it will have a
list of regular expressions in the yafDPIIRules.conf file that are
compared against the captured payload. Any matches are stored and
later exported in an IPFIX information element. If the protocol is
based on a plugin rule, it will store important information while it is
decoding the payload using the dynamically loaded plugin listed in the
yafApplabelRules.conf file. See the source code to the plugins
included with yaf for details on the specific protocol implementations.
Some plugins will allow configurable deep packet inspection from the
yafDPIRules.conf file, such as DNP 3.0, Ethernet/IP, and SCADA. See
below for specific information on these particular protocols.
In order to perform DPI on DNSSEC resource records, add "DNSSEC" to the
--plugin-opts option:
"--plugin-opts=DNSSEC"
"--plugin-opts="DNSSEC 53""
DPI CONFIG FILE FORMAT
The yafDPIRules.conf file should be in the same location as the
yafApplabelRules.conf file. The file follows a similar format to
yafApplabelRules.conf. The file is a list of label, element pair
statements. A label statement begins with the keyword 'label', and has
the following form:
label <N> element <N2> <element-rule>
where <N> is the application label (usually the well-known port) found
in the yafApplabelRules.conf file (an unsigned 16-bit decimal integer
in the range 0 to 65535), <N2> is the Information Element ID found in
the /usr/local/yaf/CERT_IE.h and below, and <element-rule> is a PCRE
regular expression and will be stored and associated with the ID number
preceding it. There can be multiple lines for a single application
label, however each should have a different <N2>. There should be
parentheses around the substring you want to capture and store. If
there is more than 1 set of parentheses in the regular expression, the
most outer set is the substring captured. (See PCRE documentation for
details on regular expressions and substring matching.)
User Defined Elements
To define your own information elements, use the following form:
label <N> user <E> name <element-name> <element-rule>
where <N> is the application label found in yafApplabelRules.conf file.
<E> is the Information Element ID in the range of 0 to 65535 to be
given to the element upon export. This number should be unique to this
file and should NOT be defined in /usr/local/yaf/CERT_IE.h. This
element will be added to the template upon processing of this file, and
must be added to the yaf collecting process in order to properly decode
the IPFIX message. <element-name> is the name you want to give to this
IPFIX Information Element. This name can consist of letters and
numbers and underscores; it can not contain special characters or
spaces. <label-rule> is the PCRE regular expression and will be stored
and associated with the Information Element ID and name preceding it.
There is a limit of 30 additional fields per protocol that YAF will
store and execute. To find out if yaf accepted your elements, run yaf
with --verbose. All user elements will be exported using the CERT
Private Enterprise Number (PEN) 6871. ONLY user labels for protocols
FTP, HTTP, IMAP, SMTP, RTSP, SSH, and SIP will be added. Elements will
be added to the template in the order they are listed in the
yafDPIRules.conf file in the form of an fbBasicList_t. By default,
HTTP exports 20 basicLists, FTP exports 5 basicLists, IMAP exports 7
basicLists, RTSP exports 12 basicLists, SIP exports 7 basicLists, SMTP
exports 11 basicLists, and 1 basicList is exported for SSH.
A "#" smybol starts a comment for the entire line. If a rule is not
properly formatted, all subsequent rules may not be processed. It is
acceptable to comment out any yaf DPI rules. yaf rules commented out
will not be executed against the payload but they will still exist in
the template and record. User-defined information elements are added
based on the configuration file at run time.
Optionally, this file may contain two limit statements to configure the
DPI plugin. A limit statement begins with the keyword 'limit', and has
the following form:
limit [field|total] <limit-value>
If the "field" label is present, the <limit-value> will be the number
of bytes yaf will export for any given field in this file. This does
not affect the DNS Deep Packet Inspection or SSL Certificate Capture.
FOr DNS, a domain name can have a maximum of 255 characters, so the
limit is not configurable.
If the "total" field is present, the <limit-value> will be the total
number of bytes yaf may export from the DPI plugin. Obviously, this
number will not be larger than the --max-payload value yaf is given at
run time.
Both the field and total limits have a maximum value of 65535. If they
are larger, they will revert back to the defaults of 200 for per-field
limit and 1000 for total limit.
There are also 2 configuration parameters related to SSL export. By
default, yaf parses the X.509 certificates and exports the information
described below under SSL/TLS. If the following line is present:
cert_export_enabled = 1
yaf will export the full X.509 certificate in the format described
below under Full Certificate Export. Setting this variable to 1
disables the traditional SSL certificate decode and export. If the
second configuration variable is present:
cert_hash_enabled = 1
yaf will export the hash of the X.509 certificate as found in the
certificate. This is typically the SHA-256 hash of the binary
certificate but it can vary on the hashing algorithm used. The hashing
algorithm can be identified by the sslCertSignature field. If both
cert_export_enabled and cert_hash_enabled are set to 1, yaf will export
both the full X.509 certificate and perform the traditional decode of
the X.509 certificate. It is not recommended to do both. If
cert_export_enabled is set to 1, super_mediator can perform the
extraction of relevant fields as is done by yaf, plus it provides the
option to perform SHA-1 or MD5 hashes of the certificate.
DPI in Action
Upon yaf startup and capture, you will be able to see if the rule files
and their regular expressions were accepted using the --verbose flag.
[2013-05-03 19:39:25] DPI Running for ALL Protocols
[2013-05-03 19:39:25] Reading packets from packets.pcap
[2013-05-03 19:39:25] Initializing Rules from DPI File
/usr/local/etc/yafDPIRules.conf
[2013-05-03 19:39:25] DPI rule scanner accepted 63 rules from the DPI
Rule File
An unacceptable regular expression will be brought to your attention
with the above statements. If you choose certain protocols for
inspection using the "--plugin-opts" flag, only the appropriate rule
statements will be loaded into the DPI Rule Scanner.
Configure Options
The following options can be given to ./configure when yaf is built to
export DNS authoritative and NXDomain Responses only.
--enable-exportDNSAuth
Enable export of DNS Authoritative Responses only. The default is to
capture and export all DNS Responses. This flag can be used in
conjunction with --enable-exportDNSNXDomain. It is only recognized if
--plugin-name is set to the DPI plugin, application labeling is
enabled, and --max-payload is set.
--enable-exportDNSNXDomain
Enable export of DNS NXDomain Responses only. The default is to
capture and export all DNS Responses. This flag can be used in
conjunction with --enable-exportDNSAuth. It is only recognized if
--plugin-name is set to the DPI plugin, application labeling is
enabled, and --max-payload is set.
DPI Data Export
DPI Templates & Information Elements by Protocol
yaf's output consists of an IPFIX message stream. yaf uses a variety
of templates for IPFIX data records; As of yaf 2.0, yaf uses a
subTemplateMultiList to export optional information elements, such as
Deep Packet Inspection fields, relating to the flow. Below are
templates that may appear in this subTemplateMultiList depending on the
application label of the flow. For more information on yaf information
elements see yaf(1). For more information on IPFIX Structured lists,
see the Internet Draft, Export of Structured Data in IPFIX, <RFC 6313>.
Most of the elements are exported as a basicList. An IPFIX basicList
represents a list of zero or more instances of any Information Element
(IE 291).
FTP
File Transfer Protocol (FTP) Deep Packet Inspection is based on RFC
959. The following information elements are exported as a template in
the subTemplateMultiList as basicLists of variable length elements in
the order they are listed in the yafDPIRules.conf file. YAF will
always export at least 5 basicLists for FTP, even if not all of the
following are enabled. By default, they will be in the following
order:
ftpReturn CERT (PEN 6871) IE 131, variable length, DPI basicList
FTP Commands or Replies.
ftpUser CERT (PEN 6871) IE 132, variable length, DPI basicList
FTP User Command Argument. This command will normally be the first
command transmitted by the user.
ftpPass CERT (PEN 6871) IE 133, variable length, DPI basicList
FTP Password Command Argument. This command must be preceded by the
user name command, and is usually required to complete
authentication.
ftpType CERT (PEN 6871) IE 134, variable length, DPI basicList
FTP Data Representation Type.
ftpRespCode CERT (PEN 6871) IE 135, variable length, DPI basicList
FTP Reply. This consists of a three digit number followed by some
text.
HTTP
HTTP Deep Packet Inspection is based on RFC 2616. The following
information elements are exported as a template in the
subTemplateMultiList as basicLists of variable length elements in the
order they are listed in the yafDPIRules.conf file. Some elements are
not enabled by default. The template will always contain at least 20
information elements even if less elements are enabled in the
configuration file. By default, the following 20 information elements
are exported in the following order:
httpServerString CERT (PEN 6871) IE 110, variable length, DPI basicList
HTTP Server Response-header field. Contains information about the
software used to handle the HTTP Request.
httpUserAgent CERT (PEN 6871) IE 111, variable length, DPI basicList
HTTP User-Agent Request-header field. Contains information about the
user agent originating the request.
httpGet CERT (PEN 6871) IE 112, variable length, DPI basicList
HTTP Method Command. Retrieves information identified by the
following Request-URI.
httpConnection CERT (PEN 6871) IE 113, variable length, DPI basicList
HTTP Connection header fields. Contains options that are desired for
a particular connection.
httpReferer CERT (PEN 6871) IE 115, variable length, DPI basicList
HTTP Referer request-header field. Address (URI) of the resource
which the Request-URI was obtained.
httpLocation CERT (PEN 6871) IE 116, variable length, DPI basicList
HTTP Location response-header field. Used to redirect the recipient
to a location to complete a request or identify a new resource.
httpHost CERT (PEN 6871) IE 117, variable length, DPI basicList
HTTP Host Request-header. The Internet host and port number of the
resource being requested.
httpContentLength CERT (PEN 6871) IE 118, variable length, DPI
basicList
HTTP Content-Length header. Indicates the size of the entity-body.
httpAge CERT (PEN 6871) IE 119, variable length, DPI basicList
HTTP Age response-header. Argument is the sender's estimate of the
time elapsed since the response.
httpResponse CERT (PEN 6871) IE 123, variable length, DPI basicList
HTTP Response Status Code. Usually a three-digit number followed by
text.
httpAcceptLanguage CERT (PEN 6871) IE 121, variable length, DPI
basicList
HTTP Accept-Language Request-Header field. Restricts the set of
natural languages that preferred.
httpAccept CERT (PEN 6871) IE 120, variable length, DPI basicList
HTTP Accept request-header field. Used to specify certain media
types that are acceptable for the response.
httpContentType CERT (PEN 6871) IE 122, variable length, DPI basicList
HTTP Content Type entity-header field. Indicates the media type of
the entity-body.
httpVersion CERT (PEN 6871) IE 114, variable length, DPI basicList
HTTP Version Number.
httpCookie CERT (PEN 6871) IE 220, variable length, DPI basicList
HTTP Cookie Header Field.
httpSetCookie CERT (PEN 6871) IE 221, variable length, DPI basicList
HTTP Set Cookie Header Field.
httpAuthorization CERT (PEN 6871) IE 252, variable length, DPI
basicList
HTTP Authorization Header Field.
httpVia CERT (PEN 6871) IE 253, variable length, DPI basicList
HTTP Via Header Field.
httpX-Forwarded-For CERT (PEN 6871) IE 254, variable length, DPI
basicList
HTTP X-Forwarded-For Header Field.
httpRefresh CERT (PEN 6871) IE 256, variable length, DPI basicList
HTTP Refresh Header Field.
Optional HTTP Elements
The following information elements are defined but not enabled by
default. To enable any of the following fields, uncomment the line in
the yafDPIRules.conf file.
httpExpires CERT (PEN 6871) IE 255, variable length, DPI basicList
HTTP Expires Header Field.
httpIMEI CERT (PEN 6871) IE 257, variable length, DPI basicList
HTTP International Mobile Station Equipment Identity ID.
httpIMSI CERT (PEN 6871) IE 258, variable length, DPI basicList
HTTP International Mobile Subscriber Identity
httpMSISDN CERT (PEN 6871) IE 259, variable length, DPI basicList
HTTP MSISDN number, a telephone number for the SIM card in a
mobile/cellular phone.
httpSubscriber CERT (PEN 6871) IE 260, variable length, DPI basicList
HTTP Mobile Subscriber Information
httpAcceptCharset CERT (PEN 6871) IE 261, variable length, DPI
basicList
HTTP Accept Charset Header Field.
httpAllow CERT (PEN 6871) IE 262, variable length, DPI basicList
HTTP Accept Encoding Header Field.
httpDate CERT (PEN 6871) IE 263, variable length, DPI basicList
HTTP Date Header Field.
httpExpect CERT (PEN 6871) IE 265, variable length, DPI basicList
HTTP Expect Header Field.
httpFrom CERT (PEN 6871) IE 266, variable length, DPI basicList
HTTP From Header Field.
httpProxyAuthentication CERT (PEN 6871) IE 267, variable length, DPI
basicList
HTTP Proxy Authentication Field.
httpUpgrade CERT (PEN 6871) IE 268, variable length, DPI basicList
HTTP Upgrade Header Field.
httpWarning CERT (PEN 6871) IE 269, variable length, DPI basicList
HTTP Warning Header Field.
httpDNT CERT (PEN 6871) IE 270, variable length, DPI basicList
HTTP DNT Header Field.
httpX-Forwarded-Proto CERT (PEN 6871) IE 271, variable length, DPI
basicList
HTTP X-Forwarded-Proto Header Field.
httpX-Forwarded-Host CERT (PEN 6871) IE 272, variable length, DPI
basicList
HTTP X-Forwarded-Host Header Field.
httpX-Forwarded-Server CERT (PEN 6871) IE 273, variable length, DPI
basicList
HTTP X-Forwarded-Server Header Field.
httpX-DeviceID CERT (PEN 6871) IE 274, variable length, DPI basicList
HTTP X-Device ID Header Field.
httpX-Profile CERT (PEN 6871) IE 275, variable length, DPI basicList
HTTP X-Profile Header Field.
httpLastModified CERT (PEN 6871) IE 276, variable length, DPI basicList
HTTP Last Modified Header Field.
httpContentEncoding CERT (PEN 6871) IE 277, variable length, DPI
basicList
HTTP Content Encoding Header Field.
httpContentLanguage CERT (PEN 6871) IE 278, variable length, DPI
basicList
HTTP Content Language Header Field.
httpContentLocation CERT (PEN 6871) IE 279, variable length, DPI
basicList
HTTP Content Location Header Field.
httpX-UA-Compatible CERT (PEN 6871) IE 280, variable length, DPI
basicList
HTTP X-UA-Compatible Header Field.
IMAP
IMAP Deep Packet Inspection is based on RFC 3501. The following
information elements are exported as a template in the
subTemplateMultiList as basicLists of variable length elements in the
order they are listed in the yafDPIRules.conf file. yaf will always
export at least 7 fields in the IMAP template and data record. By
default, yaf exports the following fields in order:
imapCapability CERT (PEN 6871) IE 136, variable length, DPI basicList
IMAP Capability Command and Response. Captures the listing of
capabilities that the server supports.
imapLogin CERT (PEN 6871) IE 137, variable length, DPI basicList
IMAP Login Command. Arguments are user name and password.
imapStartTLS CERT (PEN 6871) IE 138, variable length, DPI basicList
IMAP STARTTLS Command. Captures this command only as no arguments or
responses are related.
imapAuthenticate CERT (PEN 6871) IE 139, variable length, DPI basicList
IMAP Authenticate Command. Captures the authentication mechanism name
of the server following this command.
imapCommand CERT (PEN 6871) IE 140, variable length, DPI basicList
Captures a variety of IMAP Commands and their arguments.
imapExists CERT (PEN 6871) IE 141, variable length, DPI basicList
IMAP Exists Response. Reports the number of messages in the mailbox.
imapRecent CERT (PEN 6871) IE 142, variable length, DPI basicList
IMAP Recent Response. Reports the number of message with the Recent
flag set.
RTSP
Real Time Streaming Protocol (RTSP) Deep Packet Inspection is based on
RFC 2326. The following information elements are exported as a
template in the subTemplateMultiList as basicLists of variable length
elements in the order they are listed in the yafDPIRules.conf file.
yaf will always export at least 12 information elements in the RTSP
template and data record. By default, the following information
elements are exported in order:
rtspURL CERT (PEN 6871) IE 143, variable length, DPI basicList
RTSP URL. Captures the address of the network resources requested.
rtspVersion CERT (PEN 6871) IE 144, variable length, DPI basicList
RTSP Version Number.
rtspReturnCode CERT (PEN 6871) IE 145, variable length, DPI basicList
RTSP Status-Line. Captures the RTSP Protocol version, numeric status
code, and the textual phrase associated with the numeric code.
rtspContentLength CERT (PEN 6871) IE 146, variable length, DPI
basicList
RTSP Content-Length Header Field. Contains the length of the content
of the method.
rtspCommand CERT (PEN 6871) IE 147, variable length, DPI basicList
RTSP Command. Captures the method to be performed and the Request-
URI associated with the method.
rtspContentType CERT (PEN 6871) IE 148, variable length, DPI basicList
RTSP Content Type.
rtspTransport CERT (PEN 6871) IE 149, variable length, DPI basicList
RTSP Transport request header field. Captures the transport protocol
used and the parameters that follow.
rtspCSeq CERT (PEN 6871) IE 150, variable length, DPI basicList
RTSP CSeq field. Contains the sequence number for an RTSP request-
response pair.
rtspLocation CERT (PEN 6871)IE 151, variable length, DPI basicList
RTSP Location header field.
rtspPacketsReceived CERT (PEN 6871) IE 152, variable length, DPI
basicList
RTSP Packets Received header field.
rtspUserAgent CERT (PEN 6871) IE 153, variable length, DPI basicList
RTSP User Agent field. Contains information about the user agent
originating the request.
rtspJitter CERT (PEN 6871) IE 154, variable length, DPI basicList
RTSP Jitter Value.
SIP
Session Initiation Protocol (SIP) Deep Packet Inspection is based on
RFC 3261. The following information elements are exported as a
template in the subTemplateMultiList as basicLists of variable length
elements in the order listed in yafDPIRules.conf. yaf will always
export at least 7 information elements in the SIP template and data
record. By default, the following information elements are exported in
order:
sipInvite CERT (PEN 6871) IE 155, variable length, DPI basicList
SIP Invite Method. Contains the SIP address and SIP Version Number.
sipCommand CERT (PEN 6871) IE 156, variable length, DPI basicList
SIP Command. Contains a SIP Method, SIP address, and SIP Version
Number.
sipVia CERT (PEN 6871) IE 157, variable length, DPI basicList
SIP Via contains the SIP Version Number and the address the sender is
expecting to receive responses.
sipMaxForwards CERT (PEN 6871) IE 158, variable length, DPI basicList
SIP Max Forwards contains the limit of number of hops a request can
make on the way to its destination.
sipAddress CERT (PEN 6871) IE 159, variable length, DPI basicList
SIP Address contains the argument of the To, From, or Contact Header
Fields.
sipContentLength CERT (PEN 6871) IE 160, variable length, DPI basicList
SIP Content Length header field. Contains the byte count of the
message byte.
sipUserAgent CERT (PEN 6871) IE 161, variable length, DPI basicList
SIP User Agent Header Field. Contains information about the User
Agent Client originating the request.
SMTP
Simple Mail Transfer Protocol (SMTP) Deep Packet Inspection is based on
RFC 2821. The following information elements are exported as a
template in the subTemplateMultiList as basicLists of variable length
elements in the order they are listed in the yafDPIRules.conf file.
yaf will always export at least 11 information elements in the SMTP
template and data record. By default, the following information
elements are exported in order:
smtpHello CERT (PEN 6871) IE 162, variable length, DPI basicList
SMTP Hello or Extend Hello command. Captures the command and the
domain name of the SMTP client.
smtpFrom CERT (PEN 6871) IE 163, variable length, DPI basicList
SMTP Mail Command. Contains the reverse-path of the sender mailbox.
smtpTo CERT (PEN 6871) IE 164, variable length, DPI basicList
The SMTP Recipient (RCPT) Command. Captures the command and the
forward-path of the recipient of the mail data.
smtpContentType CERT (PEN 6871) IE 165, variable length, DPI basicList
SMTP Content Type Header Field.
smtpSubject CERT (PEN 6871) IE 166, variable length, DPI basicList
SMTP Subject. Contains the subject of the mail data.
smtpFilename CERT (PEN 6871) IE 167, variable length, DPI basicList
SMTP Filename. Contains the name of the file attached to the mail
message.
smtpContentDisposition CERT (PEN 6871) IE 168, variable length, DPI
basicList
SMTP Content-Disposition Header field.
smtpResponse CERT (PEN 6871) IE 169, variable length, DPI basicList
SMTP Replies. Consists of a three digit number followed by text.
smtpEnhanced CERT (PEN 6871) IE 170, variable length, DPI basicList
Enhanced SMTP. Contains the ESMTP command with the following
argument.
smtpSize CERT (PEN 6871) IE 222, variable length, DPI basicList
SMTP Size Header Field. Contains the size in bytes of the mail data.
smtpDate CERT (PEN 6871) IE 251, variable length, DPI basicList
SMTP Date Field. Added in version 2.3.
SSH
By default, yaf only exports 1 information element in the SSH template
and data record.
sshVersion CERT (PEN 6871) IE 171, variable length, DPI basicList
SSH Version Number
DNS
Domain Name System (DNS) Deep Packet Inspection is based on RFC 1035.
DNS Information is exported in the yaf subTemplateMultiList as a
subTemplateList of Resource Record Templates. Each resource record
entry contains generic resource record information such as type, TTL,
and name. There is also one element (subTemplateList) that contains
resource record specific information based on the type of resource
record (A Record vs NS Record, for example). The subTemplateList will
contain one entry for each resource record in the packet. Due to
alignment issues, the resource record specific element is the first
element in the template and is therefore the first item listed below.
DNSSEC information is not exported by default. To export DNSSEC
information, run yaf with --plugin-opts=DNSSEC. The following
information elements exist in the DNS resource record subTemplateList:
DNS Resource Record
The following elements (in order) are contained in the DNS Resource
Record Template.
subTemplateList IE 292, variable length
An IPFIX subTemplateList. This list contains a "DNS Resource Record
Type" Template. The type of this template depends on the type
(dnsQRType) of resource record. See the DNS Resource Record Types
listed below.
dnsQName CERT (PEN 6871) IE 179, variable length
A DNS Query or Response Name. This field corresponds with the QNAME
field in the DNS Question Section or the NAME field in the DNS
Resource Record Section.
dnsTTL CERT (PEN 6871) IE 199, 4 octets, unsigned
DNS Time To Live. This is an unsigned integer that specifies the
time interval, in seconds, that the resource record may be cached
for. This will contain a value of zero for DNS Queries.
dnsQRType CERT (PEN 6871) IE 175, 2 octets, unsigned
DNS Query/Response Type. This corresponds with the QTYPE field in
the DNS Question Section or the TYPE field in the DNS Resource
Record Section. This field determines the type of subTemplateList
found in this record.
dnsQueryResponse CERT (PEN 6871) IE 174, 1 octet, unsigned
DNS Query/Response header field. This corresponds with the DNS
header one bit field, QR. If the message is a query (0), or a
response (1).
dnsAuthoritative CERT (PEN 6871) IE 176, 1 octet, unsigned
DNS Authoritative header field. This corresponds with the DNS
header one bit field, AA. This bit is only valid in responses (when
dnsQueryResponse is 1), and specifies that the responding name
server is an authority for the domain name in the question section.
dnsNXDomain CERT (PEN 6871) IE 177, 1 octet, unsigned
DNS NXDomain or Response Code (RCODE). This corresponds with the
DNS RCODE header field. This field will be set to 3 for a Name
Error, 2 for a Server Failure, 1 for a Format Error, and 0 for No
Error. See http://www.iana.org/assignments/dns-parameters for other
valid values.
dnsRRSection CERT (PEN 6871) IE 178, 1 octet, unsigned
DNS Resource Record Section Field. This field will be set to 0 if
the information is from the Question Section, 1 for the Answer
Section, 2 for the Name Server Section, and 3 for the Additional
Section.
dnsID CERT (PEN 6871) IE 226, 2 octets, unsigned
DNS Transaction ID. This identifier is used by the requester to
match up replies to outstanding queries.
DNS Resource Record Types
· DNS A Resource Record
This entry will exist if dnsQRType is 1 and the A Record contains an
IP address.
sourceIPv4Address IE 8, 4 octets, unsigned
IPv4 address of the host.
· DNS NS Resource Record
This entry will exist if dnsQRType is 2 and the NS Record contains
an NSDNAME.
dnsNSDName CERT (PEN 6871) IE 183, variable length
An authoritative name server domain-name.
· DNS CNAME Resource Record
This entry will exist if dnsQRType is 5 and the CNAME Record
contains an CNAME.
dnsCName CERT (PEN 6871) IE 180, variable length
A domain-name which specificies the canonical or primary name for
the owner.
· DNS SOA Resource Record
This entry will exist if dnsQRType is 6 and the SOA Record contains
at least 1 of the following elements:
dnsSOAMName CERT (PEN 6871) IE 214, variable length
Corresponds to DNS SOA MNAME Field.
dnsSOARName CERT (PEN 6871) IE 215, variable length
Corresponds to DNS SOA RNAME Field.
dnsSOASerial CERT (PEN 6871) IE 209, 4 octets, unsigned
Corresponds to DNS SOA SERIAL Field.
dnsSOARefresh CERT (PEN 6871) IE 210, 4 octets, unsigned
Corresponds to DNS SOA REFRESH Field.
dnsSOARetry CERT (PEN 6871) IE 211, 4 octets, unsigned
Corresponds to DNS SOA RETRY Field.
dnsSOAExpire CERT (PEN 6871) IE 212, 4 octets, unsigned
Corresponds to DNS SOA EXPIRE Field.
dnsSOAMinimum CERT (PEN 6871) IE 213, 4 octets, unsigned
Corresponds to DNS SOA MINIMUM Field.
· DNS PTR Resource Record
This entry will exist if dnsQRType is set to 12 and PTRDNAME exists.
dnsPTRDName CERT (PEN 6871) IE 184, variable length
Corresponds to DNS PTR PTRDNAME Field.
· DNS MX Resource Record
This entry will exist if dnsQRType is set to 15 and MXExchange
exists
dnsMXExchange CERT (PEN 6871) IE 182, variable length
Corresponds to the DNS MX Exchange field.
dnsMXPreference CERT (PEN 6871) IE 181, 2 octets, unsigned
Corresponds to the DNS MX Preference field.
· DNS TXT Resource Record
This entry will exist if dnsQRType is set to 16 and TXT-DATA exists.
dnsTXTData CERT (PEN 6871) IE 208, variable length
Corresponds to DNS TXT TXT-DATA field.
· DNS AAAA Record
This entry will exist if dnsQRType is set to 28 and the IPv6 Address
exists. See RFC 3596.
sourceIPv6Address IE 27, 16 octets, unsigned
An IPv6 Address found in the data portion of an AAAA Resource
Record.
· DNS SRV Record
This entry will exist if dnsQRType is set to 33 and at least 1 of
the following elements exist. See RFC 2782.
dnsSRVTarget CERT (PEN 6871) IE 219, variable length
Corresponds to the Target Field in the DNS SRV Resource Record.
dnsSRVPriority CERT (PEN 6871) IE 216, 2 octets, unsigned
Corresponds to the Priority Field in the DNS SRV Resource Record.
dnsSRVWeight CERT (PEN 6871) IE 217, 2 octets, unsigned
Corresponds to the Weight Field in the DNS SRV Resource Record.
dnsSRVPort CERT (PEN 6871) IE 218, 2 octets, unsigned
Corresponds to the Port Field in the DNS SRV Resource Record.
· DNSSEC DNSKEY Record
This entry will exist if dnsQRType is set to 48 and at least 1 of
the following elements exist. See RFC 4034.
dnsPublicKey CERT (PEN 6871) IE 232, variable length
DNSSEC uses public key cryptography to sign and authenticate DNS
resource record sets. This field holds the public key. The format
depends on the algorithm of the key.
dnsFlags CERT (PEN 6871) IE 241, 2 octets, unsigned
The flags field in the DNSKey Resource Record. Certain bits
determine if the key is a zone key or should be used for a secure
entry point.
protocolIdentifier IE 4, 1 octet, unsigned
The protocol field in the DNSKEY RR. This should be 3 or treated
as invalid.
dnsAlgorithm CERT (PEN 6871) IE 227, 1 octet, unsigned
Identifies the public key's cryptographic algorithm, which
determines it's format.
· DNSSEC DS Record
This entry will exist if dnsQRType is set to 43, yaf was enabled to
export DNSSEC information, and at least 1 of the following elements
exist. See RFC 4034.
dnsDigest CERT (PEN 6871) IE 231, variable length
The digest of the DNSKEY RR.
dnsKeyTag CERT (PEN 6871) IE 228, 2 octets, unsigned
The Key Tag field in the DS RR.
dnsAlgorithm CERT (PEN 6871) IE 227, 2 octets, unsigned
The Algorithm number of the DNSKEY RR referred to by the DS Record.
dnsDigestType CERT (PEN 6871) IE 238, 1 octet, unsigned
The Digest Type field which identifes the algorithm used to
construct the digest.
· DNSSEC NSEC Record
This entry will exist if dnsQRType is set to 47, yaf was enabled to
export DNSSEC information, and the following field exists. See RFC
4034.
dnsHashData CERT (PEN 6871) IE 234, variable length
This item contains the Next Domain Name in the NSEC RR.
· DNSSEC NSEC3 or NSEC3PARAM Record
This entry will exist if dnsQRType is set to 50 or 51, yaf was
enabled to export DNSSEC information, and at least one of the
following fields exists. See RFC 5155.
dnsSalt CERT (PEN 6871) IE 233, variable length
The Salt Field in the DNSSEC NSEC3 or NSEC3PARAM RR.
dnsHashData CERT (PEN 6871) IE 234, variable length
The Next Hashed Owner Name in the DNSSEC NSEC3 RR. This will be
empty for NSEC3PARAM records.
dnsIterations CERT (PEN 6871) IE 235, 2 octets, unsigned
The Iterations field in the DNSSEC NSEC3 or NSEC3PARAM RR.
dnsAlgorithm CERT (PEN 6871) IE 227, 2 octets, unsigned
The Hash Algorithm field in the DNSSEC NSEC3 or NSEC3PARAM RR.
Values are described in RFC 5155.
· DNSSEC RRSIG Record
This entry will exist if dnsQRType is set to 46, yaf was enabled to
export DNSSEC information, and at least one of the following fields
exists. See RFC 4034.
dnsSigner CERT (PEN 6871) IE 229, variable length
The Signer's Name field in the RRSIG RR.
dnsSignature CERT (PEN 6871) IE 230, variable length
The Signature field in the RRSIG RR. Contains the cryptographic
signature that covers the dnsQName field.
dnsSignatureInception CERT (PEN 6871) IE 236, 4 octets, unsigned
The Signature Inception field in a RRSIG RR. The Expiration and
Inception fields specify a validity period for the signature.
dnsSignatureExpiration CERT (PEN 6871) IE 237, 4 octets, unsigned
The Signature Expiration field in a RRSIG RR. The Expiration and
Inception fields specify a validity period for the signature.
dnsTTL CERT (PEN 6871) IE 199, 4 octets, unsigned
The Original TTL Field in the RRSIG RR.
dnsKeyTag CERT (PEN 6871) IE 228, 2 octets, unsigned
The Key Tag field in a RRSIG RR.
dnsTypeCovered CERT (PEN 6871) IE 240, 2 octets, unsigned
The Type Covered field in a RRSIG RR.
dnsAlgorithm CERT (PEN 6871) IE 227, 1 octet, unsigned
The Algorithm Number field in a RRSIG RR. Identifies the algorithm
used to create the signature.
dnsLabels CERT (PEN 6871) IE 239, 1 octet, unsigned
The Labels field in a RRSIG RR. Specifies the number of labels in
the original RRSIG resource record owner name.
SSL/TLS
Secure Socket Layer (SSL)/Transport Layer Security (TLS) Deep Packet
Inspection can identify and export handshake and certificate
information if it is contained in the payload of the flow. Each
certificate identified by yaf is exported as an entry in the
subTemplateList field below. Each entry in the subTemplateList has
three nested subTemplateLists, one for issuer fields, one for subject
fields, and one for extension fields, along with other basic handshake
elements such as serial numbers and validity timestamps. Each of the
nested subTemplateLists contain an ID and a value. The IDs correspond
to the attributes associated with X.509 Certificates, object
identifiers id-ce and id-at.
sslCipher CERT (PEN 6871) IE 185, 4 octets, unsigned, DPI basicList
sslCipher is exported by yaf as a basicList that contains the list of
CipherSuites suggested by the client in the ClientHello Message.
sslServerCipher CERT (PEN 6871) IE 187, 4 octets, unsigned
sslServerCipher is the CipherSuite chosen by the server in the
ServerHello message.
sslClientVersion CERT (PEN 6871) IE 186, 1 octet, unsigned
sslClientVersion is the version it supports contained in the initial
ClientHello message.
sslCompressionMethod CERT (PEN 6871) IE 188, 1 octet, unsigned
sslCompressionMethod is the compression method chosen by the server
in the ServerHello message.
sslRecordVersion CERT (PEN 6871) IE 288, 2 octets, unsigned
sslRecordVersion is the version of ssl or tls that was used in the
flow.
subTemplateList IE 292, variable length
This contains 0 or more X.509 Certificates as available to yaf in the
captured payload. Note that most certificate chains are about 3000
bytes. In order to capture the entire certificate chain,
--max-payload should be set appropriately.
subTemplateList IE 292, variable length
The Issuer field identifies the entity that has signed and issued
the certificate. It is encoded as a sequence of Relative
Distinguished Names, which are basically type, value pairs. This
list will contains zero or more occurences of the
RelativeDistinguishedName id, value pairs pulled from the X.509
Certificate Issuer RDNSequence. There will be one entry in the list
for each pair. See below for a common list of attributes.
subTemplateList IE 292, variable length
The Subject field identifies the entity associated with the public
key stored in the subject public key field. It is encoded as a
sequence of Relative Distinguished Names, which are basically type,
value pairs. This list will contains zero or more occurences of
the RelativeDistinguishedName id, value pairs pulled from the X.509
Certificate Subject RDNSequence. There will be one entry in the
list for each pair. See below for a common list of attributes.
subTemplateList IE 292, variable length
Extensions are only defined for X.509 v3 certificates and provide
methods for associating additional attributes with the Issuer and
Subject information. Each extension includes an object identifier
and an ASN.1 structure. This list will contain zero or more
occurences of the object ids and ASN.1 values. yaf will not parse
the ASN.1 values for the string objects, it includes the entire
ASN.1 structure in the value field. However, it does not contain
the entire Extension ID. yaf only parses extensions that are
members of the id-ce arc and only exports information about the
following objects:
id-ce-subjectKeyIdentifier {id-ce 14}
id-ce-keyUsage {id-ce 15}
id-ce-privateKeyUsagePeriod {id-ce 16}
id-ce-subjectAltName {id-ce 17}
id-ce-issuerAltName {id-ce 18}
id-ce-certificateIssuer {id-ce 29}
id-ce-cRLDistributionPoints {id-ce 31}
id-ce-certificatePolicies {id-ce 32}
id-ce-authorityKeyIdentifier {id-ce 35}
id-ce-extKeyUsage {id-ce 37}
sslCertSignature CERT (PEN 6871) IE 190, variable length
The signature contained in a SSL certificate. This is typically the
hashing algorithm identifier.
sslCertSerialNumber CERT (PEN 6871) IE 244, variable length
The Serial Number from the X.509 certificate.
sslCertValidityNotBefore CERT (PEN 6871) IE 247, variable length
The notBefore field in the Validity Sequence of the X.509
Certificate.
sslCertValidityNotAfter CERT (PEN 6871) IE 248, variable length
The notAfter field in the Validity Sequence of the X.509
Certificate.
sslPublicKeyAlgorithm CERT (PEN 6871) IE 249, variable length
The algorithm, encoded in ASN.1, in the SubjectPublicKeyInfo
Sequence of the X.509 Certificate.
sslPublicKeyLength CERT (PEN 6871) IE 250, 2 octets, unsigned
The length of the public key in the X.509 Certificate.
sslCertVersion CERT (PEN 6871) IE 189, 1 octet, unsigned
The Certificate Version. This is the value contained in the
certificate v1(0), v2(1), v3(2).
sslCertificateHash CERT (PEN 6871) IE 295, variable length, optional
The hash of the X.509 certificate. This field is only populated if
the cert_hash_enabled is present and set to 1.
sslServerName, CERT (PEN 6871), IE 294, variable length
The server name from the SSL/TLS Client Hello. This is typically the
name of the server that the client is connecting to.
Issuer, Subject, and Extension Templates
Each subtemplateList for the above issuer, subject, and extension
sequences will contain zero or more entries of the below elements.
· sslObjectValue CERT (PEN 6871) IE 246, variable length
The bit strings associated with the below attribute types.
· sslObjectType CERT (PEN 6871) IE 245, 1 octet, unsigned
Above lists the extension types that yaf will export. For the
Issuer and Subject subTemplateLists, yaf only parses objects that
are members of the id-at arc {joint-iso-ccitt(2)ds(5) 4}, pkcs-9
{iso(1) member-body (2) us(840)rsadsi(113459)pkcs(1) 9}, and LDAP
dc 0.9.2342.19200300.100.1.25. This field will not contain the full
object identfier, it will just contain the member id. For example,
for an issuer common name, sslObjectType will contain 3. Below is a
list of common objects in an X.509 RelativeDistinguishedName
Sequence for X.509 Certificates:
pkcs-9-emailAddress {pkcs-9 1}
id-at-commonName {id-at 3}
id-at-countryName {id-at 6}
id-at-localityName {id-at 7}
id-at-stateOrProvinceName {id-at 8}
id-at-streetAddress {id-at 9}
id-at-organizationName {id-at 10}
id-at-organizationalUnitName {id-at 11}
id-at-title {id-at 12}
id-at-postalCode {id-at 17}
0.9.2342.19200300.100.1.25 {dc 25}
id-at-name {id-at 41}
Full Certificate Template
yaf will export the full X.509 certificate if the cert_export_enabled
variable is present and set to 1 in the configuration file. The
following information is exported as an extra entry in the
subTemplateMultiList as a basicList:
sslCertificate, CERT (PEN 6871) IE 296, variable length, DPI basicList
IRC
Internet Relay Chat (IRC) Deep Packet Inspection is based on RFC 2812.
The following information element is exported as a template in the
subTemplateMultiList as a basicList of variable length elements in the
following order:
ircTextMessage CERT (PEN 6871) IE 125, variable length, DPI basicList
IRC Chat or Join Message. This field contains any IRC Command and
the following arguments.
NNTP
Network News Transfer Protocol (NNTP) Deep Packet Inspection is based
on RFC 977. The following information elements are exported as a
template in the subTemplateMultiList in the following order:
nntpResponse CERT (PEN 6871) IE 172, variable length
NNTP Reply. This consists of a three digit status code and text
message.
nntpCommand CERT (PEN 6871) IE 173, variable length
NNTP Command. Contains an NNTP Command and following argument(s).
POP3
Post Office Protocol 3 (POP3) Deep Packet Inspection is based on RFC
1939. The following information element is exported as a template in
the subTemplateMultiList as a basicList of variable length elements:
pop3TextMessage CERT (PEN 6871) IE 124, variable length, DPI basicList
POP3 Command and Replies. Contains any command or reply message found
in POP3 payload data.
SLP
Service Location Protocol (SLP) Deep Packet Inspection is based on RFC
2608. The following information elements are exported as a template in
the subTemplateMultiList in the following order:
slpString CERT (PEN 6871) IE 130, variable length, DPI basicList
Contains the text elements found in an SLP Service Request.
slpVersion CERT (PEN 6871) IE 128, 1 octet, unsigned
SLP Version Number.
slpMessageType CERT (PEN 6871) IE 129, 1 octet, unsigned
SLP Message Type. This value should be between 1 and 11 and describes
the type of SLP message.
TFTP
Trivial File Transfer Protocol (TFTP) Deep Packet Inspection is based
on RFC 1350. The following information elements are exported as a
template in the subTemplateMultiList in the following order:
tftpFilename CERT (PEN 6871) IE 126, variable length
TFTP Name of File being transferred.
tftpMode CERT (PEN 6871) IE 127, variable length
Contains the mode of transfer. (Currently supported: netascii, octet,
mail).
MySQL
MySQL Deep Packet Inspection is based on information found at
http://forge.mysql.com/wiki/MySQL_Internals_ClientServer_Protocol.
MySQL packet capture information is exported in the yaf
subTemplateMultiList as a subTemplateList of Command Code, Command Text
pairs.
subTemplateList IE 292, variable length
An IPFIX SubTemplateList. This type represents a list of zero or
more instances of a structured data type, where the data type of each
list element is the same and corresponds with a single Template
Record. In this case, a list of MySQL Command Code, Command Text
Pairs. There will be one element in the list for each MySQL Command
found.
mysqlCommandText CERT (PEN 6871) IE 225, variable length
MySQL Command Text. For example, this can be a SELECT, INSERT,
DELETE statement. This is the first element in the MySQL
subTemplateList.
mysqlCommandCode CERT (PEN 6871) IE 224, 1 octet, unsigned
MySQL Command Code. This number should be between 0 and 28.
This is the second element in the above MySQL subTemplateList.
mysqlUsername CERT (PEN 6871) IE 223, variable length
MySQL Login User Name.
DNP3
Distributed Network Protocol (DNP3) Deep Packet Inspection is slightly
different than other plugin-based protocols. YAF will export the
following information if the yafDPIRules.conf contain regular
expressions with the label ID 20000. The regular expressions are
compared against the payload of DNP3 packets starting with the function
code in the DNP Application Layer header. YAF will loop through all
the the available DNP3 packets contained in the captured payload. For
each packet that matches one of the regular expressions listed in
yafDPIRules.conf, YAF will include an entry in the exported
subTemplateList. The subTemplateMultiList contains the following
information elements in the following order:
subTemplateList IE 292, variable length
An IPFIX SubTemplateList. This type represents a list of zero or more
instances of a structured data type, where the data type of each list
element is the same and corresponds with a single Template Record.
There will be one element in the list for each DNP3 packet that
matches one of the DNP3 regular expressions found in the
yafDPIRules.conf file.
dnp3SourceAddress CERT (PEN 6871) IE 281, 2 octets, unsigned
The DNP3 Source Address found in the Data Link Layer of the DNP
Header.
dnp3DestinationAddress CERT (PEN 6871) IE 282, 2 octets, unsigned
The DNP3 Destination Address found in the Data Link Layer of
the DNP Header.
dnp3Function CERT (PEN 6871) IE 283, 1 octet, unsigned
The DNP3 Function Code found in the first byte of the
Application Layer.
dnp3ObjectData CERT (PEN 6871) IE 284, variable length
The pattern captured from the DNP3 regular expression in
yafDPIRules.conf
Modbus
Modbus DPI is similar to DNP3 DPI. YAF will export any patterns
matched by the regular expressions labeled with the ID 502 found in the
yafDPIRules.conf file. The regular expressions are compared against
the payload of all valid Modbus packets starting right after the MBAP
header (offset 7), beginning with the Modbus function code. The
information is exported as variable length fields in a single
BasicList. All regular expressions for Modbus should use the label
502. No user-defined information elements will be accepted for Modbus.
modbusData CERT (PEN 6871) IE 285, variable length, DPI basicList
Any patterns captured from the Modbus regular expressions in
yafDPIRules.conf
Ethernet/IP
Ethernet/IP DPI is similar to Modbus DPI. YAF will export any patterns
matched by the regular expressions labeled with the ID 44818 in the
yafDPIRules.conf file. The regular expressions are compared against
the start of the payload of all valid Ethernet/IP packets (Command in
the Encapsulation Header is the first byte). The matched patterns are
exported as variable length fields in a single BasicList. All regular
expressions for Ethernet/IP should use the label 44818. No user-
defined information elements will be accepted for Ethernet/IP.
ethernetIPData CERT (PEN 6871), IE 286, variable length, DPI basicList
The pattern captured from the Ethernet/IP regular expressions in
yafDPIRules.conf
RTP
YAF will export the Payload Type in the Real-time Transport Protocol
(RTP) header if RTP DPI is enabled (yes by default). The Payload Type
indicates the format of the payload and how it should be interpreted by
the receiving application. The following two elements will be exported
for each flow labeled as RTP. If the flow is a uniflow, the reverse
element will be exported but will contain the value of 0.
rtpPayloadType CERT (PEN 6871), IE 287, 1 octet, unsigned
The payload type in the RTP header of the first payload in the
forward direction.
reverseRtpPayloadType CERT (PEN 6871), IE 288, 1 octet, unsigned
The payload type in the RTP header of the first payload in the
reverse direction.
AUTHORS
Emily Sarneso <ecoff@cert.org> and the CERT Network Situational
Awareness Group Engineering Team, http://www.cert.org/netsa
SEE ALSOyaf(1), yafscii(1), PCRE Documentation
2.8.0 19-Feb-2016 YAFDPI(1)
