Neds: Network Events Data Stream 9



Download 135.82 Kb.
Page1/2
Date conversion25.06.2018
Size135.82 Kb.
  1   2

NEDS: Network Events Data Stream v0.9

A Log Format for Network Traffic Reporting


Overview

This format is a logging format for reporting events related to network traffic. Each record starts with an Event Identifier that specifies the format for the rest of record. The format for event records is fixed and versioned.


The overall format is Comma Separate Values (http://en.wikipedia.org/wiki/Comma-separated_values). If a string value has a double quote (“) in it, it should be escaped as two double quotes (“”). There should be flexibility in handling record formats. Certain transports or logging systems may prefix the records with additional information such as a timestamp. This is outside of this specification. Consumers of these records should ignore anything that appears before the “neds.f5” or equivalent prefix for field 1.
String fields should be limited to a maximum length. This should be configured to the same number for the consumer and producer. A default maximum of 256 characters is suggested. If Unicode text is transported, it should be encoded using UTF8.
Example:

"neds.f5.conn.end.v1","1.1.1.1:53-1.1.1.1:80@1255047800.15",1255047810.47,5,5,1040,621


Note that this format is being defined with proxy semantics in mind. That means that this format tries to address the fact that network connections often go through a proxy or load balancer on their way to a server. In this case, there is a client side connection and a server side connection. There may be significant differences between these connections.

Consider that in a cacheing scenario, there may not even be a server side connection. In order to get a full picture of network traffic, network proxies need to report traffic handled directly as well as traffic passed on to other servers.

For example, the proxy may compress data on the way to the client and so the client side will have less packets and bytes transmitted. Vendors defining NEDS formats should be clear about whether information is being described for the client side connections or server side connections. Generally the client side is the “primary” connection to be reported on. Nevertheless, some of the fields below may refer to the “server side of the proxy”. Generally these fields may be blank in the case where the connection is handled by the network equipment itself.
Syslog is a valid transport for this format. Further transports may be defined that specify compression or other features.
Note this specification is in “beta” testing and should be considered subject to change until it reaches version 1.0.

Field 1: Event Identifier

The format for this field is "neds.Vendor.Context.Event.Version".

Also, this format is supported: "neds.Vendor.Context.Event.Version.Format".

neds is the literal string ‘neds’.

Vendor is a short string (e.g. ‘f5’).

Context is the protocol or area of interest (conn,http,ssl,etc)

Event is a short string.

Format is a short string (e.g. ‘AES+base64’)

Versions are specific to event and must be incremented when it changes. Only the vendor responsible for originating a version should define a subsequent version. Vendors should add fields to the end of a format so that consumers of earlier formats can try to consume the new format. New versions which are not compatible with early versions in this way should not be introduced without a long warning ahead of time (by publishing the format).

Example: neds.f5.conn.start.v1

The vendor is 'f5' networks.

The context is the connection level.

The event is 'start'.

The version is 'v1'.

Connection Events




neds.f5.conn.start.v1


The client connection has been established.
Field 2:

Name: Device

Format: String

Description: Identifies the device handling the flow

Examples: “mybigip.test.net”

Field 3:


Name: Flow

Format: "ClientIP:ClientPort-ServerIP:ServerPort@DateTimeSecs"

Description: These are the addresses and ports of the connection on

the client side in the case of a proxy.

Field 4:

Name: DateTimeSecs

Format: Floating Point Unix Time

Description: http://en.wikipedia.org/wiki/Unix_time

Field 5:

Name: Ingress Interface/VLAN

Format: Any short string

Description: Identifies the ingress Interface or VLAN

Examples: “eth0”, “vlan32”, “external”, “internal”, “myvlan”, “4023”

Field 6:


Name: Protocol

Format: Integer

Description: IP Protocol from the IP header.

Field 7:


Name: DiffServ

Format: Integer

Description: Differentiated Services. From the IP header.

Field 8:


Name: TTL

Format: Integer

Description: Time to Live, from the IP header.

Field 9:

Name: PolicyName

Format: String

Description: The name of the virtual server or traffic policy

that intercepted the connection. Although this should always

be present for F5 traffic, consumers of this format should accept

an empty string here.

Field 10:

Name: Direction

Format: String

Description: A short string that describes the directional nature of the connection, such as whether it is ‘Inbound’ or ‘Outbound’. This may be blank, which should be considered unknown. Other strings may be configured on a site by site basis (e.g. ‘DMZ’). Users should check with their vendor for support before using arbitrary strings here.


neds.F5.conn.end.v1


The client connection has been closed.
Field 2:

Name: Device

Format: String

Description: Identifies the device handling the flow

Examples: “mybigip.test.net”

Field 3:


Name: Flow

Format: "ClientIP:ClientPort-ServerIP:ServerPort@DateTimeSecs"

Field 4:

Name: DateTimeSecs

Format: Floating Point Unix Time

Description: http://en.wikipedia.org/wiki/Unix_time

Field 5:

Name: PktsIn

Format: Integer

Description: Total packets in on the client side of proxy

Field 6:

Name: PktsOut

Format: Integer

Description: Total packets out on the client side of proxy

Field 7:

Name: BytesIn

Format: Integer

Description: Total bytes in on the client side of proxy

Field 8:

Name: BytesOut

Format: Integer

Description: Total bytes out on the client side of proxy






  1   2


The database is protected by copyright ©hestories.info 2017
send message

    Main page