Changes from Version 1 of MetricStatistics

Author:

trac (IP: 127.0.0.1)

Timestamp:

06/14/07 15:50:47 (3 years ago)

Comment:

--

MetricStatistics

@@ +1 @@
+= Traffic Feature Statistics =
+
+Once an alarm is triggered in an intrusion detection system which is monitoring metrics, one of the quickest ways to determine the source of the alarm is by examining the ''top-k'' of a metric.  This is the top hosts, ports, flow size distributions, etc... for a given interval.  In the simplest case, when an alarm is generated by traffic volume the ''top-k'' addresses can be examined to see which host generated the most amount of traffic.  If no host stands out as generating a large amount of traffic individual, it could be typical of a DDoS attack and the ''top-k'' for degree in should be checked.  If it is a DDoS attack one could find a new host introduced in the top hosts for in degree which could be the host being attacked. To examine these features of each metric, we provide ordered results by rank for each.  
+
+== Methods ==
+
+Each of the following methods, which are defined in [source:scripts/ruby/include/sql_queries.rb sql_queries.rb], returns a sorted array indexed by rank descending in terms of value.  At each index is an array whose index and value varies for each method:
+
+  * stats_addr_degree_in(timestamp,table) - result indexed by ''host address'' with values being the associated ''in degree''
+
+  * stats_addr_degree_out(timestamp,table) - result indexed by ''host address'' with values being the associated ''out degree''
+
+  * stats_degree_in(timestamp,table) - result indexed by ''in degree'' with values being the ''number of hosts'' with the indexed degree
+
+  * stats_degree_out(timestamp,table) - result indexed by ''out degree'' with values being the ''number of hosts'' with the indexed degree
+
+  * stats_addr_src(timestamp,table) - result indexed by ''host address'' with values being the associated ''source packets''
+
+  * stats_addr_dst(timestamp,table) - result indexed by ''host address'' with values being the associated ''destination packets''
+
+  * stats_ports_src(timestamp,table) - result indexed by ''port'' with values being the associated ''source packets''
+
+  * stats_ports_dst(timestamp,table) - result indexed by ''port'' with values being the associated ''destination packets''
+
+  * stats_fsd(timestamp,table) - result indexed by ''flow size distribution'' with values being the ''number of flows'' with the given flow size distribution
+
+== Example Usage ==
+
+''Display the top 3 hosts in terms of degree in.'' ie. host ''191102'' had ''5393'' unique hosts contact it within the 2005-02-01 00:00:00 interval
+{{{
+irb> stats_addr_degree_in("2005-02-01 00:00:00","flows").first(3).each {|host,degree_in| puts "#{host} #{degree_in}"}
+191102 5393
+16085133 3855
+180751 3309
+}}}
+
+''How many unique hosts did 191102 contact?'' hmmm... could he be a server? ;) 
+{{{
+irb> stats_addr_degree_out("2005-02-01 00:00:00","flows").assoc(191102).last
+=> 62
+}}}
+
+''Get the number of hosts who only contacted 3 other unique hosts.''  Since the association result is an array with index being the degree and value being the number of hosts, and we only want the number of hosts, we use ''.last''
+{{{
+irb> stats_degree_out("2005-02-01 00:00:00","flows").assoc(3).last
+=> 989
+}}}
+
+''How many packets were sourced from port 80?''
+{{{
+irb> stats_ports_src("2005-02-01 00:00:00","flows").assoc(80).last
+=> 1769099
+}}}
+
+== Statistic Scripts ==
+
+
+'''[source:scripts/ruby/dp_metric_stats.rb dp_metric_stats.rb]'''
+   * ''Usage'': ./dp_metric_stats.rb
+   * ''Description'': returns the complete ranked list for a given [wiki:TrafficFeatures traffic feature] and interval.
+   * ''Output Format'':  dependent on metric, follow result formats for each method listed above
+   * ''Example usage'':
+   {{{
+$ ./dp_metric_stats.rb "2005-02-01 00:00:00" addr_src | head -n 5
+134619 1564472
+189143 1464679
+173376 1090653
+180487 915830
+133104 848064
+   }}}