Estimating flow distributions from sampled flow statistics. - Duffield, Nick
Passive traffic measurement increasingly employs sampling at the packet level. Many high-end routers form flow statistics from a sampled substream of packets. Sampling controls the consumption of resources by the measurement operations. However, knowledge of the statistics of flows in the unsampled stream remains useful, for understanding both characteristics of source traffic, and consumption of resources in the network. This paper provides methods that use flow statistics formed from sampled packet stream to infer the frequencies of the number of packets per flow in the unsampled stream. A key task is to infer the properties of flows of original traffic that evaded sampling altogether. We achieve this through statistical inference, and by exploiting protocol level detail reported in flow records. We investigate the impact on our results of different versions of packet sampling.
Keywords: Computer systems organization; Computer communication networks; Network operations; Network monitoring; Data communications; Network architecture and design; Network communications; Network protocols; TCP IP; Network management; Performance of systems; Measurement techniques; Modeling techniques; Mathematics of computing; Probability and statistics; Statistical computing; IP flows; Maximum likelihood estimation; Measurement; Measurement errors; Packet sampling; Sampling methods