TU Delft
The MultiProbe Framework
Parallel and Distributed Systems
EWI PDS MultiProbeThe MultiProbe Framework
Quick links
overview pre-processing measurement post-processing
    implementation references

An overview of the MultiProbe Framework
what is MultiProbe?

MultiProbe is a framework for large-scale underlay/overlay network measurements that has been designed, implemented, and deployed by the MultiProbe team in the Parallel and Distributed Systems group of the Faculty of Electrical Engineering, Mathematics, and Computer Science of the Delft University of Technology.

The main feature of MultiProbe is that is allows performing joint measurements of large-scale overlay networks and their underlying Internet parts. For instance, MultiProbe has been successfully used in a large-scale measurement of BitTorrent, currently the dominant P2P file sharing network in terms of traffic, and its underlying Internet parts. With its post-processing modules, the MultiProbe framework can corroborate the measurement results and characterize real users of the P2P file-sharing networks.

MultiProbe consists of pre-processing, measurement, and post-processing modules. Figure MP-1 depicts the MultiProbe framework, and its inter-connections with the BitTorrent network, during the measurement process.

The MultiProbe framework
Figure MP-1. The MultiProbe framework.

The pre-processing modules
brief description
The BitTorrent network uses web sites as content directories. The largest of these sites serve millions of users. MultiProbe can select the largest such web site at the start of the experiments (the SiteStats module, see Figure MP-1, bullet 1). The web site selection module is linked with the measurements setup module, in which the top files from the largest web site are selected, and their respective torrents are downloaded. The files are sorted according to the number of users downloading or offering chunks of these files (swarm size).


The measurement modules
brief description

For our measurements, we inserted monitoring nodes (probes) into the BitTorrent network. We define two types of measurements: active-start measurements, in which probes initiate the contact with BitTorrent peers, and passive-start measurements, in which probes wait for an externally initiated contact from BitTorrent peers. The need for the second approach is twofold. First, some measurement platforms limit the number of contacts a user can initiate during a fixed period of time. Second, the first approach does not allow to contact peers behind firewalls, while the second can overcome this limitation.

In active-start measurements, probes contact BitTorrent trackers repeatedly, acquiring lists of peer contacts (the GetPeers module, see Figure MP-1, bullet 2). They subsequently contact the corresponding peers using the BitTorrent protocol, and learn about what chunks are owned by these peers (the PingPeers module, see Figure MP-1, bullet 3). In passive-start measurements, probes register themselves to a tracker, and wait to be contacted by other peers (the ListenPeers module, see Figure MP-1, bullet 4). Discovery packets from multiple sources within the measurement infrastructure are sent to the peers discovered in this way, to characterize the Internet parts that are used by BitTorrent (the TrackPeers module, see Figure MP-1, bullet 5). We call this process multi-source traceroute.


The post-processing modules
brief description
Finally, MultiProbe has a wide-range of post-processing modules, which can process and correlate detailed information about both the overlay network and the Internet characteristics. Currently, MultiProbe can analyze for all data and per file: geographical and geopolitical location, RTT, IP path hop count, AS and ISP mapping, port traffic break-down, application-level bandwidth, application connectivity, and various time patterns.


brief description
We have implemented the MultiProbe framework in
Python [1], Ruby [2]}, and C. We used Python for the measurement scripts, except for the multi-source traceroute, which was implemented in Ruby, due to deployment restrictions. For the ListenPeers module, we adapted an open-source Python implementation of a BitTorrent client. We used Python and C for pre- and post-processing.


we refer to these links for detailed information
[1] Python official web site. http://www.python.org.
[2] Ruby official web site. http://www.ruby-lang.org.


Last modified: Thu, 13 October, 2005 6:34 PM
The newest version of this page can be found at: http://multiprobe.ewi.tudelft.nl/multiprobe.html
Copyright © 1998-2005 Alexandru Iosup. All Rights Reserved.
And the famous NedStat counter: