Enabling Large-scale Simulations: Selective Abstraction Approach to the Study of Multicast Protocols

Summary. ns simulation does not scale with respect to time or memory as the simulation scenario grows large. The authors demonstrate how details can be abstracted in certain situations to allow orders of magnitude increase in scenario size. It is critical to understand, howver, that what abstractions can be done and what effects they'll have on results is scenario specific.

More Detail

• Centralized multicast (the concept can be applied to unicast routing protocols, also) whereby the join/leave/update messages in DVMRP (the multicast protocol they chose to work with) are eliminated and the multicast tree is calculated centrally by the simultor. The effect on results is that transient effects of join/leave/link failure (i.e., anything that causes the multicast tree to be recalculated) are not faithfully reproduced. If transient effects are not important, however, this abstraction is recommended.

• Abstract packet distribution abstracts away the details of ns packet distribution which include several components: sending interface, queue, delay, TTL and interface again. While powerful and flexible, the processing can become lengthy as scenarios grow large. With the exception of the queuing discipline, the effects of all links between a source and receiver are are rolled into a single replication/delay/TTL path. The effect is that queuing cannot be studied so when cross-traffic or high-load sources are necessary in a simulation, abstract packet distribution is not recommended.

The authors present the following steps to help users conduct their own abstraction techniques:

1. Define scaling factors/dimensions (varying input) and measurement metrics (output)
2. Start from small-scale detailed simulations.
3. Profile simulations to find bottleneck.
4. Adapt techniques to avoid simulation bottleneck.
5. Verify simulations in small-scale in detailed mode.
6. If the differences between the two versions is not crucial to the research question, simulate larger-scale sims using the improved version.

The authors did a case study on SRM using both abstraction types. Though the studies' resource consumption did decrease, the shape of the curve did not change.

• Both of the above abstractions can be used in end-to-end experiments.
• How does this relate to the testbed?
  • See testbed mail archive for a more concise summary.
  • We're interested in ns for several reasons:
    1. We want to complement ns's capabilities.
    2. We are somewhat interested in ns for the delay module.
    3. We'd like to make our scripting language ns compatible.
  • About 1), ns apparently can scale in a multicast exp (remember: resource consumption is going to vary between simulations) to the several hundreds of nodes BUT even at several hundred, the simulation time is (300 nodes took 2.76 hours).These abstractions allow ns to scale to 500 -- not an order of magnitude difference, but better.

    This tells me that ns can potentially kick our scalability backside in all experiments and especially when they can apply some of these abstraction techniques. So we can't play the scalability card; rather, we have to emphasize our other offerings:

    • If an implementation already exists, no need to reimplement in the simulator.
    • More realistic measurements due to actually having to traverse interrupts, etc.. (this is yet to be proven, i know).
    • Whatever else.

  • About 2): given the numbers in this paper, ns might work just fine as delay-node software, but this might "contaminate" the "realness" of the testbed by introducing false delays. On the upside, ns could be an easy first implementation. Experimentation designed to characterize ns' overhead is in order if we want to do this (note that this is sligtly different than chris' suggestion to see how scalable ns is).
  • (3) isn't relevant to this paper.