Mahimahi
A set of lightweight tools for browser developers, website authors, and network protocol designers that provides accurate measurements when recording and replaying HTTP content over emulated network conditions.
Mahimahi is free software and is available on Ubuntu (version 14.04 or higher).
Record and replay HTTP traffic
Using Mahimahi's RecordShell, you can completely record and store real HTTP content (such as Web pages) to disk. Recorded sites can then be repeatably replayed using ReplayShell. ReplayShell accurately models a Web application's multi-server structure by locally mirroring a page's server distribution.
Emulate numerous network conditions
Mahimahi's network emulation tools can be used to emulate many different link conditions for replaying recorded HTTP traffic. Mahimahi supports emulating fixed propagation delays (DelayShell), fixed and variable link rates (LinkShell), stochastic packet loss (LossShell). LinkShell also supports a variety of queueing disciplines such as DropTail, DropHead, and active queue management schemes like CoDel. Each of Mahimahi's network emulation tools can be arbitrarily nested within one another providing more experimental flexibility.
Use unmodified client applications
Recording and replaying sites can be done using commercial browsers such as Google Chrome, Mozilla Firefox, and Safari. Additionally, other client applications including virtual machines and mobile-device emulators can be run unmodified within each of Mahimahi's tools.
Getting Mahimahi
Ubuntu 14.04 and later
$ sudo add-apt-repository ppa:keithw/mahimahi $ sudo apt-get update $ sudo apt-get install mahimahi
Operating system logos are trademarks or registered trademarks and are displayed for identification only.
The vendors shown aren't affiliated with and haven't endorsed Mahimahi.
Building from source
$ git clone https://github.com/ravinet/mahimahi $ cd mahimahi $ ./autogen.sh $ ./configure $ make $ sudo make install
Dependencies
| Name | Typical package |
|---|---|
| Protocol Buffers | protobuf-compiler, libprotobuf-dev |
| autotools | autotools-dev |
| autoreconf | dh-autoreconf |
| iptables | iptables |
| pkg-config | pkg-config |
| dnsmasq | dnsmasq-base |
| Apache2 | apache2-bin |
| debhelper (>= 9) | debhelper |
| OpenSSL | libssl-dev, ssl-cert |
| xcb | libxcb-present-dev |
| Cairo | libcairo2-dev |
| Pango | libpango1.0-dev |
Details
Each mahimahi tool spawns a lightweight container, generally connected to the outside via a synthetic network device that observes packets in transit or emulates a desired behavior.
Mahimahi's tools are composable so that a series of emulated network effects can be chained together, with mahimahi containers nested inside each other. Each tool takes an optional command to execute, so it is possible to create a series of nested containers with one command line.
Network Emulation Tools
DelayShell
Every packet entering or leaving DelayShell's container is delayed by the specified one-way delay (in milliseconds). This technique enforces a fixed delay on a per-packet basis.
LinkShell
LinkShell is a network emulation tool that emulates links using packet delivery trace files. An uplink trace file emulates the link from LinkShell's container to the Internet and a downlink trace file emulates the link from the Internet to LinkShell's container.
LinkShell can emulate both time-varying links, such as cellular links, and links with fixed link speeds. When a packet arrives at the link (from either the Internet or from LinkShell), it is directly placed into one of two packet queues depending on its intended direction: the uplink queue or the downlink queue. LinkShell releases packets from each queue based on the corresponding input packet-delivery trace.
Each line in a trace file represents a packet delivery opportunity: the time at which an MTU-sized packet can be delivered in the emulation. Accounting is done at the byte-level, and each delivery opportunity represents the ability to deliver 1500 bytes. Thus, a single line in the trace file can delivery several smaller packets whose sizes sum to 1500 bytes. Delivery opportunities are wasted if bytes are unavailable at the instant of an opportunity. When LinkShell reaches the end of an input trace file, it wraps around to the beginning of the trace file. LinkShell can be nested within DelayShell to flexibly create links with a user-supplied one-way delay and a user-supplied link rate.
LinkShell supports live graphing to visualize a process's use of the network. Supported plots include the transfer rate as well as the per-packet queueing delay for traffic leaving (uplink) and entering (downlink) LinkShell's container.
LossShell
Packets are lost at a given rate (between 0 and 1) both when leaving (uplink) and entering (downlink) the LossShell's container.
HTTP Record-and-Replay Tools
RecordShell
RecordShell transparently proxies outgoing HTTP and HTTPS connections, saving the requests, corresponding responses, and IP address of each Web server contacted in the given directory. RecordShell uses a self-signed TLS certificate in its HTTPS proxy, causing typical Web browsers to reject it. For testing or debugging purposes, this behavior can usually be turned off, e.g.: with the --no-check-certificate option to wget or the --ignore-certificate-errors option to the Chromium Browser.
ReplayShell
Replays a saved session from a previous run of RecordShell. Unlike most mahimahi tools, the ReplayShell container does not have a network connection to the outside world. Instead, it has dummy network interfaces bound to each IP address on which a Web server in the saved session had answered a request. ReplayShell runs an Apache2 Web server bound to each such IP address inside the container. Each Web server emulates the corresponding server from the saved session. When receiving a request that matches one in the directory, the corresponding Apache2 server replies with the same reply as previously captured.
ReplayShell can be used to measure the performance of Web browsers on complex websites and the effect of changes in Web protocols (e.g. HTTP, HTTP/2, SPDY, QUIC). Unlike tools like web-page-replay, ReplayShell preserves the sharded structure of a website, binds to the actual IP addresses that the real website used, and serves requests from real Web servers.
Technical Paper
The Mahimahi research paper, which provides more details about the design and evaluation of Mahimahi, was presented at the 2015 USENIX Annual Technical Conference.
Example Commands
To spawn a shell with a delayed, lossy link to the Internet:
mm-delay 50 mm-loss uplink 0.2
Running this command will result in a shell prompt of "[delay 50 ms] [loss up=0.1]" and all programs run inside this environment will have a 100 ms minimum RTT and the emulated link has a loss rate of 20%.
To record a page load from www.nytimes.com in Chromium:
mm-webrecord /tmp/nytimes chromium-browser --ignore-certificate-errors --user-data-dir=/tmp/nonexistent$(date +%s%N) www.nytimes.com
The use of "--user-data-dir=/tmp/no