How to Connect your Multi-homed Linux machine to the Internet via a Bridge

Lawrence E. Widman
April 2, 2000

Abstract

We could not get aliased IP addresses to work properly over an ADSL line and a machine running Linux 2.2.13. The problem appeared to be that ARP requests for the aliased IP addresses were dropped, and the solution appeared to be adding an additional aliased IP address that puts the machine and the Alcatel ADSL modem on the same subnet. After this solution was implemented, the previously persistent problem could not be recreated, so the validity of the solution is not completely certain. But, the aliased IP addresses now work fine.

Background

We installed an ADSL line with a series of static IP addresses, say x.x.x.32-39/24. The ADSL modem was an Alcatel Speed Touch "Home" model, with default IP address 10.0.0.138/8. A single machine running Linux 2.2.13 was configured to respond to all eight of the static addresses. This machine had a second ethernet card that connected to an internal network with IP addresses in the 192.168.0.0/16 subnetwork. All of the static IP addresses could be pinged successfully from the internal network.

The Problem

Only one of the eight static IP addresses could be pinged from other hosts on the Internet. Any of the eight would work, but only one at a time. The one that worked was whichever was the primary IP address at a given time. Programs other than ping also could not reach the other addresses.

Investigation and Analysis

It was clear that the problem arose at the level of ARP, which lets the remote host know the MAC address of the multihomed machine for each of the 8 static addresses. When the ethernet interface was shut down and retarted with each of the addresses in turn as the primary address, outside hosts could ping up to two of the addresses at a time. This suggested that the MAC address was somehow published in a permanent way when the primary IP address was activated, but not when the aliased IP addresses were activated.

Also, this finding suggested that the cache entry that retained the MAC address had a lifetime of less than 5 minutes. This was important because the Alcatel ADSL modem has a default cache entry lifetime of 5 minutes. Extending this lifetime to several hours did not change the results of cycling the primary IP address of the machine among the eight static IP addresses.

A call to the ISP confirmed that packets addressed to each of the static IP addresses were in fact being routed to the ADSL modem.

The ARP cache on the machine was augmented with the aliased IP addresses with

arp -Ds x.x.x.y eth1 pub

This did not allow remote pings to succeed.

The user-space ARP daemon (tarpd-1.6.tar.gz) was installed and configured to return the MAC address of both the internal and external ethernet cards. It was modified to print a message when it responded to an ARP request. Pings from the internal network worked fine, demonstrating correct behavior by the program, but pings from the Internet did not elicit a response.

tcpdump on the external network card showed that the machine was issuing ARP requests to the upstream router every 5 minutes, which may have accounted for the fact that the primary IP addresses always worked properly.

For a while, we could force the machine's MAC address to be associated with each aliased IP address by running a modified version ping that allowed the source IP address to be set in turn to each of the aliased IP addresses. This worked as long as we reran ping every 5 min or so, but stopped working when, for unclear reasons, this version of ping could not reach the ADSL modem with source addresses set to the aliased IP addresses.

These findings suggested that in general the problem arose somewhere between the ADSL modem and the multihomed machine, and in particular that ARP requests from the upstream router were not being answered for aliased static IP addresses because the machine was not receiving them from the modem. A search of the Web with www.google.com revealed nothing helpful.

The Solution (?)

The Alcatel Speed Touch Home ADSL modem performs bridging but not routing service. The manual does not provide much technical detail and, further, states that certain details of operation are "invisible" to the user.

We noted that the modem has a default IP address of 10.0.0.138. According to the manual, the modem passes through all packets from one port to the other. The fact that it has a unique IP address did not appear relevant because the modem's IP address did not prevent it from allowing bidirectional connections between our machine and the Internet.

We hypothesized that, in the case of ARP, the modem might behave differently in two respects:

The modem's ARP cache might associate only one IP address with an MAC address. This address might be the machine's primary IP address, and might be reset whenever the machine's network interface was shut down and restarted.
If so, then the upstream router would be keeping the MAC addresses of the aliased IP addresses. That router might have a short cache lifetime, which would explain the observation that cycling the primary IP address of the machine worked only for a short period of time.
The modem might broadcast ARP requests only on the 10.0.0.0/8 subnetwork. When the router issued an ARP request to the ADSL modem, the modem would respond from its cache only for the one MAC address it kept, which would be the machine's primary IP address. It would generate ARP requests for the aliased IP addresses. These would be broadcast only to the 10.0.0.0/8 subnetwork, and would not be received by the machine because it was not on that network.

To test this hypothesis, we added an alias IP address on the 10.0.0.0/8 subnetwork to the machine's external ethernet card. This had no immediate effect. As it was 2 AM, we then stopped work. A cron job tested the visibility of the aliased IP addresses every 5 minutes. At about 1035 AM, we ran tcpdump. We immediately saw ARP requests from the upstream router being answered correctly for each of the eight static IP addresses. Examination of the log of the cron job showed that the seven aliased IP addresses had not worked at all until 1035 AM, and worked perfectly after that.

Conclusions

The simple solution is that the ADSL modem's ARP requests can be received and answered only by a machine that is on its network (in our case, on the 10.0.0.0/8 subnetwork). For this to be true, we would have to assume that the upstream router issues ARP requests infrequently (less than every 8 hours), and that we happened to run tcpdump at one of those infrequent times. Our only problem with this solution is the delay between when we put the machine on the modem's subnetwork and when the aliased IP addresses became visible to the Internet.

An alternate solution would be that the simple solution is not correct, in which case we don't know why the aliased IP address are now working. We took down the machine's 10.0.0.0/8 address for several days: all 7 aliased IP addresses continued to be visible from another machine on the Internet that checked every two hours. We shut down the machine, powered off the router, then brought up everything the way it was, and also restored the 10.0.0.0/8 address. Everything worked fine. Basically, we have not been able to recreate the original problem, even though the original problem lasted for weeks until we implemented the fix above. So, if you have the same problem, try making sure that the ADSL modem and the machine that answers ARP requests are on the same subnetwork. It may solve your problem. If so, please let us know.