4 Steps to Finding a Mysterious Server

Few network administrators would argue that having a complete and always-current network map is an incredibly valuable and important part of their job. After all, network maps are an essential element of planning, implementing, and supporting every network.

But stories abound of lost computers, servers that cannot be physically located but can be reached on the network, printers that only exist on a spreadsheet, etc. I’ve personally encountered several varieties of this happening, so I can state with confidence that computers do, in fact, get lost. The impact of this loss could be minor if the computer is not in demand, but could also prove devastating if the computer is infected with a virus or is constantly overheating.

Do You Really Need to Find It?

Finding the missing server or computer can be quite a challenge; especially if you don’t have complete information about the system in question.

Usually at least an outdated network map from a previous administrator is available. This piece of data is similar to a treasure map drawn by a drunken pirate. It’ll get you in the neighborhood, but you will need to do research to pinpoint the treasure. And if the pirate was really drunk the map may be more of a distractor. Likewise a very out-of date network map might take you in a completely wrong direction.

Many administrators question whether they even need to find a missing system. For every reason to ignore the problem, there are a dozen that compel them to find it. You read a few of the most common reasons earlier, but the number of reasons to find the system is nearly infinite. Regulatory compliance, auditing, power conservation, hardware maintenance, fire hazard, stock exchange investigation, law enforcement investigation… the list goes on. Being unable to find a system is not acceptable.

The Process

You want to use a simple yet effective process to find missing computers using increasingly powerful tools. This approach serves well to uncover easily-found computers quickly and yet is deep enough to ferret out even the most completely lost systems. The process I illustrate here assumes that the system is powered up and responding to network traffic, which is most often the case.

The process is:

  1. Obtain the IP address of the system
  2. Scan the IP address range of the system’s subnet
  3. Identify the router port nearest the system that the system is behind
  4. Follow the cable

I’ll explain each step of the process.

Step #1: Obtain the IP Address of the System

This should be a no-brainer for most administrators. A simple ping or traceroute command from any modern operating system or network switch will resolve the system’s IP address. You should also obtain the media access control (MAC) address while you’re looking at this data. You can do this by following either the ping or traceroute with the ARP -a command, which shows all addresses in the address resolution protocol (ARP) cache. There will be a physical address that corresponds to the internet address of the lost system. Note both of these.

For use in our example lost system, let’s assume the IP address is 192.168.1.10.

Step #1½: Start a Network Map

It is never too late to begin creating or refreshing a network map. Just because you don’t have one, or it is uselessly old, doesn’t mean you can’t begin to build one now. In fact you can kick one off and let it run as you hunt for the missing system. While it might not help you find the system you’re looking for right now, it will certainly help in the future.

There are a number of powerful tools available in today’s software market to identify systems and create a thorough network map. Most of them are automated. You simply need to kick off the process and let them run. They may take hours, days, or even weeks to completely analyze a complex network and provide a comprehensive database. But once that database is built, you are in a much better position to resolve lost system issues in the future. So there’s no time to lose.

Step #2: Scan the IP Address Range of the System’s Subnet

Determining which systems are logically nearest the lost system is helpful. It doesn’t necessarily determine where the physical machine is located but it will certainly help, especially if you need to proceed to step #4. To find out what systems are logically near the target system you need to scan and identify the systems with the IP addresses closest to the target system.

There are no port scanning tools in commercial operating systems that will generate the type of data we’re looking for. You’ll need to seek out a third-party solution in order to get this data. Using one, create a custom scan that is limited to the IP address ranges and ports you’re interested in. Be careful not to extend that range too far. You don’t want to set off network intrusion defense systems! An example of a useful port scan of the IP address range 192.168.1.1 – 192.168.1.20 is shown in Figure 1.



Figure 1. A successful port scan of 20 IP addresses. Note the restrictions on the scan.

Now I must make an important caution at this point. Unfortunately some port scanning tools get a bad reputation with IT professionals as being hacker tools. That’s because the most common use of linear port scanning is, in fact, primarily an intruder’s technique. The intruder doesn’t know what systems are available for attack but he does know a few IP addresses, so he uses port scanning to actively determine the network landscape. For that reason, unrestricted port scans often get flagged as network attacks.

You should always use port scanning tools that are restricted to the target range and ports you’re probing, and either inform the network security group or temporarily disable the alerts until the scan is complete.

Step #3: Identify the Router Port Nearest the System that the Mysterious Server is Behind

In Figure 1 you can see that our target system at 192.168.1.10 is not responding to the network map. It likely has a firewall or host-based intrusion prevention system (IPS) that is blocking the scan. But you know it is out there. So you’re undeterred!

Take a look at the IP addresses immediately around the target system. You can see their resolved DNS names and which scanned ports are open. This actually tells you a great deal about the missing system. The data implies that the missing system is near the systems listed, at least logically. You should also note that there are routers on this subnet, in the immediate logical vicinity of the missing system. You likely know where those routers are as they’re physical as well as logical hubs. Systems are cabled in and out of routers.

At this stage in the investigation I’ll usually do two things. First I’ll check the network mapping process that I kicked off at Step #1½ to see if there’s any new information. If not, I’ll grab a flashlight and jacket and head into the usually-freezing data center. I know the locations of the networking gear and systems that logically surround the missing system. So I have a fairly good idea of which rack the system should be in or very near. Identifying the missing system is usually a matter of matching a serial number, but it always helps to have a laptop or access to a console nearby during this search.

Most of my missing system searches resolve at or before this stage in the process. The port scan almost always yields valuable data that I can analyze to figure out where the system is likely to be. Thankfully, I rarely have to proceed to the next stage.

Step #4: Follow the Cable

Ultimately the system you’re looking for may be very stealthy. It may be hiding in a different rack, row, virtual machine within a physical, or even a different data center entirely.

When this happens, there’s no technique more reliable or annoying than following a cable. You’ve investigated the system to the point where you know what switch it is connected to, and what port on that switch. Now you need to follow the wire from that port. This is my last resort because it often takes a very long time in dense data centers that have bound and properly routed cabling. It takes even longer, and becomes physically dangerous, if I have to crawl through a raised floor or plenum. It is also a filthy process. But following the cable invariably leads to the system.

Four Steps to Finding any Computer

Tracking down a system hidden behind a newly-built office wall or below a raised floor can really be challenging to a network administrator. These problems do arise frequently though. And there’s no putting off the process to identify the location of the system as quickly as possible. An unexpected audit or law enforcement investigation will be unforgiving of a missing system. In fact, it may arouse unwarranted suspicion.

Following a logical step-by-step process of increasing difficulty is the best way to find these systems. You should have tools and data at your disposal to help you with this effort. Some information, like a thorough and current network map, can make the process fast and painless. But even searching from a position of no information does not have to be painful. You should consider keeping a few tools in your toolbox to help you scan the network, both as necessary and ongoing. Hopefully these tools, and the process outlined here, will help keep you out of filthy plenums.

About the Author