The Unix and Internet Fundamentals HOWTO
Eric Raymond
Revision History
Revision 2.11 2010-01-05 Revised by: esr
Notice that it's mostly 64 bits now. Update the sections on booting and running programs to describe X.
Revision 1.0 1998-10-29 Revised by: esr
Initial revision.
This document describes the working basics of PC-class computers, Unix-like operating systems, and the
Internet in non-technical language.
Table of Contents
1. Introduction.....................................................................................................................................................1
1.1. Purpose of this document..................................................................................................................1
1.2. New versions of this document.........................................................................................................1
1.3. Feedback and corrections..................................................................................................................1
1.4. Related resources..............................................................................................................................1
2. Basic anatomy of your computer...................................................................................................................2
3. What happens when you switch on a computer?.........................................................................................3
4. What happens when you log in?....................................................................................................................5
5. What happens when you run programs after boot time?...........................................................................6
6. How do input devices and interrupts work?................................................................................................8
7. How does my computer do several things at once?.....................................................................................9
8. How does my computer keep processes from stepping on each other?...................................................10
8.1. Virtual memory: the simple version...............................................................................................10
8.2. Virtual memory: the detailed version.............................................................................................10
8.3. The Memory Management Unit......................................................................................................12
9. How does my computer store things in memory?......................................................................................13
9.1. Numbers..........................................................................................................................................13
9.2. Characters.......................................................................................................................................13
10. How does my computer store things on disk?..........................................................................................15
10.1. Low-level disk and file system structure......................................................................................15
10.2. File names and directories............................................................................................................15
10.3. Mount points.................................................................................................................................16
10.4. How a file gets looked up.............................................................................................................16
10.5. File ownership, permissions and security.....................................................................................16
10.6. How things can go wrong.............................................................................................................19
11. How do computer languages work?..........................................................................................................20
11.1. Compiled languages......................................................................................................................20
11.2. Interpreted languages....................................................................................................................20
11.3. P-code languages..........................................................................................................................21
12. How does the Internet work?.....................................................................................................................22
12.1. Names and locations.....................................................................................................................22
12.2. The Domain Name System...........................................................................................................22
12.3. Packets and routers.......................................................................................................................23
12.4. TCP and IP....................................................................................................................................24
12.5. HTTP, an application protocol......................................................................................................24
The Unix and Internet Fundamentals HOWTO
i
Table of Contents
13. To Learn More............................................................................................................................................26
The Unix and Internet Fundamentals HOWTO
ii
1. Introduction
1.1. Purpose of this document
This document is intended to help Linux and Internet users who are learning by doing. While this is a great
way to acquire specific skills, sometimes it leaves peculiar gaps in one's knowledge of the basics — gaps
which can make it hard to think creatively or troubleshoot effectively, from lack of a good mental model of
what is really going on.
I'll try to describe in clear, simple language how it all works. The presentation will be tuned for people using
Unix or Linux on PC-class machines. Nevertheless, I'll usually refer simply to ‘Unix’ here, as most of what I
will describe is constant across different machines and across Unix variants.
I'm going to assume you're using an Intel PC. The details differ slightly if you're running an PowerPC or some
other kind of computer, but the basic concepts are the same.
I won't repeat things, so you'll have to pay attention, but that also means you'll learn from every word you
read. It's a good idea to just skim when you first read this; you should come back and reread it a few times
after you've digested what you have learned.
This is an evolving document. I intend to keep adding sections in response to user feedback, so you should
come back and review it periodically.
1.2. New versions of this document
New versions of the Unix and Internet Fundamentals HOWTO will be periodically posted to
comp.os.linux.help and comp.os.linux.announce and news.answers. They will also be uploaded to various
websites, including the Linux Documentation Project home page.
You can view the latest version of this on the World Wide Web via the URL
http:http://www.tldp.org/HOWTO/Unix-and-Internet-Fundamentals-HOWTO/index.html.
This document has been translated into: the following languages: Polish Spanish Turkish
1.3. Feedback and corrections
If you have questions or comments about this document, please feel free to mail Eric S. Raymond, at
esr@thyrsus.com. I welcome any suggestions or criticisms. I especially welcome hyperlinks to more detailed
explanations of individual concepts. If you find a mistake with this document, please let me know so I can
correct it in the next version. Thanks.
1.4. Related resources
If you're reading this in order to learn how to hack, you should also read the How To Become A Hacker FAQ.
It has links to some other useful resources.
1. Introduction 1
2. Basic anatomy of your computer
Your computer has a processor chip inside it that does the actual computing. It has internal memory (what
DOS/Windows people call "RAM" and Unix people often call "core"; the Unix term is a folk memory from
when RAM consisted of ferrite-core donuts). The processor and memory live on the motherboard, which is
the heart of your computer.
Your computer has a screen and keyboard. It has hard drives and an optical CD-ROM (or maybe a DVD
drive) and maybe a floppy disk. Some of these devices are run by controller cards that plug into the
motherboard and help the computer drive them; others are run by specialized chipsets directly on the
motherboard that fulfill the same function as a controller card. Your keyboard is too simple to need a separate
card; the controller is built into the keyboard chassis itself.
We'll go into some of the details of how these devices work later. For now, here are a few basic things to keep
in mind about how they work together:
All the parts of your computer inside the case are connected by a bus. Physically, the bus is what you plug
your controller cards into (the video card, the disk controller, a sound card if you have one). The bus is the
data highway between your processor, your screen, your disk, and everything else.
(If you've seen references to ‘ISA’, ‘PCI’, and ‘PCMCIA’ in connection with PCs and have not understood
them, these are bus types. ISA is, except in minor details, the same bus that was used on IBM's original PCs in
1980; it is passing out of use now. PCI, for Peripheral Component Interconnection, is the bus used on most
modern PCs, and on modern Macintoshes as well. PCMCIA is a variant of ISA with smaller physical
connectors used on laptop computers.)
The processor, which makes everything else go, can't actually see any of the other pieces directly; it has to
talk to them over the bus. The only other subsystem that it has really fast, immediate access to is memory (the
core). In order for programs to run, then, they have to be in core (in memory).
When your computer reads a program or data off the disk, what actually happens is that the processor uses the
bus to send a disk read request to your disk controller. Some time later the disk controller uses the bus to
signal the processor that it has read the data and put it in a certain location in memory. The processor can then
use the bus to look at that data.
Your keyboard and screen also communicate with the processor via the bus, but in simpler ways. We'll
discuss those later on. For now, you know enough to understand what happens when you turn on your
computer.
2. Basic anatomy of your computer 2
3. What happens when you switch on a computer?
A computer without a program running is just an inert hunk of electronics. The first thing a computer has to
do when it is turned on is start up a special program called an operating system. The operating system's job is
to help other computer programs to work by handling the messy details of controlling the computer's
hardware.
The process of bringing up the operating system is called booting (originally this was bootstrapping and
alluded to the process of pulling yourself up "by your bootstraps"). Your computer knows how to boot
because instructions for booting are built into one of its chips, the BIOS (or Basic Input/Output System) chip.
The BIOS chip tells it to look in a fixed place, usually on the lowest-numbered hard disk (the boot disk) for a
special program called a boot loader (under Linux the boot loader is called Grub or LILO). The boot loader is
pulled into memory and started. The boot loader's job is to start the real operating system.
The loader does this by looking for a kernel, loading it into memory, and starting it. If you Linux and see
"LILO" on the screen followed by a bunch of dots, it is loading the kernel. (Each dot means it has loaded
another disk block of kernel code.)
(You may wonder why the BIOS doesn't load the kernel directly — why the two-step process with the boot
loader? Well, the BIOS isn't very smart. In fact it's very stupid, and Linux doesn't use it at all after boot time.
It was originally written for primitive 8-bit PCs with tiny disks, and literally can't access enough of the disk to
load the kernel directly. The boot loader step also lets you start one of several operating systems off different
places on your disk, in the unlikely event that Unix isn't good enough for you.)
Once the kernel starts, it has to look around, find the rest of the hardware, and get ready to run programs. It
does this by poking not at ordinary memory locations but rather at I/O ports — special bus addresses that are
likely to have device controller cards listening at them for commands. The kernel doesn't poke at random; it
has a lot of built-in knowledge about what it's likely to find where, and how controllers will respond if they're
present. This process is called autoprobing.
You may or may not be able to see any of this going on. Back when Unix systems used text consoles, you'd
see boot messages scroll by on your screen as the system started up. Nowawadays, Unixes often hide the boot
messages behind a graphical splash screen. You may be able to see them by switching to a text console view
with the key combination Ctrl-Shift-F1. If this works, you should be able to switch back to the graphical boot
screen with a different Ctrl-Shift sequence; try F7, F8, and F9.
Most of the messages emitted boot time are the kernel autoprobing your hardware through the I/O ports,
figuring out what it has available to it and adapting itself to your machine. The Linux kernel is extremely
good at this, better than most other Unixes and much better than DOS or Windows. In fact, many Linux
old-timers think the cleverness of Linux's boot-time probes (which made it relatively easy to install) was a
major reason it broke out of the pack of free-Unix experiments to attract a critical mass of users.
But getting the kernel fully loaded and running isn't the end of the boot process; it's just the first stage
(sometimes called run level 1). After this first stage, the kernel hands control to a special process called ‘init’
which spawns several housekeeping processes. (Some recent Linuxes use a different program called ‘upstart’
that does similar things)
The init process's first job is usually to check to make sure your disks are OK. Disk file systems are fragile
things; if they've been damaged by a hardware failure or a sudden power outage, there are good reasons to
3. What happens when you switch on a computer? 3
take recovery steps before your Unix is all the way up. We'll go into some of this later on when we talk about
how file systems can go wrong.
Init's next step is to start several daemons. A daemon is a program like a print spooler, a mail listener or a
WWW server that lurks in the background, waiting for things to do. These special programs often have to
coordinate several requests that could conflict. They are daemons because it's often easier to write one
program that runs constantly and knows about all requests than it would be to try to make sure that a flock of
copies (each processing one request and all running at the same time) don't step on each other. The particular
collection of daemons your system starts may vary, but will almost always include a print spooler (a
gatekeeper daemon for your printer).
The next step is to prepare for users. Init starts a copy of a program called getty to watch your screen and
keyboard (and maybe more copies to watch dial-in serial ports). Actually, nowadays it usually starts multiple
copies of getty so you have several (usually 7 or 8) virtual consoles, with your screen and keyboards
connected to one of them at a time. But you likely won't see any of these, because one of your consoles will
be taken over by the X server (about which more in a bit).
We're not done yet. The next step is to start up various daemons that support networking and other services.
The most important of these is your X server. X is a daemon that manages your display, keyboard, and mouse.
Its main job is to produce the color pixel graphics you normally see on your screen.
When the X server comes up, during the last part of your machine's boot process, it effectively takes over the
hardware from whatever virtual console was previously in control. That's when you'll see a graphical login
screen, produced for you by a program called a display manager.
The Unix and Internet Fundamentals HOWTO
3. What happens when you switch on a computer? 4
4. What happens when you log in?
When you log in, you identify yourself to the computer. On modern Unixes you will usually do this through a
graphical display manager. But it's possible to switch virtual consoles with a Ctrl-Shift key sequence and do a
textual login, too. In that case you go through the getty instance watching that console tto call the program
login.
You identify yourself to the display manager or login with a login name and password. That login name is
looked up in a file called /etc/passwd, which is a sequence of lines each describing a user account.
One of these fields is an encrypted version of the account password (sometimes the encrypted fields are
actually kept in a second /etc/shadow file with tighter permissions; this makes password cracking harder).
What you enter as an account password is encrypted in exactly the same way, and the login program checks to
see if they match. The security of this method depends on the fact that, while it's easy to go from your clear
password to the encrypted version, the reverse is very hard. Thus, even if someone can see the encrypted
version of your password, they can't use your account. (It also means that if you forget your password, there's
no way to recover it, only to change it to something else you choose.)
Once you have successfully logged in, you get all the privileges associated with the individual account you are
using. You may also be recognized as part of a group. A group is a named collection of users set up by the
system administrator. Groups can have privileges independently of their members’ privileges. A user can be a
member of multiple groups. (For details about how Unix privileges work, see the section below on
permissions.)
(Note that although you will normally refer to users and groups by name, they are actually stored internally as
numeric IDs. The password file maps your account name to a user ID; the /etc/group file maps group
names to numeric group IDs. Commands that deal with accounts and groups do the translation automatically.)
Your account entry also contains your home directory, the place in the Unix file system where your personal
files will live. Finally, your account entry also sets your shell, the command interpreter that login will start up
to accept your commmands.
What happens after you have successfully logged in depends on how you did it. On a text console, login will
launch a shell and you'll be off and running. If you logged in through a display manager, the X server will
bring up your graphical desktop and you will be able to run programs from it — either through the menus, or
through desktop icons, or through a terminal emulator running a shell.
4. What happens when you log in? 5
5. What happens when you run programs after
boot time?
After boot time and before you run a program, you can think of your computer as containing a zoo of
processes that are all waiting for something to do. They're all waiting on events. An event can be you pressing
a key or moving a mouse. Or, if your machine is hooked to a network, an event can be a data packet coming in
over that network.
The kernel is one of these processes. It's a special one, because it controls when the other user pro