Making hardware work is tedious. To write to a hard disk, for example, requires that you write magic numbers in magic places, wait for the hard drive to say that it is ready to receive data, and then feed it the data it wants, very carefully. To write to a floppy disk is even harder, and requires that the program supervise the floppy disk drive almost constantly while it is running.

Instead of putting code in each application you write to control each device, you share the code between applications. To make sure that that code is not compromised, you protect it from users and normal programs that use it. If you do it right, you will be able to add and remove devices from your system without changing your applications at all. Furthermore, you need to be able to load your program into memory and run it, which the operating system also does. So an operating system is essentially a priviledged, general, sharable library of low-level hardware and memory and process control functions and routines.

All versions of Unix have an abstract way of reading and writing devices. By making the devices act as much as possible like regular files, the same calls (read(), write(), etc.) can be used for devices and files. Within the kernel, there are a set of functions, registered with the filesystem, which are called to handle requests to do I/O on “device special files,” which are those which represent devices. (See mknod(1,2) for an explanation of how to make these files.)

All devices controlled by the same device driver are given the same major number, and of those with the same major number, different devices are distinguished by different minor numbers. (This is not strictly true, but it is close enough. If you understand where it is not true, you don’t need to read this section, and if you don’t but want to learn, read the code for the tty devices, which uses up 2 major numbers, and may use a third and possibly fourth by the time you read this. Also, the “misc” major device supports many minor devices that only need a few minor numbers; we’ll get to that later.)

This chapter explains how to write any type of Linux device driver that you might need to, including character, block, SCSI, and network drivers. It explains what functions you need to write, how to initialize your drivers and obtain memory for them efficiently, and what function are built in to Linux to make your job easier.

Creating device drivers for Linux is easier than you might think. It merely involves writing a few functions and registering them with the Virtual Filesystem Switch (VFS), so that when the proper device special files are accessed, the VFS can call your functions.

However, a word of warning is due here: Writing a device driver is writing a part of the Linux kernel. This means that your driver runs with kernel permissions, and can do anything it wants to: write to any memory, reformat your hard drive, damage your monitor or video card, or even break your dishes, if your dishwasher is controlled by your computer. Be careful.

Also, your driver will run in kernel mode, and the Linux kernel, like most Unix kernels, is non-pre-emptible. This means that if you driver takes a long time to work without giving other programs a chance to work, your computer will appear to “freeze” when your driver is running. Normal user-mode pre-emptive scheduling does not apply to your driver.

The POSIX thread libraries are a standards based thread API for C/C++. It allows one to spawn a new concurrent process flow. It is most effective on multi-processor or multi-core systems where the process flow can be scheduled to run on another processor thus gaining speed through parallel or distributed processing. Threads require less overhead than “forking” or spawning a new process because the system does not initialize a new system virtual memory space and environment for the process. While most effective on a multiprocessor system, gains are also found on uniprocessor systems which exploit latency in I/O and other system functions which may halt process execution. Parallel programming technologies such as MPI and PVM are used in a distributed computing environment while threads are limited to a single computer system. All threads within a process share the same address space. A thread is spawned by defining a function and its arguments which will be processed in the thread. The purpose of using the POSIX thread library in your software is to execute software faster.

Sockets are the fundamental technology for programming software to communicate on TCP/IP networks. A socket provides a bidirectional communication endpoint for sending and receiving data with another socket. Socket connections normally run between two different computers on a LAN or across the Internet, but they can also be used for interprocess communication on a single computer.

Introduction:

A thread is a semi-process, that has its own stack, and executes a given piece of code. Unlike a real process, the thread normally shares its memory with other threads (where as for processes we usually have a different memory area for each one of them). A Thread Group is a set of threads all executing inside the same process. They all share the same memory, and thus can access the same global variables, same heap memory, same set of file descriptors, etc. All these threads execute in parallel (i.e. using time slices, or if the system has several processors, then really in parallel).

The advantage of using a thread group instead of a normal serial program is that several operations may be carried out in parallel, and thus events can be handled immediately as they arrive (for example, if we have one thread handling a user interface, and another thread handling database queries, we can execute a heavy query requested by the user, and still respond to user input while the query is executed).

The advantage of using a thread group over using a process group is that context switching between threads is much faster then context switching between processes (context switching means that the system switches from running one thread or process, to running another thread or process). Also, communications between two threads is usually faster and easier to implement then communications between two processes.

On the other hand, because threads in a group all use the same memory space, if one of them corrupts the contents of its memory, other threads might suffer as well. With processes, the operating system normally protects processes from one another, and thus if one corrupts its own memory space, other processes won’t suffer. Another advantage of using processes is that they can run on different machines, while all the threads have to run on the same machine (at least normally).

Synopsys:

FTP based Client Server project using sockets and threads

Objective:

• This project is designed to ensure that students of Engineering College with academic capabilities will have the skill set needed to deal with the challenges involved in real-world linux system programming and network programming and associated protocols / technologies to meet the needs of industries both today and in the future.
• The course considers linux based network programming and communication techni

Notice that the client needs to know of the existence of and the address of the server, but the server does not need to know the address of (or even the existence of) the client prior to the connection being established. Notice also that once a connection is established, both sides can send and receive information.

The system calls for establishing a connection are somewhat different for the client and the server, but both involve the basic construct of a socket. A socket is one end of an interprocess communication channel. The two processes each establish their own socket.

The steps involved in establishing a socket on the client side are as follows:

1. Create a socket with the socket() system call
2. Connect the socket to the address of the server using the connect() system call
3. Send and receive data. There are a number of ways to do this, but the simplest is to use the read() and write() system calls.

The steps involved in establishing a socket on the server side are as follows:

1. Create a socket with the socket() system call
2. Bind the socket to an address using the bind() system call. For a server socket on the Internet, an address consists of a port number on the host machine.
3. Listen for connections with the listen() system call
4. Accept a connection with the accept() system call. This call typically blocks until a client connects with the server.
5. Send and receive data

Socket Types

When a socket is created, the program has to specify the address domain and the socket type. Two processes can communicate with each other only if their sockets are of the same type and in the same domain. There are two widely used address domains, the unix domain, in which two processes which share a common file system communicate, and the Internet domain, in which two processes running on any two hosts on the Internet communicate. Each of these has its own address format.

The address of a socket in the Unix domain is a character string which is basically an entry in the file system.

The address of a socket in the Internet domain consists of the Internet address of the host machine (every computer on the Internet has a unique 32 bit address, often referred to as its IP address). In addition, each socket needs a port number on that host. Port numbers are 16 bit unsigned integers. The lower numbers are reserved in Unix for standard services. For example, the port number for the FTP server is 21. It is important that standard services be at the same port on all computers so that clients will know their addresses. However, port numbers above 2000 are generally available.

There are two widely used socket types, stream sockets, and datagram sockets. Stream sockets treat communications as a continuous stream of characters, while datagram sockets have to read entire messages at once. Each uses its own communciations protocol. Stream sockets use TCP (Transmission Control Protocol), which is a reliable, stream oriented protocol, and datagram sockets use UDP (Unix Datagram Protocol), which is unreliable and message oriented.

The examples in this tutorial will use sockets in the Internet domain using the TCP protocol.

Sample code

C code for a very simple client and server are provided for you. These communicate using stream sockets in the Internet domain. The code is described in detail below. However, before you read the descriptions and look at the code, you should compile and run the two programs to see what they do.

Click here for the server program

Click here for the client program

Download these into files called server.c and client.c and compile them separately into two executables called server and client. They require special compiling flags as stated in their respective progarms.

Ideally, you should run the client and the server on separate hosts on the Internet. Start the server first. Suppose the server is running on a machine called cheerios. When you run the server, you need to pass the port number in as an argument. You can choose any number between 2000 and 65535. If this port is already in use on that machine, the server will tell you this and exit. If this happens, just choose another port and try again. If the port is available, the server will block until it receives a connection from the client. Don’t be alarmed if the server doesn’t do anything; it’s not supposed to do anything until a connection is made. Here is a typical command line:

server 51717

To run the client you need to pass in two arguments, the name of the host on which the server is running and the port number on which the server is listening for connections. Here is the command line to connect to the server described above:

client cheerios 51717

The client will prompt you to enter a message. If everything works correctly, the server will display your message on stdout, send an acknowledgement message to the client and terminate. The client will print the acknowledgement message from the server and then terminate.

You can simulate this on a single machine by running the server in one window and the client in another. In this case, you can use the keyword localhost as the first argument to the client.

Server code

The server code uses a number of ugly programming constructs, and so we will go through it line by line.

#include <stdio.h>

This header file contains declarations used in most input and output and is typically included in all C programs.

#include <sys/types.h>

This header file contains definitions of a number of data types used in system calls. These types are used in the next two include files.

#include <sys/socket.h>

The header file socket.h includes a number of definitions of structures needed for sockets.

#include <netinet/in.h>

The header file netinet/in.h contains constants and structures needed for internet domain addresses.

void error(char *msg)
{
    perror(msg);
    exit(1);
}

This function is called when a system call fails. It displays a message about the error on stderr and then aborts the program. Click here to see the man page for perror()

int main(int argc, char *argv[])
{
     int sockfd, newsockfd, portno, clilen, n;

sockfd and newsockfd are file descriptors, i.e. array subscripts into the file descriptor table . These two variables store the values returned by the socket system call and the accept system call.

portno stores the port number on which the server accepts connections.

clilen stores the size of the address of the client. This is needed for the accept system call.

n is the return value for the read() and write() calls; i.e. it contains the number of characters read or written.

     char buffer[256];

The server reads characters from the socket connection into this buffer.

     struct sockaddr_in serv_addr, cli_addr;

A sockaddr_in is a structure containing an internet address. This structure is defined in <netinet/in.h>. Here is the definition:

struct sockaddr_in {
        short   sin_family;
        u_short sin_port;
        struct  in_addr sin_addr;
        char    sin_zero[8];
};

An in_addr structure, defined in the same header file, contains only one field, a unsigned long called s_addr. The variable serv_addr will contain the address of the server, and cli_addr will contain the address of the client which connects to the server.

     if (argc < 2) {
         fprintf(stderr,"ERROR, no port provided\n");
         exit(1);
     }

The user needs to pass in the port number on which the server will accept connections as an arg

A compiler is a program that turns source code into executable code. Like all programs, a compiler runs on a specific type of computer, and the new programs it outputs also run on a specific type of computer.[1]

The computer the compiler runs on is called the host, and the computer the new programs run on is called the target. When the host and target are the same type of machine, the compiler is a native compiler. When the host and target are different, the compiler is a cross compiler.[2]
Why cross-compile?

In theory, a PC user who wanted to build programs for some device could get the appropriate target hardware (or emulator), boot a Linux distro on that, and compile natively within that environment. While this is a valid approach (and possibly even a good idea when dealing with something like a Mac Mini), it has a few prominent downsides for things like a linksys router or iPod:

Speed – Target platforms are usually much slower than hosts, by an order of magnitude or more. Most special-purpose embedded hardware is designed for low cost and low power consumption, not high performance. Modern emulators (like qemu) are actually faster than a lot of the real world hardware they emulate, by virtue of running on high-powered desktop hardware.[3]

Capability – Compiling is very resource-intensive. The target platform usually doesn’t have gigabytes of memory and hundreds of gigabytes of disk space the way a desktop does; it may not even have the resources to build “hello world”, let alone large and complicated packages.

Availability – Bringing Linux up on a hardware platform it’s never run on before requires a cross-compiler. Even on long-established platforms like Arm or Mips, finding an up-to-date full-featured prebuilt native environment for a given target can be hard. If the platform in question isn’t normally used as a development workstation, there may not be a recent prebuilt distro readily available for it, and if there is it’s probably out of date. If you have to build your own distro for the target before you can build on the target, you’re back to cross-compiling anyway.

Flexibility – A fully capable Linux distribution consists of hundreds of packages, but a cross-compile environment can depend on the host’s existing distro from most things. Cross compiling focuses on building the target packages to be deployed, not spending time getting build-only prerequisites working on the target system.

Convenience – The user interface of headless boxes tends to be a bit crampled. Diagnosing build breaks is frustrating enough as it is. Installing from CD onto a machine that hasn’t got a CD-ROM drive is a pain. Rebooting back and forth between your test environment and your development environment gets old fast, and it’s nice to be able to recover from accidentally lobotomizing your test system.

Why is cross-compiling hard?
Portable native compiling is hard.

Most programs are developed on x86 hardware, where they are compiled natively. This means cross-compiling runs into two types of problems: problems with the programs themselves and problems with the build system.

The first type of problem affects all non-x86 targets, both for native and for cross-builds. Most programs make assumptions about the type of machine they run on, which must match the platform in question or the program won’t work. Common assumptions include:

Word size – Copying a pointer into an int may lose data on a 64 bit platform, and determining the size of a malloc by multiplying by 4 instead of sizeof(long) isn’t good either. Subtle security flaws due to integer overflows are also possible, ala “if (x+y < size) memset(src+x,0,y);”, which results in a 4 gigabyte memset on 32-bit hardware when x=1000 and y=0xFFFFFFF0…

Endianness – Different systems store binary data iternally in different ways, which means that block-reading int or float data from disk or the network may need translation. Type “man byteorder” for details.

Alignment – Some platforms (such as arm) can only read or write ints from addresses that are an even multiple of 4 bytes, otherwise they segfault. Even the ones that can handle arbitrary alignments are slower dealing with unaligned data (they have to fetch twice to get both halves), so the compiler will often pad structures to align variables. Treating structures as a lump of data that can be sent to disk or across the network thus requires extra work to ensure a consistent representation.

Default signedness – Whether the “char” data type defaults to signed or unsigned varies from platform to platform (and in some cases from compiler to compiler), which can cause some really surprising bugs. The easy workaround for this is to provide a compiler argument like “-funsigned-char” to force the default to a known value.

NOMMU – If your target platform doesn’t have a memory management unit, several things need to change. You need vfork() instead of fork(), only certain types of mmap() work (shared or read only, but not copy on write), and the stack doesn’t grow dynamically.

Most packages aim to be portable when compiled natively, and will at least accept patches to fix any of the above problems (with the possible exception of NOMMU issues) submitted to the appropriate development mailing list.

A FIFO special file (a named pipe) is similar to a pipe, except that
it is accessed as part of the filesystem. It can be opened by
multiple processes for reading or writing. When processes are
exchanging data via the FIFO, the kernel passes all data internally
without writing it to the filesystem. Thus, the FIFO special file
has no contents on the filesystem; the filesystem entry merely serves
as a reference point so that processes can access the pipe using a
name in the filesystem.

The kernel maintains exactly one pipe object for each FIFO special
file that is opened by at least one process. The FIFO must be opened
on both ends (reading and writing) before data can be passed.
Normally, opening the FIFO blocks until the other end is opened also.

A process can open a FIFO in nonblocking mode. In this case, opening
for read-only will succeed even if no-one has opened on the write
side yet, opening for write-only will fail with ENXIO (no such device
or address) unless the other end has already been opened.

Under Linux, opening a FIFO for read and write will succeed both in
blocking and nonblocking mode. POSIX leaves this behavior undefined.
This can be used to open a FIFO for writing while there are no
readers available. A process that uses both ends of the connection
in order to communicate with itself should be very careful to avoid

Semaphores are used to protect critical regions of code or data structures. Remember that each access of a critical piece of data such as a VFS inode describing a directory is made by kernel code running on behalf of a process. It would be very dangerous to allow one process to alter a critical data structure that is being used by another process. One way to achieve this would be to use a buzz lock around the critical piece of data that is being accessed, but this is a simplistic approach that would degrade system performance.

Instead Linux uses semaphores to allow just one process at a time to access critical regions of code and data; all other processes wishing to access this resource will be made to wait until it becomes free. The waiting processes are suspended, other processes in the system can continue to run as normal.

A Linux semaphore data structure contains the following information:

count
This field keeps track of the count of processes wishing to use this resource. A positive value means that the resource is available. A negative or zero value means that processes are waiting for it. An initial value of 1 means that one and only one process at a time can use this resource. When processes want this resource they decrement the count and when they have finished with this resource they increment the count,
waking
This is the count of processes waiting for this resource which is also the number of process waiting to be awakened when this resource becomes free,
wait queue
When processes are waiting for this resource they are put onto this wait queue,
lock
A buzz lock used when accessing the waking field.

Suppose the initial count for a semaphore is 1, the first process to come along will see that the count is positive and decrement it by 1, making it 0. The process now “owns” the critical piece of code or resource that is being protected by the semaphore. When the process leaves the critical region it increments the semphore’s count. The most optimal case is where there are no other processes contending for ownership of the critical region. Linux has implemented semaphores to work efficiently for this, the most common case.

If another process wishes to enter the critical region whilst it is owned by a process it too will decrement the count. As the count is now negative (-1) the process cannot enter the critical region. Instead it must wait until the owning process exits it. Linux makes the waiting process sleep until the owning process wakes it on exiting the critical region. The waiting process adds itself to the semaphore’s wait queue and sits in a loop checking the value of the waking field and calling the scheduler until waking is non-zero.

RETURN VALUE->
on success zero is returned,if error -1 is returned