Tuesday, October 21, 2008

Why X-windows is back-to-front

This is a very basic introduction to the X-windows protocol, with the purpose of explaining why the server runs on the workstation and the client runs on the server.

Actually the naming is the right way round. Per definition clients initiate connections and servers accept connections. Without trying to be pedantic or philosophical about the difference between a server and a workstation, lets just say that for the purpose of this discussion your desktop machine is the workstation, and the server is some system "service" applications, files, printers, etc from your computer room. When you start an X-windows application on the server, it needs to open a window somewhere. In this case, that somewhere is the screen of your workstation. The X-windows application, really the X-client, will open a TCP/IP connection to port 6000 on your workstation, and then via this session it will send "instructions" to the X-server on how to draw its window.

Clearly something needs to be running on your workstation to accept a connection on port 6000. This piece of software is called the X-server, and such programs are available for most if not all operating systems. In particular it is available for MS Windows and Mac-OS, and Linux and Solaris includes X-server software in the form of Xfree86, Xorg and Xsun.

The X server actually listens on port 6000+N where N is the "Display" or "instance" number. Thus, the first server is Display 0, and listens on TCP/IP port 6000. A second Display would listen on port 6000+1 = 6001, and so on, though having more than one is not particularly common.

There are a few things which seemingly complicates this matter. The first is that in many situations the X-server and the X-client runs on the same system. Essentially this is true for all X applications used on a "local workstation". But the rule still holds - port 6000 listens and accepts connections from the clients running locally. X-clients, which can be anything from Firefox to Gnome-terminal, get a DISPLAY environment variable which tells them to connect to "localhost:0.0"

The second factor is how the application is started. It is possible to walk to the server, enter the command to set the DISPLAY variable (to point to the IP address of your workstation, then start the X-client application, and when you return to your workstation you will find that it is showing the X-application (assuming that the X-server on your desktop is running and accepting connections). However it is much more convenient and more common to log into the server remotely and then set the DISPLAY variable and then start the X-client.

If you use a strategy such as the above often it quickly becomes too much effort and a way to automate the DISPLAY environment variable setting becomes neccesary. Many X-server programs include a few "automation" settings, which often includes connecting to the remote server via rsh, telnet or ssh, logging in, setting the DISPLAY variable, and then starting an X-client program such as a terminal.

In addition SSH has got some special features whereby it can tunnel the X-server setting back to your PC over the encrypted session. This is particularly handy when you have a firewall blocking incomming connections on port 6000. When starting your SSH client up to do this, the client will ask the SSH server to start listening on port 6010, as if it is an X-server for Display nr 10. The SSH server does this (if it is configured to allow this kind of connection) and then starts the shell, usually cleverly setting the DISPLAY value to localhost:10.0. Note that "localhost" refers to the server itself, thus connections to port 6010 will be picked up by the SSH daemon on that server. When such a connection is made, eg by starting an X-client from within the SSH sessions, the SSH daemon on the server accepts the connection and it knows that the specific port is associated with your SSH tunnel, so it will forward the X-windows data back to your workstation via the existing SSH tunnel. The SSH client on your workstation distinguishes the X-windows data from other data along the channel, and makes a connection to the real X-server on the workstation, and forwards the un-encrypted packets received to the local X-server.

So in summary, the X-server runs on the workstation where the display is rendered. The X-client establishes the TCP/IP session to this X-server based on the value of the DIPSLAY setting. In normal situations you will observe the connection as incomming back towards yourself, but usually we associate the "server" of a client-server application with the software running on the remote machine (relatively to where we are sitting), but in this case it is obviously back-to-front.

Note: This brief introduction glosses over the many complex issues and details. Desktop Environments, Display Managers, Composting, Direct Rendering, the Driver model, connections via other than TCP/IP protocols, etc. For a slightly more complete though still very digestible introduction, see the Wikipedia Article on the X-Windowing System. For all the information you can handle head over to the official steering group web site at http://x.org

Saturday, October 18, 2008

Making the most of Solaris Man pages

Solaris man pages (manual pages) are well written, consistent, complete, and generally a great source of information. Here are a few tips to help you get the most out of them. Of course this applies to all Solaris derivatives, including proper Solaris, OpenSolaris and other Open Solaris distributions like Solaris Express, Belenix and Nexenta. It even applies to other Unix flavors like BSD and AIX, and Unix derivatives like Linux.

The simplest (and most obvious) way to use the manual pages are to enter "man command" and then read the entire page.

My first hint however is to change the "PAGER" environment variable. I suggest you set it like this in your .profile file:

PAGER="less -iMsq"; export PAGER

(or for csh and its family members, use setenv PAGER "less -iMsq")

The reasons for this are

a) "less" supports more useful options than does "more". In particular, it supports highlighting, as well as all of the below!

b) The "-i" causes less to ignore case in searches. A definite advantage because you can simply enter /user and it will find the string "user" even when capitalized for use at the start of a sentence.

c) The -M causes "less" to show a more verbose status at the bottom of the screen.

d) -q ... Stop irritating me.

e) -s to "squeeze" blank lines (because otherwise less will "format" pages with blank spaces, as if it is a printed page)

f) You can return to the first line (top) of the page by taping "1" followed by "G"

g) less is more.

The PAGER environment variable is automatically used by the man command, so you can now test it and will find that you can scroll up and down in man pages. To find a word in the man page, tap the forward-slash / followed by the string to search for, and the word wherever it is found will be highlighted in inverse text.

Note: less does have some limitations over more, but these do not apply to viewing man pages. For the curious, the limitation is in the handling of control characters in the input file. In particular, more does a better job of displaying "captured" sessions recorded from a system console or from a shell using the "script" command. Regardless, I solve these by reading capture files using cat -nvet | less -iMq (Note - I like seeing line numbers)

Second Hint: Generate the Manual Pages keyword index database (The so-called windex or apropos information). To do this, you need to run the following command once.

$ catman -w

This will run for a while as it generates and stores the man page keyword index for future use.

Once it completes, the man command's -k option and the "whatis" command will work. This allows you to find what you need much easier. For example to find all man pages for tools, commands and drivers related to Wifi and Wireless networking, you can use

$ man -k wifi wireless

Third hint: Some man pages are not automatically searched or found. In older (proper) Solaris releases, the man command has got a "default list" of locations where to find man pages. In Recent Open Solaris releases, man constructs a list of locations to search. The man manual page states:

  Search Path
     Before searching for a given name, man constructs a list  of
     candidate directories and sections. man searches for name in
     the directories specified by the MANPATH  environment  vari-

     In the absence of MANPATH, man constructs  its  search  path
     based  upon the PATH environment variable, primarily by sub-
     stituting man for the last component of  the  PATH  element.
     Special  provisions  are added to account for unique charac-
     teristics  of   directories   such   as   /sbin,   /usr/ucb,
     /usr/xpg4/bin, and others. If the file argument contains a /
     character, the dirname portion of the argument  is  used  in
     place of PATH elements to construct the search path.

Note: The (currently apparently undocumented) -p option to man will show the effective man search path.

As implied, you can set in MANPATH a list of directories containing man pages you use regularly. It is also possible to overide the man search path using the man command's -M option. This is particularly true for packages that install under /usr/local, /opt/SUNW...., etc. So to view the man page for "explorer" you would run:

man -M /opt/SUNWexplo/man explorer

(Assuming you have SUNWexplorer installed)

Finally, it helps to understand the structure of a man page. The individual manual pages are divided up into a common set of sections, and knowing them you will often skip straight to a specific (sub)section in the page when looking for specific information.

Some of these subsections are optional, but the names used are consistent. Here is a quick summary of some important sections.

NAMEWhat this page is about. This is the "whatis" information.
SYNOPSISThe summary, eg how to use a command.
DESCRIPTIONA more complete description of what this component is.
OPTIONS, OPERANDS and USAGEThese section details how a command is used, eg what command-line options are available for a specific command.
ENVIRONMENT VARIABLESAs the name suggests, this section details Environment variables which affects the behavior of the command.
EXIT STATUSAs the name implies, it explains the possible exit status values. Useful to know that the section exists though.
FILESA list of files which are relevant to the command or subsystem. In particular here you will find out where a program keeps its configuration files. For a good example, see man dumpadm.
EXAMPLESOften a great place to skip to when a manual page is complicated.
ATTRIBUTESAn often overlooked section, it explains the status of the item. For commands it tells you which packages the files are in. This section of man pages are fully explained in the "attributes" manual page.
SEE ALSOOne of the most useful parts of man pages, this gives you hints about other pages which are related to the same topic.
NOTESVarious Additional notes.

Other sections, such as SECURITY, ERRORS, SUBCOMMANDS, etc exists in some man pages, but the above are probably the most useful sections.

The entire collection of man pages are divided up into manual sections, such as Section 1 - User Commands, Section 1M - Maintenance commands for sys-admins, Section 3C, the C programming reference information and section 7 - information about device drivers. This is not hugely important, except that you need to understand that some pages occur in more than one section. For example "read" or "signal" By default, man will display the first manual page which matches the file name. If you want to see one of the other pages you need to specify the specific manual section explicitly.

For example

$ man -f signal

signal  signal (3c) - simplified signal management for application processes
signal  signal (3ucb) - simplified software signal facilities
signal  signal.h (3head)    - base signals

We can see that file "signal" has got manual entries in sections 3c (C programming reference), 3ucb (UCB/Posix version), and 3head (the Headers section)

Simply entering ||man signal|| will show the man page from "3C". To see the man page in the "3head" section, enter either

$ man -s 3head signal


$ man signal.3head

I often end up using the second form simply because after viewing a man page I often see that there is another related man page from the SEE ALSO section. I then exit the man page, recall the previous command, and append .section to the command line, which is to say that it is just a matter of convenience.

One last tip: There are online versions of the manual pages at http://docs.sun.com.

You don't need to learn every command and every option by heart. Knowing that you have manual pages, and that you can look up the information you need quickly by looking at related commands from SEE ALSO, and from the keyword index using man -k, and then being able to quickly search through man pages for the right section or keyword absolutely WILL make you a clever, more efficient and over all better sysadmin!