This is a very basic introduction to the X-windows protocol, with the purpose of explaining why the server runs on the workstation and the client runs on the server.
Actually the naming is the right way round. Per definition clients initiate connections and servers accept connections. Without trying to be pedantic or philosophical about the difference between a server and a workstation, lets just say that for the purpose of this discussion your desktop machine is the workstation, and the server is some system "service" applications, files, printers, etc from your computer room. When you start an X-windows application on the server, it needs to open a window somewhere. In this case, that somewhere is the screen of your workstation. The X-windows application, really the X-client, will open a TCP/IP connection to port 6000 on your workstation, and then via this session it will send "instructions" to the X-server on how to draw its window.
Clearly something needs to be running on your workstation to accept a connection on port 6000. This piece of software is called the X-server, and such programs are available for most if not all operating systems. In particular it is available for MS Windows and Mac-OS, and Linux and Solaris includes X-server software in the form of Xfree86, Xorg and Xsun.
The X server actually listens on port 6000+N where N is the "Display" or "instance" number. Thus, the first server is Display 0, and listens on TCP/IP port 6000. A second Display would listen on port 6000+1 = 6001, and so on, though having more than one is not particularly common.
There are a few things which seemingly complicates this matter. The first is that in many situations the X-server and the X-client runs on the same system. Essentially this is true for all X applications used on a "local workstation". But the rule still holds - port 6000 listens and accepts connections from the clients running locally. X-clients, which can be anything from Firefox to Gnome-terminal, get a DISPLAY environment variable which tells them to connect to "localhost:0.0"
The second factor is how the application is started. It is possible to walk to the server, enter the command to set the DISPLAY variable (to point to the IP address of your workstation, then start the X-client application, and when you return to your workstation you will find that it is showing the X-application (assuming that the X-server on your desktop is running and accepting connections). However it is much more convenient and more common to log into the server remotely and then set the DISPLAY variable and then start the X-client.
If you use a strategy such as the above often it quickly becomes too much effort and a way to automate the DISPLAY environment variable setting becomes neccesary. Many X-server programs include a few "automation" settings, which often includes connecting to the remote server via rsh, telnet or ssh, logging in, setting the DISPLAY variable, and then starting an X-client program such as a terminal.
In addition SSH has got some special features whereby it can tunnel the X-server setting back to your PC over the encrypted session. This is particularly handy when you have a firewall blocking incomming connections on port 6000. When starting your SSH client up to do this, the client will ask the SSH server to start listening on port 6010, as if it is an X-server for Display nr 10. The SSH server does this (if it is configured to allow this kind of connection) and then starts the shell, usually cleverly setting the DISPLAY value to localhost:10.0. Note that "localhost" refers to the server itself, thus connections to port 6010 will be picked up by the SSH daemon on that server. When such a connection is made, eg by starting an X-client from within the SSH sessions, the SSH daemon on the server accepts the connection and it knows that the specific port is associated with your SSH tunnel, so it will forward the X-windows data back to your workstation via the existing SSH tunnel. The SSH client on your workstation distinguishes the X-windows data from other data along the channel, and makes a connection to the real X-server on the workstation, and forwards the un-encrypted packets received to the local X-server.
So in summary, the X-server runs on the workstation where the display is rendered. The X-client establishes the TCP/IP session to this X-server based on the value of the DIPSLAY setting. In normal situations you will observe the connection as incomming back towards yourself, but usually we associate the "server" of a client-server application with the software running on the remote machine (relatively to where we are sitting), but in this case it is obviously back-to-front.
Note: This brief introduction glosses over the many complex issues and details. Desktop Environments, Display Managers, Composting, Direct Rendering, the Driver model, connections via other than TCP/IP protocols, etc. For a slightly more complete though still very digestible introduction, see the Wikipedia Article on the X-Windowing System. For all the information you can handle head over to the official steering group web site at http://x.org