Semisane's writeup on crystals is quite informative. I wish to give a more precise definition to which I can refer in other writeups on solid state physics.

Definition: A crystal is a set of one or more atoms called a basis that is positioned at every point in a Bravais lattice (see Bravais lattice).

It is helpful to consider an example. A very common and important example is the diamond structure, into which both carbon and silicon crystallize (carbon also crystallizes into other forms, as noted by Semisane). The Bravais lattice of the diamond structure is the face-centered cubic, which looks like this (actually it extends forever in all directions):

Face-centered cubic Bravais lattice


            .
            . 
            .
            .

    X----------X----------X
   /|         /|         /|
  / |  o     / |  o     / |
 /  |    o  /  |    o  /  |
X---|------X---|------X   |   . . . to infinity  
| o |      | o |      | o |
|   X------|---X------|---X
|  /| o    |  /| o    |  /|
| / |   o  | / |   o  | / | 
|/  |      |/  |      |/  |    Some o's are missing
X---|------X---|------X   |    here for clarity.
|   |      |   |      |   |       
|   X------|---X------|---X
|  /|      |  /|      |  /|
| / |  o   | / |  o   | / |
|/  |    o |/  |    o |/  |
X---|------X---|------X   |   . . . to infinity  
| o |      | o |      | o |
|   X------|---X------|---X
|  /  o    |  /  o    |  /
| /     o  | /     o  | /  
|/         |/         |/    
X----------X----------X   
     
           .
           .
           .
           .
   

The X's and the o's are completely equivalent positions in the face-centered cubic lattice. The o's can be viewed as the "face centers" of the cubes formed by the X's. This is how I visualized the lattice when I drew the dashed lines. However, it is equivalent to view the X's as the face centers of the cubes formed by the o's.

Now, if we add a two-atom carbon basis to every Bravais lattice point, we'll form diamond. The two-atom carbon basis is oriented such that one carbon atom is at every point X and o and one carbon atom is a quarter of a cube diagonal up from each X and o. A way to think of the diamond structure is as two interpenetrating face-centered cubic lattices, offset by a quarter of a cube diagonal. This would be hard to draw with ASCII art. However, if you're picturing it correctly, you'll notice that every atom in the diamond structure has four nearest-neighbor atoms located symmetrically in space, forming the vertices of a tetrahedron. This is the sp3 hybridized bonding structure.

Notice that when the two-atom basis is added to the Bravais lattice, the cubic 90-degree rotational symmetries are lost! Can you see the axes about which there are still 120-degree rotational symmetries?