Confusingly, man is not only the command to browse a manual page on a UN*X system, it is also the name of the troff and nroff (and therefore groff, too) macros "to lay out the reference pages in this [the UNIX] manual".
man macros exist to typeset the title and section headings of the manpage. Additionally, because it's just a simple text file, other commands such as whatis peek into the man-based "source code" for the manpage to fetch their data.
The man command is usually a horrible shell script (and it's completely different on every UN*X system!). It generally looks for a preformatted manpage; if it doesn't find one (and if the system has a *roff command...), it searches MANPATH for matching manual page, then formats it with a long pipeline. Somewhere in the pipeline you'll always find
... | nroff -man | ...
That's the man package coming into play.
gorgonzola points out that the name "man" is a horrible pun. All troff and nroff macro packages have names beginning with "m". "-m" is the switch used to select a macro package, but by punning the name is considered to start with an "m" -- see ms and me for more examples.