When most people use the term "serial port" they're referring to a DB-25 or DE-9 port you find on older computers or USB dongles. It's also seen in 8P8C (aka "RJ45") form sometimes, especially in industrial equipment. It can send and receive "characters" (anywhere from 5-8 bits each) one at a time at a fixed rate, either half duplex or full duplex. They usually implement one or more of the RS232, RS422, or RS485 standards.
Originally, you communicated with the computer using a teletype or video terminal connected to a serial port. Whatever you typed went to the computer, and whatever the computer sent back was printed on your terminal screen (or paper in the case of a teletype).
The UNIX (and thus, Linux) command line environment still works this way, except the serial line is virtual.