Some routers/switches/hubs call it an uplink port. The device would consider it an upstream connection as if it's connected to an ISP etc. On older devices they weren't self-detecting (for the type of connection - crossover or straight through (MDI/MDI-X)) so there's sometimes a little button next to the uplink port to toggle the uplink functionality or standard port use.

When you connect two switches it's called cascading or daisy-chaining if you want to look it up further. You shouldn't notice any degradation in speed. If there's a management interface just make sure you're running the ports at 100Mbit Full Duplex so there's no chance of them just using 10Mbit.

Will the computers connected to the second ethernet switch notice any slow down at all compared to those connected directly to one port each on the router?

If the 4 ports of the router are parts of an unique internal switch, then you can insert another switch in a router port, and connect all cabled computers to switch.