Some routers/switches/hubs call it an uplink port. The device would consider it
an upstream connection as if it's connected to an ISP etc. On older devices they weren't
self-detecting (for the type of connection - crossover or straight through (MDI/MDI-X)) so
there's sometimes a little button next to the uplink port to toggle the uplink functionality
or standard port use.
When you connect two switches it's called cascading or daisy-chaining if you want to look it
up further. You shouldn't notice any degradation in speed. If there's a management interface
just make sure you're running the ports at 100Mbit Full Duplex so there's no chance of them
just using 10Mbit.
Will the computers connected to the second ethernet switch notice any slow down at all
compared to those connected directly to one port each on the router?
If the 4 ports of the router are parts of an unique internal switch, then you can insert
another switch in a router port, and connect all cabled computers to switch.