Ok, for your home network, it’s time to do some exercises, here is how I do physical network setup. I will use D-Link's DI-604 broadband router as an example. You can use other type of router according to your needs. Please prepare some straight network cable as well.
Connect the WAN port on router to your cable/DSL modem using straight cable, then connect computers’ network card to router’s LAN ports using straight cable also. You can connect up to 4 computers to this router. Power on the router after finish connecting, you should be able to see the WAN and LAN lights on the router.
Also you need to ensure that your DSL/Cable modem is configured in bridge mode, so that it can work well after connecting to router. If you need more idea, here is an example of configuring DSL modem in bridge mode, you can have a look if you like.