How do I set up security on my wireless modem/router

If You want WPA (or WPA2) not WPA. A "RADIUS" server provides even better security, but you don't have a RADIUS server, so ignore that part.

Choose a distinctive name and enter it in the SSID field. You will select this name when you go to connect from one of your wireless computers.

Select WPA. If you have an option for AES, select that. If not, select PSK

Leave the Group Key Interval at its default (typically 3600).

If you are located far from your neighbors, leave the channel at its default (probably 6 or 7). Otherwise, set it to 1 or 11. (Channel selection really doesn't much matter, unless you experience interference
from nearby wifi networks).

Pick a long, a non-dictionary-word password phrase for better security  and enter it in "PSK String." You will need to remember this to enter in your wifi connection later.

Acronyms

WPA - Wi-Fi Protected Access
PSK -  Pre Shared Key
SSID  - Service set identification