it's pretty obvious. place the frst waypoint using the waypoint button. then you can drag it to where you want the waypoint to start. Then Shift+click the waypoint to create another. Drag this new waypoint to the next location you want the waypoint to go to. Continue this process as if you were plotting a route on a map.
then use a waypoint related script for the character AI.
for more detailed information, and how to link 2 waypoints/waypoints across different floors read the manual. To be honest if you cant grasp it from the (very clear) explanation in the manual, then... well... it's plain english dude.
"Guns for show, knives for a pro."