Go to the venues page in the left-hand navigation

Click the name of the venue you want to delete

Click the delete button at the bottom of the venue details

It will ask for confirmation, and it won't delete the venue if it has games assigned.