to Americans, a 'button down shirt' means a dress shirt; one with a collar and buttons down the front. it's usually worn for work or 'events' - I don't know many men that chose to wear a button down shirt for just kicking back and relaxing.

you don't make that kind of distinction in the UK? Interesting!