I attach my AP to one of my router port directly. I don't have a Switch since the 4 ports from the N5105 that I have is enough to cover my home needs. And actually I only use 2 of the 4.
I use the first port for the WAN and the second for the LAN which is the AP connected. The AP have 4 more additional ports which I connect only one other device which is low in throughput and that's it. Mostly my entire house relies on the AP wireless if you know what I mean.
If I had way more devices or computers that requires cable access, I would consider doing a 2 tier network architecture.
EDIT: I am actually changing a bit how I implement stuff and made a small diagram on how I want things now:
View attachment 26691