Almost every host will offer PHP and MySQL these days; check the Wiki for a “Known Good Hosts” list.
That said, if you want really easy WordPress and lots of bandwidth, check out DreamHost (https://www.dreamhost.com). They’ve got a one-click WordPress installer on their control panel. I signed up with them a couple of weeks ago, and by using the promo code 777 (found via Google) I’m getting 800 MB disk space and 120GB monthly bandwidth with shell access for two years for about a hundred bucks. That’s a little over four bucks a month.
I haven’t been with them long enough to have a good sense of their speed or uptime, but they’ve been around since 1997 and they’ve responded promptly to my questions. I can’t say for sure that they’re the best, but they’re worth checking out.