query_posts and pagination problem

Hi.

I am developing a magazine style theme for WordPress. I want to display the latest 3 posts on the front page and on the upcoming pages the default number of posts.

The problem is, it is displaying the first 3 posts on the front page and then on the next page it displays posts 11 to 20 or a 404 error if there are less then 11 posts published.

Here’s the code I’m using:

<?php if(!is_paged()) { ?>
  <?php $page = (get_query_var('paged')) ? get_query_var('paged') : 1;
    query_posts("paged=$page&posts_per_page=3"); ?>
  // The_Loop
<?php } ?>
<?php else { ?>
  <?php $page = (get_query_var('paged')) ? get_query_var('paged') : 1;
    query_posts("paged=$page"); ?>
  // The_Loop
<?php } ?>

So technically posts 4 to 10 are never displayed. Besides from that, the last page always displays a 404 error, although the link to it is displayed.

Any ideas what i’m doing wrong?