Pagination: posts limited only on first page

What I’m wondering about is why you are using query_posts in your index file. I think that may be part of the problem. Remove that and use a filter instead.

You could try:

add_action( 'pre_get_posts', 'custom_pre_get_posts', 1 );
function custom_pre_get_posts( $query ){
    if ( $query->is_home && !$query->is_paged ){
        $query->set( 'ignore_sticky_posts', true );
        $query->set( 'posts_per_page', 5 );
    } else {
        $query->set( 'posts_per_page', 10 );
        return;
    }
}

So in your index it would just be a simple loop:

while( have_posts() ): the_post();
    // inside the loop
    the_content();
endwhile;
// create navigational links
echo paginate_links();

Obviously you can make the changes accordingly.

Unfortunately the same issue:

add_action( 'pre_get_posts', 'custom_pre_get_posts', 1 );
function custom_pre_get_posts( $query ){
	if ( $query->is_main_query() ) :

		if ( $query->is_home && !$query->is_paged ){
			$query->set( 'ignore_sticky_posts', true );
			$query->set( 'posts_per_page', 5 );
		} else {
			$query->set( 'posts_per_page', 10 );

			$offset = ( (get_query_var('paged') - 2) * 10 + 5 );
			$query->set( 'offset', $offset );
			return;
		}

	endif;
}

It’s looks like because offset. To show excluded 5 posts from first page it have to be here. However it also moves last posts to overflow so they are no attached on last (41) page.

I am definetely sure that will be used only in front page, so I think it’s better to include code only in index.php due my functions.php are going little large. Also I am not wrong, php hooks could be called also after code that is destination (at least in downloaded for my own and <title> tag).

+ for cleaner set query variables. Just note that you lose $query->is_main_query() so code took effect also for my slider’ and widget’ posts.

Solution for future readers.

index.php:

global $query_string;
$my_query = '';

//Change it to number of posts that should appear on non-paginated main page
$front_limit = 5;

//I am not using sticky posts at all in main loop (only for slider), but haven't tested how it works with them, you could comment (add // on line start) and test
$my_query .= '&ignore_sticky_posts=1';

$paged = (get_query_var('paged')) ? get_query_var('paged') : 1;
$posts_per_page = get_option('posts_per_page');

$offset = ( is_paged() ? ( ($paged - 2) * $posts_per_page + $front_limit ) : 0 );
$my_query .= '&offset='.$offset;
$my_query .= '&paged='.$paged;

if ( !is_paged() ) :
	$my_query .= '&posts_per_page='.$front_limit;
else :
	if ( $paged == $wp_query->max_num_pages ) $posts_per_page += $front_limit;

	$my_query .= '&posts_per_page='.$posts_per_page;
endif;

query_posts( $query_string . $my_query );

Pagination (show number of pages properly), generally could be found in inc/template-tags.php, you should look for function like twentyfourteen_paging_nav(), then you have to add:

global $front_limit; 

	$total_posts = $wp_query->found_posts;
	$posts_per_page = get_option('posts_per_page');
	$page = get_query_var('paged');

	if ( is_front_page() ) {
		$args = array_merge( $wp_query->query_vars, array( 'showposts' => $posts_per_page ) );
		query_posts( $args );
	}

	$total_pages = $wp_query->max_num_pages;

before $something = paginate_links([...])

Note: you may need change variables names in pagination to match those in paginate_links() args.

Example for TwentyFourteen theme:

if ( ! function_exists( 'twentyfourteen_paging_nav' ) ) :
/**
 * Display navigation to next/previous set of posts when applicable.
 *
 * @since Twenty Fourteen 1.0
 *
 * @global WP_Query   $wp_query   WordPress Query object.
 * @global WP_Rewrite $wp_rewrite WordPress Rewrite object.
 */
function twentyfourteen_paging_nav() {
	global $wp_query, $wp_rewrite;

	// Don't print empty markup if there's only one page.
	if ( $wp_query->max_num_pages < 2 ) {
		return;
	}

	$paged        = get_query_var( 'paged' ) ? intval( get_query_var( 'paged' ) ) : 1;
	$pagenum_link = html_entity_decode( get_pagenum_link() );
	$query_args   = array();
	$url_parts    = explode( '?', $pagenum_link );

	if ( isset( $url_parts[1] ) ) {
		wp_parse_str( $url_parts[1], $query_args );
	}

	$pagenum_link = remove_query_arg( array_keys( $query_args ), $pagenum_link );
	$pagenum_link = trailingslashit( $pagenum_link ) . '%_%';

	$format  = $wp_rewrite->using_index_permalinks() && ! strpos( $pagenum_link, 'index.php' ) ? 'index.php/' : '';
	$format .= $wp_rewrite->using_permalinks() ? user_trailingslashit( $wp_rewrite->pagination_base . '/%#%', 'paged' ) : '?paged=%#%';

	global $front_limit; 

	$total_posts = $wp_query->found_posts;
	$posts_per_page = get_option('posts_per_page');
	$page = get_query_var('paged');

	if ( is_front_page() ) {
		$args = array_merge( $wp_query->query_vars, array( 'showposts' => $posts_per_page ) );
		query_posts( $args );
	}

	$total_pages = $wp_query->max_num_pages;

	// Set up paginated links.
	$links = paginate_links( array(
		'base'     => $pagenum_link,
		'format'   => $format,
		'total'    => $total_pages,
		'current'  => $paged,
		'mid_size' => 1,
		'add_args' => array_map( 'urlencode', $query_args ),
		'prev_text' => __( '← Previous', 'twentyfourteen' ),
		'next_text' => __( 'Next →', 'twentyfourteen' ),
	) );

	if ( $links ) :

	?>
	<nav class="navigation paging-navigation" role="navigation">
		<h1 class="screen-reader-text"><?php _e( 'Posts navigation', 'twentyfourteen' ); ?></h1>
		<div class="pagination loop-pagination">
			<?php echo $links; ?>
		</div><!-- .pagination -->
	</nav><!-- .navigation -->
	<?php
	endif;
}
endif;