I just checked the test page with multiple browsers and smooth scrolling, offset etc. works for me every time.
The issue you’re having is most likely because you’re using a plugin or theme that is loading images on-the-fly while the page is scrolling (lazy loading images).
When each image is loaded, it changes the documents length while page is scrolling. This means that the position of targets is also changing while page scrolling takes place (e.g. if the targets are placed after an image, this newly loaded image will “push” them further down).
The problem is that your image placeholders do not have the same height (and probably width) with the actual image that’s going to be loaded.
For example in your test page, the image placeholder at the bottom has an initial height of 677 pixels but the image loaded has a height of 540 pixels. This means that any target below this image will have an “incorrect” scrolling position of 127 pixels. This “incorrect” position will become “correct” after the image has loaded, that’s why the 2nd time you click the link it works correctly.
Lazy loading images affects all kinds of same-page links (i.e. anchor points). Even if you didn’t use “Page scroll to id”, you’d have the same issue. The page would “jump” to an id but if you had an image right before it, it would push this id further down after the image was loaded and rendered.
The only way to “fix” this is to have image placeholders that have the correct dimensions (esp. height) set (the same height as the actual image to be loaded). This of course might be complicated to do because the actual image elements are affected by certain CSS rules (like max-width: 100%;) that modify their height when rendered.
Hope all the above makes sense and explains what happens.
