I set up a temporary solution to the problem. I made a script to manually set the correct links to the “First” and “Previous” buttons, which is why they are going to the correct pages currently, but that needs to be updated when new pages are uploaded.
By default those buttons are still behaving the way that I described in the OP, and I am still looking for a permanent solution. Unfortunately I don’t think there is a good way to display the bug now without breaking the navigation again. The biggest indicator are the classes being added to the arrows by the plugin. On the “Last” page, the “First” and “Previous” buttons are being assigned the “current-webcomic” by the plugin which indicates those arrows should be hidden, but I am overriding that behavior with CSS (that class should only be appearing on those arrows on the “First” page).