There is definitely an improvement. Now the backslashes are not escaped anymore. But the semicolon still is.
In the no-cache version, this string looks like this:
\\\;\\p
With the updated wp-fastest-cache version, the source is changed to this:
\\;\\p
This causes KaTex to display the semicolon, where it should otherwise be just an empty line.
Thanks!