links missing ‘index.php’

I just installed WP and everything seems to be more or less working. However, all links are missing ‘index.php’ in the path. In other words, in the admin, I’ll get ‘https://mydomain.com/wp/?cat=2’ instead of ‘https://mydomain.com/wp/index.php?cat=2′.

Is there an area in the config where I should be specifying the index file’s name?