one option:
upload the featured image seperately. then create a custom field column in csv as ‘_thumbnail_id’ the value of this cell should be the id of uploaded image. one can get the id of image by editing it. observe the url on edit options. you can find the id.