programatically assign images to Twitter Publicize

I use Jetpack publicise to Facebook and Twitter. With Facebook I use this
function to add the right image:

function jeherve_custom_image( $media, $post_id, $args ) {...}

This works fine and Facebook shows the post with the image I specify in the
function.

But Twitter always shows the site logo. Note that most posts do NOT have any
featured image or even image in the body of the post.

How can I programmatically set the image which shows with Twitter?

Many thanks!