I believe it's a balance. If you really dislike the person's looks, to the extent that you feel you won't be able to fufil his rights upon you, then it would be advisable not to go ahead with the nikkah.
If he is not the "ideal" but nonetheless, his deen, his character and his akhlaq are to pleasing to you, and you feel you are able to fufil his rights fully, then make Istikhara and mashwara and work on that

.
Ultimately, as a blunt answer, no, you do not "have" to find them attractive. Shariah has allowed leaway here. However, it would also not be incorrect to reject someone if you are not happy with their looks. Just keep a balance; realise that looks will eventually go with the passage of time, but someone established in deen

will be a source of khair throughout the marriage.
Wallahu Alam.

Bookmarks