"i am not at all physically attracted to you"
is an absolutely valid reason to not want to date someone.
People had the nerve to call me shallow for this.
By the way, it’s also totally cool to turn someone down without explaining your reasons. You are not interested, no will suffice. Do not feel pressured to explain your decisions to someone else.