Better do so now than to have this professor sabotage your research.
And, if you fixate on his weird relationship with the other student, it's not going to help you-it will only hurt you in the long run.
They aren't going to ignore 4 people with the same complaint. There are another 2 Ph D students who graduated last year who had problems with him as well. I had a friend who once had to go to the provosts because his department kicked him out (they were racists) of his doc program and then tried to ghost him out of being able to sit for exams for a master's degree.
OP here: We have all decided to arrange a meeting with the chairman. We spoke to other people in our department but it didn't help much. He doesn't want to advise any of us and we have no choice but to go to the chair at this moment. After being ignored for weeks, one email to the provost got a same day reply and he had his exam scheduled nearly immediately by the department head who'd been pretending he didn't exist.
Given the many rules on colleges about relationships between teachers and students, this can be a bit tricky.
But, if you play your cards right during the semester, you can get your professor interested, learn a little more about him or her, and start a relationship with an educated professional.
The fact that he's married really pisses me off too. First - if there is something going on, the chances are very good that the university has regulations against this kind of thing.It doesn't matter whether the professor is married.I talked to him about changing his screen saver because of the impression it might give his students.And before people accuse me of homophobia, I would have said the same to a male professor with a screensaver full of gorgeous young women.