Detect perspective based on image

Hi, my problem is that I have a photo of fuel prices pylon. Photo can be took from side or underneath and I need to correct the perpective like when you "scan" documents with smartphone. I know everything what I need including fitgeotrans but the problem is that I need to automaticaly detect anchor points. How should I even start? I though I could use Hough transform do higlight edges but it gives me rather rubbish then lines.

Example image for reference:

Answer 1

[yanqi liu]

yes，sir，what is target，is it to rectification image，such as

[Tomasz Klapyta]

That is pretty what I need. But it must be done automaticaly.

[yanqi liu]

should choose 4 point by hand to make transform