I mean it happened to me. I grew up as a Fortuna Düsseldorf fan, then lost interest in the sport altogether as a young teenager. Then I watched a few BVB games because Fredi Bobic played for them, and my friends were like "he supports BVB" and I was like "okay, I guess I do". But 10ish years ago my interest in Fortuna Düsseldorf came back. I visited a Fortuna game with friends, and while we were singing Düsseldorf songs in the stands like "Fortuna is my club, my heart belongs to Düsseldorf on the Rhine" I was like "hey...wait a minute... that is true."
I never had a connection to the city of Dortmund, and was born and lived in Düsseldorf all my life (apart from a few months in England). I was even embarrassed to wear BVB gear around here haha. So yea, I would say I was rather misguided for a time. But I found my way back home.
I really regret that I didn't support Fortuna at the time they were relegated to the 4th division. They played in the original small stadium, and the togetherness among the hardcore supporters must have been amazing. It is a shame that I missed that because I was a characterless glory hunting asshole.