Had the same bug... would enter the tower, and nothing would happen - couldn't complete the quest, and worse, it left me trapped in the tower with no way to get back out... :)
This is what worked for me... The wooden door on the outside of the tower is the quest-breaker... If it is open when you arrive, close the wooden door first, then open it again. After you do that, enter the tower, and approach the portal... Geralt should grumble about how much he hates portals, you step through, and then the portal should vanish behind you... the quest should continue normally from that point onwards... :)
Geralt talking and the portal closing did not happen for me while the outside door to the tower was open... instead, Geralt would walk through the portal without uttering a word, and once inside the tower the portal would remain instead of vanishing -- but it also be non-functional... you couldn't teleport back outside (essentially leaving you trapped inside the tower with no way out except to re-load and older saved game.)
