There was two gates of time in the game, one in the desert "Temple of Time" which is situated in Lanayru (where castle town is located in future game, at a time it is no longer a desert), one in the Temple of Hylia in the forest.
Desert Temple of Time >> OOT Temple of Time
Temple of Hylia >> TP Temple of Time
Desert Temple of Time >> OOT Temple of Time
Temple of Hylia >> TP Temple of Time