Today, traffic congestion presents a real challenge in urban areas, with consequences beyond affecting commuters by increasing travel times and causing frustrating experiences. There has been consistent research using reinforcement learning methods to approach this issue. This manuscript considers a shift in this direction and proposes a novel intelligent controller based on a GPT-based decision transformer. Using transformer models will open significant opportunities for exploiting the available pre-trained transformer models. This paper prepared an offline dataset using the DQN agent and published it online, followed by training a decision transformer. The evaluation results show that the proposed method outperforms the DQN agent used for gathering the dataset; this manifests that transformers are promising in revealing knowledge that reinforcement learning agents cannot discover during their interaction with the environment.