In the unappealing town of Suraxani, about an hour's drive from Baku is an astonishing fire temple, yet another example of Azerbaijan's rich geothermal connections.
The structure is not the most intricate religious monument, but a central plaza is surrounded by a cloister of pilgrim cells. The main fire comes from a small shrine in the middle of the courtyard.
The gas pocket below the shrine has become exhausted (sold by the last Hindu priest to the Baku Oil company in 1879) and so gas is piped in -- as needed. When it's lit, flames shoot from the four corners of the central hearth roof. Unfortunately, it isn't lit often, and this kind of situation has been reported as common by travelers dating all the way back to 1881.
There isn't much material on its history - some claim it was 6th or 7th Century Zoroastrians (who came from this region), while others say it was not a temple until 13th C. Ironically, the present is not Zoroastrian at all. Its present structure built by Indian merchants in the early 18th Century and it was used by Punjabi mystics (probably devotees of Jawala-ji, flame-faced incarnation of Devi) this this temple was probably a sub temple to those in Himachal Pradesh's Kangra Valley.
Infamous Caucasus writer Essad Bey (1905-47) "reports that it was briefly the centre of a weird pantheist religion in which reverence to mother earth was symbolized by kissing the breasts of naked female devotees" although his historical authenticity isn't really relied on!
The temple was visited by those as famous as Alexandre Dumas, and it remains a popular tourist site for visitors to Azerbaijan.
Source: Mark Elliot's Azerbaijan 4th Edition p 154-155