So I was playing through BG1 using the BGT mod.

This time I'm using an evil party. (Me, Kagain, Viconia, Montaron & Xzar)
Currently my reputation is 4 by the time I was heading to Nashkel. =)

So, apparently, at that stage all the Nashkel Guards turn hostile and start attacking me the minute I enter Nashkel. I killed them all, sending my rep to the abyss. Then I reloaded and went to the temple to up my reputation.

But, and here's the problem, no matter how much I donate, I cannot increase my reputation and it stays at 4. Anyone knows exactly how to raise reputation?

Donations aren't cumulative, you need to donate a certain amount in one go to get to a higher level. Keep in mind that having Viconia in your party gives -2 to party reputation, so your rep is in fact 6.
I think you need to donate something like 500 gold or so to raise rep from 6 to 7. There must be an accurate table somewhere, maybe even in the manual.

Either way, even if you want to roleplay an evil character, you should try not to let your rep get too low as very low rep makes BG 1 unplayable by spawning endless amounts of tough guards. In BG 2, it's still punitive, but not too much so.

Edit: according to the BG 2 manual, you need 500 gold to get from 4 to 5. I may be different in BG 1, I'm not sure.

Yeah. I tried donating around 2k in one shot. And I still didn't get enough to raise my rep.

Man, both types of evil parties I like to run are unplayable in BG. =\

Generally, I think an evil villain should either be a manipulative Magnificent Bastard
or Godzilla going to town.

Just embrace the idea of an evil PC who knows how to trick the populace into thinking he's a good guy by helping people and 'reluctantly' taking their money. :p

Also, theft, theft, theft.

In BG1 you should never have to donate more than 500 gold to a church at a time to get a rep increase. For rep 4-6, you only need to give 200. So if the church was actually taking your money but not increasing your rep, then it's either a bug or the effect of some mod.

You should only need 4200 gold to get from 1 rep to 18 rep if you donate it at the correct increments (which are in the back of the manual) so there's definitely something strange going on. (In BG2 you'd need 20800 gold, but you also don't have to worry about random guards until 3 rep, as opposed to FLAMING FIST EVERYWHERE at 6 rep for BG1.)

BGT uses the BG2 engine so you need the higher donation amounts from the BG2 manual. If your reputation is 4 1000 GP should be enough to raise it by 1 if the manual is correct. Try if dismissing Viconia for a second while you're donating helps.

That's exactly how I typically play my Good-aligned Characters. :p

I tried that once with an evil character, but my evil party members kept whining,
and I had to kill them. =) Yup. BGT uses the BG2 values. After asking Viconia to wait outside, the Gods had accepted my 1k donations.

If you want, you can probably find some random guy out by himself and whack him for some rep loss. That oughta keep your party happy. :p