Thanks for the advice. This is all very helpful.

To update, i ran another can of diesel purge directly through the injectors and it came out all black. The car runs a lot better now, but the idle fluctuation persists, although now it only happens when the car is not in gear. So, it will fluctuate while idling in neutral or park, but will smooth out and be almost perfect as soon as i shift into drive. This makes me think transmission vacuum issue.

Another thought was that it might be that there's a small head or head gasket leak and that coolant is getting into one of the cylinders at certain RPM. There are some bubbles in the radiator when i fill it all the way to the top, ie when it's overfilled and the water is just below overflowing. When it's at normal level there are no bubbles. If it is a head gasket, say, then it seems strange that it would be worse in different gears. it also smokes at idle intermittently, but today after i warmed it up a little bit it ran with no visible smoke at all for the rest of the day.

Very strange indeed.