I decided to call my indy to ask him about my transmission issues and he said he was very familiar with all of them and could solve them for just $50. I took him up on the offer, took the car to his shop and he softened the shifts a little but not too much to avoid damage to the clutches. Turned out my transmission modulator is fine, but there was such a big hole on my vacuum line that it was getting no vaccum. He solved it and the downshift clunk is no more.
Damn, I feel like kicking myself for living with that annoying clunk for so long when it could be fixed for just $50. The car drives like never before. Even my girlfriend noticed the absence of those clunks tonight when we drove to the movies.