I did remove the exhaust manifold, too
Although others followed up on Hilton's question about whether he should remove the exhaust manifold on his 104 motor (different animal), I thought I would reply with what I actually did on my 103 motor. I completely removed both exhaust manifolds before I pulled the head. Probably I could have just disconnected the "down pipe" bolts and removed the manifold on the bench instead of in the car, but I think that taking the exhaust manifolds off makes the head easier to handle both for removal and reinstallation. For example, with the exhaust manifolds removed as I lowered the head back on, I did not have to worry about getting the down pipe bolts to align while I was worried about all the other alignments. Getting the exhaust manifolds back on does take a firm hand in flexing things to get it all to line back up, but I don't remember it being all that terrible.