After months with nightmares, I found it was caused by the conflict between Safari and SafariStand.
I deleted SafariStand, and I got very smooth web surfing with Safari 4.
My fault was that I do not updated the SafariStand after I installed Safari 4.
SafariStand is installed on /Library/Application Support/SIMBL/Plugins/SafariStand.bundle .
I was mistakenly used SafariStand 3.1T161 with Safari 4.
Current version of SafairStand is 4.0T167 and it is compatible to Safari 4.