just a new chain? no
conversion kit means you go from a 525 to a 520 pitch chain, and new sprockets matching the new pitch. the sprockets (front & rear) might be different # of teeth also for different ratio, e.g. -1/+2 (popular sizing). i've seen some conversion kits called that sell same pitch chain but different size gears. but typically, you go for a smaller pitch for less weight, especially less rotating mass/unsprung weight.
does it gain you anything if you replace chain with same pitch and change the gears but same size? no
does it gain you anything if you replace the chain with same pitch and change the gears with different size? yes (going down 1 gear in front or increasing size in back will gain you better acceleration).
the question in here (for going from 525 to 520) is "does it really matter or are the performance gains worthwhile riding in the street"
"Leaders are like tea; put them in hot water to find out how strong they are." Anonymous