I was thinking this today. Had a play over 30 odd miles with a B5 RS4. They are classic cars, but there was virtually nothing in it in terms of speed - mine is running 350+ - milltek, AMD remap, filter - his was standard - running 380 i guess if still pushing out that power.
As said above, the B6 S4 is newer, better specced, safer and sounds better - and aside from the RS4 being a classic I really think the S4 looks as good in B6 form.
The RS4 can get more power easier and cheaper, but it's reliability is not as good as the S4.
Day to day, the S4 edges it for me.