Yeah, if that chain is spinning while not in a cut, you're going to have oil all over the place. It's slinging off the end of the bar, and up into the clutch cover. So of course it will drip out afterword. Don't worry about that. The first, and easiest thing to look at is the LA or idle screw on your carb. All it does is preset the carb's gate to a certain open position. It doesn't mess with the jets at all. Find that screw-it'll be the one that screws onto the throttle lever and opens the carb. You should be able to see it plain as day. Back that screw out until the chain stops spinning. If that doesn't work, or doesn't work enough, you'll have to retune the carb. You likely should retune it anyway. That carb was probably adjusted way out of spec to compensate for it being old and worn, so you'll have to bring it back into spec.
Jeff