After I release the gas pedal, the engine takes about four seconds to return to its normal RPM. It also does this if I pull on and then release the throttle under the hood. It does return to normal after three-four seconds.

It is a 1988 Plymouth Voyager 2.5L.

Take the air cleaner assembly or tube off and take a look at the throttle valve.
Pull on the throttle lever then release. Does the throttle blade snap shut right away, or does it close slowly.
If it's slow, try lubricating the blade and linkage with WD-40.
It it snaps closed right away, the trouble is elsewhere, such as an intake manifold vacuum leak, bad/dirty IAC,