You have to remove it all if you truely want it gone, as in no risk that any parts fail. Turning it off and replacing the vally cover still means the DoD lifter are still there and could still fail.

The intake manifold is not specific to the DoD/AFM either, there is only one intake for the LS4. Everything else comes from another motor GM makes if you wish to swap it.

Just turn it off and forget about it untill you actually want to tear deep into the engine.