Never chase him when he is playing this "game". That makes it super rewarding for him even if you are angry, yelling. If he is eating dirt, distract him into a different game. Make yourself more interesting than dirt!
Play lots of "trade" games with him all throughout the day. Any time he has something in his mouth, trade him for treats or toys, and then give the thing back to him and let him keep it. If it is something he is not supposed to have, let him end with keeping something else. When he is outside and has something, turn and run away from him. Usually a puppy will run after you, often dropping whatever he has in his mouth. If he doesn't drop it, try to get him to trade for a treat, toy, or even another piece of bark. The object is to get him interacting with you with it instead of playing keep away. Try to always end with letting him keep something, whether it is a treat or a toy.
Try to get into the habit of keeping a pocketful of treats on you, and having toys handy where you can grab them.
Bottom line is you have to watch him and distract him when he starts these behaviors, and show him that if he comes to you he will be rewarded with a fun game or a tasty treat, and he doesn't have to worry about you "stealing" whatever it is he has found. Sometimes you will take it, but you will always leave him with something good.