It's kind of strange question but let's give it a try. Several times after paying a bill in a restaurant in Japan, my friend was going to take a toothpick to carry it outside but a waiter said/warned something which we couldn't understand. As I know, it's nothing wrong to use it at the table if you hide your mouth. Is there some unwritten rule for carrying them out?
I did a bit of research and as far as I can make out, Japanese etiquette dictates that you do not use toothpicks in public i.e, you find a secluded spot to use toothpicks. However, since that is not practical, people try to cover their mouth with the other hand while using a toothpick (reference). I couldn't find any other reason for why your waiter might have stopped you from taking a toothpick. Your friend might have been picking his teeth in full view of everyone in the restaurant and your waiter probably did not want him to do the same outside the restaurant. A history of toothpicks in Japan can be found in this very well written Quora answer (reference).