Yes it can, in combination with other factors.
I'm Canadian and I crossed from the US into Mexico by motorcycle about five years ago. I was in Mexico for about four months and during that time, I took a flight to Bogota, Colombia to attend a tech conference.
When I crossed back from Mexico to the US at El Paso, I was handcuffed, detained, searched, and questioned. According to the agent, there were a combination of factors that caused them detain me... Mexico, Colombia, my mode of travel, etc. that added up to a red flag. I had no contraband was was admitted to the US after a couple of hours.
Since this event, when flying to the USA, I am always pulled in for secondary screening and in one case they searched all of my luggage carefully and questioned me about Colombia and narcotics for about 30 minutes.
I've heard of others in my extended network who've experienced similar.
I'd say that while you likely won't suffer any problems just for visiting Colombia, you may run into problems if there is anything else about your profile that attracts attention.
It's a shame, since as others have pointed out, Colombia is a great destination.