"The first time, I was like, 'This is going nowhere.' Literally, I think it was a 30-second conversation," Stephen Colbert recalled of meeting wife Evie

"As America's dad, I want to know what your intentions are with America's daughter, right over here," the host said