Make a joke about it

You lean back to him and whisper "It's nice to know that you're happy to see me" You playfully push him back and get out of the position. He laughs softly and later when you walk into the break room he is standing with a couple of your colleagues. He calls out to you "Hey, we're having a dinner tonight with some colleagues, do you want to come?"

Go to the dinner

Don't go to the dinner