Do you speak German?

I am often asked this question "Do you speak German now?".  As of right now I dont really know how I should answer this question and be factually accurate at the same time, for my dilemma is this.
If someone who doesn't speak German at all (which is the majority of the people that I know in my life) listens to me having a conversation with someone in German, he would think I am fluent whereas in  essence I would have to actually learn as much more German as I already do now to be fluent.

So the point I want to make is this. This question is invalid because there is no clear answer that will both satisfy the listener and myself. . All I can hope for is that the listener knows about the "common European Framework of Referance for Languages" and then I can simply say. B2 or whatever I want to say.

But it makes me wonder in how many others things does the same thing apply?  Maybe we are always on a line but instead choose to answer many questions with a "Yes I am fluent" without fully thinking about it.