I've had someone say the same, and also "you don't look like you're in pain." I think anxiety is probably under diagnosed in men, because we are generally less expressive. The only sign I show is a tightening in the throat and a slightly strained voice. Maybe a restless leg if it's really badI quite recently started taking Propranolol which is a beta blocker. It works fairly well for me. It took a long time for me to persuade anyone that the anxiety I had been experiencing for a long time had really gotten too much for me to cope with or distract myself from.
I think I have a similar problem describing what's going on in my head to others. I also have had people tell me that on the outside I don't "look" like I'm really struggling. I don't seem like I'm "displaying" the typical signs that I'm really struggling with anxiety. My family will often tell me to not appear so composed. I suppose it's a form of masking that's just become a reflex.
I think in the past this has meant that I didn't get the help I needed because the doctors etc think "well you don't look anxious...".
I would recommend perhaps taking a family member or someone else you trust with you to your appointment with the doctor to help you advocate for yourself if you have a similar issue with looking too calm on the surface. That's what I had to do myself to finally get help with my anxiety.
Until the anxiety became so overwhelming, my natural tendency to get super focused on projects or solving problems would just kick in and distract me from the anxiety. For some reason I started to struggle doing this. But still, when I can focus on something, it's still the best way to keep the anxiety at bay. So don't give up trying to do things that help distract from the anxiety and if you do get some beta blockers, try to use their effects as an opportunity to get focused on something that helps distract you from the anxiety.
I really hope that you feel better soon! Anxiety is absolutely awful.