I just found out that my boyfriend gets cold sores and I'm scared because I have autoimmune problems.?

I have been having panic attacks after finding out that my boyfriend of 6-months has a lifelong history of cold sores. He has never had one while we've been together and says he gets one a year. I have never had one, never saw them growing up other than in health cl (seriously, and I'm 30) and am scared to death because I have a long history of autoimmune problems including past hospitalizations. Everything I read scares me as it always refers to the health of your immune system when describing how bad it could be. I feel like if I got it I would have terrible outbreaks. I am a very self-conscious and emotional person as well and feel the stress of it would be emotionally damaging. I care about my boyfriend and was beginning to feel like he was "the one," but feel like our trust is broken, especially because he knows my medical history and how concerned I am about my health and never told me about it. I have been very careful my whole life, waited a long time to have , had only one ual partner before him and always asked the STD questions. I'm seeing a doctor next week and will get tests for HSV-1 but I don't know if I test negative if it hasn't shown up yet or if I test positive if its because I was exposed years ago. We've had little (now I'm wondering if this is why) but I know it can be spread that way as well. I'm just scared. I feel like I'm going to lose this relationship and/or develop a terrible health condition I will have to deal with forever. :(