Rapidly-approaching dread: Chronic Illnesses & Tamoxifen

First, some background:

So in August, I (40f) had my first routine mammogram and was dxged with IDC, grade 1, stage 1a, ++- in my right breast. Very hormone responsive. No genetic abnormalities. 5mm tumor, removed via lumpectomy. My oncotype score was 13. No chemo required, and I'm 3/4 of the way through 20 rounds of hypofractionated radiation. Of course Tamoxifen is the next step: it's a miracle drug that could cut my relative risk of distant recurrence in almost half.

My personal reasons for Tamoxifen dread:

I have had CFS/ME & Fibromyalgia since I was 15 years old, and it has worsened over time. At this point, I'm disabled to the point I can no longer work a part time job outside the home. To illustrate where I'm at, after 15 rounds of radiation, the fatigue/exhaustion is indistinguishable from my baseline, everyday quality of life.

It's been that bad since about 3 years ago, when I was told to try various hormone therapies to treat my hypertonic pelvic floor. I then tried various combined BC, progesterone-only, and finally Mirena IUD. It all went...poorly. I experienced bizarre neuromuscular side effects that made my muscles tremble & weakened me to the point that I bruised a bone in my knee from falling on it too many times, while just trying to take a walk outside. I had my IUD removed in February, and barely had time to adjust & attempt to recover before I was diagnosed with the BC. Unsurprisingly, I am extremely deconditioned, and have gained a ton of weight.

Doctors just made puzzled faces and shrugged it off. Women's health is under-researched, and under-funded. I feel that my oncologist does not appreciate the potential carnage of hormone therapy on my chronic illnesses.

Obviously the thought of starting Tamoxifen terrifies me. I would much, MUCH rather leave things alone, and slowly work on my PT & exercise to build myself back up again. I'm struggling to see the benefit of being weaker and even heavier, more sedentary, and sicker for 5 of my prime years-- but with a few percentage points of risk shaved off. Meanwhile, I'm going to die of about 5 other things if I even make it another 20 years?

Sorry for the length of this rant-- but does anyone have any experiences/insight to share with me? Thanks in advance!