Does he stay...or does he go?

This is Lynard, one of two cats that I adopted when my sister passed away. They lived peacefully with Ann because there were no other cats in her household.

Lynard is extremely skittish and rarely comes out of hiding. He won't come down the stairs to eat or, as it turns out, to use the cat box. He's apparently using my husband's closet to do his business, as evidenced by the pile of poo that I found this morning. (Husband has been in the Bay Area and I rarely go into his closet.)

In the old house, the territories were established and Nelson and Lynard lived in my bedroom. Food, water, cat box and all. It wasn't fair to them to be cooped up. But I thought the move to Colorado would wipe the slate clean and that some degree of peace could be achieved.

I originally took these cats into my home because I knew that my sister would have wanted me to do so, and I wanted to honor her in a way that would be important to her. I really, really don't want six cats. Four was my limit and was working out just fine. I am strained to keep Lynard because of the constant fighting with my other cats...they get along splendidly with one another. I'm awakened every morning at 3am to Lynard skreeching/hissing/growling/fighting with another cat. It's getting old.

This poor little guy didn't ask for any of these things to happen to him, I know. And the fact that he's being bullied constantly breaks my heart. The Three Insurgents seek him out, not the other way around, and I'm at a loss to know how to stop it. He's really only trying to defend himself.

Oy Vey.

Do I continue to try to make this work?
Do I give Lynard his very own cat box upstairs so he won't continue to go in the closet?
Do I put him up for adoption so he can live in a house without other cats to bully him?
What about his brother Nelson who is doing fine?
Do I separate them or put them both up for adoption?
How do I manage the guilt of doing this?
Will time resolve the territorial issues and can they ever "just get along?"
And how do I preserve the sanctity and cleanliness of my (new) home?

I just can't seem to find a direction of What's Best....for Lynard. For my sister's memory. For me.