I grew up in a family owned fish restaurant - my Dad owned a seafood market and cafe - and not only was it my first solid food, but I ate fish almost every day of my life for 13 years. Since I am writing this, you can assume I have not died from Mercury poisoning.
"Selenium, an essential element in our diet, is vital to the body's antioxidant system and proper immune function. It has anti-cancer effects and is known to detoxify the body of mercury"
You can get selenium from other foods such as Brazil nuts, sunflower seeds and onions.
You should be more concerned about the sustainability of the fish population than mercury toxicity.