Yes, those odors could definitely permeate your beer which would really bum you out (unless you're going for that flavor). I'm all about doing things on the cheap but for $20 or less you can get a new 6.5 gallon food grade fermentation bucket with lid and spigot. On craigslist you can find them for less as local homebrewers are always selling off their equipment as they upgrade.
If you do try to clean out the smell I'd start by soaking it in oxyclean or pbw (professional brewers wash). You don't want anything too harsh that could etch the plastic making it harder to sanitize before fermenting.