I've recolored a smb1 spiny to smb3 but I noticed that it had no black outlines like the other npcs on the screenshots did.
I want to know how to add black outlines to the npcs correctly so it won't look strange and weird
You add them through Paint.NET. If you just recolor it as normally without adding any 2x2 black squares around the outside then it will appear like that in SMBX. If you add 2x2 squares around the NPC then it will show as an outline when you play the level.