I'm only using 3 buttons: select start, D-pad, and A, but I won't be using B. So, can I not use it?

Sure, no problem. There's no requirement to use them all.

it would make more sense to use B instead of Start. the Start and Select buttons were generally only used for menus and stuff rather than gameplay. it probably doesn’t make a big difference though.