Black and orange bugs on milkweed plants are usually milkweed bugs and not box elder bugs. In big enough numbers, they can do significant plant damage.

You may see a black bug with orange stripes, called a Boxelder bug, on the trees of the same name, before they try to go inside. Other typical invaders include Asian Lady Beetles and Stink Bugs.