What plugin do I need to upload a large number of categories into WooCommerce?

If you want to import a large number of taxonomies (categories, tags, attributes, etc) to your site, you need the WP Sheet Editor – Taxonomies spreadsheet.

How does the import work?

You can find a complete guide on how to import Taxonomies with Hierarchy here.