You need to have a name on your insurance certificate. You sorta need a name so people know what you go by, but I'm not certain if you need to register that name. Mebbe that's a state by state thing, but I can't even answer that for my own state. All I know is I never registered my business name until I incorporated and that wasn't until 7 years into the business.
You WILL need to pay taxes. A business name needs to go on the tax forms, but whether or not that name is registered somewhere, I dunno. You pay quarterly installments, and at the end of the year you compare how much you've paid with how much you owe. Gross income, minus the expenses to create that income is the amount upon which you are taxed (net income). You'll pay tax on around 30% of that net income. This IS a necessary evil.
Certified Public Accountant. He/she will be very helpful in getting you dialled in from the onset. It may cost you $300 or whatever to have him/her do your taxes at the end of the year, but they'll get done, and they'll get done right. Pay it and don't even worry about it.
I feel for you Bro. Getting started seems to involve a million details, as well as gear choices and learning the actual care of the tree (don't lose that focus).
I'm a one-man operation, and we share that. Experience in the ring is the only difference between us. I'm going to PM you with my insurer's name and #, not because I get a commission (I don't) but because you want to know what it will take to get insured, and what it will cost. Between the accountant, the insurer and Arboristsite, you are <i> there!</i> Stay confident. It seems complicated only because you're in the middle of it and have never walked through these doors.
With your insurer, you are wanting <b>commercial general liability</b>, $300,000 per occurence, which is generally (I think) the minimum you can purchase, though ask the price for half a million in coverage, just to know. Good luck.