Again, apples and oranges.....As there appears to be legal action pending already, the owner of the property would have been served along with the people responsible for running the farm. If so, the owner is aware of what is going on and has taken no action, so yes the owner can be held responsible.
It also seems that the owner does come around regularly, so its a good bet he has some knowledge of what the farmer is doing.
If you have the permission to cut trees on a farm and the owner is aware of your intent, yes, it should be partly on him to make sure you aren't going to burn the place down by being negligent, ie have the proper training and equipment to work safely without creating a hazard for others. However, its his property, so if he allows you work in a manner that causes damage, ie a fire, he can have you charged with negligence. When you set the place on fire and somebody else gets hurt is when the owner can also be charged because he allowed you to do what you were doing.
And this is exactly why the focus needs to be on the legal stuff, not what was sprayed or if the farmer followed the directions. Negligence covers a broad spectrum that is not part of what the AG station has jurisdiction over. The don't even need to be involved except to confirm that the damage was herbicide related from the farm. This is a simple matter of a farmer spraying on a bad day and doing collateral damage to his neighbors property, not a fight over which herbicide does what or anything else related to farming regulations. Spraying herbicides into the wind so they carry onto adjoining properties and cause damage on those properties is simply negligent. I don't think there are negligence exemptions in the AG regulations, even in TN.....
That being said, it may take a good deal of time for the final outcome of this to be revealed. Its very important to document everything. Changes in the foliage over time, which plants and trees are effected and the extent of the effect etc. Try to keep a log, with lots of pictures, dates and times, have the inspectors return on a regular basis to monitor the damage and get written reports. Get weather reports from the NWS for the time period when the spraying occurred and onward. Most importantly, don't try to change or fix anything or do anything that could alter the appearance of culpability on the part of the farmer. In other words, let nature take its course here......there is probably little that you can do to stop whats happening anyhow......