Hmmm... The twigs kind of remind me of apple, and so does the bark. But you say it isn't so. I don't see any obvious rays on the end grain, so, to me, that rules out oak.
I can see a little of elm in it, especially how the ends are a little fibery, and the color of the inner wood. Maybe slippery elm? However, I think I can barely see a white ring just under the bark on the top of one pic. That's a dead giveaway for American Elm. .
Do you have a pic of the bark that is not covered in moss?