I think the biggest challenge people are having with ActivityPub is that they are, for the first time, generally encountering a "structured open world system" for the first time.
Closed world systems let you control (and prove) a lot, but you can only do the things your own "world" has built an idea up about. Also "easier" to build DB schemas for, etc. But hopeless for extensibility when talking with the rest of the world.
Meanwhile, completely free-for-all-json, coordinating on meaning is nigh impossible.
The main thing you can and should do: compact incoming json-ld to your local world knowledge with your local json-ld context. (This is a one-line operation with any json-ld library.) Then you can treat it as "just json" and all the fields will be things you know. Store the fields you don't know about somewhere so you can get to them if you need them but they can be off to the side a bit.