This is a mis-feature of kresd when listening on ::0 (which Turris does by default, unfortunately). Upstream work-around is to bind to addresses explicitly; I’m not sure how to best do that on Turris.
Listening on ::0 results into OS selecting source address depending on the target address instead of using the address which accepted the query. In most cases the right address is selected even so, but not in all setups.