Breaking news, everyone! I did some tests and it turns out that the power meter (Favero Assioma Pro MX1) I used for the problematic BT rides actually shows values ca. 6,2% higher than my indoor wahoo trainer. That seems to be a pretty substantial difference. Luckily, the Favero app has an offset option. After the offset, the tests came out pretty close.
So hopefully this will solve my problems with the fitness signature.
In the meantime thanks to everyone who contributed and helped me troubleshoot the problem. A very helpful and responsive community here.