Crops are ready when they are yellow, but you also have a chance to not gather anything. I think it breaks a pace a bit and that’s what may make you feel that you’re gathering them not in the right time!
Thank you for your honesty!