Weak-to-Gold Label Training of Spatio-Temporal Deep Learning Models for Cross-Region Coffee and Forest Segmentation

Deep learning models can effectively segment classes of interest from remote sensing data, which has applications for the real-time monitoring of agricultural crops. The European Union regulation on deforestation-free products has amplified the importance of monitoring coffee plantations and forests. However, there are limited previous works on automated coffee classification utilising deep learning methodologies, particularly with regards to separating it from forest, as well as across multiple different regions. One of the reasons for this is the lack of reliable coffee and forest labels available on which to train deep learning models. In order to overcome this limitation, we created a dataset containing a large number of poor-quality ‘weak’ labels and a very small number of high-quality gold-standard hand-annotated labels, associated with time series remote sensing patches. We then developed a weak-to-gold label pre-training and fine-tuning strategy to teach a spatio-temporal model both generalisability (from the weak label pre-training) and precision (from the fine-tuning). Our model achieved a per-class ‘coffee’ F1 score of 0.886 and ‘forest’ F1 score of 0.874. The model was also able to make accurate predictions in regions falling outside of our weak label dataset, demonstrating its capacity to be deployed for deforestation-free coffee monitoring tasks.