Reliable Rules for Relation Extraction in a Multimodal Setting

Relation extraction for automated knowledge base construction typically requires much training data. If these are not available for a specific information need, relations must be extracted manually, or by hand-crafted extraction rules [Wu18]. Data Programming can be used to define heuristics that generate noisy labels for many instances, but this requires programming knowledge [Di19]. We present an approach to extract relations from multimodal documents using a few training data. Furthermore, we derive explanations in the form of extraction rules from the underlying model to ensure the reliability of the extraction. Finally, we will evaluate how reliable (high model fidelity) extracted rules are and which type of classifier is suitable in terms of F1 Score and explainability.