Búsqueda avanzada

Page-Level Main Content Extraction from Heterogeneous Webpages

(Artículo ya publicado)


The main content of a webpage is often surrounded by other boilerplate elements related to the template, such as menus, advertisements, copyright notices, comments, etc. For crawlers and indexers, isolating the main content from the template and other noisy information is an essential task, because processing and storing noisy information produce a waste of resources such as bandwidth, storage space, computing time, etc. Besides, the detection and extraction of the main content is useful in different areas, such as data mining, web summarization, content adaptation to low resolutions, etc. This work introduces a new technique for main content extraction. In contrast to most techniques, this technique not only extracts text, but also other types of content, such as images, animations, etc. It is a DOM-based page-level technique, thus it only needs to load one single webpage to extract the main content. As a consequence, it isefficient enough as to be used online (in real-time). We have empirically evaluated the technique using a suite of real heterogeneous benchmarks producing very good results compared with other well-known content extraction techniques.Publicado en: ACM Transactions on Knowledge Discovery from Data. Año 2021

Palabras Clave:

Block Detection - Content Extraction - Information retrieval - Template Extraction - Web Mining





Acceso a los detalles haciendo click aquí.