Artículo:
Page-Level Main Content Extraction from Heterogeneous Webpages

Fecha

2021-09-22

Editor

Sistedes

Publicado en

Actas de las XX Jornadas de Programación y Lenguajes (PROLE 2021)

Licencia Creative Commons

Resumen

The main content of a webpage is often surrounded by other boilerplate elements related to the template, such as menus, advertisements, copyright notices, comments, etc. For crawlers and indexers, isolating the main content from the template and other noisy information is an essential task, because processing and storing noisy information produce a waste of resources such as bandwidth, storage space, computing time, etc. Besides, the detection and extraction of the main content is useful in different areas, such as data mining, web summarization, content adaptation to low resolutions, etc. This work introduces a new technique for main content extraction. In contrast to most techniques, this technique not only extracts text, but also other types of content, such as images, animations, etc. It is a DOM-based page-level technique, thus it only needs to load one single webpage to extract the main content. As a consequence, it is efficient enough as to be used online (in real-time). We have empirically evaluated the technique using a suite of real heterogeneous benchmarks producing very good results compared with other well-known content extraction techniques. Publicado en: ACM Transactions on Knowledge Discovery from Data. Año 2021

Descripción

Acerca de Alarte, Julián

Palabras clave

Block Detection, Content Extraction, Information Retrieval, Template Extraction, Web Mining
Página completa del ítem
Notificar un error en este artículo
Mostrar cita
Mostrar cita en BibTeX
Descargar cita en BibTeX