From Extractive to Abstractive Summarization: A Journey

This book describes recent advances in text summarization, identifies remaining gaps and challenges, and proposes ways to overcome them. It begins with one of the most frequently discussed topics in text summarization –  ‘sentence extraction’ –, examines the effectiveness of current techniques in domain-specific text summarization, and proposes several improvements. 

In turn, the book describes the application of summarization in the legal and scientific domains, describing two new corpora that consist of more than 100 thousand court judgments and more than 20 thousand scientific articles, with the corresponding manually written summaries. The availability of these large-scale corpora opens up the possibility of using the now popular data-driven approaches based on deep learning. The book then highlights the effectiveness of neural sentence extraction approaches, which perform just as well as rule-based approaches, but without the need for any manual annotation. As a next step, multiple techniques for creating ensembles of sentence extractors – which deliver better and more robust summaries – are proposed. In closing, the book presents a neural network-based model for sentence compression. Overall the book takes readers on a journey that begins with simple sentence extraction and ends in abstractive summarization, while also covering key topics like ensemble techniques and domain-specific summarization, which have not been explored in detail prior to this.

2.1 Extractive Summarization

2.1.1 Legal Document Summarization

2.1.2 Scientific article Summarization

2.2 Ensemble techniques for extractive summarization

2.3 Sentence compression

3.1 Corpora

3.2 Legal document Summarization

3.2.1 Boosting legal vocabulary using a lexicon

3.2.2 Weighted TextRank and LexRank

3.2.3 Automatic key phrase identification

3.2.4 Attention based sentence extractor

3.3 Scientific article summarization

3.4 Experiment Details

3.4.1 Results

3.5 Conclusion

4.1 Motivation for rank aggregation

4.2 Analysis of existing extractive systems

4.2.1 Experimental Setup

4.3 Ensemble of extractive summarization systems

4.3.1 Effect of Informed fusion

4.4 Discussion

4.4.1 Determining the robustness of candidate systems

4.4.2 Qualitative analysis of summaries

5.1 Limitations of consensus based aggregation

5.2 Proposed approach for content based aggregation

5.3 Document level aggregation

5.3.1 Experimental results

5.4 Sentence Level aggregation

5.4.1 SentRank

5.4.2 GlobalRank

5.4.3 LocalRank

5.4.4 HybridRank

5.4.5 Experimental Results

5.5 Conclusion

6.1 Sentence compression by deletion

6.2 Sentence compression using Sequence to Sequence model

6.2.1 Sentence Encoder

6.2.2 Context Encoder

6.2.3 Decoder

6.2.4 Attention module

6.3 Exploiting SMT techniques for sentence compression

6.4 Results for sentence compression

6.5 Limitations of sentence compression techniques

6.6 Overall System

Dr. Parth Mehta completed his M.Tech. in Machine Intelligence and his Ph.D. in Text Summarization at Dhirubhai Ambani Institute of ICT (DA-IICT), Gandhinagar, India. At the DA-IICT he was part of the Information Retrieval and Natural Language Processing Lab. He was also involved in the national project “Cross Lingual Information Access”, funded by the Govt. of India, which focused on building a cross-lingual search engine for nine Indian languages. 

Dr. Mehta has served as reviewer for the journals Information Processing and Management and Forum for Information Retrieval Evaluation. Apart from several journal and conference papers, he has also co-edited a book on text processing published by Springer. 

Prof. Prasenjit Majumder is an Associate Professor at Dhirubhai Ambani Institute of ICT (DA-IICT), Gandhinagar and a Visiting Professor at the Indian Institute of Information Technology, Vadodara (IIIT-V). Prof. Majumder completed his Ph.D. at Jadavpur University in 2008 and worked as a postdoctoral fellow at the University College Dublin, prior to joining the DA-IICT, where he currently heads the Information Retrieval and Language Processing Lab. His research interests lie at the intersection of Information Retrieval, Cognitive Science and Human Computing Interaction. He has headed several projects sponsored by the Govt. of India. 

He is one of the pioneers of the Forum for Information Retrieval Evaluation (FIRE), which assesses research on Information Retrieval and related areas for South Asian languages. Since being founded in 2008, FIRE has grown to become a respected conference, drawing participants from across the globe. Prof. Majumder has authored several journal and conference papers, and co-edited two special issues of Transactions in Information Systems (ACM). He has co-edited two books: ‘Multi Lingual Information Access in South Asian Languages’ and ‘Text Processing,’ both published by Springer.