Welcome!

Java IoT Authors: Elizabeth White, Liz McMillan, Pat Romanski, Yeshim Deniz, Carmen Gonzalez

Related Topics: Java IoT

Java IoT: Article

Making PDFs Portable: Integrating PDF and Java Technology

Making PDFs Portable: Integrating PDF and Java Technology

Since Adobe released the first public PDF Reference in 1993, a number of PDF utilities and libraries, supporting all kinds of languages and platforms, have been made available to users and developers alike. However, support for Adobe's technology has lagged in Java application development. And this is curious because PDF documents tend to be a popular way of storing and interchanging information when dealing with enterprise information systems - an application domain that Java technology is particularly well suited to. Yet it seems that, until recently, mature, capable PDF support wasn't readily available to Java applications developers.

PDFBox (an Open Source project released under the BSD license) is a pure Java library that lets developers read and create PDF documents. It has features such as:

  • Extracting text, including Unicode characters
  • Easy integration with text search engines like Jakarta Lucene
  • Encryption/Decryption of PDF documents
  • Importing/Exporting of form data in FDF and XFDF formats
  • Appending to existing PDF documents
  • Splitting a single PDF into multiple documents
  • Overlaying one PDF document on top of another
PDFBox API
PDFBox has been designed to represent PDF documents using familiar object-oriented paradigms. The data contained in a PDF document is a collection of basic object types: arrays, booleans, dictionaries, numbers, strings and binary streams. PDFBox captures these basic object types in the org.pdfbox.cos package (the COS Model). While it's possible to create any desired interactions with a PDF document using only these objects, it requires an intimate knowledge of the internals of PDF documents and the techniques used to represent higher-level concepts. For example, objects such as pages and fonts are represented as dictionaries with specialized attributes; deciphering all these various attributes and their types requires tedious consultation of the PDF Reference.

For this reason, the org.pdfbox.pdmodel package (the PD Model) sits on top the COS Model and provides a high-level API that accesses PDF document objects in a more familiar manner (see Figure 1). Objects such as PDPage and PDFont can be found in this package, which encapsulates their lower-level COS model counterparts.

A word of caution to developers: the PD Model offers many nice features but is still a work in progress. In some instances, use of the COS Model may be required to access a particular piece of PDF functionality. Consequently, all PD Model objects can retrieve the corresponding COS Model object that they represent, so it's always possible to start with the PD Model and drop down to the COS Model when the required piece of functionality is found to be missing.

Now that the general capabilities of PDFBox have been discussed a few examples of its use are appropriate. We will start by reading an existing PDF document:


PDDocument document =
PDDocument.load( "./test.pdf" );

This operation will cause the PDF file to be parsed and an in-memory representation of the document will be created. To facilitate the efficient handling of large documents, PDFBox only stores the document structure in memory; objects such as images, embedded fonts and page content are cached in a temporary file.

Note: When finished using a PDDocument object, care should be taken to invoke the close() method on the document object to release resources used during its creation.

Text Extraction and Lucene Integration
In an information retrieval age when applications are expected to have searching and indexing capabilities regardless of the medium, the ability to organize and catalog information into a searchable format is critical. This is simple for text and HTML documents, but PDF documents have more structure and meta-information that makes it difficult to extract the underlying text. The PDF language is similar to Postscript in that objects are drawn as vectors on the page at certain positions. For example:


/Helv 12 Tf
0 13.0847 Td
(Hello World) Tj

This set of instructions changes the font to Helvetica size 12, moves the caret to the next line and renders the string "Hello World." These command streams are usually compressed and the order in which the glyphs are displayed on the screen is not necessarily the order in which the characters appear in the file, so it isn't always possible to simply extract text strings directly from the raw PDF document. However, PDFBox has a sophisticated text-extraction algorithm that deals with this and other complexities, letting a developer get the text of the document as if reading off its rendered form.

Lucene, which is part of the Apache Jakarta project, is a popular Open Source search engine library. Lucene lets developers create an index and do complex searches on a large volume of textual content based on that index. Since Lucene has adopted text as the common denominator for content, it's the developer's responsibility to convert the data contained in other desired file formats to text to use Lucene. For example, file formats such as Microsoft Word and StarOffice documents have to be converted to text before they can be added to a Lucene index.

PDF files are no exception, but PDFBox makes it easy to include a PDF document in a Lucene index by supplying a special object that does the integration. A basic PDF document can be converted to a Lucene document with a single statement:


Document doc = LucenePDFDocument.getDocument( file );

This operation parses the PDF document, extracts the text and creates a Lucene document object that can then be added to the index. As mentioned above, PDF documents also contain metadata such as author information and keywords that are important to track when indexing PDF documents. Table 1 shows the fields that PDFBox will populate while creating the Lucene document.

This integration makes it easy for developers to support simple searching and indexing of PDF documents with Lucene. Of course, some applications require more sophisticated text-extraction methods. In that case, the PDFTextStripper class can be used directly, or extended to handle these complex requirements.

By extending this class and overriding the showCharacter() method, many aspects of text extraction can be controlled. For instance, an implementation of this method can use the x, y positioning information to limit the inclusion of certain blocks of text in the extraction. One use might exclude all of the text above a certain y-coordinate value effectively excluding an unwanted document header.

Another example: Oftentimes a group of PDF documents may have been created from forms and the source data are no longer available. In other words, the documents all have some interesting text at similar locations on the page, but the form data used to fill the document out are no longer available. For example, a collection of cover letters that have the name and address at the same location in the document. In this case, an extension of the PDFTextStripper class can be used as a sort of screen-scraping device to extract the desired fields.

More Stories By Ben Litchfield

Ben Litchfield is a business systems consultant within the development & integration practice at LPA Systems. He has been the lead developer of PDFBox for the past two years. Ben holds a BS in Software Engineering from the Rochester Institute of Technology. He has been providing solutions for enterprise applications for the past five years.

Comments (3) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
Lucious 03/23/05 02:27:55 PM EST

I can't believe I found this!! I was searching for tools to update pdf files in my java programs. All I could find was commercial tools that would charge for single/multiple CPU and development liscenses and then charge MORE for deployment!! I actually gave up and downloaded the pdf specs (over 1200 pages) to develop tools of my own. I can't wait to start using these tools!

Maulik 03/23/05 08:09:43 AM EST

Great easy-to-follow article for someone who knows next to nothing about integrating Java and PDF. Good job.

Richard Bouchard 03/12/05 09:15:57 PM EST

Excellent article and has great utility.

@ThingsExpo Stories
SYS-CON Events announced today that MangoApps will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. MangoApps provides modern company intranets and team collaboration software, allowing workers to stay connected and productive from anywhere in the world and from any device. For more information, please visit https://www.mangoapps.com/.
WebRTC is bringing significant change to the communications landscape that will bridge the worlds of web and telephony, making the Internet the new standard for communications. Cloud9 took the road less traveled and used WebRTC to create a downloadable enterprise-grade communications platform that is changing the communication dynamic in the financial sector. In his session at @ThingsExpo, Leo Papadopoulos, CTO of Cloud9, will discuss the importance of WebRTC and how it enables companies to fo...
The IoT is changing the way enterprises conduct business. In his session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, discuss how businesses can gain an edge over competitors by empowering consumers to take control through IoT. We'll cite examples such as a Washington, D.C.-based sports club that leveraged IoT and the cloud to develop a comprehensive booking system. He'll also highlight how IoT can revitalize and restore outdated business models, making them profitable...
In his session at 18th Cloud Expo, Bruce Swann, Senior Product Marketing Manager at Adobe, will discuss how the Adobe Marketing Cloud can help marketers embrace opportunities for personalized, relevant and real-time customer engagement across offline (direct mail, point of sale, call center) and digital (email, website, SMS, mobile apps, social networks, connected objects). Bruce Swann has more than 15 years of experience working with digital marketing disciplines like web analytics, social med...
IoT generates lots of temporal data. But how do you unlock its value? How do you coordinate the diverse moving parts that must come together when developing your IoT product? What are the key challenges addressed by Data as a Service? How does cloud computing underlie and connect the notions of Digital and DevOps What is the impact of the API economy? What is the business imperative for Cognitive Computing? Get all these questions and hundreds more like them answered at the 18th Cloud Expo...
SYS-CON Events announced today that ContentMX, the marketing technology and services company with a singular mission to increase engagement and drive more conversations for enterprise, channel and SMB technology marketers, has been named “Sponsor & Exhibitor Lounge Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York City, New York. “CloudExpo is a great opportunity to start a conversation with new prospects, but what happens after the...
SYS-CON Events announced today the How to Create Angular 2 Clients for the Cloud Workshop, being held June 7, 2016, in conjunction with 18th Cloud Expo | @ThingsExpo, at the Javits Center in New York, NY. Angular 2 is a complete re-write of the popular framework AngularJS. Programming in Angular 2 is greatly simplified. Now it’s a component-based well-performing framework. The immersive one-day workshop led by Yakov Fain, a Java Champion and a co-founder of the IT consultancy Farata Systems and...
Customer experience has become a competitive differentiator for companies, and it’s imperative that brands seamlessly connect the customer journey across all platforms. With the continued explosion of IoT, join us for a look at how to build a winning digital foundation in the connected era – today and in the future. In his session at @ThingsExpo, Chris Nguyen, Group Product Marketing Manager at Adobe, will discuss how to successfully leverage mobile, rapidly deploy content, capture real-time d...
SYS-CON Events announced today that BMC Software has been named "Siver Sponsor" of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. BMC is a global leader in innovative software solutions that help businesses transform into digital enterprises for the ultimate competitive advantage. BMC Digital Enterprise Management is a set of innovative IT solutions designed to make digital business fast, seamless, and optimized from mainframe to mo...
What a difference a year makes. Organizations aren’t just talking about IoT possibilities, it is now baked into their core business strategy. With IoT, billions of devices generating data from different companies on different networks around the globe need to interact. From efficiency to better customer insights to completely new business models, IoT will turn traditional business models upside down. In the new customer-centric age, the key to success is delivering critical services and apps wit...
Join us at Cloud Expo | @ThingsExpo 2016 – June 7-9 at the Javits Center in New York City and November 1-3 at the Santa Clara Convention Center in Santa Clara, CA – and deliver your unique message in a way that is striking and unforgettable by taking advantage of SYS-CON's unmatched high-impact, result-driven event / media packages.
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, will provide an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life ...
SYS-CON Events announced today that MobiDev will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. MobiDev is a software company that develops and delivers turn-key mobile apps, websites, web services, and complex software systems for startups and enterprises. Since 2009 it has grown from a small group of passionate engineers and business managers to a full-scale mobile software company with over 200 develope...
SoftLayer operates a global cloud infrastructure platform built for Internet scale. With a global footprint of data centers and network points of presence, SoftLayer provides infrastructure as a service to leading-edge customers ranging from Web startups to global enterprises. SoftLayer's modular architecture, full-featured API, and sophisticated automation provide unparalleled performance and control. Its flexible unified platform seamlessly spans physical and virtual devices linked via a world...
Companies can harness IoT and predictive analytics to sustain business continuity; predict and manage site performance during emergencies; minimize expensive reactive maintenance; and forecast equipment and maintenance budgets and expenditures. Providing cost-effective, uninterrupted service is challenging, particularly for organizations with geographically dispersed operations.
SYS-CON Events announced today TechTarget has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. TechTarget is the Web’s leading destination for serious technology buyers researching and making enterprise technology decisions. Its extensive global networ...
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management...
SYS-CON Events announced today Object Management Group® has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
As cloud and storage projections continue to rise, the number of organizations moving to the cloud is escalating and it is clear cloud storage is here to stay. However, is it secure? Data is the lifeblood for government entities, countries, cloud service providers and enterprises alike and losing or exposing that data can have disastrous results. There are new concepts for data storage on the horizon that will deliver secure solutions for storing and moving sensitive data around the world. ...
The essence of data analysis involves setting up data pipelines that consist of several operations that are chained together – starting from data collection, data quality checks, data integration, data analysis and data visualization (including the setting up of interaction paths in that visualization). In our opinion, the challenges stem from the technology diversity at each stage of the data pipeline as well as the lack of process around the analysis.